Portable electronic telecommunication device having capabilities for the hearing-impaired

[0002] This invention relates generally to portable computer systems. More particularly, this invention relates to portable electronic telecommunication devices, such as cellular telephones, and other portable telecommunications computers that include displays and have computer platforms that can execute programs for speech-to-text translation.

[0003] For hearing-impaired persons to communicate though a voice communications pathway, they typically use desk-top computer units that either receive a communication data stream from the communicating party or translate the actual words spoken by the communicating party and display the text to the hearing-impaired person. The hearing-impaired person has the text of the speech displayed to them, and can then either talk back to the communicating party if they are able, or the hearing-impaired can also have a text-to-speech conversion device that allows the person to type in text and the device translates the text into audible speech. The communication data stream can contain information that is meant to be interpreted by the device of the hearing-impaired person, such as data in a “closed-captioned” format. The devices that allow this type of communication are quite large and typically are interconnected with a home telephone.

[0004] Moreover, computers such as home computers are often used to provide this interaction with telecommunications for the hearing-impaired. The home computer has a platform with software resident to interpret the speech or data stream and make the appropriate translation. The home computer can further include hardware and/or software to interpret the text input at the keyboard and translate the text into audible speech across the communication line.

[0005] In regard to speech-to-text conversion, there are several commercial software packages available that convert speech to text particularly for purposes of dictation and document generation. Examples of such software products that perform speech-to-text translation are “Dragon” and “Via Voice” of IBM.

[0006] Portable telecommunication devices, such as cellular telephones, do not typically include means for usage by the hearing-impaired. Thus, the hearing-impaired cannot enjoy the benefits of readily portable telecommunications devices as they have no way to at least receive the speech communications from the communicating (calling) party.

[0007] Therefore, it would be advantageous to provide a portable electronic telecommunication device that can use existing software and products to allow a hearing-impaired person to use the device to communicate.

[0008] FIG. 1 is a perspective view of the portable electronic telecommunication device embodied as a cellular telephone.

[0009] FIG. 2 is a block diagram of the electronic organizer of the cellular telephone being a computer platform and interacting with the other components of the cellular telephone.

[0010] FIG. 3 is a flowchart illustrating the cellular telephone receiving speech through a communication link with a communicating party and then translating the speech into text.

[0011] FIG. 4 is a flowchart illustrating text input at the keypad interface being translated into audible speech and sent to the communicating party.

[0012] FIG. 5 is a flowchart illustrating speech from the user being translated into text data and being sent to the communicating party.

[0013] A preferred embodiment of the invention is now described in detail. Referring to the drawings, like numbers indicate like parts throughout the views. As used in the description herein and throughout the claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise: the meaning of “a,” “an,” and “the” includes plural reference, the meaning of “in” includes “in” and “on.”

[0014] Referring now to FIG. 1, illustrated therein is a portable electronic telecommunication device embodied as a cellular telephone 10 which has text-to-speech and speech-to-text translation capabilities so that it can be used by a hearing-impaired user. The cellular telephone 10 includes a keypad 12 having an electronic keypad interface, and a graphic display 14 that can display alphanumeric text and other graphics to the user of the cellular telephone 10 as well as others who can view the display 14. The cellular telephone 10 further has one or more speaker apertures 16 that provide audio access to a speaker which is typically placed next to the user's ear during conversation, and a microphone aperture 18 which provides audio access to a microphone such that the user can speak into the aperture 18 and the microphone converts the speech into electronic signals for transmission from the cellular telephone 10.

[0015] The cellular telephone 10 also may have a computer platform in the form of an electronic organizer 22, as shown in FIG. 2, that can interface with some or all of the other components of the cellular telephone 10. Any cellular telephone or other portable electronic telecommunication device, such as personal digital assistants, that are well known in the art can be used in the present invention as long as the computer platform has the requisite software processing capability as is further described herein.

[0016] The electronic organizer 22 of the cellular telephone 10 and its interaction with the other components is particularly shown in FIG. 2. The electronic organizer 22 has storage for one or more programs and interacts with the various components of the cellular telephone 10. The electronic organizer 22 particularly interfaces with the communications interface 24 that ultimately receives and transmits communication data to a communication network 26, such as a cellular network or a broadcast wide area network (WAN). The electronic organizer 22 may interface with the keypad interface 28, the graphic display 30, and the audio interfaces 32. The keypad interface 28 may receive and interpret the input from the keys of the keypad 12 of the cellular telephone 10. The graphic display screen 30 displays alphanumeric text and other graphics on the display 14 of the cellular phone 10, and displays the text of the translated speech of the communicating party to the hearing-impaired user. The audio interfaces 32 are electronic interfaces for the physical components of the speaker and microphone (behind respective apertures 16 and 18 on cellular phone 10), each translating electronic signals either from or to audible speech.

[0017] The electronic organizer 22 may include the speech recognition translator program module 34 resident and selectively executable thereupon, and when called, the program module 34 translates speech from either the communicating party or the user of the telephone into a data stream of text comprised of text words ideally for each spoken word. The process of the speech translation is particularly shown in FIG. 3. There is also a text-to-speech translator program module 36 that is resident on the electronic organizer 22, and the program module 36 translates text input from the keypad interface 28 into speech though the process shown in the flowchart of FIG. 4, as is further discussed herein. The electronic organizer can further have additional program modules that allow the cellular telephone 10 to receive communication data streams from the communication network 26 and the communications interface 24 and display the text of the information from the communication data stream on the graphic display 14 with graphic display screen control 30. The communication data stream can be in known formats such as closed-captioning or other data formats commonly used by the devices of the hearing impaired.

[0018] In use, the cellular telephone 10 can either make or receive calls and selectively activate the speech-to-text translator 34 or text-to-speech translator 36 on the electronic organizer 22 to have the communicating party receive either a speech or text stream from the user. Further, if the communicating party is a calling party, the call itself can prompt one of the translators to be executed at the connection of the incoming phone call. This is especially important to the hearing-impaired user as he or she cannot identify the source of the call, i.e. voice or data, immediately upon connecting the call.

[0019] With reference to FIG. 3, there is shown a flowchart illustrating the cellular telephone 10 receiving speech through a communication link from the communication network 26 with a communication party and then translating the speech into text. A communication connection is established between the cellular telephone 10 and a communicating party, as shown at step 40. The communication connection can be either making or receiving a call from the cellular telephone 10. If the communication connection has a voice stream being sent to the cellular telephone 10, then the electronic organizer 22 received the voice stream, as shown at step 42, and calls the speech-to-text translator 34. Each word in the voice stream is then parsed, as shown at step 44, and then a determination is made as to whether the parsed word is known a resident dictionary on the electronic organizer 22, as is shown at decision 46.

[0020] The term dictionary is simply a data store of the electronic signatures of words. To identify a word, the electronic signatures of each word are compared in the dictionary to determine the text equivalent. Other speech-to-text conversion programs such as Dragon and Via Voice can be used on the computer platform (here electronic organizer 22) as well.

[0021] Referring again to FIG. 3, if the word is not located in the dictionary, then the electronic organizer stores the word for later review, as shown at step 48. While the simple storage of the unknown word is not a necessary step, it is advantageous because the voice stream will continue to be processed and the continuity of conversation is not lost. The stored words can later be reviewed to determine if there was an error in interpretation or if the words are new and should be added to the dictionary. If the word is located after comparison in the dictionary, then the text word is obtained from the dictionary, as shown at step 50, and then the text word is sent to the graphic display screen control 30 and ultimately displayed on the graphic display 14 of the cellular telephone 10, as is shown at step 52. If sufficient memory is present in the electronic organizer, the entire text from the communication can be saved and selectively recalled.

[0022] With reference to FIG. 4, there is shown a flowchart illustrating text input at the keypad 12 and is interpreted by the keypad interface 28 being translated into audible speech and sent to the communicating party. The text being typed by the user is received at the electronic organizer 22 from the keypad interface 28, as shown at step 56. The electronic organizer 22 calls the text-to-speech translator 36 and then each word of text is parsed, as shown at step 58.

[0023] A decision is made as to whether the parsed word is in the dictionary, as shown by decision 60, and if not, the user is prompted to reenter the word, as shown by step 63. Then control of the executing text-to-speech translator 34 is returned to step 56 so that the electronic organizer 22 waits until it receives the reentered word. If the text word is recognized at decision 60, then the text word is translated to is speech equivalent, as shown at step 64, and the electronic signal to create the audible word from the textual word is sent to the communication interface 24, as shown at step 66. There are several software programs in the art which can generate the electronic signals to speakers that can recreate speech sufficient to audibly communicate words.

[0024] FIG. 5 illustrates a flowchart for the process of translation of speech from the user of the cellular telephone 10 into text data and, the text data sent to the communicating party. The speech-to-text translator 34 is activated on the electronic organizer 22, as shown in step 70, and then the voice stream is received in electronic form at the electronic organizer 22 from the audio interface 32 for the microphone. Each word in the voice stream is parsed, as shown at step 74, and then a decision is made as to whether the word is known in the dictionary, as shown at decision 76.

[0025] If the word is not known (i.e. located) then the user is prompted to restate the word, either at the display 14 or audibly at the speaker (through speaker apertures 16), as shown at step 78. Then the program returns to step 72 to one again receive the word spoken by the user. If the word is known in the dictionary, then the textual equivalent of the word is obtained from the dictionary, as shown at step 80, and the text is transmitted to the communication interface 24 for ultimate transmission across the communication network 26 to the communicating party, as shown at step 82.

[0026] While the preferred embodiments of the invention have been illustrated and described, it is clear that the invention is not so limited. Numerous modifications, changes, variations, substitutions, and equivalents will occur to those skilled in the art without departing from the spirit and scope of the present invention as defined by the following claims. For example, while one preferred embodiment is a cellular telephone, the invention could equally be applied to two-way radios, two-way pagers, and the like.