Distributed voice capture and recognition system

[0002] The present invention relates to a distributed voice capture and recognition system.

[0003] Laptop computers were developed, at least in part, to permit users to perform computing at locations remote from their desktop computers. Such locations include hotels, airports, and business meetings. Laptops normally include scheduler and address book software which allows users to manage such information, while maintaining the capability of synchronizing such information with others and their desktop computer. Typically, data entry on laptops is performed using a keyboard or voice recognition software. Unfortunately, laptops are bulky and weigh several pounds which limits their portability.

[0004] Market demands for greater portability than that achievable with laptops have created a new class of electronic devices, known generally as personal data appliances (PDAs). PDAs are small hand-held electronic devices that include many features, such as an address book, a calculator, a date book, a memo pad, and a to-do list.

[0005] US Robotics has developed a hand held PDA, sold under the trademark Palm Pilot, having a touch screen display that includes handwriting recognition. The handwriting recognition is based on using a stylus (plastic stick) to draw each character on a portion of the display. The use of handwriting recognition requires the user to learn a scripting character set. Unfortunately, handwriting recognition lacks the accuracy and speed of a traditional computer keyboard. Also, entering letters using the stylus is especially difficult to perform in moving vehicles. In addition, PDA's lack the computing resources necessary to effectively perform speech recognition. Nonetheless, users of PDA's have demonstrated a willingness to perform data entry (creation of new appointments or contacts) on portable devices, such as PDAs, using these traditional data entry techniques (handwriting recognition, small keypad text entry, etc.) in spite of the cumbersome nature of these entry methods.

[0006] The application of database synchronization technology to the PDA has dramatically reduced the amount of data which must be entered directly into the PDA. Specifically, corporate phone lists can be automatically “synchronized” to the target PDA as can schedule information from the user's desktop time management software. Given this situation, a class of mainstream users who are unwilling to tolerate the peculiarities of PDA data entry are still able to benefit from the PDA and chose to simply scribble notes which are carried until they can be entered into a host personal computer which will then be used to synchronize with the PDA.

[0007] There exists credit card sized PDAs that do not even support on-device data entry. All updates to the credit card sized PDA must be performed on a personal computer which is later synchronized to the credit card sized PDA. The advantage of such a device is that it eliminates the computing resources necessary to support PDA-based data entry, and the size and cost of the resulting PDA can be dramatically reduced. Unfortunately, without the ability to update information on the PDA while away from the personal computer, the user is forced to carry handwritten notes which outline changes to the stored schedule. The longer the user waits between synchronization sessions, the less relevant the stored PDA data will likely become.

[0008] What is desired, therefore, is an input system for a hand-held electronic device that is convenient for users.

[0009] The present invention overcomes the aforementioned drawbacks of the prior art by a hand-held data acquisition device that includes a display presenting at least one of an address book, a date book, a memo pad, a to-do list, a contact manager, an expense tracker, an e-mail client, and a project manager, at least one of which contains multiple data items. An input device is operatively connected to the device is suitable to receive voice data from the user. The data acquisition device stores the voice data and associates the voice data with at least one of the data items.

[0010] The association of the voice sample with the data object(s) is an easy data entry technique requiring no textual entry for user of the device. The recording and retrieval of the voice sample alleviates the need for the user to use a stylus and the limitations of handwriting recognition to enter data into the device. In this manner, those users who are unwilling to tolerate the peculiarities of held-held device data entry are still able to more fully benefit from the hand-held device.

[0011] In the preferred embodiment the voice data is synchronized with a host system which may use voice recognition software to interpret the voice data. The textual representation of the voice data is then re-synchronized with the hand-held device to supplement or replace the voice samples. This allows for easier data entry and/or speech recognition capability, and permits the conversion of voice samples to textual information. In this manner, the hand-held device does not need to include substantially more computing resources because acquiring and storing voice samples is a relatively non-computationally intensive task.

[0012] The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention, taken in conjunction with the accompanying drawings.

[0013] FIG. 1 is an exemplary embodiment of an information appliance of the present invention.

[0014] FIG. 2 is an illustration of a daily schedule for the information appliance of FIG. 1.

[0015] FIG. 3 is an illustration of a weekly schedule for the information appliance of FIG. 1.

[0016] FIG. 4 is an illustration of a monthly schedule for the information appliance of FIG. 1.

[0017] FIG. 5 is an illustration of a listing of addresses for the information appliance of FIG. 1.

[0018] FIG. 6 is an illustration of a detailed information for each address for the information appliance of FIG. 1.

[0019] FIG. 7 is an illustration of a memo list for the information appliance of FIG. 1.

[0020] FIG. 8 is an illustration of a to-do list for the information appliance of FIG. 1.

[0021] FIG. 9 is a flow chart of an exemplary embodiment of the use of the information appliance of FIG. 1.

[0022] FIG. 10 is a block diagram of an information appliance system.

[0023] The present inventor came to the realization that hand held data devices, generally referred to herein as information appliances, include sufficient computing resources to facilitate the acquisition, storage, and retrieval of voice samples. Such information appliances include, for example, personal data assistants (PDAs), digital phones, and other hand held electronic devices. Further, the present inventor came to the realization that information appliances are suitable for associating the stored speech samples with other data elements stored within the information appliance.

[0024] Referring to FIG. 1, an exemplary embodiment of an information appliance includes a power on/off button 10 and a touch-sensitive display 12. A microphone 14 and a speaker 16 are included within the information appliance. Alternatively, the microphone and speaker may be separate devices that are connected to the information appliance. A record button 18 is included on the information appliance. A date book is activated by touching the applications icon 20 followed by touching the date book icon 24 appearing on the display, or alternatively the date book button 22. The date book includes a daily schedule (FIG. 2), a weekly schedule (FIG. 3), and a monthly schedule (FIG. 4). Each entry in the date book is associated with a particular time period and day which is observable by the user. An address book is activated by touching the applications icon 20 followed by touching the address icon 30, or alternatively the address book button 32. The address book includes a listing of addresses (FIG. 5) and detailed information for each address (FIG. 6). A memo pad is activated by touching the applications icon 20 followed by touching the memo pad icon 33 or alternatively the memo pad button 34. The memo pad includes a list of memos entered by the user (FIG. 7). A to-do list is activated by touching the applications icon 20 followed by touching the to-do list icon 36, or alternatively the to-do list button 38. The to-do list includes a list of items with a priority, a category, and a date due (FIG. 8). A portion of the display 40 may be used for data entry with a stylus, if desired. One. suitable information appliance is a US Robotics Palm Pilot as described in the Palm Pilot handbook, which is incorporated by reference herein. The information appliance may include additional functionality, such as a contact manager, an expense tracker, an e-mail client, and a project manager, as desired.

[0025] Referring to “FIG. 1 in combination with FIG. 9”, to use the information appliance the user selects one or more information appliance data objects (items), such as, for example, a person in the address book, an appointment in the date book, an item in the to-do list, and an item in the memo list. The user then initiates a voice recording session by pressing the record button 18 and speaking into the microphone 14 a memo to be attached to the data object(s). The information appliance acquires the voice and stores it as a voice sample, preferably in a digital format. The voice sample is associated with the data object(s). The user may listen to the voice sample through the speaker 16 by selecting the particular data objects with which the voice sample is associated. Subsequent viewing of the data object(s) will show the item as used or otherwise associated with a voice sample. The association of a voice sample with a data object(s) is an easy data entry technique requiring no textual entry for users of the information appliance (device). The information appliance, at this point, may replay the voice sample upon request but there is no associated textual representation. The recording and retrieval of the voice sample alleviates the need for the user to use a stylus and the limitations of handwriting recognition to enter data into the information appliance. In this manner, those users who are unwilling to tolerate the peculiarities of PDA data entry are still able to more fully benefit from the PDA.

[0026] In a further aspect of the present invention, the present inventor came to the realization that while the information appliance may lack the capability of interpreting the speech sample, the host system to which the information appliance is synchronized with may have speech recognition capability. When the user initiates a subsequent synchronization session after acquiring one or more speech samples associated with a data object(s), the speech samples are transmitted to the host system together with the remaining data. The host system process the voice samples to create a text-based representation. The user then edits and/or approves the text based representation. Alternatively, the host system may present the sampled speech to the user and allow manual entry of textual data representative of the sampled speech. The host system then synchronizes with the information appliance to replace the voice samples within the information appliance with the textual representation. As a result, the converted textual entry becomes indistinguishable from a traditional text based entry. Alternatively, the speech samples may be supplemented with the textual representation.

[0027] The integration of voice samples (memos) obtained by the information appliance together with later synchronization with the host computer system, which typically allows for easier data entry and/or speech recognition capability, permits the conversion of voice samples to textual information. In this manner, the information appliance does not need to include substantially more computing resources because acquiring and storing voice samples is a relatively non-computationally intensive task.

[0028] Referring to FIG. 10, a system includes the information appliance which is interconnected to the host system through any suitable communication link for the synchronization of data.

[0029] The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention, in the use of such terms and expressions, of excluding equivalents of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.