Technique for recognizing telephone numbers and other spoken information embedded in voice messages stored in a voice messaging system

[0002] The present invention relates to the field of natural language speech recognition. More particularly, the present invention relates to the field of recognizing telephone numbers and other spoken information embedded in natural language voice messages stored in a voice messaging system.

[0003] Natural language is written or spoken language in a form that a person would use instinctively, as when communicating with another person. Natural language recognition systems are known which permit a user to interface with a computer system using natural language. The natural language recognition system receives spoken input from the user, interprets the input, and then translates it into a form that the computer system understands.

[0004] For example, a voice recognition application program is available from Dragon Systems, Inc., which enables a user to enter text into a written document by speaking the words to be entered into a microphone attached to the user's computer system. The application interprets the spoken words and translates them into typographical characters which then appear in the written document displayed on the user's computer screen.

[0005] Natural language recognition systems are known which provide a telephonic interface between a caller and a customer service application. For example, the caller may obtain information regarding flight availability and pricing for a particular airline and may purchase tickets utilizing natural spoken language and without requiring service from an airline reservations clerk.

[0006] Telephone voice messaging systems are known which enable a caller to leave a voice message for a system subscriber who is temporarily unavailable to take the call. Often, the voice message conveys specific and important information required by the subscriber along with other information. As a first example, a voice message may contain a personal greeting from caller, the reason for the call, and a telephone number at which the caller can be reached. As a second example, a voice message may contain a personal greeting from the caller, a reminder that a meeting is to take place, an agenda for the meeting, a time of day for the meeting and a street address for the meeting. In each of these examples, the recipient is likely to immediately assimilate all of the information conveyed by the voice messages, except that, in the first example, the recipient is likely to need to write down the telephone number of the caller and, in the second example, the recipient is likely to need to write down the time of day and the street address. Transcribing this information can be inconvenient and time consuming for the recipient, especially if the recipient needs to rewind or replay the message repeatedly to accurately transcribe information from the voice message.

[0007] Therefore, what is needed is a technique for automatically identifying and interpreting or outputting telephone numbers and other information embedded in a voice messages stored in a voice messaging system.

[0008] The invention is a method and apparatus for recognizing telephone numbers and other information embedded in voice messages stored in a telephone voice messaging system. A voice recognition system in accordance with the present invention is coupled to the telephone voice messaging system. A voice message stored in the voice messaging system is transferred from the voice messaging system to the voice recognition system. Alternately, the voice message can be provided to the voice messaging system while the message is being recorded. The voice recognition system identifies potential speech utterances in the voice message and segments the voice message into a plurality of such utterances. Alternately, the voice recognition system 100 acts on the message as a whole. The voice recognition system then searches the segments or the message as a whole for a predetermined speech reference model which is expected to contain information of importance to the recipient of the message. This reference model is called a grammar. In the preferred embodiment, a predetermined grammar is a numeric grammar which specifies a sequence of numbers occurring in the voice message which correspond to a telephone number. In alternate embodiments, the grammar can specify a date, a time, an address, a person's name and so forth. According to an aspect of the present invention, the grammar can be modified or selected by the recipient of the voice message so that the voice recognition system searches for information of particular interest to the recipient.

[0009] Once the predetermined grammar is identified, the voice recognition system outputs a portion of the stored voice message which includes the grammar. The output can be a display of the information contained in the grammar, such as a telephone number or an address. Alternately, the output can be an audible replay of just the portion of the stored voice message which includes the telephone number or address.

[0010] Important information included in the stored message may be accompanied by an indicator that important information is to follow or that important information was just given. For example, the voice message may contain a telephone number. In which case, the caller is likely to have preceded the telephone number, by saying, “you can reach me at”, “my phone number is”, “call me at”, “my extension is”, or the like. Accordingly, the indicator can be one of these phrases. As another example, the voice message may contain an address which is important. In which case, the caller is likely to have preceded a street number and street name by saying, “the address is”. Alternately, the speaker may have followed a street number and street name by saying “street”, “road”, “avenue”, “lane”, or the like. Accordingly, the indicator can be one of these phrases.

[0011] In accordance with the present invention, the predetermined grammar can specify an indicator of important information along with the important information. In which case, the voice recognition system searches the voice message for the indicator in conjunction with the important information. Alternately, separate grammars are specified for the indicator and the important information. In which case, the voice recognition system first searches the voice message for the predetermined indicator. Assuming a predetermined indicator is identified, the voice recognition system then searches for the important information in the vicinity of the indicator. Thus, assuming the indicator is “my phone number is”, then a numeric grammar which follows the indicator is expected to include a string of numbers, such as “555-1234”. As another example, assuming the indicator is “road”, then a grammar which preceded the indicator is expected to include a number and a street name, such as “1380 Willow”. Finally, the recognized grammars are outputted in a form that the recipient of the message can readily utilize. For example, the telephone number, “555-1234” or the address “1380 Willow” can be displayed or audibly reproduced for the user. In addition, the indicator can be combined with the important information. Thus, the entire utterance, “my phone number is 555-1234”, or “1380 Willow Road” can be provided to the recipient. Use of an indicator of the important information is expected to increase the recognition accuracy of the invention.

[0012] The invention provides an advantage to users of voice mail and other audio messaging systems by recognizing telephone numbers and other information embedded in a stored voice message and providing this information in a form which is readily accessible to the user.

[0013] FIG. 1 illustrates a speech recognizing system in conjunction with a telephone voice mail system in accordance with the present invention.

[0014] FIG. 2 illustrates flow diagram for recognizing telephone numbers and other information embedded in a voice message in accordance with the present invention.

[0015] FIG. 3 illustrates a flow diagram for recognizing information selected by a recipient of a voice message in accordance with the present invention.

[0016] FIG. 4 illustrates a flow diagram for searching a voice message for a predetermined indicator in accordance with the present invention.

[0017] FIG. 1 illustrates a speech recognizing system 100 in conjunction with a telephone voice mail system 150 in accordance with the present invention. The speech recognizing system 100 includes a general purpose processor 102, a system memory 104, a mass storage medium 106, and input/output devices 108, all of which are interconnected by a system bus 110. The processor 102 operates in accordance with machine readable computer software code stored in the system memory 104 and mass storage medium 106 so as to implement the present invention. The input/output devices 108 can include a display monitor, a keyboard and an interface coupled to the telephone voice mail system 150 for receiving voice messages therefrom. Though the speech recognizing system illustrated in FIG. 1 is implemented as a general purpose computer, it will be apparent that the speech recognizing system can be implemented so as to include a special-purpose computer or dedicated hardware circuits. In which case, one or more of the hardware elements illustrated in FIG. 1 can be omitted or substituted by another.

[0018] The voice mail system 150 can be coupled to telephone system (not shown), such as a central office of a telephone service provider or a private branch exchange, for receiving and recording messages from callers intended to be retrieved at a later time by a subscriber to the voice mail system 150. It will be apparent that the voice mail system 150 can alternately be incorporated into the voice recognizing system 100 and implemented by the processor 102 operating according to a stored software program. In which case, voice messages are stored in the memory 104 or mass storage 106, rather than externally to the speech recognizing system 100.

[0019] A caller leaves a voice message for an intended recipient subscriber who is temporarily unavailable to receive the call. The voice message is then stored by the voice mail system 150 as natural language speech. The message can then be audibly replayed by the intended recipient at the recipient's convenience. The stored message is expected to contain specific and important information required by the recipient along with other information. As a first example, the voice message may contain a personal greeting from caller, the reason for the call, and a telephone number at which the caller can be reached. As a second example, the voice message may contain a personal greeting from the caller, a reminder that a meeting is to take place, an agenda for the meeting, a time of day for the meeting and a street address for the meeting. In each of these examples, the recipient is likely to immediately assimilate all of the information conveyed by the voice messages upon a single playback of each message, except that, in the first example, the recipient is may not remember the entire telephone number of the caller and, in the second example, the recipient may not remember the time of day and the street address.

[0020] Conventionally, the recipient replays the message repeatedly until able to write down all of the important information or commit it to memory. In accordance with the present invention, the important information is provided to the user in a more convenient and readily accessible form. More particularly, the voice recognition system 100 of the present invention receives the voice message from the voice mail system 150 and extracts the important information from the message, such as telephone numbers or other information, required by the recipient of the message and provides this information to the recipient in a form that is convenient and readily accessible by the recipient.

[0021] FIG. 2 illustrates a flow diagram for recognizing telephone numbers and other information embedded in voice messages in accordance with the present invention. The flow diagram of FIG. 2 illustrates graphically operation of the speech recognizing system 100 in accordance with the present invention. Program flow begins in a start state 200. From the state 200, program flow moves to a state 202. In the state 202, the voice recognition system 100 receives a voice message from the voice messaging system 150. The voice message can be received while the voice message is being recorded, upon playback of the message by the voice messaging system 150 or upon initiation by the intended recipient of the voice message. Note that the voice recognition system 100 preferably acts upon the voice message while the recipient is listening to the voice message being played back. This allows the extracted important information to be provided to the recipient prior to playing back the message, immediately upon conclusion of the message or soon thereafter.

[0022] From the state 202, program flow moves to a state 204. In the state 204, the voice recognition system 100 identifies potential speech utterances in the stored voice message and segments the voice message into a plurality of such utterances. Alternately, the voice recognition system 100 acts on the message as a whole. The voice recognition system 100 then searches the segments or the entire message for a predetermined speech reference model which is expected to contain information of importance to the recipient of the message. This reference model is called a grammar. In the preferred embodiment, a predetermined grammar is a numeric grammar which specifies a sequence of numbers occurring in the voice message. It will be apparent that the predetermined grammar could be modified to specify a date, a time, an address, a person's name, or any other information of interest that may be contained in a stored voice message. Alternately, the predetermined grammar can be replaced with a plurality of predetermined grammars. In which case, the voice recognition system 100 can search for a plurality of information types in the state 204. For example, voice recognition system can search for a telephone number and time of day in the state 204.

[0023] In addition to specifying the information of interest, the predetermined grammars may also model superfluous speech or sounds surrounding the information bearing speech in the voice message. For instance, a telephone number grammar may specify a sequence of zero or more words of any kind, followed by a sequence of numbers, followed by a sequence of zero or more words of any kind. Alternatively, it could specify a sequence of zero or more sounds of any kind, followed by a sequence of numbers, followed by a sequence of zero or more sounds of any kind, where the sounds may be, for example, all the phonemes of the English language. When the grammar matches a portion of the voice message, the sequence of numbers matched would correspond to the information of interest (e.g., the telephone number), and the possible initial and final sequences would correspond to superfluous speech surrounding the telephone numbers in the message. In such an approach, a weighting is preferably placed in the grammar to prevent the part of the grammar modeling the superfluous speech or sounds in the voice message from matching the information bearing part of the voice message.

[0024] If the search performed in the state 204 is successful in locating the predetermined grammar (with at least a predetermined level of certainty), this results in the identification of a portion of the voice message which includes the predetermined grammar or grammars (e.g., the portion containing the telephone number is identified.)

[0025] Then program flow moves from the state 204 to a state 206. In the state 206, a determination is made as to whether the search performed in the state 204 was successful in locating a predetermined grammar. If the search was not successful, program flow is aborted for corrective action. For example, program flow returns to the state 202 to await reception of a next voice message. Alternately, the corrective action can include repeating the search.

[0026] Otherwise, if the search was successful, program flow moves from the state 206 to a state 208. In the state 208 the voice recognition system 100 outputs a portion of the stored voice message which includes the grammar. In the case where multiple predetermined grammars are identified, then multiple portions of the message can be outputted. The output can be a display of the important information contained in the grammar, such as a display of the caller's telephone number. In which case, the voice recognition system 100 interprets the information contained in the grammar in accordance with conventional voice recognition techniques and displays this information for the recipient as a series of alphanumeric characters. For example, the alphanumeric characters can appear on the monitor included in the input/output devices 108. Alternately, the output can be the information spoken by a synthesized voice. If the information is a telephone number, the output can be dual-tone, multi-frequency (DTMF) tones which are applied to an outgoing telephone line so as to initiate return of the call to the caller.

[0027] A disadvantage of interpreting the important information and then displaying it for the recipient is that it is possible for an error to be introduced during the interpretation process depending upon a variety of factors, such as environmental noise, the caller's speech habits, and so forth. For example, if the caller states that the telephone number is “555-1234”, it is possible for the voice recognition system 100 to interpret this as “555-1235”. In which case, an erroneous telephone number would be displayed for the recipient. Accordingly, rather than attempting to display the telephone number, the voice recognition system 100 can audibly replay the portion of the voice message which includes the grammar. For example, upon completion of replaying the entire voice message, which includes the caller's telephone number along with other information, the user would then again hear just the portion of the telephone message which includes the telephone number.

[0028] As an alternative, the voice recognition system 100 plays back the important information to the person who is leaving the message. In which case, the person leaving the message can be prompted to verify the accuracy of the identification of such information performed by the voice recognition system 100.

[0029] According to another aspect of the present invention, the grammar can be modified or selected by the recipient of the voice message so that the voice recognition system 100 searches for information of particular interest to the recipient. FIG. 3 illustrates a flow diagram for recognizing information selected by a recipient of a voice message in accordance with the present invention. The flow diagram of FIG. 3 illustrates graphically operation of the speech recognizing system 100 in accordance with the present invention. Program flow begins in a start state 300. From the state 300, program flow moves to a state 302. In the state 302, the voice recognition system 100 receives a voice message from the voice messaging system 150.

[0030] From the state 302, program flow moves to a state 304. In the state 304, the voice recognition system 100 awaits input from the recipient selecting a type of information the recipient is interested in obtaining from the voice message. In the preferred embodiment, the recipient plays the message back and then, based upon the content of the message, the recipient makes this selection. For example, the voice message may contain a phone number, an address, or a date that is of interest to the recipient.

[0031] Assuming the recipient selects a phone number in the state 304, program flow moves to a state 306. In the state 306, the voice recognition system 100 identifies potential speech utterances in the stored voice message and segments the voice message into a plurality of such utterances. Alternately, the voice recognition system 100 acts on the voice message as a whole. The voice recognition system 100 then searches the segments or the entire message for a predetermined numeric grammar which specifies a sequence of numbers occurring in the voice message, such as the caller's telephone number. Assuming the search performed in the state 306 was successful in locating the predetermined grammar, a portion of the voice message which includes the predetermined grammar is identified. Otherwise, program flow is aborted for corrective action.

[0032] Program flow then moves to a state 308. In the state 308, the voice recognition system 100 outputs a portion of the stored voice message which includes the grammar. The output can be a display of the caller's telephone number, a synthesized voice, DTMF tones, or the voice recognition system 100 can audibly replay the portion of the voice message which includes the caller's telephone number.

[0033] Assuming the recipient selects an address in the state 304, program flow moves to a state 310. In the state 310, the voice recognition system 100 identifies potential speech utterances in the stored voice message and segments the voice message into a plurality of such utterances. Alternately, the voice recognition system 100 acts on the voice message as a whole. The voice recognition system 100 then searches the segments or the entire message for a predetermined grammar which specifies an address occurring in the voice message. For example, the predetermined grammar can specify a street number and a street name. Assuming the search performed in the state 310 was successful in locating the predetermined grammar, a portion of the voice message which includes the predetermined grammar is identified. Otherwise, program flow is aborted for corrective action.

[0034] Program flow then moves to a state 312. In the state 312, the voice recognition system 100 outputs a portion of the stored voice message which includes the grammar. The output can be a display of the caller's address, a synthesized voice or the voice recognition system 100 can audibly replay the portion of the voice message which includes the caller's address.

[0035] Assuming the recipient selects a date in the state 304, program flow moves to a state 314. In the state 314, the voice recognition system 100 identifies potential speech utterances in the stored voice message and segments the voice message into a plurality of such utterances. Alternately, the voice recognition system 100 acts on the voice message as a whole. The voice recognition system 100 then searches the segments or the entire message for a predetermined grammar which specifies a date occurring in the voice message. For example, the predetermined grammar can specify a name of a month and a day of the month. Assuming the search performed in the state 314 was successful in locating the predetermined grammar, a portion of the voice message which includes the predetermined grammar is identified. Otherwise, program flow is aborted for corrective action.

[0036] Program flow then moves to a state 316. In the state 316, the voice recognition system 100 outputs a portion of the stored voice message which includes the grammar. The output can be a display of the date, a synthesized voice or the voice recognition system 100 can audibly replay the portion of the voice message which includes the date.

[0037] In an alternate embodiment, the recipient can select more that one type of information at a time. For example, the recipient could request a telephone number and an address in the state 304. In which case, the voice recognition system 100 searches for, identifies and outputs each selected type of information that can be found in the voice message.

[0038] In another alternate embodiment, the voice recognition system 100 can search for all of the specified types of information (e.g. phone numbers, addresses, and dates) while the recipient is listening to the voice message a first time, while the caller is leaving the message or after. The recipient makes a selection of which information is desired prior to, or after, listening to the message. Because the voice recognition search is completed before the recipient makes the selection, the selected information can be immediately provided to the recipient. Further, the voice recognition system 100 can volunteer specified types of information without being prompted.

[0039] Important information included in the stored message may be accompanied by an indicator that import information is to follow or that important information was just given. For example, the voice message may contain a telephone number. In which case, the caller is likely to have preceded the telephone number by saying, “you can reach me at”, “my phone number is”, “call me at”, “my extension is”, or the like. Accordingly, the indicator can be one of these phrases. As another example, the voice message may contain an address which is important. In which case, the caller is likely to have preceded a street number and street name by saying, “the address is”, or followed a street number and street name by saying “street”, “road”, “avenue”, “lane” or the like. Accordingly, the indicator can be one of these phrases. It will be apparent that these indicators and exemplary only and the particular indicator utilized can be selected according to the type of information sought.

[0040] In accordance with the present invention, the predetermined grammar can specify an indicator of important information along with the important information. In which case, the voice recognition system searches the voice message for the indicator in conjunction with the important information. In such case, program flow proceeds as described in reference to FIG. 2.

[0041] Alternately, separate grammars are specified for the indicator and the important information. In which case, the voice recognition system 100 first searches the voice message for the predetermined indicator. Assuming a predetermined indicator is identified, the voice recognition system 100 then searches for the important information in the vicinity of the indicator. Use of the indicator of the important information is expected to increase the recognition accuracy of the invention.

[0042] FIG. 4 illustrates a flow diagram for searching for a predetermined indicator in accordance with the present invention. The flow diagram of FIG. 4 illustrates graphically operation of the speech recognizing system 100 in accordance with the present invention. Program flow begins in a start state 400. From the state 400, program flow moves to a state 402. In the state 402, the voice recognition system 100 receives a voice message from the voice messaging system 150. The voice message can be received while the voice message is being recorded, upon playback of the message by the voice messaging system 150 or upon initiation by the intended recipient of the voice message. From the state 402, program flow moves to a state 404. In the state 404, the voice recognition system 100 searches for a predetermined indicator. Note that the voice recognition system can search for several different indicators in the state 404. For example, the voice recognition system 100 can simultaneously search for “my phone number is”, “you can call me at” or the like. Assuming the predetermined indicator is found, program flow moves to a state 406. Otherwise, if the predetermined indicator is not found, program flow is aborted for corrective action. Corrective action can include repeating the search for the indicator, searching only for the important information or returning to the state 402 to await a next message.

[0043] In the state 406 the voice recognition system 100 searches for a predetermined grammar which specifies the important information in the vicinity of the indicator. In the preferred embodiment, the predetermined grammar is selected based upon the indicator. Thus, if the phrase “my phone number is” is located in the state 404, the voice recognition system searches for a numeric grammar which specifies a telephone number following the indicator. As another example, assuming the indicator is “road”, then a grammar which preceded the indicator is expected to include a number and a street name, such as “1380 Willow”. Assuming the predetermined grammar is located, the predetermined grammar is identified. Then program flow moves to a state 408.

[0044] In the state 408, a portion of the voice message which includes the grammar is outputted. For example, the telephone number, “555-1234” or the address “1380 Willow” can be displayed or audibly reproduced for the user. In addition, the indicator can be combined with the important information. Thus, the entire utterance, “my phone number is 555-1234”, or “1380 Willow Road” can be provided to the recipient.

[0045] Assuming the grammar is not located in the state 406, corrective action can include outputting the indicator along with a predetermined length of the voice message which precedes or follows the indicator, as appropriate.

[0046] The present invention has been described in terms of specific embodiments incorporating details to facilitate the understanding of principles of construction and operation of the invention. Such reference herein to specific embodiments and details thereof is not intended to limit the scope of the claims appended hereto. It will be apparent to those skilled in the art that modifications may be made in the embodiment chosen for illustration without departing from the spirit and scope of the invention.