Discrete speech recognition system with ballooning active grammar

[0002] This invention relates to discrete speech recognition systems. More particularly, this invention relates to discrete speech recognition systems with ballooning grammars. This invention further relates to vehicle computer systems that implement such discrete speech recognition systems.

[0003] Two common types of speech recognition systems are continuous and discrete. Continuous speech recognition systems detect and discern useful information from continuous speech patterns. In use, an operator may speak phrases and sentences without pausing and the continuous speech recognition system will determine the words being spoken. Continuous speech recognition systems are used, for example, in voice-input word processors that enable operators to dictate letters directly to the computer.

[0004] In contrast, discrete speech recognition systems are designed to detect individual words and phrases that are interrupted by intentional pauses, resulting in an absence of speech between the words and phrases. Discrete speech recognition systems are often used in “command and control” applications in which an operator speaks individual commands to initiate corresponding predefined control functions. In a typical use, the operator speaks a command, pauses while the system processes and responds to the command, and then speaks another command. The system detects each command and performs the associated function.

[0005] This invention is directed to the discrete class of speech recognition systems.

[0006] A discrete speech recognition system employs a complete list of recognized words or phrases, referred to as the “vocabulary.” A subset of the vocabulary that the recognition system is attempting to detect at any one time is known as the “grammar.” In general, the smaller the active grammar, the more reliable the recognition because the system is only focusing on a few words or phrases. Conversely, the larger the active grammar, the less reliable the recognition because the system is attempting to discern a word or phrase from many words or phrases.

[0007] Accordingly, one design consideration for discrete speech recognition systems is to devise grammars that present useful command options, while being reliably detectable.

[0008] One conventional approach is to construct a large grammar that encompasses each command option. FIG. 1 shows how this conventional approach might be applied to control an automobile radio. In this example, suppose the system is designed to allow the user to control the radio and access his/her favorite radio stations using voice commands. Using a large-size active grammar, a default radio grammar 20 might include the radio control words-“AM”, “FM”, “Seek”, and “Scan”-and all of the preset radio stations. A corresponding command function is associated with each grammar word, as represented in Table 1.

[0009] The speech recognition system actively tries to recognize one of these words when the operator speaks. When a grammar word is detected, the speech recognition system performs the appropriate function. Suppose the operator says is the word “AM”. The discrete speech recognition system detects the active word 22 and performs the corresponding function 24 to set the radio to the AM band.

[0010] As noted above, a drawback with presenting a large all-encompassing grammar is that there is a greater likelihood of false recognition by the speech system. For instance, the system may experience trouble distinguishing between the words “FM” and “Seven” when both are spoken rapidly and/or not clearly enunciated.

[0011] Another conventional approach is to construct a small default grammar and to switch to a new grammar upon detection of one or more keywords. FIG. 2 shows how this conventional approach might be applied to control an automobile radio. With this approach, a default radio grammar 30 might include only the radio control words-“AM”, “FM”, “Seek”, “Scan”, and “Preset”. A corresponding command function is associated with each grammar word, as represented in Table 2.

[0012] Upon recognition of the keyword “preset”, the speech recognition system changes to a new grammar 32 for detecting the preset station numbers. Table 3 lists the new preset station grammar.

[0013] The speech recognition system actively tries to recognize one of these words from the preset station grammar. Suppose the operator says the word “One”. The discrete speech recognition system detects the active word 34 and performs the corresponding function 36 to set the radio to the preset station 1.

[0014] A drawback with this system is navigation of the grammars. An operator may call out a keyword in one grammar, causing the system to switch to a different grammar, and then subsequently be interrupted (e.g., driving in traffic) and forget which grammar is currently active upon returning his/her attention to the radio. For instance, suppose the operator had called out “preset” to get the preset station grammar of Table 3 and then subsequently became interrupted. The operator may then wish to seek or scan, but the system will not recognize these commands because the active grammar is currently looking for a preset station number.

[0015] Accordingly, there is a need to improve techniques for presenting grammars in discrete speech recognition systems for such applications as operating a vehicle radio.

[0016] This invention concerns a discrete speech recognition system with a ballooning grammar. The system begins with a default grammar that has both keywords and non-keywords. Upon detecting a word that is not a keyword in the default grammar, the speech recognition system simply performs the function associated with the detected word. Upon detecting a keyword in the default grammar, the speech recognition system expands its active grammar list from the default grammar to a ballooned grammar that include both the words in the default grammar and the additional words triggered by detection of the keyword. In this manner, the operator still has the option to select a word from the original grammar, or choose a word from the additional list.

[0017] The ballooned grammar remains active until the operator makes a new selection. With a bi-level system (meaning the system balloons the grammar only once), the speech recognition system returns to the default grammar. In a higher level system in which multiple ballooned grammars may be used, the operator can work his/her way through the grammars, with each level including the words from the default grammar and words that are added along the way. In one implementation of a higher level system, words that are effectively rendered useless by subsequent selections can be removed from the active ballooned vocabulary. At the end of the traversal through the levels, the speech recognition system returns to the default grammar.

[0018] This system advantageously offers a flexible balance between small-size default grammars and greater functionality enabled by large size grammars to thereby mitigate the possibility of false or erroneous recognition.

[0019] FIG. 1 shows a conventional approach of employing a large all-encompassing default grammar.

[0020] FIG. 2 shows a conventional approach of using a small default grammar and switching to a new grammar upon selection of a keyword.

[0021] FIG. 3 is a functional block diagram of a discrete speech recognition system.

[0022] FIG. 4 is a diagrammatic illustration of a vehicle computer system that implements the discrete speech recognition system.

[0023] FIG. 5 illustrates how a default grammar is expanded to a ballooned grammar according to an aspect of this invention.

[0024] FIG. 6 is a flow diagram showing steps in a method for ballooning an active grammar.

[0025] FIG. 7 is a block diagram of a more general computer device used to Implement the speech recognition system.

[0026] FIG. 8 illustrates how a default grammar is expanded multiple times to create multiple ballooned grammars according to another aspect of this invention.

[0027] FIG. 9 illustrates how a default grammar is expanded through multiple times to create multiple ballooned grammars, but with active removal of unwanted terms, according to another aspect of this invention.

[0028] FIG. 3 shows a discrete speech recognition system 40 that recognizes individual words or phrases. As used herein, the term “utterances” means a word, phrase, or other spoken sound that is detectable as a discrete voice input to the speech recognition system. The discrete speech recognition system 40 includes an application 42, a vocabulary 44, an active grammar 46, and a speech recognition engine 48. The application 42 may be essentially any kind of application that supports voice-input commands for operation. One suitable type of application is the command and control application where discrete commands are issued to cause a particular function to occur. As possible examples, the application 42 might be a program to operate a vehicle radio, or a program to locate an address or destination, or a program to operate a telephone, or a program to open files on a computer, and so forth.

[0029] The vocabulary 44 is the complete list of utterances that are recognized by the application. The vocabulary is stored in memory that can be accessed by the application. The active grammar 46 is a subset of the vocabulary that the recognition system is attempting to recognize at any one time. During operation, the application 42 extracts various sets of utterances from vocabulary 44 and registers them as the active grammar. The active grammar 46 is stored in short-term memory or cache, which is accessible by the application 42 and the speech recognition engine 46.

[0030] The speech recognition engine 46 performs the actual task of recognizing utterances in the active grammar. It compares the voice input from the operator with utterances in the active grammar 46. Upon detection of an utterance listed in the active grammar 46, the speech recognition engine 46 informs the application 42 of the particular utterance that has been recognized. The application 42 then performs the function associated with the utterance.

[0031] The speech recognition system 40 begins with a default grammar as the active grammar. The default grammar has both keywords and non-keywords. Upon detecting a word that is not a keyword in the default grammar, the speech recognition system 40 simply performs the function associated with the detected word. Upon detecting a keyword in the default grammar, however, the speech recognition system 40 temporarily expands its active grammar list from the default grammar to a ballooned grammar that includes both the words in the default grammar and the additional words triggered by detection of the keyword. That is, the application 42 extracts additional words from the vocabulary 44 and adds them to the active grammar 46. In this manner, the operator still has the option to select a word from the original grammar, or choose a word from the additional list.

[0032] The ballooned grammar remains active until the operator makes a new selection. With a bi-level system (meaning the system balloons the grammar only ii once), the speech recognition system 40 returns to the default grammar. In a higher level system in which multiple ballooned grammars may be used, the operator can work his/her way through the grammars, with each level including the words from the default grammar and words that are added along the way. At the end of the traversal through the levels, the speech recognition system 40 returns to the default grammar.

[0033] The discrete speech recognition system 40 can be used implemented in many different contexts. Two exemplary contexts are provided in this disclosure: an vehicle computer/entertainment accessory and a computing device. These implementations are discussed separately below.

[0034] FIG. 4 shows an in-dash vehicle accessory 50 that implements the discrete speech recognition system 40. In the illustrated implementation, the vehicle accessory 50 is a vehicle computer system that includes a variety of different elements such as a security subsystem, a navigation subsystem, and an entertainment subsystem. In other embodiments, the accessory might be a car radio, cassette player, CD player, or similar device. The speech recognition system 40 facilitates voice activation of the accessory 50, or features thereof, to permit the driver to verbally enter commands in a hands-free, eyes-free environment.

[0035] Vehicle accessory 50 has a base module 52 and a detachable faceplate 54. Base module 52 includes a housing 56, which is sized to be mounted in a vehicle dashboard similar to a conventional car stereo. Preferably, housing 56 has a form factor of a single DIN (Deutsche Industry Normen). The base module could alternatively be housed in a 2 DIN unit or other special form factor for an OEM.

[0036] In the described implementation, base module 52 is a computer that runs a multi-tasking operating system to support multiple applications. One preferred operating system is a Windows brand operating system sold by Microsoft Corporation, such as Windows 95, Windows CE, Windows NT or other derivative versions of the Windows family of operating systems. The computer and operating system are configured to support after-market peripherals including both hardware and software components.

[0037] The computer 52 includes at least one storage drive that permits the vehicle user to download programs and data from a storage medium. In the illustrated implementation, the computer 52 has a CD ROM drive 58 that reads application-related CDs, as well as musical, video, game, or other types of entertainment CDs. In this manner, the CD ROM drive 58 performs a dual role of storage drive and entertainment player. Also, a hard disk drive (not shown in FIG. 4) that can be used for storing both application programs and user data is included on the computer module 52. The computer base module 52 has an optional 3.5″ floppy diskette drive 60, a smart card reader 62, and dual PCMCIA card sockets 64 which accept PC (or PCMCIA) card types II and III. The faceplate 54 can be rotated to expose and permit easy access to the storage drives. In other implementations, the faceplate 54 has slots to permit access to one or more of the memory drives.

[0038] Faceplate 54 has a keypad 66 and a graphical display 68. The display 68 is preferably a backlit LCD having a rectangular array of pixels that are individually selectable for illumination or display. The display element is a medium-resolution, bit-mapped display system having at least 10,000 pixels. In the described implementation, the array of pixels has a size of at least 64 by 256 pixels. The operating system of base module 52 interacts with faceplate keypad 66 and faceplate display 68 as peripheral devices when the faceplate 54 is attached to the base module 52.

[0039] The faceplate 54 has a start button 70 that provides the familiar “Start” functions of a Windows brand operating system. The faceplate 54 also has a power button 72, a four-position actuator 74, an “enter” button 76, an “escape” button 78, a “help” button 80, a “menu” button 82, and a volume control input 84.

[0040] The accessory 50 has a voice input port 86 that picks up voice commands for entry to the speech recognition system. The accessory also supports an IrDA (infrared developers association) transceiver port 88 mounted on the faceplate 54 to transmit and receive data and programs using infrared signals.

[0041] The vehicle accessory 50 can be used to integrate multiple vehicle-related systems onto one open platform. For instance, the accessory can serve as a multimedia entertainment system, a navigation system, a communications system, a security system, and a diagnostics system. Moreover, the accessory provides additional functionality traditionally associated with desktop and laptop personal computers. For instance, it can support word processing applications, spreadsheet applications, database applications, and appointment/schedule applications. Furthermore, the vehicle accessory 50 can be configured to operate as a server to other computing units in a vehicle to distribute games, video movies, and the like to passengers.

[0042] Vehicle accessory 50 may support a number of components and features that are not discussed herein. For more detail regarding such features refer to a U.S. patent application Ser. No. 08/564,586, entitled “Vehicle Computer System”, which was filed Nov. 29, 1995 now U.S. Pat. No. 5,794,164 and is assigned to Microsoft Corporation. This application is hereby incorporated by reference.

[0043] For purpose of continuing discussion, suppose the discrete speech recognition system 40 is implemented to control the radio functionality of the computer/entertainment accessory 50. It is noted that the speech recognition system can be used to control many other functions in addition to the radio. For instance, the system can be used to control an address book program, as described below with reference to FIGS. 7-9. Other controllable functions include temperature control, operating a cellular phone, operating a CD player, locking and unlocking doors, lowering and raising windows, and so forth.

[0044] In the radio example, the speech recognition system 40 includes an application 42 that is configured for controlling the radio functionality and a vocabulary 44 with radio command utterances such as “AM”, “FM”, “Seek”, “Scan”, “Preset”, “One”, “Two”, and so forth.

[0045] FIG. 5 shows the process of ballooning the active grammar. This process is explained with additional reference to the steps shown in FIG. 6. At step 100 in FIG. 6, the radio command application extracts a default grammar from the vocabulary 44 and loads the default grammar into the active grammar 46. The default active grammar, referenced by number 90 in FIG. 5, contains five utterances: “AM”, “FM”, “Seek”, “Scan”, and “Preset”. Table 4 contains the default grammar and associated functions.

[0046] The first four utterances-“AM”, “FM”, “Seek”, and “Scan”-are non-is keyword utterances and the fifth“Preset”-is a keyword utterance.

[0047] At step 102 in FIG. 6, the speech recognition engine begins listening to speech patterns in an effort to detect an utterance contained in the default grammar 90. In the event the operator (e.g., vehicle driver) speaks a non-keyword utterance, such as “AM” (i.e., the “no” branch from step 104), the speech recognition system performs the associated function of setting the radio to the AM band (step 106 in FIG. 6). The default active grammar remains the same (step 108).

[0048] When the operator speaks the keyword “Preset” (i.e., the “yes” branch from step 104), the speech recognition system temporarily expands the default grammar to include an additional set of utterances triggered by the keyword (step 110 in FIG. 6). In this case, the extra terms are preset channel numbers that are activated by the keyword “Preset”. Table 5 lists the ballooned grammar.

[0049] As shown in FIG. 5, the extra words “One”, “Two”, . . . , “Ten” are added to the ballooned active grammar 92. Notice that the original default words are still present. If the operator made a mistake and would still like to enter a default command, like “AM” or “Scan”, the operator is able to speak such a command and have the system respond.

[0050] After the grammar is expanded, the system continues to monitor for any utterance in the ballooned grammar (step 102). The operator can speak one of the original default commands, or one of the added commands. The system maintains this ballooned grammar until the system detects one of the utterances listed therein. Suppose the operator says the word “One”. The speech recognition system recognizes the utterance “One”, as represented by numeral 94 in FIG. 5. Since this is not a keyword (i.e., the “no” branch from step 104 in FIG. 6), the system performs the associated function of setting the radio to the preset station 1, as represented by number 96 in FIG. 5 (i.e., step 106 in FIG. 6).

[0051] After the associated operation is performed, the speech recognition system returns to the small default grammar (step 108 in FIG. 6). This is represented diagrammatically in FIG. 5 by the dashed return arrow to the default grammar 90.

[0052] The speech recognition system may be implemented in other contexts than a vehicle accessory. To provide another example for discussion purposes, FIG. 7 is a simplified block diagram of a computing device 120 that implements the discrete speech recognition system (DSRS) 40. The computing device 120 has a processor 122, associated addressable memory 124 (including non-volatile memory and high-speed volatile memory), and a voice input device 126. In this implementation, the speech recognition system 40 is embodied as a software program that is stored in memory 124 and executes on the processor 122. The program has an application 42, a predefined vocabulary 44, an active grammar 46, and a recognition engine module 48.

[0053] FIG. 8 shows a higher level process that allows multiple ballooning tiers of the active grammar. This process is explained with reference to the steps in FIG. 6. For this example, the application 42 is an address book program that manages addresses and provides directions to locate the address. The default grammar 130 is initially loaded as the active grammar (step 100 in FIG. 6). The default active grammar 130 contains four utterances: “Place”, “Name”, “Directions”, and “Address”. Suppose the word “Place” is a keyword and the speech recognition engine detects the keyword “Place” (step 102 in FIG. 6). Since this is a keyword (i.e., the “yes” branch from step 104 in FIG. 6), the default grammar is temporarily expanded to a first ballooned grammar 132 that adds the utterances “City”, “County”, and “Street” (step 110).

[0054] Suppose that word “City” in the first ballooned grammar 132 is also a keyword. When the speech recognition engine detects the keyword “City” (step 102 in FIG. 6), the first ballooned grammar is further expanded to a second ballooned grammar 134 that adds the names of various U.S. cities, such as “Albany”, “Anchorage”, “Atlanta”, “Baltimore”, “Boise”, “Boston”, “Buffalo”, and so forth (step 110). The second ballooned grammar 134 thus contains the original words in the default grammar 130, the added words in the first ballooned grammar 132, and the new city names.

[0055] Suppose the operator says the word “Atlanta”. The speech recognition system recognizes the utterance “Atlanta”, as represented by numeral 136 in FIG. 8. Since this is not a keyword (i.e., the “no” branch from step 104 in FIG. 6), the system performs the associated function of filtering the address book to entries pertaining to the city Atlanta, as represented by number 138 in FIG. 8 (i.e., step 106 in FIG. 6).

[0056] After the associated operation is performed, the speech recognition system returns to the small default grammar (step 108 in FIG. 6). This is represented diagrammatically in FIG. 8 by the dashed return arrow to the default grammar 130.

[0057] FIG. 9 shows a variation of the FIG. 8 process. In the FIG. 9 ballooning process, words that are effectively rendered useless by subsequent selections are removed from the active ballooned grammar. The default grammar 130 and the first ballooned grammar 132 are the same as in FIG. 8. However, when the system detects the keyword “City” and the first ballooned grammar 132 is expanded to include the city names, the speech recognition system also removes certain words that are no longer useful. In this case, the words “Name” and “Address” from the default grammar are removed and the word “County” from the first ballooned grammar is removed. This selective reduction helps keep the active grammar as small as possible.

[0058] The discrete speech recognition system is advantageous because it offers a flexible balance between small-size default grammars and greater functionality enabled by large size grammars to thereby mitigate the possibility of false or erroneous recognition. In general, the smaller the active grammar, the more reliable the recognition; whereas the larger the active grammar, the less reliable the recognition. In the described system, the default grammar is kept to a few words. The grammar is then expanded to a ballooned grammar that includes the original default words and extra words that enable a greater range of functionality. After a subsequent selection, the process is returned to the small default grammar.

[0059] Although the invention has been described in language specific to structural features and/or methodological steps, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or steps described. Rather, the specific features and steps are disclosed as preferred forms of implementing the claimed invention.