Method of Implicit Confirmation for Automatic Speech Recognition

1. Field of the Invention

The present invention relates to automated speech recognition. More particularly, the invention is directed to a method for automatically confirming verbal commands issued by a user and received by an automatic speech recognition system.

2. Description of the Related Art

Two general categories of speech recognition systems are well known in the art. The first category includes speech recognition systems used for automating data acquisition ("speech data systems"). Speech data systems typically recognize speech by converting a sequential set of speech data segments vocalized by the user into a corresponding set of data elements. The data elements may then be transmitted to a secondary computer system for processing or storage. For example, many banks permit a customer calling a customer service telephone number to vocalize, for automated recognition by a speech recognition system, the customer's credit card number. Each numeral of the credit card number vocalized by the customer is a speech data segment which is recognized as a corresponding data element. The recognized credit card number is then used to retrieve information about the customer's account. Other speech data systems are used to recognize a user's speech and convert it into text, which may be stored in a secondary computer system for subsequent manipulation.

The second category of speech recognition systems includes systems used for interpreting and executing predefined verbal commands ("speech command systems"). A verbal command is typically a collection of speech data segments with an associated data processing instruction that is executable by a data processing system or another electronic device to perform a particular function. For example, an electronic clock equipped with a speech command system may have a predefined verbal command of "current time". When the user speaks the phrase "current time", the speech command system interprets the verbal command and executes a data processing instruction to inform the user of the current time. Typically, a speech command system will have several predefined verbal commands stored in a command set.

Speech command systems use similar recognition techniques to those of speech data systems; however, instead of treating and recording recognized speech data segments as pure data, a speech command system compares the set of recognized speech data segments to the verbal commands in the command set. If the recognized speech data segments match a speech data component of one of the stored verbal commands, then the speech command system executes the instruction associated with that command. Otherwise, the speech data system ignores recognized speech data segments that do not match any of the verbal commands.

Like all automatic speech recognition systems, speech command systems are subject to an error commonly known as false recognition. False recognition occurs when a speech command system incorrectly identifies a user's utterance as a particular verbal command. For example, if the user is speaking to another person while using the speech command system, one of the user's phrases may be incorrectly interpreted by the speech command system as a verbal command. In cases where a verbal command includes or activates an additional command operator, false recognition of verbal commands can have significant effects. For example, a telephone equipped with a speech command recognition system for placing calls will typically have a verbal command "call" that directs the telephone system to place a call and a command operator set of one or more names with associated telephone numbers to which the call is to be placed. Thus, to place a call to a person named Bill King using a speech recognition-equipped telephone, the user simply speaks "call Bill King" to thereby instruct the telephone to dial Bill King's telephone number. However, if the same user were to tell another person "You should call Bill King tomorrow" when the user is in close proximity to the telephone, then this speech may be incorrectly interpreted and acted upon as a verbal command to dial Bill King's telephone number, causing the user (and perhaps Bill King) inconvenience and unnecessary expense.

To address the problem of false recognition, prior art speech data systems have incorporated a required second or confirmation verbal command to confirm that a particular identified utterance by the user is in fact intended to be an actual verbal command. In the previous example, before dialing Bill King, the telephone equipment would prompt the user to confirm, by saying "yes" or "no", whether a call to Bill King should be made. While this approach provides a workable solution to the false recognition problem, it is flawed in two respects. First, if the user did in fact intend an actual verbal command to place a call, then the user is forced to go through an extra step of confirming the command even though the original command was correctly recognized. Moreover, when an utterance is falsely interpreted as a verbal command, the user's activity is unnecessarily interrupted by the speech command system request for confirmation of a verbal command that was never given.

A method for automatically confirming verbal commands issued by a user and recognized by a speech command system is provided. When a user issues a verbal command, the speech command system recognizes or identifies the command and retrieves a set of verbal command operators associated with the initial identified verbal command. The speech command system then initiates operation a timer of a pre-determined maximum timing period and activates an indicator to alert the user that the system is awaiting a verbal command operator. The indication is preferably performed without interrupting the user's activity. For example, the indication may be accomplished by a visual indicator (e.g., a light) and/or a low volume tone. If the user vocalizes a recognized verbal command operator from the retrieved verbal command operator set before the timer reaches its maximum timing period, then the speech command system executes the initial verbal command using the verbal command operator and issues a confirmation to the user that the command has been executed. If, on the other hand, the user does not vocalize an appropriate verbal command operator from the retrieved verbal command operator set before the timer reaches its maximum timing period, then the speech command system deactivates the indicator and returns to its idle state, ready to receive new verbal commands.

Other objects and features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the invention, for which reference should be made to the appended claims.

The drawing FIG. 1 is a flow chart depicting a mehtod for automatically confirming verbal commands issued by a user and recognized by a speech comand system implemented in accordance with the teachings of the present invention.

The present invention is broadly directed to a method for automatically confirming verbal commands issued by a user and recognized by a speech command system. The method of the invention is preferably implemented in a digital microprocessor-based speech recognition system ("speech command system") operatively capable of interpreting and executing verbal commands, and of storing verbal commands in a memory. It should be noted that "executing a verbal command" refers to the execution of a data-processing instruction or instructions associated with that verbal command. Preferably, each verbal command is pre-defined as a primary command "COMMAND.sub.-- (X)" and a command operator "OPERATOR.sub.-- (X).sub.-- (Y)" selected from a command operator set "OPERATOR.sub.-- SET.sub.-- (X)" associated with the COMMAND.sub.-- (X). The variable "(X)" identifies a particular primary command and its associated command operator set. For example, a third primary command and its associated command operator set are identified as COMMAND.sub.-- 3 and OPERATOR.sub.-- SET.sub.-- 3, respectively. The variable "(Y)" identifies a particular command operator stored in the OPERATOR.sub.-- SET.sub.-- (X). For example, a fourth command operator in the OPERATOR.sub.-- SET.sub.-- 3 is identified as OPERATOR.sub.-- 3.sub.-- 4. Thus, in order to execute a verbal command, the user must first vocalize the primary command COMMAND.sub.-- (X), and then vocalize command operator OPERATOR.sub.-- (X).sub.-- (Y) selected by the user from the set OPERATOR.sub.-- SET.sub.-- (X). For example, COMMAND.sub.-- 1 for telephone equipped with a speech command system may be defined as "make a call". The OPERATOR.sub.-- SET.sub.-- 1 would then include a list of names (e.g., Jeff Gold, Bill Smith, etc) each defined as OPERATOR.sub.-- 1.sub.-- (1 . . . Y). To place a call to Bill Smith, the user would say "make a call" and then say "Bill Smith." The speech command system will then dial a telephone number associated with Bill Smith. If the user does not vocalize an OPERATOR.sub.-- 1.sub.-- (Y) following COMMAND.sub.-- 1, then the speech command system does not execute COMMAND.sub.-- 1. Preferably the OPERATORs.sub.-- (X).sub.-- (Y) are defined by the user. Thus, in the previous example, prior to operating the speech command system, the user would define each OPERATOR.sub.-- 1.sub.-- (Y) as a name of a particular individual or institution to be dialed along with an associated telephone number. In addition, the user may also define one or more COMMANDs.sub.-- (X). For example, a personal computer-based speech command system enables the user to define one or more COMMANDs.sub.-- (X) to control various application programs.

In accordance with the method of the present invention, after the user vocalizes a COMMAND.sub.-- (X), the speech command system awaits vocalization of a required OPERATOR.sub.-- (X).sub.-- (Y) for a pre-determined period of time. During this waiting period a non-intrusive indicator such as a light, a low volume tone, or a character-based display notifies the user that the speech command system is ready to receive a command operator OPERATOR.sub.-- (X).sub.-- (Y). If the user vocalizes an OPERATOR.sub.-- (X).sub.-- (Y) during the waiting period, then the speech command system executes the primary command COMMAND.sub.-- (X) using the command operator OPERATOR.sub.-- (X).sub.-- (Y) and issues a confirmation to the user that COMMAND.sub.-- (X) has been executed. Otherwise, the speech command system returns to its idle state awaiting further verbal commands.

Referring now to the drawing Figure, the inventive method begins at block 100 where the speech command system enters and remains in its idle or "command spotting" state, ready to receive verbal commands. By way of example, a user may manually turn on a speech command system feature on a telephone set, or the speech command system may activate automatically when the user is in close physical proximity to the telephone. The method flow then proceeds to block 102 where the speech command system waits until the user speaks. When an utterance is then received from the user, flow proceeds to block 104 where the speech command system recognizes or identifies the utterance. An utterance is preferably a word or phrase spoken by the user in proximity to the speech command system. Recognition or identification of the particular utterance may be carried out by the speech command system using any available or well-known speech recognition techniques. At block 106, the speech command system determines whether the recognized utterance is a known COMMAND.sub.-- (X). If so, then flow proceeds to block 108; otherwise, flow returns to block 102 where the speech command system continues to wait for further speech from the user. When a COMMAND.sub.-- (X) is recognized, the speech command system identifies a corresponding data processing instruction to perform a particular task. For example, if the COMMAND.sub.-- 1 is "make a call", then the speech command system will recognize that verbal command as an instruction to dial a particular telephone number yet to be identified.

At block 108, the speech command system retrieves a stored OPERATOR.sub.-- SET.sub.-- (X) that is associated with the COMMAND.sub.-- (X) recognized at block 104. The OPERATOR.sub.-- SET.sub.-- (X) includes one or more OPERATORs.sub.-- (X).sub.-- (Y) for use with the COMMAND.sub.-- (X). For example, if the COMMAND.sub.-- (X) was "make a call", the OPERATORs.sub.-- (X).sub.-- (Y) may include names of people or institutions defined by the user and to which calls are to be made.

At block 110, the speech command system next initializes a COUNTER variable, which is preferably implemented as a timer of a predefined maximum timing period "N". The time period "N" is preferably of at least sufficient duration for the user to vocalize an operator command OPERATOR.sub.-- (X).sub.-- (Y). At block 112, the speech command system activates a LISTENING.sub.-- INDICATOR to inform the user that the speech command system is waiting for the user to vocalize an appropriate command OPERATOR.sub.-- (X).sub.-- (Y). Preferably, the LISTENING.sub.-- INDICATOR is non-intrusive so that the user's activity is not interrupted. Examples of non-intrusive LISTENING.sub.-- INDICATORs include, without limitation, visual indicators such as an LED light or character-based display, audible indicators such as a low volume tone, and a combination of visual and audible indicators.

At block 114, the speech command system then waits for the user to vocalize an appropriate OPERATOR.sub.-- (X).sub.-- (Y). If an utterance from the user is not received in a pre-determined timing interval (e.g., 1 millisecond), the flow proceeds to block 116 at which the COUNTER variable is incremented. The magnitude of the timing interval is dependent on the resolution of the timer used for implementing the COUNTER variable and may be determined as a matter of design choice. For example, a high resolution timer typically has a low interval, such as 1 microsecond. At block 118, the speech command system determines whether the COUNTER variable is equal to "N", i.e. whether the timer has reached the end of the predefined maximum timing period (i.e. timed out). If the COUNTER variable is not equal to "N", then the flow returns to block 114 where the speech command system continues to wait for the user to vocalize an OPERATOR.sub.-- (X).sub.-- (Y). If, at block 118, the COUNTER variable is determined to be equal to "N", then the flow first proceeds to block 120 where the LISTENING.sub.-- INDICATOR is deactivated, and next proceeds to block 100 at which the speech command system reenters the primary command-spotting state.

If an utterance is, on the other hand, received before the COUNTER variable reaches "N", then the flow proceeds from block 114 to block 122 where the speech command system recognizes or identifies the utterance. At block 124, the speech command system determines whether the recognized utterance is a known OPERATOR.sub.-- (X).sub.-- (Y). If so, then flow proceeds to block 126; otherwise, flow proceeds to block 116 where the COUNTER is incremented.

At block 126, the speech command system issues a confirmation to the user that it is about to execute COMMAND.sub.-- (X) using OPERATOR.sub.-- (X).sub.-- (Y) recognized at block 122. For example, if the COMMAND.sub.-- (X) is "make a call" and the OPERATOR.sub.-- (X).sub.-- (Y) is "Bill King", at block 126 the speech command system would inform the user (e.g. via speech synthesis, a character based visual indicator, or a similar technique) that Bill King's number is about to be dialed. It should be noted, however, that the confirmation step represented by the confirmation block 126 may optionally be eliminated without departing from the spirit of the present invention, particularly if the speech command system with which the present invention is implemented lacks speech synthesis or character-based data output capability. At block 128, the speech command system executes the primary command COMMAND.sub.-- (X) using the OPERATOR.sub.-- (X).sub.-- (Y). Thus, if the speech command system is instructed to place a call as in the previous example, the system will dial the appropriate telephone number at block 128. Flow then returns to block 100 where the speech command system returns to the idle or command-spotting state.

While there have shown and described and pointed out fundamental novel features of the invention as applied to a preferred embodiment thereof, it will be understood that various omissions and substitutions and changes in the form and details of the herein-disclosed method, and in its operation, may be made by those skilled in the art without departing from the spirit of the invention. For example, it is expressly intended that all combinations of those method steps which perform substantially the same function in substantially the same way to achieve the same results are within the scope of the invention. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.