METHOD AND APPARATUS TO GENERATE A SPEECH RECOGNITION LIBRARY

[0001] This application is a continuation of U.S. patent application Ser. No. 12/120,369 filed May 14, 2008 by Hisao Chang, entitled “Methods and Apparatus to Generate a Speech Recognition Library”, now U.S. Patent Publication No. 2009/0287486. All sections of the aforementioned application are incorporated herein by reference in its entirety.

[0002] This disclosure relates generally to speech recognition systems and, more particularly, to methods and apparatus to generate a speech recognition library for use by a speech recognition system.

[0003] Speech recognition systems allow a user to interact with a device by speaking words and/or commands. For example, when a command is spoken, the speech recognition system translates the spoken command into text that can be used and/or manipulated by the device to, for example, change a state of the device.

[0012] The use of the same reference symbols in different drawings indicates similar or identical items.

[0013] The numerous innovative teachings of the present application will be described with particular reference to the presently preferred exemplary embodiments. However, it should be understood that this class of embodiments provides only a few examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed inventions. Moreover, some statements may apply to some inventive features but not to others.

[0014] Example methods and apparatus to generate a speech recognition library for use by a speech recognition system are disclosed. A disclosed example method includes identifying a plurality of video segments having closed caption data corresponding to a phrase, the plurality of video segments associated with respective ones of a plurality of audio data segments, computing a plurality of difference metrics between a baseline audio data segment associated with the phrase and respective ones of the plurality of audio data segments, selecting a set of the plurality of audio data segments based on the plurality of difference metrics, identifying a first one of the audio data segments in the set as a representative audio data segment, determining a first phonetic transcription of the representative audio data segment, and adding the first phonetic transcription to a speech recognition library when the first phonetic transcription differs from a second phonetic transcription associated with the phrase in the speech recognition library.

[0015] A disclosed example apparatus includes an audio segment selector to identify a plurality of video segments having closed caption data corresponding to a phrase, the plurality of video segments associated with respective ones of a plurality of audio data segments, an audio comparator to compute a plurality of difference metrics between a baseline audio data segment associated with the phrase and respective ones of the plurality of audio data segments, an audio segment grouper to identify a set of the plurality of audio data segments based on the plurality of difference metrics, a phonetic transcriber to determine a first phonetic transcription corresponding to the set of audio data segments, and a database manager to add the first phonetic transcription to a speech recognition library when the first phonetic transcription differs from a second phonetic transcription associated with the phrase in the speech recognition library.

[0016] Another disclosed example method includes identifying a plurality of video segments having closed caption data corresponding to a phrase, the plurality of video segments associated with respective ones of a plurality of audio data segments, determining a plurality of phonetic transcriptions for respective ones of the plurality of audio data segments, identifying a set of the plurality of audio data segments having a first phonetic transcription different from a second phonetic transcription associated with the phrase in a speech recognition library, and adding the first phonetic transcription to the speech recognition library.

[0017] The example speech recognition libraries described herein can be used by any number and/or type(s) of speech recognition systems, and/or any number and/or type(s) of systems having an included speech recognition system. Example systems that incorporate a speech recognition system are described in U.S. patent application Ser. No. 12/120,368, entitled “Methods and Apparatus to Generate Relevance Rankings for Use By A Program Selector of A Media Presentation System,” which was filed on May 14, 2008; U.S. patent application Ser. No. 11/520,092, entitled “Authoring System for IPTV Network,” which was filed on Sep. 12, 2006; U.S. patent application Ser. No. 11/475,551, entitled “Biometric and Speech Recognition System and Method,” which was filed on Jun. 27, 2006; U.S. patent application Ser. No. 11/226,646, entitled “Wireless Multimodal Voice Browser for Wireline-based IPTV Services,” which was filed on Sep. 14, 2005; U.S. patent application Ser. No. 11/106,361, entitled “System and Method of Locating and Providing Video Content via an IPTV Network,” which was filed on Apr. 14, 2005; U.S. patent application Ser. No. 11/866,873, entitled “System for Managing Media Services,” which was filed on Oct. 3, 2007; and U.S. patent application Ser. No. 11/106,016, entitled “Wireless Device to Access Network-based Voice-Activated Services Using Distributed Speech Recognition,” which was filed on Apr. 14, 2005. U.S. patent application Ser. No. 12/120,368, U.S. patent application Ser. No. 11/520,092, U.S. patent application Ser. No. 11/475,551, U.S. patent application Ser. No. 11/226,646, U.S. patent application Ser. No. 11/106,361, U.S. patent application Ser. No. 11/866,873, and U.S. patent application Ser. No. 11/106,016 are hereby incorporated by reference in their entirety.

[0018] FIG. 1 is a schematic illustration of an example system 100 to generate a speech recognition library 105 for use by a speech recognition system (not shown). To generate the example speech recognition library 105, the example system 100 of FIG. 1 includes a pronunciation library generator 110. The example pronunciation generator 110 of FIG. 1 uses closed caption information associated with audio tracks of a video library 115 to generate the speech recognition library 105. For a phrase of one or more words (e.g., “Alan Alda”), the example pronunciation library generator 110 uses closed caption information to identify audio tracks that contain the phrase. The identified audio tracks are then analyzed to identify one or more pronunciations of the phrase (e.g., “Alan-ALL-da” versus “Alan-el-da”). The pronunciations library generator 110 associates each of the identified pronunciations with the phrase in the speech recognition library 105. By analyzing the video library 115, the example pronunciation library generator 110 generates a speech recognition library 105 that encompasses as many distinct pronunciations of a given phrase as possible. For example, a proper name or location (e.g., “Alan Alda,” “Beijing,” “Qatar,” etc.) may be pronounced differently depending on the nationality or geographic location of the speaker. By incorporating distinct pronunciations of a phrase in the speech recognition library 105, a speech recognition system that uses the example speech recognition library 105 is able to more often or more correctly determine what phase is spoken regardless of how the phrase is pronounced. An example data structure that may be used to implement the example speech recognition library 105 is described below in connection with FIG. 3. An example manner of implementing the example pronunciation library generator 110 of FIG. 1 is described below in connection with FIG. 4.

[0019] As shown in FIG. 2, the example video library 115 contains a plurality of videos and/or video clips 205 (television shows, movies, newscasts, documentaries, sport shows, home movies, etc.) received from and/or associated with any number and/or type(s) of sources (e.g., broadcasters, television stations, governments, schools, studios, individuals, web sites, countries, etc.). Each of the example videos 205 has an associated audio track, two of which are designated at reference numerals 210 and 211. The example audio tracks 210 and 211 of FIG. 2 have respective closed caption information 215 and 216 corresponding to the various portions of the audio tracks 210 and 211. In the illustrated example of FIG. 2, a first audio data segment 220 associated with the example audio track 210 and a second audio data segment 221 associated with the example audio track 211 both correspond to the same closed caption text “Alan Alda.” However, the example audio data segments 220 and 221 represent the same or different pronunciations of the phrase “Alan Alda.” The example audio data segment 220 of FIG. 2 can be delimited and/or identified by times t1 and t2 of the audio track 210.

[0020] FIG. 3 illustrates an example data structure that may be used to implement the example speech recognition library 105 of FIG. 1. The example data structure of FIG. 3 includes a plurality of entries 305 for respective ones of a plurality of phrases. To represent a phrase, each of the example entries 305 of FIG. 3 includes a phrase field 310. Each of the example phrase fields 310 of FIG. 3 contains one or more letters and/or words that represent, for example, a name, a location, and/or a title.

[0021] To represent pronunciations, each of the example entries 305 includes a plurality of pronunciation entries 315 for respective ones of pronunciations of the phrase represented by the field 310. To store a pronunciation, each of the example pronunciation entries 315 of FIG. 3 includes a phonetic transcription field 320. Each of the example phonetic transcription fields 320 of FIG. 3 contains one or more letters, symbols and/or annotations that collectively represent a phonetic pronunciation of the phrase represented by the field 310. For example, the field 320 may contain “Alan-el-da” for the name “Alan Alda.”

[0022] To store audio data, each of the example pronunciation entries 315 of FIG. 3 includes an audio data field 325. Each of the example audio data fields 325 of FIG. 3 stores and/or represents audio data corresponding to the phonetic transcription 320. The example audio data 325 may be stored, for example, as linear predictive coding (LPC) encoded audio data.

[0023] In some examples, when a new phrase 310 is added to the speech recognition library 105 (e.g., a new entry 305 added to the library 105), corresponding baseline audio data 325 and a baseline phonetic transcription 320 are automatically created using a text-to-speech (TTS) synthesis engine. Additionally or alternatively, the baseline audio data 325 can be recorded as the phrase 310 is spoken by a person, and the baseline phonetic transcription 320 automatically created from the recorded audio data 325. For example, a user may use a computer system to specify (e.g., type) a phrase 310 to be added to the speech recognition library 105, and then use, for example, a microphone to record the baseline audio data 325. The example computer system implements a phonetic transcriber, such as the example phonetic transcriber 425 of FIG. 4, to generate the baseline phonetic transcription 320 based on the recorded baseline audio data 325.

[0024] While an example data structure that can be used to implement the example speech recognition library 105 of FIG. 1 is illustrated in FIG. 3, a speech recognition library 105 may be implemented using any number and/or type(s) of other and/or additional data structures, fields and/or data. Further, the fields and/or data illustrated in FIG. 3 may be combined, divided, re-arranged, eliminated and/or implemented in any way. Moreover, the example data structure of FIG. 3 may include fields and/or data in addition to, or instead of, those illustrated in FIG. 3, and/or may include more than one of any or all of the illustrated fields and/or data.

[0025] FIG. 4 illustrates an example manner of implementing the example pronunciation library generator 110 of FIG. 1. To select audio data segments from, for example, one or more audio/video programs, the example pronunciation library generator 110 of FIG. 4 includes an audio segment selector 405. The example audio segment selector 405 of FIG. 4 searches the video library 115 to identify programs having closed caption information that match a presently considered phrase 410. The phrase 410 may, for example, be provided to the pronunciation library generator 110 by a user and/or be automatically identified from, for example, an electronic program guide. The audio segment selector 405 further identifies the audio data segments (e.g., 5 or 10 second intervals) of the identified programs that roughly correspond to the phrase 410. The identified audio data segments are then delimited (e.g., starting and ending times identified) to more precisely identify the specific portions of the identified audio data segments that correspond to the phrase 410.

[0026] To compare the identified audio data segments with pronunciations already stored in the speech recognition library 105, the example pronunciation library generator 110 of FIG. 4 includes an audio comparator 415. The example audio comparator 415 of FIG. 4 compares each of the identified audio data segments with audio data previously associated with the phrase 410 in the speech recognition library 105 (e.g., the example audio data 325 of FIG. 3). For example, the audio comparator 415 can compute a difference metric between audio data segments. Example difference metrics include, but are not limited to, a mean-squared error, a difference in formants (i.e., sounds made by the human vocal tract), an LPC coefficient difference, or any combination thereof.

[0027] To group identified audio data segments, the example pronunciation library generator 110 of FIG. 4 includes an audio segment grouper 420. The example audio segment grouper 420 of FIG. 4 groups the identified audio data segments into one or more sets based on their differences. For example, a set may contain identified audio data segments that each differ from a baseline audio segment 325 in a similar manner. For example, they may all contain one or more particular formants that differ from the baseline audio segment 325.

[0028] To perform phonetic transcriptions, the example pronunciation library generator 110 of FIG. 4 includes a phonetic transcriber 425. For each set of audio data segments identified by the example audio segment grouper 420, the example phonetic transcriber 425 performs a phonetic description of a representative one of the group. A representative one of each group may be selected by, for example, identifying the audio data segment that has the smallest average difference when compared to all the other members of the group.

[0029] To manage the speech recognition library 105, the example pronunciation library generator 110 of FIG. 4 includes a database manager 430. For each group identified by the example audio segment grouper 420, the example database manager 430 compares the corresponding representative phonetic transcription with the phonetic transcriptions 320 previously associated with the phrase 410 in the speech recognition library 105. If the representative phonetic transcription differs from those previously associated with the phrase 410, the example database manager 430 adds the new phonetic transcription to the speech recognition library 105 by, for example, adding a new transcription entry 315 to a phrase entry 305 (FIG. 3). The example database manager 430 also adds the audio data segment 325 associated with the representative phonetic transcription for the group to the new transcription entry 315.

[0030] The example pronunciation library generator 110 of FIG. 4 may be operated when, for example, additional video and/or video clips are added to the video library 115, and/or when a new phrase 410 has been added to the speech recognition library 105. Thus, the same phrase 410 may be processed multiple times as, for example, new programs are added to the video library 115. Additionally or alternatively, the example pronunciation library generator 110 may be operated iteratively to refine the speech recognition library 105 such that, for example, the audio data 325 represents a more representative recording of the corresponding phonetic transcription 320. For example, a phrase 310 and baseline audio data 325 created using a TTS engine may initially be added to the speech recognition library 105. The example pronunciation library generator 110 may then be operated to identify more representative audio data for the phrase 410 based on closed caption information contained in the video library 115. The more representative audio data is used to replace the original baseline audio data 325 with audio data spoken by an actual person from the video library 114, and/or to add additional pronunciations of the phrase 410 to the library 105, if any are present in the video library 115.

[0031] While an example manner of implementing the example pronunciation library generator 110 of FIG. 1 has been illustrated in FIG. 4, one or more of the interfaces, data structures, elements, processes and/or devices illustrated in FIG. 4 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. For example, the phonetic transcriber 425 may perform a phonetic transcription for each of the audio data segments identified by the audio segment selector 405, and the thus generated phonetic transcriptions may be used by the audio comparator 415 and the audio segment grouper 420 to compare and group the identified audio data segments. Further, the example audio segment selector 405, the example audio comparator 415, the example audio segment grouper 420, the example phonetic transcriber 425, the example database manager 430 and/or, more generally, the example pronunciation library generator 110 of FIG. 4 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any or all of the example audio segment selector 405, the example audio comparator 415, the example audio segment grouper 420, the example phonetic transcriber 425, the example database manager 430 and/or, more generally, the example pronunciation library generator 110 may be implemented by one or more circuit(s), programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field-programmable logic device(s) (FPLD(s)), etc. When any of the appended claims are read to cover a purely software and/or firmware implementation, at least one of the example audio segment selector 410, the example audio comparator 415, the example audio segment grouper 420, the example phonetic transcriber 425, the example database manager 430 and/or, more generally, the example pronunciation library generator 110 are hereby expressly defined to include a tangible medium such as a memory, a digital versatile disc (DVD), a compact disc (CD), etc. storing the software and/or firmware. Further still, a pronunciation library generator may include interfaces, data structures, elements, processes and/or devices instead of, or in addition to, those illustrated in FIG. 4 and/or may include more than one of any or all of the illustrated interfaces, data structures, elements, processes and/or devices.

[0032] FIGS. 5 and 6 illustrates example machine accessible instructions that may be executed to implement the example pronunciation library generator 110 of FIGS. 1 and/or 4. The example machine accessible instructions of FIGS. 5 and/or 6 may be carried out by a processor, a controller and/or any other suitable processing device. For example, the example machine accessible instructions of FIGS. 5 and/or 6 may be embodied in coded instructions stored on a tangible medium such as a flash memory, a read-only memory (ROM) and/or random-access memory (RAM) associated with a processor (e.g., the example processor P105 discussed below in connection with FIG. 7). Alternatively, some or all of the example machine accessible instructions of FIGS. 5 and/or 6 may be implemented using any combination(s) of ASIC(s), PLD(s), FPLD(s), discrete logic, hardware, firmware, etc. Also, some or all of the example machine accessible instructions of FIGS. 5 and/or 6 may be implemented manually or as any combination of any of the foregoing techniques, for example, any combination of firmware, software, discrete logic and/or hardware. Further, although the example operations of FIGS. 5 and 6 are described with reference to the flowcharts of FIGS. 5 and 6, many other methods of implementing the operations of FIGS. 5 and/or 6 may be employed. For example, the order of execution of the blocks may be changed, and/or one or more of the blocks described may be changed, eliminated, sub-divided, or combined. Additionally, any or all of the example machine accessible instructions of FIGS. 5 and/or 6 may be carried out sequentially and/or carried out in parallel by, for example, separate processing threads, processors, devices, discrete logic, circuits, etc.

[0033] The example machine accessible instructions of FIG. 5 begin with the example audio segment selector 405 of FIG. 4 comparing a phrase with closed caption text of the example video library 115 to identify one or more segments of video having closed caption text matching the phrase (block 505). The phrase may be provided by, for example, a user of the pronunciation library generator 110. The audio segment selector 405 then delimits the portion(s) of the audio tracks associated with the identified video segments that correspond to the closed caption text (block 510).

[0034] The example audio comparator 415 of FIG. 4 compares each of the delimited audio data segments with audio data associated with the phrase in the speech recognition library 105 (e.g., baseline audio data computed from the phrase using a TTS module and/or baseline audio data identified using the example process of FIG. 5) (block 515). The example audio segment grouper 420 groups the audio data segments based on the differences (block 520).

[0035] The example phonetic transcriber 425 of FIG. 4 selects a representative one of a first set of audio data segments (block 525) and computes a phonetic transcription of the selected representative audio data segment (block 530). If the representative phonetic transcription is not the same as a phonetic transcription already associated with the phrase in the library 105 (block 535), the example database manager 430 adds the phonetic transcription to the speech recognition library 105 (block 540). If there are more groups to process (block 545), control returns to block 525 to process the next group. If there are no more groups to process (block 545), control exits from the example machine accessible instructions of FIG. 5.

[0036] Returning to block 535, if the phonetic transcription is already associated with the phrase in the library (block 535), control proceeds to block 545 to determine whether there are more groups to process.

[0037] In comparison to the example machine accessible instructions of FIG. 5, the example machine accessible instructions of FIG. 6 group identified audio data segments based on their phonetic transcriptions rather than based on computed difference metrics. The example machine accessible instructions of FIG. 6 begin with the example audio segment selector 405 of FIG. 4 comparing a phrase with closed caption text of the example video library 115 to identify one or more segments of video having closed caption text matching the phrase (block 605). The audio segment selector 405 then delimits the portion(s) of the audio tracks associated with the identified video segments that correspond to the closed caption text (block 610).

[0038] The example phonetic transcriber 425 of FIG. 4 computes a phonetic transcription for each of the delimited audio data segments (block 615). The example audio segment grouper 420 groups the audio data segments based on the phonetic transcriptions (block 620). For example, each group would contain all delimited audio data segments having the same phonetic transcription.

[0039] If the phonetic transcription of a first group is not already associated with the phrase in the speech recognition library 105 (block 625), the audio comparator 415 selects a representative one of the group of audio data segments (block 630), and the database manager 430 adds the phonetic transcription of the presently considered group and the representative audio data segment to the library 105 (block 635). If there are more groups to process (block 640), control returns to block 625 to process the next group. If there are no more groups to process (block 640), control exits from the example machine accessible instructions of FIG. 6.

[0040] Returning to block 625, if the phonetic transcription of the presently considered group is already associated with the phrase in the library (block 625), control proceeds to block 640 to determine whether there are more groups to process.

[0041] FIG. 7 is a schematic diagram of an example processor platform P100 that may be used and/or programmed to implement any or all of the example pronunciation library generators 110 disclosed herein. For example, the processor platform P100 can be implemented by one or more general-purpose processors, processor cores, microcontrollers, etc.

[0042] The processor platform P100 of the example of FIG. 7 includes at least one general-purpose programmable processor P105. The processor P105 executes coded instructions P110 and/or P112 present in main memory of the processor P105 (e.g., within a RAM P115 and/or a ROM P120). The processor P105 may be any type of processing unit, such as a processor core, a processor and/or a microcontroller. The processor P105 may execute, among other things, the example machine accessible instructions of FIGS. 5 and/or 5 to implement the example methods and apparatus described herein.

[0043] The processor P105 is in communication with the main memory (including a ROM P120 and/or the RAM P115) via a bus P125. The RAM P115 may be implemented by dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and/or any other type of RAM device, and ROM may be implemented by flash memory and/or any other desired type of memory device. Access to the memory P115 and the memory P120 may be controlled by a memory controller (not shown). The memory P115, P120 may be used to, for example, implement the example video library 115 and/or the example speech recognition library 105.

[0044] The processor platform P100 also includes an interface circuit P130. The interface circuit P130 may be implemented by any type of interface standard, such as an external memory interface, serial port, general-purpose input/output, etc. One or more input devices P135 and one or more output devices P140 are connected to the interface circuit P130.

[0045] Of course, the order, size, and proportions of the memory illustrated in the example systems may vary. Additionally, although this patent discloses example systems including, among other components, software or firmware executed on hardware, it will be noted that such systems are merely illustrative and should not be considered as limiting. For example, it is contemplated that any or all of these hardware and software components could be embodied exclusively in hardware, exclusively in software, exclusively in firmware or in some combination of hardware, firmware and/or software. Accordingly, the above described examples are not the only way to implement such systems.

[0046] At least some of the above described example methods and/or apparatus are implemented by one or more software and/or firmware programs running on a computer processor. However, dedicated hardware implementations including, but not limited to, an ASIC, programmable logic arrays and other hardware devices can likewise be constructed to implement some or all of the example methods and/or apparatus described herein, either in whole or in part. Furthermore, alternative software implementations including, but not limited to, distributed processing or component/object distributed processing, parallel processing, or virtual machine processing can also be constructed to implement the example methods and/or apparatus described herein.

[0047] It should also be noted that the example software and/or firmware implementations described herein are optionally stored on a tangible storage medium, such as: a magnetic medium (e.g., a disk or tape); a magneto-optical or optical medium such as a disk; or a solid state medium such as a memory card or other package that houses one or more read-only (non-volatile) memories, random access memories, or other re-writable (volatile) memories; or a signal containing computer instructions. A digital file attachment to e-mail or other self-contained information archive or set of archives is considered a distribution medium equivalent to a tangible storage medium. Accordingly, the example software and/or firmware described herein can be stored on a tangible storage medium or distribution medium such as those described above or equivalents and successor media.

[0048] To the extent the above specification describes example components and functions with reference to particular devices, standards and/or protocols, it is understood that the teachings of the invention are not limited to such devices, standards and/or protocols. Such systems are periodically superseded by faster or more efficient systems having the same general purpose. Accordingly, replacement devices, standards and/or protocols having the same general functions are equivalents which are intended to be included within the scope of the accompanying claims.

[0049] Although certain example methods, apparatus and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the appended claims either literally or under the doctrine of equivalents.

[0050] The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of the various embodiments. The illustrations are not intended to serve as a complete description of all of the elements and features of apparatus and systems that utilize the structures or methods described herein. Many other embodiments may be apparent to those of skill in the art upon reviewing the disclosure. Other embodiments may be utilized and derived from the disclosure, such that structural and logical substitutions and changes may be made without departing from the scope of the disclosure. Additionally, the illustrations are merely representational and may not be drawn to scale. Certain proportions within the illustrations may be exaggerated, while other proportions may be minimized. Accordingly, the disclosure and the FIGS. are to be regarded as illustrative rather than restrictive.

[0051] The Abstract of the Disclosure is provided with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.