US 7,319,958 B2Grant

Polyphone network method and apparatus

Issue Date:2008-01-15

•25 Claims

•2 Drawing Sheets

Abstract

Acoustic phones (preferably drawn 12 from a plurality of spoken languages) are provided 11. A hierarchically-organized polyphone network (20) organizes views of these phones of varying resolution and phone categorization as a function, at least in part, of phonetic similarity (14) and at least one language-independent phonological factor (15). In a preferred approach, a unique transcription system serves to represent the phones using only standard, printable ASCII characters, none of which comprises a special character (such as those characters that have a command significance for common script interpreters such as the UNIX command line).

Metadata

Assignee

Motorola, Inc.

Inventors

Lynette Melnar
Jim Talley
Yuan-Jun Wei
Chen Liu

Application Information

Application Number:US 10/365,820

Filing Date:2003-02-13

Priority Date:2003-02-13

Art Unit:2626

Classifications

IPC:

G10L15/04

Patent Drawings (2 sheets)

Description

Technical Field

[0001] This invention relates generally to acoustic phones and more particularly to networks of multiple phones.

Background

[0002] Speech recognition techniques are known in the art. Many such techniques utilize models that are selected as a function of a polyphone network. For example, a network of phones (wherein a “phone” is generally understood to typically comprise a speech sound considered as a physical event without regard to its possible phonological status in the sound system of a language) that comprise at least the primary phonemes (wherein a “phoneme” is generally understood to comprise an abstract unit of the phonological system of a language that corresponds to a set of similar speech sounds that are perceived to be a single distinctive sound in the language) of a given spoken language (or specific dialect thereof can comprise an acoustic foundation upon which one derives a set of models that a corresponding speech recognition engine can utilize to effect recognition of a sample of uttered speech.

[0003] Speech recognition platforms (such as, for example, a cellular telephone) that support recognition of multiple languages are also known. Unfortunately, current multilingual (and/or multi-dialect) automatic speech recognition technologies face a number of practical constraints that appear to significantly limit widespread commercial application of such an approach. One problem involves the cost of acquiring and/or the relative availability of relevant language resources and expertise. For example, while many hundreds of different acoustic phones are known to be utilized globally in one language or another, any given language (or dialect) tends to use as phonemes only a relatively small subset of this larger pool. The limited availability of resources and technology-savvy expert knowledge required to linguistically characterize a language for speech recognition engineering purposes (including, but not limited to, specifically identifying those phonemes that appropriately characterize a given language) are an impediment to broad language coverage. Further, the time and expense of creating, finding, or otherwise acquiring appropriate acoustic speech data (with associated transcriptions, lexica, and so forth) of acceptable quality and quantity to permit training of speech models often make such endeavors commercially unfeasible, especially for consumer populations that represent a relatively small speaker group and/or an emerging market.

[0004] Computational resource limitations present another problem. A not atypical prior art approach combines a plurality of monolingual speech recognition systems to consolidate sufficient capability to support multiple languages and/or dialects. With such an approach, however, requirements for both the necessary language resources and computational resources increase substantially proportionally with each incremental supported language and/or dialect. The costs and/or form factor constraints associated with such needs can again influence designers away from including languages that correspond to smaller speaker populations and/or smaller present marketing opportunities.

[0005] To attempt to meet such problems, some effort has been made to consider sharing acoustic models across a plurality of languages and/or dialects. Such an approach typically requires alteration to the fundamental approach by which models are developed (for example, while models that result from the approaches noted earlier tend to preserve substantially all defined phonetic contrasts in a given set of targeted languages or dialects, such models also poorly support any attempt to exploit any cross-language allophonetic coincidences that might also exist; in addition, the set of models is often too large and too finely differentiated for cost-effective and efficient multilingual or multi-dialect automatic speech recognition needs). For example, one approach employs an acoustic feature-based data-driven method to achieve a kind of phone merger. Acoustic models from a collection of monolingual speech recognition systems are compared and acoustically similar models are merged. Such an approach indeed tends to at least reduce to some extent total memory requirements, but such an approach is also relatively indiscriminate with respect to the grouping of acoustically similar phones (for example, this approach tends to readily permit the merger of acoustically similar but phonologically contrastive models). This approach also fails in large measure to supply acoustic models of phones for which little or no data is conveniently available. One suggested alteration to this latter approach has been to constrain the acoustic data-driven method as a function of language knowledge. This approach seeks to retain improved memory requirements while also attempting to address acoustic confusability of distinct phones. Unfortunately, however, this attempt at improvement still remains largely dependent upon the availability and quality of language data.

[0006] Another impediment to fielding a commercially useful result is the present practice of tending to represent phone information with a transcription system that favors unique font sets and/or unusual characters that are typically non-alphanumeric, printable characters that are commonly used for special purposes in command line scripting as they have special control interpretations (such as, for example, ‘@’, ‘>’, ‘\’, ‘|’, ‘=’, ‘{’, ‘^’, ‘?’, ‘*’, ‘(’, and so forth) in one or more relevant computer command and control protocols. For example, some characters used by some transcription systems present phone information that is also interpreted by Unix command line interpreters as specific Unix commands. This unfortunate proclivity further complicates the matter of attempting to provide a flexible, efficient polylingual speech recognition method and apparatus.

Brief Description of the Drawings

[0007] The above needs are at least partially met through provision of the polyphone network method and apparatus described in the following detailed description, particularly when studied in conjunction with the drawings, wherein:

[0008] FIG. 1 comprises a flow diagram as configured in accordance with an embodiment of the invention;

[0009] FIG. 2 comprises a schematic representation of a hierarchical phone correspondence network as configured in accordance with an embodiment of the invention; and

[0010] FIG. 3 comprises a block diagram of a speech recognition platform as configured in accordance with an embodiment of the invention.

[0011] Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of various embodiments of the present invention. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are typically not depicted in order to facilitate a less obstructed view of these various embodiments of the present invention.

Detailed Description

[0012] Generally speaking, pursuant to these various embodiments, phonological similarities across a plurality of languages (and/or dialects) are leveraged in a way that is at least partially based upon language-independent factors. More particularly, a plurality of phones that correspond to a plurality of spoken languages can serve to provide a polyphone network as a function, at least in part, of at least one of phonetic similarity as may exist between the phones and at least one language-independent phonological factor. In a preferred embodiment, the plurality of phones correspond to a plurality of languages including at least some languages that are substantially typologically diverse with respect to one another. Such diversity enriches the resultant polyphone network and tends to render such a network more likely to provide satisfactory models even for languages that are not otherwise specifically represented. Pursuant to another preferred approach, at least some of the plurality of phones are based, at least in part, upon at least some phonemes that are themselves substantially unique to only a few (or one) of the represented languages (in one embodiment, all phonemes that are unique to a particular language amongst the plurality of languages are so represented).

[0013] In a preferred embodiment, the polyphone network comprises a hierarchical phone correspondence network that preferably organizes merged phone selections in a tree. In one embodiment, the tree can comprise at least one binary branching tree. Depending upon the embodiment, phone information as contained by the polyphone network can be viewed at varying levels of resolution (such that, for example, higher resolution information may include at least some phone information that comprises phonemes for at least some of the plurality of spoken languages and lower resolution information may include phone information that is representative, at least in part, of cross-linguistic and language-independent mutual phonetic similarity and non-contrastiveness of corresponding higher resolution phone information views).

[0014] Depending upon the embodiment, the language-independent phonological factor can comprise information such as language-independent phonological contrast, cross-linguistic phone frequency, predetermined linguistic tendencies, and predetermined linguistic universals.

[0015] In general, the resultant polyphone network is preferably substantially unbiased towards any particular language, language family, and/or language type.

[0016] In a preferred embodiment, the phones are represented in the polyphonic network using a transcription system that comprises only standard ASCII characters (none of which comprises a control character or other special character (hereinafter referred to generally as “special characters”) having a corresponding computer operation command meaning nor to which Unix command line interpreters ascribe special significance) such that the transcription system characters only require a standard font set and substantially avoid programming or scripting collisions.

[0017] So configured, a polyphonic network can be provided that will tend to well support polylingual purposes, including at least substantial accommodation of languages that are not specifically represented by the network. The resultant network (and speech recognition models based thereon) can also be well accommodated by relatively modest computational resources including reduced memory needs as compared to prior art counterparts.

[0018] Referring now to the drawings, and in particular to FIG. 1, in a preferred approach, the process begins with provision 11 of a plurality of phones. In some cases, these phones may be at least partially provided through use of one or more previously compiled/selected phones as drawn from a plurality of spoken languages. If desired, of course, one may optionally first begin by selecting 12 a plurality of specifically selected spoken languages. If desired, the selected languages can constitute a significant portion of all currently known and spoken languages (such as, for example, at least five or even twenty-five percent of all such candidate languages). Regardless, however, it is preferred that the provided phones be drawn from a plurality of languages wherein at least two, and preferably more, of the languages are substantially typologically diverse with respect to one another (in general, significant diversity as between utilized languages will tend to better facilitate the provision of a rich and enabling polyphone network than mere quantity of typologically similar languages). Also in a preferred approach, when providing 11 the phones, at least some of the phones should correspond to phonemes that are themselves substantially unique to each of at least some of the plurality of spoken languages (and where preferably all such unique phonemes are so represented by such phones). Again, capturing such instances of uniqueness will tend in the aggregate to enrich the acoustic qualities of the resultant polyphone network.

[0019] Various transcription systems exist by which to express such phones. It is therefore possible to provide 11 the phones as described above using any of a wide variety of known or hereafter developed transcription systems. As noted earlier, however, most existing transcription systems tend to require either unique fonts and/or include characters that have additional instructional meaning for many processing platforms. In a preferred approach, therefore, the phones are represented using a transcription system that comprises only standard, printable ASCII characters and wherein none of the characters utilized comprise a special character having a corresponding computer operation command meaning with respect to the desired operating environment (for example, when using Unix to implement the relevant platform(s), none of the transcription characters should have a command meaning to a Unix command line interpreters). A description of a new preferred transcription system that accords with these design criteria follows.

[0020] This transcription system is an ASCII-encoded speech symbol inventory restricted to lower-case alphabetic characters, numerals, and the non-alphanumeric symbol ‘_’. Each individual symbol is associated with a specific phonetic feature (or feature constellation) dependent on the type of phone represented (consonant or vowel) and its syntactic position in the phone symbol string. In this embodiment, the basic character length of any consonant or vowel phone representation is two; this obligatory symbol pair is referred to as a phone's base sign. Pursuant to a preferred embodiment, a phone's phonetic features are directly encoded into the label. The first symbol of a base sign unambiguously marks the phone as either a consonant or vowel. All non-tonal diacritic symbols are sorted alphabetically behind the base sign and are framed by the marker ‘_’. Tonal diacritics are suffixed to the right diacritic marker of vowel phone strings. An example of a simple consonant phone representation, one with only the two obligatory base-sign positions filled, is ‘kp’, where the first position ‘k’ is associated with the consonant class of phones and indexes ‘velar articulation’ and the consonant second position ‘p’ indexes ‘voiceless plosive’. Thus, ‘kp’ is a voiceless velar plosive (/k/ in the International Phonetic Alphabet “IPA”). A complex vowel symbol string, that is, a vowel representation that includes diacritics suffixed to the base sign, is exemplified by ‘ii_j_—4’, where vowel first position ‘i’ indexes ‘unrounded front’, vowel second position ‘i’ signifies ‘close’, vowel third position ‘j’ is ‘palatal offglide’, and forth position ‘4’ is ‘low tone’ (/i^j/ in the IPA). The consonant and vowel position classes are defined below, along with an inventory of the position class constituents.

[0021] A preferred consonant phone string has the following position structure, from left to right:

- [0022] 1^stposition (obligatory): symbol representing primary place(s) of articulation;
- [0023] 2^ndposition (obligatory): symbol representing primary manner(s) of articulation and voice (+/−v); and
- [0024] 3^rdposition: sorted diacritic symbols, framed left and right by the diacritic marker ‘_’.

[0025] The consonant phone symbols and their interpretations are presented by position class in Table 1.

[0026] Within a given consonant symbol string, the first and second positions are only occupied by one symbol each. The first position symbol of a consonant phone label comprises a consonant symbol in traditional Romanized orthography (excluding ‘y’ as noted below). This convention unambiguously marks the label as a consonant phone.

[0027] The third position is not limited in length, but the diacritic set will preferably be sorted. Note that all symbols in the consonant inventory are lower-case alphabetic characters and that any particular character may be reused with a different meaning in a different syntactic position in the phone string. Thus, a first position ‘p’ means ‘bilabial’ while a second position ‘p’ means ‘voiceless plosive’. The sign ‘pp’, therefore, represents a voiceless bilabial plosive. Secondary places and manners of articulation are given in the third position diacritic class. As an example, an aspirated voiceless bilabial plosive is ‘pp_c_’.

[0028]

TABLE 1

Consonant Symbol Inventory by Position Class
	2^ndPosition (MOA,
1^stPosition (POA)	+/−v)	3^rdPosition (diacritics)

Bilabial	p	plosive/−v	p	aspirated/breathy	c
labio-dental	f	plosive/+v	b	lateralized	d
labio-palatal	m	nasal (+v)	n	laryngealized	h
labio-velar	b	fricative/−v	s	palatalized	j
Dental	d	fricative/+v	z	velarized	k
Alveolar	t	lateral fricative/−v	j	long	l
alveo-palatal	g	lateral fricative/+v	y	prenasalized	m
alveo-lateral	l	approximate (+v)	w	nasal(ized/release)	n
Retroflex	r	lateral-approx. (+v)	l	devoiced	o
Palatal	j	affricate/−v	f	labialized	p
Velar	k	affricate/+v	v	imploded	q
Uvular	q	tap/flap (+v)	d	syllabic	s
Pharyngeal	x	nasal tap/flap (+v)	m	ejected	t
Epiglottal	c	trill (+v)	r	non-sibilant	v
Glottal	h			pre-aspirated	w
				pharyngealized	x
				not released	z

[0029] A preferred vowel phone string has the following position structure, from left to right:

- [0030] 1^stposition (obligatory): symbol representing backness/roundness (back, +/−r);
- [0031] 2^ndposition (obligatory): symbol representing openness;
- [0032] 3^rdposition: sorted diacritic symbols, framed left and right by the diacritic marker ‘_’; and
- [0033] 4^thposition: number representing tone.

[0034] The preferred constituents of the vowel position classes are presented in Table 2.

[0035]

TABLE 2

Vowel Symbol Inventory by Position Class
1^stPosition	2^ndPosition	3^rdPosition
(back, +/−r)	(openness)	(diacritics)	4^thPosition (tones)

front/−r	i	close	i	primary stress	1	extra-high	1
front/+r	y	close, lax	h	secondary	2	high	2
				stress
central/	e	close-mid	e	tertiary stress	3	mid	3
−r
central/	o	mid	x	breathy voiced	c	low	4
+r
back/−r	a	open-mid	o	central offglide	e	extra-low	5
back/+r	u	open2-mid	c	laryngealized	h	rising	32
		open	a	palatal offglide	j	falling	34
		schwar	r	velar stricture	k	high-rising	21
		(w/e)
				long	l	high-	24
						falling
				labial inglide	m	low-rising	42
				nasalized	n	low-falling	45
				devoiced	o	rising-	424
						falling
				retracted/	r
				retroflexed
				overshort	s
				coronal inglide	t
				labial offglide	w
				pharyngealized	x
				palatal inglide	y

[0036] A vowel phone label preferably begins with either a traditional vowel symbol from Romanized orthography or the character ‘y’. Thus, any label beginning with ‘a’, ‘e’, ‘i’, ‘o’, ‘u’, or ‘y’ unambiguously marks a vowel phone.

[0037] Like the consonant position-class inventories, within a given phone symbol string each constituent of the first and second obligatory positions are in a disjunctive relation to other constituents in their same class. Furthermore, each constituent of these positions is a lower-case alphabetic character and any particular character may be reused in a different position where it is associated with a distinct meaning. Thus, an unrounded open back vowel is represented by the string ‘aa’, where ‘a’ indexes ‘back unrounded’ in vowel first position and ‘open’ in vowel second position. Non-tonal diacritics are sorted and framed by the diacritic marker ‘_’ and all tonal diacritics are represented by numbers and follow the right diacritic marker. So, for example, a long unrounded open-mid front vowel associated with high tone would be symbolized as ‘io_l_—1’.

[0038] So configured, this transcription system is designed to be usable from both a processing and a human-factors perspective. Because the symbol string is structurally defined and delimited, and because its symbol inventory lacks potentially ambiguous signs in a programming environment, these transcription strings are easy to parse and manipulate. Because individual symbols are consistently used within their type (consonant or vowel) and class (position number) and were chosen to be as congruent as possible with symbols from pre-existing phonetic symbol inventories, and because string syntax is well defined, this transcription system is relatively easily learned and expanded (unlike, for example, the IPA system).

[0039] As noted above, each symbol string preferably consists of a two character base sign to which diacritic markers may be suffixed. Suffixes only terminate in either the right-frame diacritic marker ‘_’ or a right frame diacritic marker followed by one or more numerals. In this embodiment, the first position character of a base sign always marks the string as either consonant or vowel, i.e., there is no overlap of characters between the first position consonant and vowel symbol inventories. Finally, there is no use of symbols that are employed in a special way in programming. Thus, every symbol-string label indexing a distinct phone can be easily identified and parsed with minimal programming effort, and it is not generally necessary (as it is with many other phonetic alphabets) to build a special-purpose scanner designed to convert these labels to other label types (numbers, for example).

[0040] Each individual character used in the above described preferred transcription system comprises a sign, or form-meaning pair. The relationship between a character's graphic form and its phonetic meaning is one-to-many and uniquely dependent on the sign's context, that is, its place in the symbol-string label. For example, the form ‘w’ is associated with three distinct articulation features or feature constellations: 1) voicing and approximation, 2) pre-aspiration, and 3) labial offgliding. Notwithstanding the above, the meaning of any particular ‘w’ is never ambiguous: ‘w’ is associated only with meaning (1) as a second position consonant sign; it indexes meaning (2) as a third position consonant diacritic; and as a third position vowel diacritic, it is uniquely linked to meaning (3).

[0041] Most alphabetic characters are strongly associated with phonetic meanings that users literate in Romanized orthographies learned during childhood. Because of this, there is an attempt in most multilingual phonetic inventories to conform as much as possible to pre-existing form-meaning associations, and the above described preferred transcription system mirrors this viewpoint. Thus the symbol ‘i’ is typically associated with the vowel features ‘close’, ‘front’ and ‘unrounded’ and this proposed transcription system observes this conventional sense as well. Only these features are divided between two classes: vowel position 1 (where ‘i’ means ‘front’ and ‘unrounded’) and vowel position 2 (where it means ‘close’). By unpacking a character form's typically associated feature set, it becomes possible to use the form in several distinct signs, each associated with just one or two features of the source sign.

[0042] There are presently several pre-existing phonetic alphabets in common use within the speech research community. XSAMPA and Worldbet, for example, are relatively widely employed for multilingual applications (though again, these systems tend to suffer drawbacks associated with computer processing applications and conflicts). Appendix A attached hereto presents an abbreviated base-sign inventory for the preferred transcription system (denoted as “new” in the appendix) described above and further maps those contents to the XSAMPA, Worldbet, and the IPA phonetic alphabets for the convenience of the reader.

[0043] With continued reference to FIG. 1, the provided phones 11 are then used to provide 13 a polyphone network as a function, at least in part, of phonetic similarity 14 as may exist between certain of the provided phones and at least one language-independent phonological factor 15 (such as, for example, at least one of language-independent phonological contrast, cross-linguistic phone frequency, predetermined linguistic tendencies, and predetermined linguistic universals). So configured, the resultant polyphone network will tend to be substantially unbiased towards any particular language, language family, and/or language type.

[0044] In a preferred approach, the polyphone network includes and/or comprises a hierarchical phone correspondence network. In one embodiment, that hierarchy can be configured as a plurality of merged phone selections organized in at least one tree (such as, for example, a binary branching tree hierarchical arrangement). For example, and referring now to FIG. 2, all of the phones (such as, for example, PHONE 1 and PHONE 2) that comprise the network can share a common highest resolution view 21 or level within the polyphone network 20. A next level or view 22 up from this highest resolution view 21 in turn comprises phones that represent a merger of two or more lower level phones (to put this another way, this lower resolution view provides a lower resolution view of phone information comprised of phone categories). To illustrate, PHONE 1, PHONE 2, and PHONE 3 from the first highest resolution level 21 have been merged into PHONE 1¹at the next level of resolution 22 while PHONE 5 and PHONE 6 have been merged into PHONE 5¹. In general, the resultant lower resolution phone will comprise a selected one of the merged higher resolution phones. Details regarding merging criteria are provided below. It should also be noted that not all of the higher resolution phones are necessarily merged with another phone at this second level 22 in the hierarchy. Instead, to illustrate, PHONE 4 and PHONE 9 both remain unmerged at this second hierarchical level 22.

[0045] In a similar fashion, the resultant phone categories (and remaining unmerged phones of the first hierarchical level 21) are again merged as and how appropriate at a next yet-lower-resolution hierarchical level 23. For example, PHONE 1¹and PHONE 4 are merged at this level to yield PHONE 1². In a preferred embodiment, fifteen such levels are provided (with the intervening levels being indicated by the ellipses 24 depicted), with the final lowest resolution view 25 preferably comprising a final category PHONE X^Y. Conceptually, this final hierarchical level 25 comprises a single sound that purports to grossly characterize the entire polyphone network,

[0046] So configured, these various views provide phone categories that correspond to views of varying resolution wherein the phone information essentially represents, at least in part, cross-linguistic and language-independent mutual phonetic similarities and non-contrastiveness of corresponding higher resolution phone information views. As the degree of resolution drops by moving higher in the tree, at least some of the merged phone nodes each comprise a merger of a plurality of phone categories, which phone categories are themselves each most representative of a given plurality of phone categories.

[0047] For many tasks, of course, it is not feasible to use a narrow phone transcription as represented at the highest resolution levels of the suggested polyphone network 20. All languages collapse distinct phones that are non-distinct phonologically into phonemes. Using the preferred transcription system detailed above, consider that in English, for example, a flapped alveolar stop (td), a voiceless alveolar stop (tp) and a voiceless aspirated alveolar stop (tp_c_) are all allophones of the phoneme represented by the English alphabetic symbol ‘t’. Likewise, it is possible for distinct phonemes across languages (though not distinct phonemes internal to a language) to be collapsed into a merged ‘superphoneme’ classification system. Thus, for example, it is usually the case that a voiceless dental stop phoneme (dp) and a voiceless alveolar stop phoneme (tp) may share acoustic models without losing crucial cross-linguistic phonological distinctions. Such knowledge can be exploited when reducing a multilingual model inventory to a size that is more compatible with recognition systems in embedded devices.

[0048] The polyphone network proposed herein comprises a phone merger specification tool for shared multilingual and multi-dialect acoustic modeling that takes advantage of phonetic similarities (as inherently and conveniently defined by the preferred transcription system described above) while maximally preserving known phonological contrasts. In a preferred embodiment, the polyphone network comprises a binary branching tree that has as its leaves a nearly exhaustive inventory of core phones derived from a rich set of typologically diverse languages (where preferably at least all major language families are represented) (and wherein core phone should be understood to refer to an allophone that is most representative of a particular phoneme, defined cross-linguistically). Lower resolution nodes on the tree represent superphonetic categories with merger being preferably based entirely on phonetic similarity, core phone frequency, and language-independent (universal) phonological laws and tendencies.

[0049] As an illustration of how language-independent factors can contribute to such a hierarchical arrangement of phone classes, a detailed example is provided below:

[0050] Consider the following set of language universals, tendencies, and frequency observations:

[0051] Language universal (1): all languages have stop consonants;

[0052] Language tendency (1): languages tend to have a plain voiceless stop consonant series;

[0053] Frequency observation (1): p, t and k are the most frequently occurring stop consonants in the world's inventories, with t being the most common and p the least common of the three;

[0054] Language universal (2): t<k<p; and

[0055] Language tendency (2): language phoneme inventories exploit the maxim of sufficient discriminability—that is, contrastive sounds in any given language must be sufficiently distinct from one another to be distinguishable in articulation and perception.

[0056] The first language universal points to the relative importance of stops as a consonant class. Other consonant classes (fricatives, liquids, nasals, and so forth) are not universally present in the world's sound inventories. Thus among the consonants, stops are preferably ranked higher than the other classes. This indicates that the stop class is the highest level consonant class in the phone hierarchy. Language tendency (1) provides additional information about stops. in addition to whatever other stop series a language may have—voiced, ejective, implosive, and so forth—it is likely to have a plain voiceless series. This means that within the stop class, the plain voiceless series is ranked highest in relative cross-linguistic importance.

[0057] The provided frequency observation records the overall importance of the three plain voiceless stops p, t, and k. These three stops occur more frequently than any other stops in the world's inventories. And within this set, there is a frequency hierarchy, where t is ranked the highest. The second language universal, t<k<p, signifies that if a language has the stop p, it must also have the stops k and t; and if a language has k, it necessarily has t.

[0058] Finally, the second language tendency suggests that contrastive sounds in a given language tend to be distributed proportionally among a range of articulators (lips, alveolar ridge, palate, velum, larynx, and so forth) and to not cluster together at the same region. Therefore, for example, no language contrasts palato-alveolar and palatal phonemes within a given manner series, and contrasts among dentals and alveolars within the same manner series are extremely rare. Notice that the three most common plain voiceless stops, p, t, and k observe this maxim; p is a bilabial stop, t is a dental or alveolar stop, and k is a velar stop.

[0059] These above factors contribute to the following preferred high-level hierarchical arrangements for stop consonants in the proposed polyphone network (set forth in the notation of the earlier described preferred transcription):

- [0060] 1. tp^14[2]->tp^13[2] kp^11[2]
- [0061] 2. tp^13[2]->tp^12 pp^10[2]
  where a caret (‘^’) indexes a node in the tree and the numbers signify a hierarchical level, beginning with ‘0’ at the leaves. Therefore, tp^14 is the parent node of the children nodes tp^13 and kp^11, and tp^13 has the children nodes tp^12 and pp^10.

[0062] At each node in the tree a confidence score between ‘1’ and ‘10’ appears in square brackets, where ‘10’ indicates very high confidence in the merge and ‘1’ indicates very low confidence in the merge. In the above examples all nodes have a confidence score of ‘2’. These scores are preferably derived from three knowledge sources: 1) relative phonetic similarity (defined automatically by the transcription structure), 2) relative phonological contrastiveness (that is, how often do the phones included in the children nodes contrast phonologically), and 3) frequency (that is, how frequently the phones included in the children nodes occur cross-linguistically).

[0063] The above example illustrates how the preferred polyphone network provides a novel, linguistically informed framework within which the phone model inventory size and search space issues inherent to embedded multilingual or multi-dialect automatic speech recognition can be accommodated while minimizing the negative performance impact of the resource restrictions. The complete proposed preferred polyphone network appears in Appendix B attached hereto.

[0064] Such a polyphone network is well suited for multi-language phone transcription, and especially for computational applications. Such a network explicitly encodes phonetic features that both make the inventory maximally extensible and contributes to automatic phonetic similarity metrics in the polyphone network. It can be seen that such a polyphone network further comprises a universal phone merger tool that surpasses other strategies for phone merger by taking into account not only phonetic similarity, but also language-independent universals. Using such a polyphone network, it is possible to derive a language-inclusive phone inventory which has a minimal number of distinct units but which maximally preserves both language-internal and cross-language contrasts. Use of such an inventory greatly reduces corresponding speech recognition search efforts.

[0065] As illustrated in FIG. 3, a not-uncommon platform such as a two-way wireless communications device 34 (such as, for example, a cellular telephone) often includes a speech recognition engine 31 that interacts one way or another with a wireless transceiver 35 as well understood in the art. Pursuant to these teachings, such a speech recognition engine 31 can now be partially or fully enabled through use of, for example, models 33 that are based, at least in part, upon a polyphone network as described above (such models 33 can of course be stored in off-board memory 32 as illustrated and/or in on-board memory integral to the speech recognition engine 31). So configured, the benefits of speech recognition are more easily realized for a wide variety of languages, including the languages of small speaker populations and/or the languages indigenous to markets that are otherwise presently relatively modest.

[0066] Those skilled in the art will recognize that a wide variety of modifications, alterations, and combinations can be made with respect to the above described embodiments without departing from the spirit and scope of the invention, and that such modifications, alterations, and combinations are to be viewed as being within the ambit of the inventive concept.

Claims

We claim:

1. A method to provide a polyphone network, comprising:

selecting a plurality of spoken languages;

identifying at least some phones for each of the plurality of spoken languages;

providing a polyphone network that is a hierarchical phone correspondence network comprising the at least some phones, and that includes a low resolution level of phone information, wherein the phone information of the low resolution level is determined from corresponding higher resolution phone information levels as a function of at least:

articulatorily defined phonetic similarity as between the higher resolution phone information level phones; and

at least one language-independent phonological factor.

2. The method of claim 1, wherein selecting a plurality of spoken languages comprises selecting a plurality of spoken languages that includes at least two languages that are substantially typologically diverse with respect to one another.

3. The method of claim 2, wherein selecting a plurality of spoken languages that includes at least two languages that are substantially typologically diverse with respect to one another includes selecting at least five percent of all currently spoken languages.

4. The method of claim 3 wherein selecting at least five percent of all currently spoken languages includes selecting at least twenty-five percent of all currently spoken languages.

5. The method of claim 1 wherein identifying at least some phones for each of the plurality of spoken languages includes identifying at least some phonemes that are substantially unique to each of at least some of the plurality of spoken languages.

6. The method of claim 5 wherein identifying at least some phonemes that are substantially unique to each of at least some of the plurality of spoken languages includes identifying substantially all unique phonemes for substantially all of the plurality of spoken languages.

7. The method of claim 1 wherein providing a hierarchical phone correspondence network includes organizing merged phone selections in at least one tree.

8. The method of claim 1 wherein providing a hierarchical phone correspondence network includes organizing merged phone selections in at least one binary branching tree.

9. The method of claim 1 wherein providing a polyphone network includes providing a higher resolution view of phone information.

10. The method of claim 9 wherein providing a higher resolution view of phone information includes providing a higher resolution view of phone information wherein at least some of the phone information comprises phonemes for at least some of the plurality of spoken languages.

11. The method of claim 9 wherein providing a polyphone network further includes providing a high resolution view of phone information that is a less high resolution view than the higher resolution view of phone information.

12. The method of claim 9 wherein providing a polyphone network further includes providing a lower resolution view of phone information comprised of phone categories.

13. The method of claim 9 wherein providing a high resolution view of phone information includes providing a binary branching tree hierarchy.

14. The method of claim 13 wherein providing a hierarchical phone correspondence network comprising at least one binary branching tree includes providing a hierarchical phone correspondence network comprising at least one binary branching tree having at least one level of non-merged phones and at least five potential levels of hierarchically merged phone nodes.

15. The method of claim 14 wherein providing a hierarchical phone correspondence network comprising at least one binary branching tree having at least one level of non-merged phones and at least five potential levels of hierarchically merged phone nodes includes providing at least some merged phone nodes that each comprise a merger of a plurality of phone categories, which phone categories are themselves each most representative of a given plurality of phone categories.

16. The apparatus of claim 15 wherein the polyphone network is represented by standard, printable ASCII characters, none of which comprises a special character commonly having a corresponding computer operation command meaning nor to which Unix command line interpreters prescribe special significance, such that the polyphone network only requires a standard font set and substantially avoids programming collisions.

17. The method of claim 1 wherein the at least one language-independent phonological factor comprises at least one of language-independent phonological contrast, cross-linguistic phone frequency, predetermined linguistic tendencies, and predetermined linguistic universals.

18. The method of claim 1 wherein providing a polyphone network includes providing a polyphone network that is substantially unbiased towards any particular language.

19. The method of claim 18 wherein providing a polyphone network includes providing a polyphone network that is substantially unbiased towards any particular language family.

20. The method of claim 19 wherein providing a polyphone network includes providing a polyphone network that is substantially unbiased towards any particular language type.

21. The method of claim 1 wherein identifying at least some phones for each of the plurality of spoken languages includes representing the phones using a transcription system that comprises only standard, printable ASCII characters, none of which comprise a special character commonly having a corresponding computer operation command meaning nor to which Unix command line interpreters prescribe special significance, such that the transcription system characters only require a standard font set and substantially avoid programming collisions.

22. The method of claim 1 where identifying at least some phones includes representing the phones using a transcription system that comprises only standard, printable ASCII characters, none of which comprise a special character commonly having a corresponding computer operation command meaning nor to which Unix command line interpreters prescribe special significance, such that the transcription system characters only require a standard font set and substantially avoid programming collisions.

23. An apparatus comprising:

a speech recognition engine;

a memory operable coupled to the speech recognition engine and containing a set of models selected as a function, at least in part, of a polyphone network that is a hierarchical phone correspondence network comprised of a plurality of phones, which phones correspond to phones as used in a plurality of spoken languages and wherein relationships between the phones at lower resolution levels are represented hierarchically by nodes that correspond to merged phones from higher resolution levels, wherein merging of the phones to form the nodes is based, at least in part, upon articulatory phonetic similarity as between the phones and at least one language-independent phonological factor.

24. The apparatus of claim 23 wherein the apparatus comprises a wireless communications device.

25. The apparatus of claim 23 wherein the wireless communications device comprises a two-way wireless communications device.

Patent Citations (2)

Patent	Date	Inventor	Cited By
US6085160(A)	2000-07-01	D'hoore et al.	Examiner
US6912499(B1)	2005-06-01	Sabourin et al.	Examiner

Non-Patent Literature (18)

Cheng, Y.M., Chen Liu, Yuan-Jun Wei, Lynette Melnar, Changxue Ma. 2003. “An Approach to Multilingual Acoustic Modeling for Portable Devices”, Eurospeech, Geneva (pp. 3121-3124).Applicant
Chomsky, N., and Halle, M. 1968. “The Sound Pattern of English”, Harper & Row, Publishers New York, Evanston, and London, 4-6, (pp. 298-329).Applicant
Crothers, John. 1978. “Typology and Universals of Vowel Systems”, Universals of Human Language, vol. 2, Phonology. Joseph H. Greenburg, Charles A. Ferguson, Edith A. Moravesik, editors, Stanford: Stanford University Press (pp. 93-152).Applicant
Fikkert, Paula. 1995. “Acquisition of Phonology”, Glot International, 1:8, 3-8 (25 pages).Applicant
Gamkrelidz, Thomas V. 1978. “Stops and Fricatives”, Universals of Human Language, Vol. 2, Phonology. Joseph H. Greenburg, Charles A. Ferguson, Edith A. Moraveski, editors, Stanford: Stanford University Press (pp. 9-46).Applicant
Hieronymus, J. L. 1994. “ASCII Phonetic Symbols for the World's Languages: Worldbet”, Technical Report, AT&T Bell Labs (48 pages).Applicant
Hieronymus, J. L. 1997. “Worldbet Phonetic Symbols for Multilanguage Speech Recognition and Synthesis”, Technical Report, AT&T Bell Labs (32 pages).Applicant
Hockett, Charles F. 1966. “The Problem of Universals in Language”, Universals of Language, 2ed. Cambridge, MA: MIT Press (pp. 1-29).Applicant
IPA. 1999. “Handbook of the International Phonetic Association”, Cambridge: Cambridge University Press, 27-38.Applicant
Jakobson, R. 1941/68. Child Language, Aphasia and Phonological Universals. The Hague & Paris: Mouton, 68-81, 87-91.Applicant
Kaiser, Ed. 1997. “IPA, Worldbet, and OGlbet”, Course Manual : Spoken Language Systems Laboratory, Center for Spoken Language Understanding, Department of Computer Science & Engineering, Oregon Graduate Institute of Science & Technology (1 page).Applicant
Maddieson, I. 1984. “Patterns of Sounds”, Cambridge Studies in Speech Science and Communication, Cambridge: Cambridge University Press, 5-24.Applicant
Melnar, Lynette and Jim Talley. 2003. “Phone Merger Specification for Multilingual ASR: The Motorola Polyphone Network”, ICPhS, Barcelona ( pp. 1337-1340).Applicant
Ophala, John and Carrie Lang. 1996. “Temporal Cues for Vowels and Universals of Vowel Inventories”, ICSLP, Philadelphia (4 pages).Applicant
Rodman, M., B. Petek, T. Brøndsted. 2002. “SPE-Based Selection of Context-Dependent Units for Speech Recognition”, LREC 2002 WS on Portability Issues in Human Language Technologies, (pp. 78-84).Applicant
Stuker, S., F. Metze, T. Schultz, A. Waibel. 2003. “Integrating Multilingual Articulatory Features into Speech Recognition”, Eurospeech, Geneva (pp. 1033-1036).Applicant
Schultz, T., A. Waibel. 2000. “Polyphone Decision Tree Specialization for Language Adaptation”. ICASSP, Istanbul (4 pages).Applicant
Wells, John. 1995. “Computer-coding the IPA: a proposed extension of SAMPA”, London: Phonetics and Linguistics Dept., UCL (18 pages).Applicant