Method for Extracting Formant Frequencies

1. Field of the Invention

The present invention relates to a method for extracting formant frequencies from vowels or voiced consonants of a speech sound.

2. Description of the Prior Art

It is known that the formant frequency is one of the important characteristics of speech sounds such as vowels and voiced consonants. Specifically in the case of identifying a vowel, it is known that it is generally sufficient if a first formant frequency (F1) and a second formant frequency (F2) are known. In order to achieve the extraction of such two formant frequencies using an inexpensive system such as an 8-bit personal computer, a high speed formant extraction method is desired.

FIG. 7 is a processing flow chart in a prior art formant extraction method. In a voice signal input step 11, the voice is stored in a RAM through a microphone and an A/D converter. In a linear prediction coefficient calculation step 12, a p-th degree of linear prediction coefficients for one frame length, for example, of 20 ms is calculated. An inverse filter using linear prediction coefficients is given by ##EQU1## where a.sub.1, a.sub.2, . . . , a.sub.p are linear prediction coefficients. A root solving step 40 solves all roots of the filter of an all-zero type by the Newton-Raphson method. The frequency F and the bandwidth B corresponding to a root z.sub.i are obtained by

where f.sub.s is the sampling frequency. In a postprocessing step 41, from all the roots obtained in the root solving step 40, roots whose bandwidths B are less than a threshold value or which have continuity in frequency from and to the results in the preceding and following frames are selected, and the lowest frequency root is determined to be the first formant frequency and the next lowest is determined to be the second formant frequency.

In such a method, however, it takes a considerable length of time to obtain the roots, about which an explanation will be given with reference to FIG. 8. FIG. 8 is a flow chart of the operations of the root solving step 40. In step 70, a constant z.sub.0 is substituted for Z.sub.i as an initial value of a root candidate. In step 71, A(z.sub.i) and A'(z.sub.i), the linear differential of A(z.sub.i) are calculated. At step 72, a determination is made as to whether or not the absolute value of A(z.sub.i)/A'(z.sub.i), i.e., the difference between the values after renewal and before renewal of Z.sub.i is smaller than a threshold value. If the absolute value is not smaller than the threshold value, z.sub.i is renewed in step 73 and goes back to step 71. But, if the absolute value is smaller than the threshold value, z.sub.i is judged to have converged to a correct root value and is determined to be a root in step 74. Then, in step 75, A(z) is divided by a quadratic expression of z.sub.i and its conjugate complex number z.sub.i *, (z-z.sub.i).multidot.(z-z.sub.i *), whereby A(z) is renewed. In step 76, a determination is made as to whether or not A(z) has become zero-order, and if it not, the flow returns to step 70 where the calculation to substitute z.sub.0 for z.sub.i is performed again. If A(z) is zero-order, the formant frequency and bandwidth are obtained for all roots using the aforementioned equations (1) and (2) in step 77 and the calculation is ended.

In the above described method, since an approximate value of the root is not known at the start, all of the roots are obtained with the same initial value. Hence, the loop from step 76 to step 70 is traversed p/2 times. In order to keep high accuracy even when the desired root is therefore obtained at the later looping, each of the roots obtained must be of high accuracy. Therefore, the threshold value must be made small enough and as a result, the loop 72.fwdarw.73.fwdarw.71 has to be traversed many times. Thus, if such a high volume of calculations is to be performed by an 8-bit personal computer, there arises a difficulty in that the processing time becomes very great and therefore such method is impractical.

A primary object of the present invention is to provide a high speed formant extraction method.

To achieve the above mentioned object, the formant extraction method of the present invention comprises the steps of: calculating linear prediction coefficients of an input speech signal; extracting a coarse formant frequency by making a linear combination of feature parameters of the speech and multiple regression coefficients of the feature parameter and formant frequency obtained in advance; and solving a root of an inverse filter formed of the linear prediction coefficients by an approximation method in which the coarse formant frequency is set up as an initial value of root of the inverse filter and an approximation of root is recursively calculated until it converges to the root.

The present invention enables the processing time to be reduced by virtue by obtaining the formant frequency through the root solving method in which a calculated coarse formant frequency is set up as the initial value .

FIG. 1 is a flow chart of a formant extraction method according to an embodiment of the present invention;

FIG. 2 is a block diagram showing hardware used for performing formant extraction;

FIG. 3 is a block diagram showing hardware used for calculating formant estimation coefficients for use in formant extraction;

FIG. 4 is a flow chart of an example of the coarse formant frequency extraction method shown in FIG. 1;

FIG. 5 is a flow chart of another example of the coarse formant frequency extraction method shown in FIG. 1;

FIG. 6 is a flow chart of the root solving step shown in FIG. 1;

FIG. 7 is a flow chart of a prior art formant extraction method; and

FIG. 8 is a flow chart of the root solving step shown in FIG. 7.

FIG. 2 shows a block diagram of hardware used for performing formant extraction. As much hardware, a small computer such as a personal computer may be used. Referring to the figure, element 20 is a CPU; element 21 is a speech input microphone; element 22 is an A/D converter; element 23 is a CRT; element 24 is a video display processor; element 25 is a keyboard; element 26 is a ROM for storing programs; element 27 is a ROM for coefficients; element 28 is a RAM used as a storing work area, and element 29 is a RAM used for speech storage. The microphone 21 is for converting the speech of a speaker to an analog electrical signal. The A/D converter 22 is for converting the analog signal to a digital signal. The CRT 23 is for displaying images for interacting with the speaker. The video display processor (VDP) 24 is for converting data from the CPU 20 to an image signal. The keyboard 25 is for the input of instruction (for example, gender information) from the speaker. The ROM 26 is for storing therein programs for formant extraction. The ROM 27 is for storing various coefficients used in extracting formants. The RAM 28 is a memory for calculation or holding data temporarily. The RAM 29 is for holding input speech data.

FIG. 3 is a block diagram of the hardware used for calculating the coefficients to be stored in the ROM for coefficients 27 in FIG. 2. As such hardware, a larger-sized computer such as a minicomputer is used. Element 30 is a CPU; element 31 is a speech input microphone; element 32 is an A/D converter; element 33 is a CRT; element 34 denotes a video display processor; element 35 is a keyboard; element 36 is denotes a storage disk; element 37 is a RAM used as a main memory, and element 38 is a ROM writer 38. The CPU 30 is for executing control of the overall hardware. The microphone 31 is for converting speech of a speaker to an analog electrical signal. The A/D converter 32 is for converting the analog signal to a digital signal. The CRT 33 is for displaying images for interacting with the speaker. The video display processor (VDP) 34 is for converting data from the CPU 30 to an image signal. The keyboard 35 is for the input of instruction from the speaker. The disk 36 is a memory for storing various data as files. The RAM 37 is a memory for holding data temporarily. The ROM writer 38 is for writing various coefficients stored in the disk 36 to a ROM used as the ROM 27 in FIG. 2.

Now, the flow chart for the formant extraction method of the present invention as shown in FIG. 1 will be described. Step 10 is a gender information inputting step; element 11 is speech signal inputting step; element 12 is linear prediction coefficient calculating step; element 13 is coarse formant frequency extracting step, and step 14 is root solving step. In the speech signal inputting step 11, the CPU 20 inputs speech into the speech storage RAM 29 through the microphone 21 and A/D converter 22 in accordance with an instruction from the ROM 26. In the linear prediction coefficients calculating step 12, the CPU 20 takes out the speech signal from the RAM 29, processes the speech signal by preemphasis, window and auto-correlation calculations, obtains linear prediction coefficients by the Durbin method, and stores the result in the RAM 28. As the method for obtaining linear prediction coefficients using the Durbin method, a known method, for example, as described in L. R. Rabiner and R. W. Schafer: "Digital Processing of Speech Signals", Prentice-Hall, pp. 411-413, is used.

A description of an example of the coarse formant frequency extracting step 13 will be made referring to FIG. 4. Step 50 is a vowel discriminating step; step 51 is a formant estimation coefficient selecting step, and step 52 is a formant estimating step. In the vowel discriminating step 50, the linear prediction coefficients loaded in the RAM 28 are sorted into nine vowels (i:, i, e, , , a, :, u, u:) using vowel discriminant coefficients stored in the ROM 27 and the result is stored again in the RAM 28.

Here, the method for the discrimination of vowels will be explained. Each vowel is first sorted into either of two categories of the nine vowels and then distinguished as a specific vowel.

First, the method for obtaining discriminant coefficients between two categories of vowels used here will be described. After speech data of vowels given by many speakers are stored in the RAM 37 through the microphone 31 and the A/D converter 32, in the FIG. 3 hardware, the linear prediction coefficients thereof are calculated and the results are stored in the disk 36. The number of orders of the linear prediction coefficients here is assumed to be p. The two categories of vowels will here be called group I and group II. We express average vectors and covariance matrixes of the linear prediction coefficients of the group I and group II as ##EQU2## and set .SIGMA..sub.1 =.SIGMA..sub.2 =.SIGMA.. The discriminant function z of the sample a=(a.sub.1, . . . , a.sub.p) of the linear prediction coefficients z can be expressed as

where ##EQU3## When z.gtoreq.0, a is distinguished as a group I vowel, whereas when z<0, a is distinguished as a group II vowel. In deciding at this step whether an input set of linear prediction coefficients belongs to a vowel A or to the remaining eight vowels, if the same is positively decided to belong to the vowel A, the linear prediction coefficients are distinguished as the vowel A. In the case of vowels, the values of the linear prediction coefficients are largely different between male and female speakers. Therefore, data are separated into those for male speakers and those for female speakers, and separate discriminant coefficients are used according to input gender information. The thus obtained discriminant coefficients are written into a ROM, which is used as the ROM 27 in the FIG. 2 hardware, by the ROM writer 38.

In the formant estimation coefficient selecting step 51, the formant estimation coefficients corresponding to the vowel specified in the vowel discriminating step 50 are selected from the ROM 27 and output to the RAM 28.

The method for obtaining the formant estimation coefficients is as follows. Out of linear prediction coefficients for vowel data of many speakers stored in the disk 36, F1 and F2 are obtained. The formant frequencies are obtained by a conventional method. The linear prediction coefficients and F1 as well as F2 are stored in the RAM for the main memory. Representing a known formant frequency about some data of some vowel by F, estimated formant frequency by f, and linear prediction coefficients by (a.sub.1, a.sub.2, . . . , a.sub.p), we set

Then, d.sub.0, d.sub.1, d.sub.2, . . . , d.sub.p represent desired formant estimation coefficients. At this time the estimation error is the difference between F and f. By multiple regression analysis of a large amount of data belonging to the same vowel and made with the linear prediction coefficients taken as the predictor variables and with the estimated formant frequencies taken as the criterion variables, the formant estimation coefficients that will minimize the overall estimation error are obtained and stored in the disk 36. In like manner, the formant estimation coefficients are obtained for all vowels. It is preferable, specifically concerning vowel data, that male voices; and female voices; are separated and that different estimation coefficients are provided for male and female voices. The thus obtained formant estimation coefficients classified by vowels and by sexes are written into the ROM, which is used as the ROM 27, by the ROM writer 38.

In the formant estimating step 52, a coarse formant frequency is estimated by the linear combination of the formant estimation coefficients selected in the formant estimation coefficient selecting step 51 and the linear prediction coefficients. The estimated coarse formant frequency is stored in the RAM 28.

The following is a description of the root solving step 14 using the flow chart shown in FIG. 6. In step 80, as the initial value of Z.sub.i, exp {(-.pi.B+j2.pi.f)/f.sub.s } is provided, where f represents the coarse formant frequency, B represents a suitable constant, and f.sub.s represents the sampling frequency. In step 71, calculations of A(Z.sub.i) and A'(Z.sub.i) are executed. In step 81, a determination is made as to whether or not the absolute value of the difference of Z.sub.i after renewal and before renewal, A(Z.sub.i)/A'(Z.sub.i) is smaller than the threshold value. If the absolute value of the difference is not smaller than the threshold value, Z.sub.i is renewed in step 73 and the flow goes back to step 71. But, if the absolute value of the difference is smaller, Z.sub.i is judged to have converged to a right value of the root in step 82 and is considered to be as the expected root. In step 83, the formant frequency is obtained from this root by using the aforementioned equation (1).

At this time, obtaining the root only for one position is sufficient. Since, further, another root is not required to be obtained, the accuracy needs not be so high. Therefore, the number of times the converging loop (81.fwdarw.73.fwdarw.71) is traversed can be made smaller.

The method of convergence used here is known as the Newton-Raphson method. Even if another method of convergence is used, the calculation speed can of course be made higher by using a coarse formant frequency as the initial value.

So far an example has been shown in which only linear prediction coefficients are used as the feature parameters of speech.

The following is a description of a second example of a procedure for the coarse formant frequency extracting step 13 with reference to FIG. 5. Step 60 is a first feature parameter calculating step; step 61 is a vowel discriminating step; step 62 is a formant estimation coefficient selecting step; step 63 is a second feature parameter calculating step, and step 64 is a formant estimating step. The first feature parameters and the second feature parameters are the feature parameters indicating the form of the speech spectrum. The same can be any of linear prediction coefficients, LPC cepstrum coefficients, PARCOR coefficients, and Log Area Ratio coefficients. For particulars of these feature parameters, refer, for example, to L. Rabiner and R. W. Schafer: "Digital Processing of Speech Signals", Prentice-Hall, pp. 442-444. A band-pass filter bank output may also be used as the feature parameters.

In the first feature parameter calculating step 60, the first feature parameters are obtained from the speech data stored in the RAM 29 and stored in the RAM 28. In the vowel discriminating step 61, the first feature parameters stored in the RAM 28 are sorted into the nine vowels (i:, i, e, , , a, , u, u:) using vowel discrimination coefficients stored in the ROM 27 and the result is stored again in the RAM 28. The method for vowel discrimination is the same as in the above described case with the linear prediction coefficients.

In the formant estimation coefficient selecting step 62, the formant estimation coefficients corresponding to the vowel specified in the vowel discriminating step 61 are selected from the ROM 27 and stored in the RAM 28. The formant estimation coefficients used here are obtained from the second feature parameters in advance in the same way as were obtained from the linear prediction coefficients in the first embodiment.

In the second feature parameter calculating step 63, the second feature parameters are obtained from the speech data stored in the RAM 29 and restored in the RAM 28.

In the formant estimating step 64, the coarse formant frequency is estimated by the linear combination of the formant estimation coefficients selected in the formant estimation coefficient selecting step 62 and the second feature parameters, such as

where (b.sub.1, b.sub.2, . . . , b.sub.p) are the second feature parameters. The estimated coarse formant frequency is stored in the RAM 28.

So far, the cases where the formant frequencies of vowels are obtained have been described, but those of voiced consonants can be obtained by executing appropreate speech category decisions instead of the above mentioned vowel discrimination. The third or further formant frequencies can be obtained in the same way as described above.