Speech Recognition Method

The present invention relates to a speech recognition method utilizing the pattern matching method.

In the speech recognition method according to the pattern matching method, the speech information is generally recognized by matching the information of a spectrum obtained from input speech with standard patterns. On the other hand, it has been attempted to positively adopt the information of speech power to improve the recognition accuracy. Recently, a satisfactory result has been reported on a speech recognition of speech patterns made by unspecified talkers (Aikawa, K, et. al.: An Isolated Word Recognition Method Using Power-Weighted Spectral Matching Measure; Transactions of the Committee on Speech Research, Acoust. Soc. Jpn., S81-59 (1981)).

A problem encountered when information of speech power is used to recognize a speech pattern is the difficulty in comparing speech power by use of the absolute values thereof. To solve this problem, it is proposed to normalize the speech power by using the maximum and minimum values of the speech power in the input speech interval, which is also utilized in the method discribed in the above-mentioned report. In this case, relevant processing cannot be initiated until the end of the speech interval because the maximum and minimum values of speech power are needed, that is, the processing cannot be started is principle at the same time when a speech pattern in inputted. This causes the output of the processed recognition results to be delayed, and furthermore, a buffer memory is necessary to store information to be outputted afterward; thus, the size and cost of the speech recognition equipment will be increased.

On the other hand, a pattern matching method according to the dynamic programming (to be abreviated as DP hereinafter) method has been proposed. Especially, a continuous DP matching method has been disclosed as a realtime matching method suitable for continuous speech. (Refer to the Japan Patent Laid-open No. 55-2205 for details.) This method has a feature that the results obtained by matching the input speech with the relevant standard pattern are continuously outputted. However, since the matching results reflect only the evaluation of the average degree of similarity between the input speech and the standard patterns, a problem that the error of recognition therebetween is increased in principle arises for input words including a similar portion. To overcome this difficulty, the inventors of the present invention have proposed a method in which each of the standard patterns is subdivided into a plurality of partial standard patterns and each of these partial standard patterns is compared independently. (See for example, Japan Patent Laid-open No. 58-58598 dated Apr. 7, 1983, only for reference). According to this method, if an input speech is matched with standard patterns and partial patterns thereof under a predetermined condition, the input speech is assumed to fall into the same category as the standard pattern. In this method, however, since each of the standard patterns is matched independently of the partial standard patterns thereof, the standard pattern memory and the load imposed on the matching block are increased.

An object of the present invention is to provide a speech recognition method which makes it possible to improve the recognition accuracy by use of the information of speech power without deteriorating the real time speech recognition operation.

Another object of the present invention is to provide a speech recognition method which makes it possible to implement a highly accurate recognition performance even for similar input speech patterns.

The present invention is characterized in that the degree of similarity is determined as a speech pattern is inputted by using information of the difference between speech power of the input speech, that is, whether the speech power is increasing or decreasing; thereby realizing real time speech recognition processing.

Furthermore, at least one pattern (partial standard pattern to be referred to simply a partial pattern) is specified beforehand for the pattern to be used as the standard (standard pattern) in the present invention. As a speech input takes place, the degree of matching between the input speech and the partial patterns of the standard pattern is simultaneously calculated in synchronism with the calculation of the degree of matching between the input speech and the standard pattern. These calculation results are comprehensively evaluated to reduce the error in recognizing similar speech inputs, thereby improving the speech recognition ratio.

FIG. 1 is a block diagram of an embodiment of a speech recognition system according to the present invention;

FIG. 2 illustrates a detailed block diagram of the similarity calculation block depicted in the block diagram of FIG. 1;

FIG. 3 shows a block diagram of another embodiment of the speech recognition system according to the present invention;

FIG. 4 depicts an example of the dynamic programming time alignment;

FIG. 5 is a graph illustrating a standard pattern and the matching with partial patterns thereof; and

FIGS. 6 and 7 illustrate the matching degree of an input speech with respect to the standard patterns and that of input speech with respect to the partial patterns, respectively.

The present invention will be explained in detail according to the following embodiments. FIG. 1 is a block diagram of an embodiment of a speech recognition system according to the present invention, In this embodiment, the calculation of the degree of similarity between an input speech and each standard pattern is initiated before the end of the input speech interval by use of information as to the difference in the speech power of the sequential input speech, that is, the information indicating whether the speech power is increasing or decreasing. Then the matching operation between the input speech and each standard pattern is performed on the basis of the obtained degree of similarity, thereby realizing realtime speech recognition processing.

In FIG. 1, the input speech is inputted from a microphone 1. In some cases, a telephone set or the like is utilized in place of a microphone. The feature of input speech is analyzed in the speech analysis block 2 to obtain a time series of feature parameters.

Various frequency analysis methods have been widely utilized to analyze speech features, for example, the analysis by band pass filters, zero crossing analysis, Fourier transformation, correlation analysis, Cepstrum analysis, analysis by synthesis, and linear predictive coding (LPC). Although various parameters are used in these frequency analysis methods, it should be appreciated that the types of parameters to be used for speech recognition are not essential to the present invention. For the explanation of the present invention, it is assumed that the input speech is analyzed according to the LPC method by using the autocorrelation coefficient and the normalized residual power of degree 10 as the measure of the similarity degree.

An autocorrelation coefficient of degree 10 is obtained as follows from the speech analysis block 2 for each analysis frame period (typically, 10 ms):

where t is an analysis frame number corresponding to time. A matching block 3 comprises a similarity calculation block 31 for calculating the degree of similarity with respect to the feature parameters between frames of the input speech and the standard pattern stored in a standard pattern memory 4 (to be explained by use of the normalized residual power as an example in this embodiment) and a matching process block 32. According to the present embodiment, each of a plurality of standard patterns e.g. ten patterns is represented by a time series of inverse spectrum parameters as follows:

where .tau. indicates the analysis frame number of a standard pattern. The following formula is used to calculate a normalized residual power .gamma. as the degree of similarity between the frames of the input speech and standard pattern: ##EQU1## where .gamma..sub.o is the residual power of input speech frame.

In the matching process block 32, the similarity matrix (normalized residual power matrix) obtained from a similarity calculation block 31 is referenced to perform a nonlinear time warping matching based on the dynamic programming (DP) method, thereby obtaining the matching degree. In a decision block 5, the matching degree with respect to each standard pattern is compared to determine the most satisfactory matching degree and a recognition result 6 is outputted.

FIG. 2 is a block diagram illustrating in detail the similarity calculation block 31 shown in FIG. 1. A feature parameter 311 outputted for each analysis frame period of input speech is inputted to a similarity calculation means 312 which has been commonly used in the conventional method, then an ordinary similarity degree (.gamma.) 314 is calculated from the formula (3) by using the inputted feature parameters 311 and the inverse spectrum parameters 313 obtained from the standard pattern memory 4. At the same time, an input speech power (V.sub.0t) 315 which is a portion of the feature parameters of the input speech is inputted to a comparator 316. Another input 317 to the comparator 316 is an input speech power (V.sub.0t-1) of the previous analysis frame, which can be easily produced from the input speech power (V.sub.0t) 315 by means of a shift register 318. An output 318' from the comparator 316 is information as to the difference in the input speech power (V.sub.0t) 315 and is inputted to a correction circuit 320 together with difference information 319 of the speech power of the standard patterns, the difference information 319 being calculated and stored in the standard pattern memory 4 in advance. In the correction circuit 320, each difference information between the input speech and standard pattern is referenced and the similarity degree (.gamma.) 314 is corrected to be a new similarity degree (.gamma.') 321.

There are various methods for correcting the similarity degree .gamma., for example, a multiplication type method and addition type method. A typical example of a multiplication type correction method will be explained. According to the multiplication type correction method, the new similarity degree .gamma.' is obtained from the following expression:

where .alpha. is a kind of weight to be determined by a combination of the speech power difference information of the input speech and the standard pattern. It will be assumed that a mark .sym. is indicated if the speech power is increasing for the pertinent frame and that a mark .crclbar. is indicated if it is decreasing. (A mark .phi. is indicated if the speech power does not change.) As an example, the weight .alpha. is determined according to a combination of the input speech and the standard pattern as depicted in Table 1.

As the value of normalized residual power decreases, the the degree of similarity becomes higher. (The minimum value is 1.0 which indicates the degree of similarity obtained by matching the same pattern.) Consequently, if a frame in which the input speech power is increasing (or decreasing) is combined with a frame in which the speech power of standard pattern is decreasing (or increasing), the normalized residual power is equivalent to the similarity degree .gamma. to which a penalty is added. That is, the spectrum information is weighted since the similarity degree .gamma. indicates the similarity between spectrum information.

Although various modifications are possible for the correction circuit 320, it is apparant that the correction circuit 320 can be implemented by use of simple circuits. A higher flexibility can be provided for the correction circuit 320 by adopting a general-purpose microcomputer or the like.

The control block for controlling various timing is not illustrated in FIG. 2, but it should be appreciated that the circuitry can be realized by use of simple circuits. Moreover, the equivalent functions can be apparently implemented by means of the program control based on a general-purpose microcomputer.

According to the present embodiment, it is not necessary to detect the maximum and minimum values of speech power to normalize the speech power, thus a spectrum matching distance weighted by the speech power can be obtained in real-time before the end of input speech is detected, which makes it possible to effectively utilize the speech power information having an invariant characteristic for various talkers, thereby contributing to the improvement of recognition ratio for recognizing speech made by unspecified talkers. In the experiments effected on ten numeric characters pronounced by 50 talkers by adopting the method according to the present invention, more than 30% of the erroneous recognition has been improved.

Next, an embodiment will be explained in which the recognition ratio is further improved by utilizing partial patterns in the processing for matching the input speech and the standard patterns. In this method, at least one partial pattern is beforehand specified for each standard pattern, and the matching degree with respect to the partial patterns is calculated simultaneously in synchronism with the calculation of matching degree with respect to the standard pattern. It is a characteristic of the present invention to improve the recognition ratio by reducing the recognition error for similar speech based on the conprehensive evaluation of the results obtained from the above-mentioned calculations.

FIG. 3 depicts a block diagram of another embodiment of a speech recognition system according to the present invention. In this configuration, output values of band pass filters are adopted as the feature parameters of speech analysis and the distance is utilized as the measure of the degree of similarity.

In FIG. 3, the input speech inputted from the microphone 1 is converted into a feature parameter in the speech analysis block 2. In an example in which the speech analysis block 2 comprises an n-channel filter bank, the number of filter output values obtained as follows for each analysis frame period (for example, 10 ms) is n:

where t indicates an analysis frame number corresponding to time.

The matching block 3 comprises a distance calculation block 310 and the matching process block 32. In the distance calculation block 310, the distances between the feature parameters of said input speech and the standard patterns are calculated. The standard patterns are stored in the standard pattern memory 4 in the form of a time series of feature parameter 41 as follows:

where, .tau. is the analysis frame number of a standard pattern. In this case, it is assumed to calculate the Euclidean distance as the measure of the degree of similarity. Then, the following distance matrix is obtained: ##EQU2## This indicates that the degree of similarity becomes higher as the value of distance d decreases. It is also naturally possible to weight the distance d according to the speech power like the first embodiment. Although the relevant processing is omitted in FIG. 3, the distance d in the following explanations is assumed to have been weighted according to the speech power. In the matching process block 32, the above-mentioned distance matrix is referenced to perform a nonlinear time warping matching by use of the DP method. In this case, information of the beginning and ending points 42 of partial patterns stored in the standard pattern memory is referenced to calculate the matching degree with respect to partial patterns in synchronism with the calculation of the degree of matching with respect to the standard patterns.

The matching degrees obtained with respect to the standard patterns and partial patterns are delivered to the decision block 5, which then outputs the recognition result 6.

An arithmetic processing in the matching process block 32 will be as follows in an example in which the time alignment is carried out based on the continuous DP method.

The distance D between the input speech and the standard pattern is represented as follows: ##EQU3## where min indicates that the minumum value is selected from (a), (b), and (c). A pass a, b, or c depicted in FIG. 4 is selected corresponding to the selected value (a), (b), or (c). Assuming that a register C is used for the difference between passes as compared with the case when the pass b is selected (that is, the time extension or shrinkage does not take place), C(t, .tau.) is expressed in accordance with (a),(b) and (c) of formula (5) as follows: ##EQU4##

If the length of the standard pattern is S frames, the length L(t) of the pass to the frame t of input speech is represented as:

The matching degree M in the frame t is expressed as follows:

Next, a calculation for obtaining the matching degree with respect to partial patterns will be described. Although one or more partial patterns can be specified for a standard pattern, only one partial pattern is used here to simplify the explanation in which the beginning and ending points are expressed as .tau..sub.1 and .tau..sub.2 (1.ltoreq..tau..sub.1 .ltoreq..tau..sub.2 .ltoreq.S). FIG. 5 shows the relationship between the standard pattern and the input speech. Letter l indicates a pass selected for the frame t of input speech and l.sub.p is a partial pass corresponding to the partial pattern (.tau..sub.1 .ltoreq..tau..ltoreq..tau..sub.2). The matching degree M.sub.p on l.sub.p is obtained for the frame t of input speech as follows.

Assuming that the cumulative value of distance on l.sub.p is D.sub.p and the number of distance values to be accumulated in N.sub.p : ##EQU5##

To obtain the matching degree M.sub.p of partial pattern for the frame t of input speech:

and frames t.sub.1 and t.sub.2 corresponding to the beginning and ending points .tau..sub.1 and .tau..sub.2 of partial pattern (see FIG. 5 for details), it is necessary only to perform calculations similar to formula (6) and to utilize the resultant information. (In FIG. 5, l.sub.0 indicates a pass to be selected when the extension or shrinkage of time base does not take place.) For the calculations of above-listed formulas (5)-(15), it is necessary only to hold information concerning two frames up to the second last frame and the real-time processing is possible by use of a shift register and a comparator. It need not be described that the real-time processing can be implemented by using a general-purpose microprocessor.

Next, a concrete example will be explained in which Japanese numeric characters are recognized according to the present embodiment. FIG. 6 depicts the matching degrees M.sub.1 (t) and M.sub.p1 (t) of the portion of /hachi/ in a continuous speech with respect to a standard pattern /ichi/ and its partial pattern (/i/ in the first syllable is specified). FIG. 7 shows the matching degree M.sub.8 (t) and M.sub.ps (t) of the same input speech with respect to the standard pattern /hachi/ and its partial pattern (/ha/ in the first syllable is specified). When the minimal values of the matching degrees M.sub.1 (t) and M.sub.8 (t) with respect to the respective standard patterns are compared, it is found that the minimal value of M.sub.1 (t) with respect to the standard pattern /ichi/ is smaller and the input speech is erroneously recognized to be /ichi/. This is a phenomenon which occurs because the portion of /chi/ in the input speech is quite similar to the portion of /chi/ in the standard pattern /ichi/.

The recognition performance can be stabilized by utilizing the matching degrees M.sub.p1 (t) and M.sub.p8 (t) with respect to the partial patterns in the decision block. Although there are various decision logics, one of the simplest example will be explained in which the matching degree with respect to the overall pattern is compared with the sum of the matching degrees with respect to the partial patterns. Subscripts 1 and 8 indicate the standard patterns ichi and hachi, respectively hereinafter. Assume that the matching degree M.sub.1 (t) with respect to the standard pattern /ichi/ is minimized at time t.sub.1 in FIG. 6 and that the matching degree M.sub.8 (t) with respect to the standard pattern /hachi/ is minimized at time t.sub.8 in FIG. 7. Since the matching degrees with respect to the standard patterns and partial patterns there are as follows in this example: ##EQU6## the following relationship is obtained.

Consequently, the input speech is correctly recognized to be /hachi/.

In the example described above, the present invention has been explained in conjunction with a case in which Japanese numeric characters are recognized as an example of similar input speech; however, it should be appreciated that the present invention is not limited to numeric character speech recognition and that the present invention is also effective to recognize the speech made in other languages, such as English and French.

According to the present invention, the degree of similarity of a specified portion can be simultaneously evaluated together with the average degree of similarity with respect to a standard pattern, and the recognition performance can be improved without deteriorating the real-time operability of speech recognition.