Method and Apparatus for Speech Analysis and Synthesis by Sampling a Power Spectrum of Input Speech

1. Field of the Invention

The present invention relates to a speech analyzing and synthesizing method, for analyzing speech into parameters and synthesizing speech again from the parameters.

2. Related Background Art

As a method for speech analysis and synthesis, there is already known the mel cepstrum method.

In this method, speech analysis for obtaining spectrum envelope information is conducted by determining a spectrum envelope by the improved cepstrum method, and converting it into cepstrum coefficients on a non-linear frequency scale similar to the mel scale. Speech synthesis is conducted using a mel logarithmic spectrum approximation (MLSA) filter as the synthesizing filter, the speech is synthesized by entering the cepstrum coefficients, obtained by the speech analysis, as the filter coefficients.

The Power spectrum envelope method is also known in this field (PSE).

In the speech analysis using this method, the spectrum envelope is determined by sampling a power spectrum, obtained from the speech wave by FFT, at positions of multiples of a basic frequency, and smoothy connecting the obtained sample points with consine polynomials. Speech synthesis in conducted by determining zero-phase impulse response waves from thus obtained spectrum envelope and superposing the waves at the basic period (reciprocal of the basic frequency).

Such conventional methods, however, have been associated with following drawbacks.

(1) In the mel cepstrum method, at the determination of the spectrum envelope by the improved cepstrum method, the spectrum envelope tends to vibrate depending on the relation between the order of the cepstrum coefficient and the basic frequency of the speech. Consequently, the order of the cepstrum coefficient has to be regulated according to the basic frequency of the speech. Also this method is unable to follow a rapid change in the spectrum, if it has a wide dynamic range between the peak and the zero level. For these reasons, speech analysis in the mel cepstrum method is unsuitable for precise determination of the spectrum envelope, and gives rise to a deterioration in the tone quality. On the other hand, speech analysis in the PSE method is not associated with such drawback, since the spectrum is sampled with the basic frequency and the envelope is determined by an approximating curve (cosine polynomials) passing through the sample points.

(2) However, in the PSE method, speech synthesis by the superposition of zero-phase impulse response waves requires a buffer memory for storing the synthesized wave, in order to superpose the impulse response waves symmetrically to a time zero. Also, since the superposition of impulse response waves takes place in the synthesis of a voiceless speech period, a cycle period of superposition inevitably exists in the synthesized sound of such voiceless speech period. Thus the resulting spectrum is not a continuous spectrum, such as that of white noise, but becomes a line spectrum having energy only at multiples of the superposing frequency. Such a property is quite different from that of actual speech. For these reasons speech synthesis using the PSE method is unsuitable for real-time processing, and the characteristics of the synthesized speech are not satisfactory. On the other hand, the speech synthesis in the mel cepstrum method is easily capable of real-time processing for example with a DSP because of the use of a filter (MLSA filter), and can also prevent the drawback in the PSE method, by changing the sound source between a voiced speech period and an unvoiced speech period, employing white noise as the source for the unvoiced speech period.

In consideration of the foregoing, the object of the present invention is to provide an improved method of speech analysis and synthesis, which is not associated with the drawbacks of the conventional methods.

According to the present invention, the spectrum envelope is determined by obtaining a short-period power spectrum by FFT on speech wave data of a short period, sampling the short-period power spectrum at the positions corresponding to multiples of a basic frequency, and applying a cosine polynomial model to the thus obtained sample points. The synthesized speech is obtained by calculating the mel cepstrum coefficients from the spectrum envelope, and using the mel cepstrum coefficients as the filter coefficients for the synthesizing (MLSA) filter. Such a method allows one to obtain high-quality synthesized speech in a more practical manner.

FIG. 1 is a block diagram of an embodiment of the present invention;

FIG. 2 is a block diagram of an analysis unit shown in FIG. 1;

FIG. 3 is a block diagram of a parameter conversion unit shown in FIG. 1;

FIG. 4 is a block diagram of a synthesis unit shown in FIG. 1;

FIG. 5 is a block diagram of another embodiment of the parameter conversion unit shown in FIG. 1; and

FIG. 6 is a block diagram of another embodiment of the present invention.

[An embodiment utilizing frequency axis conversion in the determination of mel cepstrum coefficients]

FIG. 1 is a block diagram best representing the features of the present invention, wherein shown are an analysis unit 1 for generating logarithmic spectrum envelope data by analyzing a short-period speech wave (a unit time being called a frame), determining whether the speech is voiced or unvoiced, and extracting the pitch (basic frequency); a parameter conversion unit 2 for converting the envelope data, generated in the analysis unit 1, into mel cepstrum coefficients; and a synthesis unit 3 for generating a synthesized speech wave from the mel cepstrum coefficients obtained in the parameter conversion unit 2 and the voiced/unvoiced information and the pitch information obtained in the analysis unit 1.

FIG. 2 shows the structure of the analysis unit 1 shown in FIG. 1 and includes: a voiced/unvoiced decision unit 4 for determining whether the input speech of a frame is voiced or unvoiced; a pitch extraction unit 5 for extracting the pitch (basic frequency) of the input frame; a power spectrum extraction unit 6 for determining the power spectrum of the input speech of a frame; a sampling unit 7 for sampling the power spectrum, obtained in the power spectrum extraction unit 6, with a pitch obtained in the pitch extraction unit; a parameter estimation unit 8 for determining coefficients by applying a cosine polynomial model to a train of sample points obtained in the sampling unit 7; and a spectrum envelope generation unit 9 for determining the logarithmic spectrum envelope from the coefficients obtained in the parameter estimation unit 8.

FIG. 3 shows the structure of the parameter conversion unit shown in FIG. 1. There are provided a mel approximation scale forming unit 10 for forming an approximate frequency scale for converting the frequency axis into the mel scale; a frequency axis conversion unit 11 for converting the frequency axis into the mel approximation scale; and a mel cepstrum conversion unit 12 for generating cepstrum coefficients from the logarithmic spectrum envelope.

FIG. 4 shows the structure of the synthesis unit shown in FIG. 1. There are provided a pulse sound source generator 13 for forming a sound source for a voiced speech period; a noise sound source generator 14 for forming a sound source for an unvoiced speech period; a sound source switching unit 15 for selecting the sound source according to the voiced/unvoiced information from the voiced/unvoiced decision unit 4; and a synthesizing filter unit 16 for forming a synthesized speech wave from the mel cepstrum coefficients and the sound source.

The function of the present embodiment will be explained in the following.

In the following explanation there are assumed the following speech data:

sampling frequency: 12 kHz

frame length: 21.33 msec (256 data points)

frame cycle period: 10 msec (120 data points)

At first, when speech data of a frame length are supplied to the analysis unit 1, the voiced/unvoiced decision unit 4 determines whether the input frame is a voiced speech period or an unvoiced speech period.

The power spectrum extraction unit 5 executes a window process (a Blackman window or a Hunning window, for example) on the input data of a frame length, and determines the logarithmic power spectrum by an FTT process. The number of points in the FTT process should be selected to be a relatively large value (for example 2048 points) since the resolving power of the frequency should be selected fine for determining the pitch in the ensuing process.

If the input frame is a voiced speech period, the pitch extraction unit 6 extracts the pitch. This can be done, for example, by determining the cepstrum by an inverse FFT process of the logarithmic power spectrum obtained in the power spectrum extraction unit 5 and defining the pitch (basic frequency: fo(Hz)) by the reciprocal of a cefrency (sec) giving a maximum value of the cepstrum. As the pitch does not exist in an unvoiced speech period, the pitch is defined as a sufficiently low constant value (for example 100 Hz).

Then the sampling unit 7 executes sampling of the logarithmic power spectrum, obtained in the power spectrum extraction unit 5, with the pitch interval (positions corresponding to multiples of the pitch) determined in the pitch extraction unit 6, thereby obtaining a train of sample points.

The frequency band for determining the train of sample points is advantageously in a range of 0-5 kHz in case of a sampling frequency of 12 kHz, but is not necessarily limited to such a range. However it should not exceed 1/2 of the sampling frequency, based on the rule of sampling. If a frequency band of 5 kHz is needed, the upper frequency F (Hz) of the model and the number N of sample points can be defined by the minimum value of fo.times.(N-1) exceeding 5000.

Then the parameter estimation unit 8 determines, from the sample point train y.sub.i (i=0, 1, . . . N-1) obtained in the sampling unit, coefficients Ai (i=0, 1, . . . , N-1) of cosine polynomial of N terms: ##EQU1## However the value y.sub.0, which is the value of logarithmic power spectrum at zero frequency, is approximated by y.sub.1, because the value at zero frequency in FFT is not exact. The value Ai can be obtained by minimizing the sum of square of the error between the sample points y.sub.i and Y(.lambda.): ##EQU2## More specifically the values are obtained by solving N simultaneous first-order equations obtained by partially differentiating J with A.sub.0, A.sub.1, . . . , A.sub.N-1 and placing the results equal to zero.

Then the spectrum envelope generation unit 9 determines the logarithmic spectrum envelope data from A.sub.0, A.sub.1, . . . , A.sub.N-1 obtained in the parameter estimation unit, according to an equation:

The foregoing explains the generation of the voiced/unvoiced information, pitch information and logarithmic spectrum envelope data in the analysis unit 1.

Then the parameter conversion unit 2 converts the spectrum envelope data into mel cepstrum coefficients.

At first the mel approximation scale forming unit 10 forms a non-linear frequency scale approximating the mel frequency scale. The mel scale is a psychophysical quantity representing the frequency resolving power of hearing ability, and is approximated by the phase characteristic of a first-order all-passing filter. For the transmission characteristic of the filter: ##EQU3## the frequency characteristics are given by: ##EQU4## wherein .OMEGA.=.omega..DELTA.t, .DELTA.t is the unit delay time of the digital filter, and .omega. is the angular frequency. It is already known that a non-linear frequency scale .OMEGA.=.beta.(.OMEGA.) coincides well with the mel scale by selecting the value .alpha. in the transmission function H(z) arbitrarily in a range from 0.35 (for a sampling frequency of 10 kHz) to 0.46 (for a sampling frequency of 12 kHz).

Then the frequency axis conversion unit 11 converts the frequency axis of the logarithmic spectrum envelope determined in the analysis unit 1 into the mel scale formed in the mel approximation scale forming unit 10, thereby obtaining mel logarithmic spectrum envelope. The ordinary logarithmic spectrum G.sub.1 (.OMEGA.) on the linear frequency scale is converted into the mel logarithmic spectrum G.sub.m (.OMEGA.) according to the following equations: ##EQU5##

The cepstrum conversion unit 12 determines the mel cepstrum coefficients by an inverse FFT operation on the mel logarithmic spectrum envelope data obtained in the frequency axis conversion unit 11. The number of orders can be theoretically increased to 1/2 of the number of points in the FFT process, but is in a range of 15-20 in practice.

The synthesis unit 3 generates the synthesized speech wave, from the voiced/unvoiced information, pitch information and mel cepstrum coefficients.

At first, sound source data are prepared in the noise sound source generator 13 or the pulse sound source generator 14 according to the voiced/unvoiced information. If the input frame is a voiced speech period, the pulse sound source generator 14 generates pulse waves of an interval of the aforementioned pitch as the sound source. The amplitude of the pulse is controlled by the first-order term of the mel cepstrum coefficients, representing the power (loudness) of the speech. If the input frame is an unvoiced speech period, the noise sound source generator 13 generates M-series white noise as the sound source.

The sound source switching unit 15 supplies, according to the voiced/unvoiced information, the synthesizing filter unit either with the pulse train generated by the pulse sound source generator 14 during a voiced speech period, or the M-series white noise generated by the noise sound source generator 13 during an unvoiced speech period.

The synthesizing filter unit 16 synthesizes the speech wave, from the sound source supplied from the sound source switching unit 15 and the mel cepstrum coefficients supplied from the parameter conversion unit 2, utilizing the mel logarithmic spectrum approximation (MLSA) filter.

[Embodiment utilizing equation in determining mel cepstrum coefficients]

The present invention is not limited to the foregoing embodiment but is subject to various modifications. As an example, the parameter conversion unit 2 may be constructed as shown in FIG. 5, instead of the structure shown in FIG. 3.

In FIG. 5, there are provided a cepstrum conversion unit 17 for determining the cepstrum coefficients from the spectrum envelope data; and a mel cepstrum conversion unit for converting the cepstrum coefficients into the mel cepstrum coefficients. The function of the above-mentioned structure is as follows.

The cepstrum conversion unit 17 determines the cepstrum coefficients by applying an inverse FFT process on the logarithmic spectrum envelope data prepared in the analysis unit 1.

Then the mel cepstrum conversion unit 18 converts the cepstrum coefficients C(m) into the mel cepstrum coefficients C.sub..alpha. (m) according to the following regression equations: ##EQU6## [Apparatus for ruled speech synthesis]

Although the foregoing description has been limited to an apparatus for speech analysis and synthesis, the method of the present invention is not limited to such an embodiment and is applicable also to an apparatus for ruled speech synthesis, as shown by an embodiment in FIG. 6.

In FIG. 6 there are shown a unit 19 for generating unit speech data (for example monosyllable data) for ruled speech synthesis; an analysis unit 20, similar to the analysis unit 1 in FIG. 1, for obtaining the logarithmic spectrum envelope data from the speech wave; a parameter conversion unit 21, similar to the unit 2 in FIG. 1, for forming the mel cepstrum coefficients from the logarithmic spectrum envelope data; a memory 22 for storing the mel cepstrum coefficient corresponding to each unit speech data; a ruled synthesis unit 23 for generating a synthesized speech from the data of a line of arbitrary characters; a character line analysis unit 24 for analyzing the entered line of characters; a rule unit 25 for generating the parameter connecting rule, pitch information and voiced/unvoiced information, based on the result of analysis in the character line analysis unit 24; a parameter connection unit 26 for connecting the mel cepstrum coefficients stored in the memory 22 according to the parameter connecting rule of the rule unit 25, thereby forming a time-sequential line of mel cepstrum coefficients; and a synthesis unit 27, similar to the unit 3 shown in FIG. 1, for generating a synthesized speech, from the time-sequential line of mel cepstrum coefficients, pitch information and voiced/unvoiced information.

The function of the present embodiment will be explained in the following, with reference to FIG. 6.

At first the unit speech data generating unit 19 prepares data necessary for the speech synthesis by a rule. More specifically the speech constituting the unit of ruled synthesis (for example speech of a syllable) is analyzed (analysis unit 20), and a corresponding mel cepstrum coefficient is determined (parameter conversion unit 21) and stored in the memory unit 22.

Then the ruled synthesis unit 23 generates synthesized speech from the data of an arbitrary line of characters. The data of input character line are analyzed in the character line analysis unit 24 and are decomposed into information of a single syllable. The rule unit 25 prepares, based on the information, the parameter connecting rules, pitch information and voiced/unvoiced information. The parameter connecting unit 26 connects necessary data (mel cepstrum coefficients) stored in the memory 22, according to the parameter connecting rules, thereby forming a time-sequential line of mel cepstrum coefficients. Then the synthesis unit 27 generates rule-synthesized speech, from the pitch information, voiced/unvoiced information and time-sequential data of mel cepstrum coefficients.

The foregoing two embodiments utilize the mel cepstrum coefficients as the parameters, but the obtained parameters become equivalent to the cepstrum coefficients by providing the condition .alpha.=0 in the equations (4), (6), (9) and (10). This is easily achievable by deleting the mel approximation scale forming unit 10 and the frequency axis conversion unit 11 in case of FIG. 3 or deleting the mel cepstrum conversion unit 18 in case of FIG. 5, and replacing the synthesizing filter unit 16 in FIG. 4 with a logarithmic magnitude approximation (LMA) filter.

As explained in the foregoing, the present invention provides an advantage of obtaining a synthesized speech of higher quality, by sampling the logarithmic power spectrum determined from the speech wave with a basic frequency, applying a cosine polynomial model to thus obtained sample points to determine the spectrum envelope, calculating the mel cepstrum coefficients from said spectrum envelope, and effecting speech synthesis with the LMSA filter utilizing said mel cepstrum coefficients.