Apparatus and method of speech coding and decoding using multiple frames

[0002] The present inventions relate to a communication system and more particularly to a speech compression method for a communication system.

[0003] Many speech compression systems are known. Generally, these systems may be divided into three types: time domain, frequency domain and hybrid codecs. However, in case of the low bit-rate coding, multi-band excitation (MBE) compression technique provides the best quality of the decoded speech.

[0004] The MBE vocoders encode the obtained speech signal by first dividing the input speech into constrained frames. These frames are transformed from the time domain to the frequency domain. Thereafter, a frequency spectrum of the framed and windowed signal is calculated, and an analysis of the frequency spectrum is performed. Speech model parameters such as a pitch value, a set of voiced/unvoiced decisions for the frequency bands, a set of spectral magnitudes and corresponding phase values are necessary for the speech synthesis in MBE vocoders. Usually, the phase values are not transmitted for low bit-rate coding.

[0005] There are numerous ways of spectrum approximation, all of which are based on an approximation of the frequency bands by some excitation function. The most traditional kind of an excitation function is a frequency response of the Hamming window. However, the Hamming window only obtains a good approximation of the original spectrum for stationary speech signals. For non-stationary speech signals, a predetermined kind of excitations function does not match well enough to the real shape of the spectrum for an accurate approximation. For example, a pitch frequency change during the analysis period may cause a widening of the peaks in the spectral magnitude envelope. Thus, the width of the peaks of the predetermined excitation function would no longer correspond to the width of the real peaks. Moreover, if the analyzed speech frame is a blend of two different processes, the spectrum would have a very complex shape, which is rather difficult to accurately approximate by means of a predetermined simple excitation function.

[0006] There are also many techniques for encoding the MBE parameters. Typically, a simple scalar quantization is used for encoding a pitch value and a band grouping method is used for encoding the voiced/unvoiced decisions. The most difficult task is the encoding of the spectral magnitudes, for which a Vector Quantization (VQ), a Linear Prediction and the like are used. Numerous high efficiency compression methods have been proposed based on VQ, one of which is a method of hierarchical structured codebook used for encoding spectral magnitudes.

[0007] Although the VQ technique allows an accurate quantizing in some problem area, it is generally effective for data close to those which has been included in the “learning sequences”. Other effective methods for encoding spectral magnitudes are intra-frame and inter-frame linear prediction. The intra-frame method allows for an adequate encoding of spectral magnitudes, but its effectiveness is substantially deteriorated at low bit-rate coding. The inter-frame prediction method is also fairly good, but its usage is reasonable only for stationary speech signals.

[0008] The speech synthesis in the related art is carried out according to an accepted speech model. Generally, the two components of the MBE vocoders, the voiced and unvoiced parts of speech, are synthesized separately and combined later to produce a complete speech signal.

[0009] The unvoiced component of the speech is generated for the frequency bands, which are determined to be unvoiced. For each speech frame, a block of random noise is windowed and transformed to the frequency domain, wherein the regions of the spectrum corresponding to the voiced harmonics are set to zero. The remaining spectral components corresponding to the unvoiced parts of speech are normalized to the unvoiced harmonic magnitudes.

[0010] A different technique is used for generating the voiced component of the speech in the MBE approach. Since the voiced speech is modeled by its individual harmonics in the frequency domain, it can be implemented at the decoder as a bank of tuned oscillators. An oscillator is defined by its amplitude, frequency and phase, and is assigned to each harmonic in the voiced regions of a frame.

[0011] However, the variations in the estimated parameters of the adjacent frames may cause discontinuities at the edges of the frames, resulting in a significant degradation of speech quality. Thus, during the synthesis, both the current and previous frames' parameters are interpolated to ensure a smooth transition at the frame boundaries, resulting in a continuous voiced speech at the frame boundaries.

[0012] Different implementations of interpolation schemes (for amplitude, frequency and phase) are possible. However, the interpolation schemes are generally only satisfactory under steady pitch. In case of sharp changing pitch, implementing processing rules do not lead to satisfactory results due to the traditional lacing of harmonics relating to the same number of frequency bands of the neighboring speech frames. In case of a pitch frequency change, a difference of frequencies of the laced harmonics appears and under conventional correspondence of harmonic bands, this difference is more significant for higher band numbers and for higher degree of pitch change. As a result, annoying artifacts in the decoded speech appear.

[0013] Accordingly, an object of the present invention is to solve at least the problems and disadvantages of the related art.

[0014] Another object of the present invention is to provide a method, which improves the quality of the speech spectrum approximation, for both voiced and unvoiced bands.

[0015] Another object of the present invention is to improve the encoding efficiency of the spectral magnitude set, regardless of the bit-rate for encoding.

[0016] A further object of the present invention is to improve the quality of speech synthesis.

[0017] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and advantages of the invention may be realized and attained as particularly pointed out in the appended claims.

[0018] To achieve the objects and in accordance with the purposes of the invention, as embodied and broadly described herein, the speech spectrum approximation is performed on the spectrum divided into plural bands according to the pitch frequency of the speech frame. The pitch frequency of the speech signal is determined, the frequency bands are built, and a voiced/unvoiced discrimination of the frequency bands is performed. Thereafter, an Analysis by Synthesis method of the speech spectrum approximation is used for calculating the magnitudes.

[0019] A more precise evaluation of the harmonic magnitudes at the encoder side results in an increase of quality for the voiced part of the signal reconstruction at the decoder side. Also, a more precise calculation of magnitudes for the unvoiced bands of spectrum results in a quality increase for the noise part of the reconstructed signal. The usage of the Analysis by Synthesis method both for the voiced and unvoiced bands provides a correct correspondence between the voiced and unvoiced parts of the reconstructed signal.

[0020] Also, the present invention improves the encoding efficiency of the spectral magnitudes set. In case of the low bit-rate encoding, the problem is to represent the spectral magnitudes data by a fixed number of bits. The present invention with respect to the spectral magnitudes encoding is divided into two main tasks: to reduce an original quantity of spectral magnitudes to the fixed number and to encode the reduced set. The present method solves the first task effectively by usage of Wavelet Transform (WT). Also, applying an inter-frame prediction effectively solves the second task, if the speech signal is stationary.

[0021] However, at time intervals containing non-stationary signals, no prediction is rather effective. Applying the Wavelet Transform technique effectively solves the encoding task in this case. The increase of encoding efficiency allows either an improved quality of reconstructed speech signal under the same bit-rate or a reduced bit-rate required for the same quality level.

[0022] Furthermore, the present invention improves the quality of speech synthesis. The speech synthesis is carried out sequentially for every frame. As a fundamental frequency is a base of the whole band division of the spectrum to be approximated, a difference of frequencies of the laced harmonics appears in case of the pitch change. The present invention uses a frequency correspondence between the laced bands of current and previous frames. This provides a correct and reliable speech synthesis process in conditions of the pitch frequency changes and the pitch frequency jumps. Even obvious troubles (errors) of pitch determination do not lead to dramatic consequences as in conventional schemes.

[0023] The invention will be described in detail with reference to the following drawings in which like reference numerals refer to like elements. wherein:

[0024] FIG. 1 is a block diagram of an encoder according to a preferred embodiment of the present invention;

[0025] FIG. 2 illustrates the Hamming window response scaling;

[0026] FIG. 3 illustrates a direct method of the speech model parameters determination according to the present invention;

[0027] FIG. 4 illustrates an Analysis by Synthesis method of the speech model parameters determination according to the present invention;

[0028] FIG. 5 is a block diagram of Analysis by Synthesis spectral magnitudes determination according to the present invention;

[0029] FIG. 6 is a block diagram hybrid encoding of the spectral magnitudes vector according to present invention;

[0030] FIG. 7 is a block diagram of decoder according to the present invention;

[0031] FIG. 8 is a block diagram of the voiced speech synthesis according to the present invention;

[0032] FIG. 9 an example of the band/frequency correspondence under the conventional voiced speech synthesis in a related art;

[0033] FIG. 10 illustrates frequency responses of some bands under the conventional voiced speech synthesis in a related art;

[0034] FIG. 11 is an example of the excitation spectrum under the conventional voiced speech synthesis in a related art;

[0035] FIG. 12 is an example of the band/frequency correspondence for the voiced speech synthesis scheme according to the present invention;

[0036] FIG. 13 illustrates frequency responses of some bands to the voiced speech synthesis scheme according to the present invention;

[0037] FIG. 14 is an example of the voiced excitation spectrum obtained by means of the voiced synthesis procedure according to the present invention; and

[0038] FIG. 15 is a block diagram of the unvoiced speech synthesis according to the present invention.

[0039] The preferred embodiments of the invention will be described in context of the MBE encoding scheme. The MBE vocoder has been disclosed by D. W. Griffin and J. S. Lim in “Multiband Excitation Vocoder,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, No. 8, August 1988, pp. 1223-35, fully incorporated herein. Typically, the vocoder operates with a speech signal sampling rate of 8 kHz. An application of the speech signal encoder and decoder according to the present invention will be discussed below.

[0040] FIG. 1 shows a preferred embodiment of an encoder according to the present invention. As shown, the encoder of the present invention includes a speech model parameter determination unit 1 and a parameter encoding unit 2. The speech model parameter determination unit 1 includes a Rectangular Windowing unit 10, a Hamming Windowing unit 20, a Frame Classification unit 30, a Fast Fourier Transform (FFT) unit 40, a Pitch Detection unit 3, a V/UV (Voiced/Unvoiced) Discrimination unit 80, and a Spectral Magnitudes Determination unit 90, all operatively coupled. These components are used for determining the MBE model parameters such as a pitch frequency fo, a set of voicing decisions V, a set of spectral magnitudes M, and phase values (due to a very low bit-rate, the phase values are not transmitted in the present embodiment).

[0041] The parameter encoding unit 2 includes a Scalar Quantization unit 100, a Spectral Magnitudes Wavelet Reduction unit 110, a Spectral Magnitudes Hybrid Encoding unit 120, and a Multiplexer unit 130. These components are used for encoding the MBE model parameters into a plurality of bits. Moreover, the Pitch Detection unit 3 includes a Pitch Candidates Set Determination unit 50, Best Candidate Selection unit 60, and a Best Candidate Refinement unit 70.

[0042] For the parameter determination, the speech signal is first divided into overlapping segments of 32 ms with an advance equal to 20-24 ms. At the Rectangular Windowing unit 10, the signal is multiplied by a rectangular window function W_Rfor frame classification performed by the Frame Classification unit 30. At the Hamming Windowing unit 20, the signal is also multiplied by the Hamming window function W_Hfor spectrum calculation performed by the FFT unit 40. To increase the frequency resolution, a series of zeroes are added to the processed frame before performing the FFT to produce a FFT_LENGTH array. A good frequency resolution may be achieved with the array FFT_LENGTH=2048, but for a real-time application, an array value of FFT_LENGTH=512 was used.

[0043] The Frame Classification unit 10 is an auxiliary unit relative to the MBE model in the related art. This unit processes the speech frame in the time domain and generates the frame classification characteristics T_fused for a more reliable and robust signal processing at the Pitch Candidates Set Determination unit 50 and at the V/UV Discrimination unit 80. The frames are classified in two ways, first by a range and character of varying signal value along the frame, and second by the characters of the signal oscillation inside the frame.

[0044] In the first classification, the types of the signal in the frames were defined as shown in TABLE 1 below. This classification way is based upon a simultaneous study of the signal sample values and character changes inside the frame. Also, the types of the signal oscillation inside the frame were defined as shown in TABLE 2 below. This second classification way is based upon values of zero crossing the first and second parts of a current frame.

[0045] As a result, 32 (i.e. 8×4) combined types of frames are derived and defined. A derived type of frame is specified from the combined types by means of logical operations. An example of the derived type of frame is as follows:

[0046] SOME_OF_VOWEL ‘AND’ ‘NOT’ SOME_OF_VIBRATION, where

[0047] SOME_OF_VOWEL=VOWEL

[0048] ‘OR’ VOWEL_FADING

[0049] ‘OR’ VOWEL_RISING

[0050] ‘OR’ PAST_VOWEL

[0051] ‘OR’ BEFORE_VOWEL, and

[0052] SOME_OF_VIBRATION=VIBRATION

[0053] ‘OR’ PAST_VIBRATION

[0054] ‘OR’ BEFORE_VIBRATION.

[0055] The obtained frame classification characteristics are used for the pitch detection and for the voicing discrimination. The operation of the Pitch Detection unit 3 utilizing the frame classification characteristics will next be discussed.

[0056] The problem of a reliable pitch frequency detection is paramount and is one of the most difficult tasks, especially for a real-time application. The pitch detection method of the present invention provides an effective and reliable solution based on the analysis in both time and frequency domains. The pitch frequency detection is performed in three stages. First, the Pitch Candidates Set Determination unit 50 determines the set of pitch candidates using an auto-correlation function (ACF) analysis in time domain. Second, the Best Candidate Selection unit 60 estimates all candidates in the frequency domain and a best candidate is selected. Thereafter, the Best Candidate Refinement unit 70 refines the best candidate value in the frequency domain.

[0057] A more reliable pitch detection can be obtained if the short-time center clipping is used before the calculation of the ACF. After the center clipping, the low-pass filtering of the processed frame is performed. Either direct or inverse order for the ACF calculation is used depending on the frame type determined by the Frame Classification unit 30. Although the same formula is used for both the direct and inverse orders for the ACF calculation, the direct order involves sample couples located in the beginning of the frame whereas the inverse order operates with sample couples located in the end of the frame. For example, the inverse order of ACF calculation is applied for frames of types VOWEL_RISING, BEFORE_VOWEL, etc. and the direct order of ACF calculation is used for frames of types VOWEL_FADING, PAST_VOWEL, etc.

[0058] The Pitch Candidates Set Determination unit 50 determines the pitch candidate set wherein the set of the candidates includes all the local maximums of the ACF located at the left side of the time lag corresponding to the global maximum ACF value. The set may include various numbers of the candidates for different frames. The range of the precise search in the frequency domain is defined by the frequency corresponding to the global maximum ACF value.

[0059] In the Best Candidate Selection unit 60, an estimation of every candidate from the obtained set is performed and the best candidate is selected. Particularly, the best value of the pitch frequency is found in a small vicinity of the pitch candidate value using the criterion of the minimum of the summarized square error (SSE) of the approximation with the weights. The approximation is performed in the frequency domain and the estimation of a quality approximation is performed as follows. According to the examined pitch frequency p, the whole frequency range is divided into n frequency bands with width p Hz. The speech spectrum in each band is approximated by the scaled Hamming window response and the SSE of the approximation is calculated by equation (1) below: $\begin{array}{cc}\mathrm{SSE}=\sum _{i=1}^n{\mathrm{SSE}}_i,& \left(1\right)\end{array}$

[0060] where $\mathrm{SSE}=\sum _{k=a_i}^{b_i}{Q}_i·{\left[S\left(k\right)-A_i·W\left(k-a_i\right)\right]}^2$

[0061] The amplitude value A_ifor the i-th band is calculated as follows $\begin{array}{cc}{A}_i=\frac{\sum _{k=a_i}^{b_i}S\left(k\right)·W\left(k-a_i\right)}{\sum _{k=a_i}^{b_i}{W}^2\left(k-a_i\right)},& \left(2\right)\end{array}$

[0062] where S is a speech spectrum, W is a scaled Hamming window response, a_iand b_iare numbers of harmonics corresponding to the beginning and the end of the: i-th band.

[0063] Traditionally, the Hamming window response with a constant width is used in the MBE scheme. However, extensive experiments show that using a fixed shape of the Hamming window response results in an unjustified odds for lower pitch frequencies (e.g. sub-harmonics of a true pitch value).

[0064] In the preferred embodiment, a special scale factor corresponding to the examined pitch value is used for scaling the Hamming window response as shown in FIG. 2. The scaling is performed for frequencies lower than a fixed frequency F_scale. The value of the frequency F_scale=140 Hz was determined experimentally. Particularly, the scaling is performed as follows. For a given FFT_LENGTH value of the total number of harmonics in the spectrum obtained by a FFT transform, the original Hamming window response has N_origcomponents deviating significantly from zero. For FFT_LENGTH=2048, Nong was accepted to be equal to 31. For low fundamental frequencies F_0exam<F_scale, a scaled Hamming window response used as an excitation function should have a sharper shape.

[0065] Therefore, the array of the response values should have N₀<N_origcomponents deviating significantly from zero. The number of these component N₀is calculated as follows N₀=int[N_orig(FFT_LENGTH/2048)·(F_0exam/F_scale)]. A procedure of the proportional sharpening based upon a linear interpolation is applied to the original Hamming window response in order to obtain a scaled response.

[0066] The present invention results in a better approximation of the frequency bands corresponding to low pitch frequency. The adequate sharpness of the approximating function in every band provides the true pitch candidate selection. For real-time application, all scaled Hamming window responses corresponding to different F_0exam<F_scalemay be tabled and may be used as a look-up table. On the other hand, to avoid giving unjustified odds at higher pitch frequencies (e.g. multiple harmonics of true pitch value), the first band is expanded under the SSE, calculation compared with other bands, such that a₁=1.

[0067] However, during the best candidate selection, the importance of the different parts of the spectra may be unequal. In consideration of this problem, the weight coefficients Q are introduced into the SSE calculation. A piecewise-linear weight Q was used as in equation (3) below $\begin{array}{cc}{Q}_k=\{\begin{array}{cc}1,& 0<k\le \mathrm{bf};\\ \left(\mathrm{ef}-k\right)/\left(\mathrm{ef}-\mathrm{bf}\right),& \mathrm{bf}<k\le \mathrm{ef};\\ 0,& \mathrm{otherwise};\end{array}& \left(3\right)\end{array}$

[0068] where bf is a harmonic number corresponding to the beginning of the weights' fading and ef is a harmonic number corresponding to end of the weights' fading and where (0≦bf<ef). The obtained value of the best pitch frequency is refined in the Best Candidate Refinement unit 70 by finding the best value of the pitch frequency within a small vicinity of the pitch candidate value using a minimum of the approximation without weights.

[0069] An important feature of the MBE approach is a generation of the voiced/unvoiced decisions for every frequency band of the original spectrum rather than for the whole frame. Also, the harmonic components may exist in a fricative frame and the vocal frame may contain some noise bands. The generation of voiced/unvoiced decisions for every frequency band of the original spectrum is carried out in the V/UV Discrimination unit 80. For a low bit-rate implementation of the MBE vocoder, the generation of decisions is performed on groups of the adjacent frequency bands. The adaptive band division (relatively to the bands' number value) is used in the preferred-embodiment.

[0070] The voicing discrimination process starts when a predetermined value of the pitch is obtained. For the discrimination, the original spectrum is divided into frequency bands according to the predetermined pitch value and every frequency band is approximated by the scaled frequency response of the Hamming window. Also, the frequency response scaling is performed for the same reason and by the same technique as was described in the Best Candidate Selection unit 60. The scaling provides a correct relation between the width of the frequency band and the approximating window. Moreover, it is very important to correctly adjust the position of the approximating window and the location of the frequency band peak.

[0071] The value of the Noise to Signal Ratio (NSR) of the approximation quality defines the voiced/unvoiced property of a frequency band group. The threshold of the NSR value depends on the classification characteristics of the current frame. For example, if the amplitude characteristic of a frame belongs to both the VOWEL types and but does not belong to any type of VIBRATION, the threshold is increased by a factor of 2, forcing the voiced decisions for evident voiced frames. However, for evident consonant frames when the amplitude characteristic of a frame does not belong to the VOWEL types but belongs to one of the VIBRATION types, the threshold is decreased by a divisor equal to 2, forcing the unvoiced decisions for evident consonant frames. If the classification is unclear, the threshold value is not changed and has a predefined value.

[0072] The estimation of the approximation quality incorporating the NSR is calculated by the following equation (4): $\begin{array}{cc}{\mathrm{NSR}}_i=\frac{\sum _{m=n_i}^{n_{i+1}-1}{\mathrm{Err}}_m}{\sum _{k=a_i}^{b_i}{\left(\left[S\left(k\right)\right]\right)}^2},& \left(4\right)\end{array}$

[0073] where NSR_iis a noise to signal ratio of i-th band group, including bands from n_ito n_i+1−1, wherein n_iis a band number of the first frequency band in the i-th group; Err_mis a summarized square error of the approximation for the m-th band; S(k) is a magnitude of the k-th harmonic of the approximated spectrum; a_iand b_iare harmonic numbers corresponding to the beginning and the end of i-th band group, wherein a_iis a harmonic number of the first harmonic in n_i-th band and b_iis a harmonic number of the last harmonic in (n_i+1−1)-th band.

[0074] A determination of Err_mis performed separately for every band included in the group. For the determination, a position tuning of the scaled Hamming window response relatively to the frequency band peak is performed for the voiced frames. This provides a correct voiced/unvoiced decision generation and is made by the following way. The Err_mvalue is calculated for various positions of the approximating window, relatively to the center of the frequency band. Thereafter, the position of the approximating window corresponding to the minimal Err_mvalue is selected and the best NSR_ivalue for the whole band group is obtained from the minimal Err_mvalues for every band included in the group. Thus, the voiced/unvoiced decision is generated by means of the NSR criterion as discussed above.

[0075] The determination of the spectral magnitudes using the AbS (Analysis by Synthesis approach will be next described. Generally, a purpose of analyzing and coding operations is to obtain at the sending side the data required for speech generating at the receiving side. According to the MBE model, a speech generation is performed with the speech model parameters including a pitch value, which induces a harmonic band system; a set of voiced/unvoiced decisions for the frequency bands; a set of spectral amplitudes; and corresponding phase values.

[0076] The speech model parameters may simply be calculated or explicitly and then output to an encoder as shown in FIG. 3. However, the Analysis by Synthesis implicitly defines all the speech model parameters or a part of the parameters before outputting the parameters to the encoder. Referring to FIG. 4, a conceptual scheme of the AbS approach to the parameter determination includes a Quality Evaluation component, a Synthesis component and a Search component. An identical Synthesis component is used for both the speech generation at the sending side and at the receiving side. The set P model parameters is searched, providing a synthesized signal {tilde over (s)} (t), which is closest to the real speech signal s{tilde over ( )}(t) depending upon certain criterion. The search of the optimal set P can be carried out as an iterative process wherein the value of vector P varies at every iteration and the value of an object function E=Ψ(M) is estimated. The optimal vector of the model parameters is subjected to encoding and is transmitted to the Synthesis component.

[0077] In one embodiment of the present invention, the spectral amplitudes are estimated using the AbS approach, based upon the pitch frequency value and the voiced/unvoiced decisions which were estimated by a direct calculation. The AbS approach for estimating the spectral amplitudes will be interpreted in terms defined above. At the sending side, a synthesis unit identical to the synthesis unit at the receiving side is used for the speech generation. Accordingly, the same rule for interpolation of the amplitude, phase and frequency from a previous frame to the current frame is used at both the sending and the receiving side.

[0078] Under a specified pitch value f₀and a set of voiced/unvoiced decisions V, the set of spectral amplitudes M is searched, providing a synthesized signal {tilde over (s)}(t) having a spectrum {tilde over (S)}, which is the closest to the real speech signal spectrum S. The criterion is a minimum SSE of the approximation spectrum S by the spectrum {tilde over (S)}. The search of the optimal spectral magnitudes can be carried out as an iterative process. At every iteration, the value of vector M varies, and the value of an object function SSE=Ψ(M) is estimated. The optimal values found are subjected to the encoding and transmission.

[0079] In the preferred embodiment of the present invention, one-iteration of the magnitude determination is proposed because this scheme is suitable for a real time implementation. The magnitude determination according to the present invention is based on the linearity of the Fourier Transform and the linearization of speech signal processing.

[0080] A model set of the spectral magnitudes is formed by means of assigning a fixed value M_m=e (in particular, these values are equal to unit values: M_m=1) to every magnitude to be determined. Under specified pitch values f_o^ph, F_o^cand the sets V^p, V^cof voicing decisions for the previous and current frames, the model speech signal {tilde over (s)}(t) is synthesized for the assigned etalon (unit) values of spectral amplitudes. The spectrum {tilde over (S)} of the synthesized signal is calculated and compared with the spectrum {tilde over (S)} of the real speech signal. Such comparison is separately performed in every band.

[0081] Similar to an analysis of a response of the unit disturbance in the linear system theory, a part of the spectrum {tilde over (S)}_mrelated to the m-th band is interpreted as a response of the linearized system under the action of the m-th spectral component of unit amplitude. A part of real spectrum S_mrelated to m-th band may be approximated as

[0082] S_m=μ{tilde over (S)}_m+E_m,  (5)

[0083] where E_mis an error of approximation. The value μ_mmay be found as the factor under which S_mis approximated in the best way. Thus, the values μm for all bands are calculated using the Least Square Method.

[0084] As a result, the approximation coefficients, which minimize the summarized square error of approximation of the spectrum S by the spectrum {tilde over (S)}, are determined. By virtue of the linearity (or quasi-linearity) property, these multiplicative coefficients may be treated as the values of spectral magnitudes (M_m=e·μ_mor M_m1·μ_m) for which the synthesized signal has the spectrum {tilde over (s)} being closest to the spectrum S of the real speech signal. These values μ_mare subjected to encoding and transmitted. At the receiving side, the values are used for assigning the spectral amplitude values for the synthesis of the output speech signal.

[0085] A detailed block diagram of the spectral magnitude determination unit 90 according to the present invention is shown in FIG. 5. The computation of the voiced and unvoiced magnitudes is separately performed. Particularly, the calculation of the voiced spectral magnitudes is performed by a Bands' Correspondence Map Building unit 91, a Voiced Synthesis unit 92, a Hamming Windowing unit 93, a FFT unit 94, and a Voiced Magnitudes Evaluation unit 95. The Bands' Correspondence Map Building unit 91 and the Voiced Synthesis unit 92, used for the production of the voiced excitation spectrum, are identical with a Bands' Correspondence Map Building unit 160 and a Voiced Synthesis unit 170 used for the voiced speech synthesis at the decoder side, as shown in FIG. 7. As the excitation signal is synthesized at the encoder side in the way it is generated at the decoder side, the frequency response is very suitable to use as an approximating function.

[0086] As discussed above, the input parameter set for the Voiced Synthesis unit 92 includes a pitch frequency f_o^cfor the current frame, a voicing decision vector V^cfor the current frame, a spectral magnitude vector M^cfor the current frame, and a bands' correspondence map built by the Bands' Correspondence Map Building unit 91. A detailed operation of the Bands' Correspondence Map Building unit 91 and the Voiced Synthesis unit 92 will be described later in reference to the decoder side (See description of the Bands' Correspondence Map Building unit 160 and the Voiced Synthesis unit 170, correspondingly). However, it is necessary to note that these units synthesize the output speech signal in the time domain under a given input parameter set for a current frame and a similar parameter set f_o^c, V^p, M^pfor the previous frame which is stored in a Previous Frame Parameters Accumulator unit built-in into the Voiced Synthesis unit 92.

[0087] For the production of the voiced excitation spectrum, the spectral amplitudes for the voiced bands are determined by assigning fixed values, which are equal to one. Assuming that the components of voicing decision vector are equal to 1 for the voiced bands and are equal to 0 otherwise, the assignment can be written as

[0088] M^c=V^c, M^p=V^p  (6)

[0089] The signal output by the Voiced Synthesis unit 92 is subjected to windowing by the Hamming Windowing unit 93 and to processing by the FFT unit 94. After the transformation, the output signal represents the voiced excitation spectrum S^v−e. An example of a voiced excitation spectrum obtained by the voiced synthesis procedure according to the present invention is shown in FIG. 14. The unvoiced part of the spectrum is nearly equal to zero while the voiced part of the spectrum has a regular structure. Even under the condition of changing pitch frequency and voicing decisions, the resulting spectrum would have similar properties, which are important for a correct spectrum approximation.

[0090] The voiced excitation spectrum obtained is used for a voiced magnitudes evaluation in the Voiced Magnitudes Evaluation unit 95. The Voiced Magnitudes Evaluation unit 95 performs a magnitudes estimation using the Least Square Method to approximate separately the voiced bands of real spectrum S by the excitation spectrum S^ve. The position of the excitation spectrum clip relatively to the frequency band is tuned for the voiced frames by shifting the spectrum on both sides relatively to the band center. Afterwards, the position of an excitation spectrum clip providing the best NSR of approximation is selected for the magnitude evaluation, which is carried out by the Least Square Method.

[0091] The obtained set of voiced magnitude values M^(v)is only a part of the M spectral magnitude vector. The set of unvoiced spectral magnitudes is the other part of the M spectral magnitude vector. A calculation of the unvoiced spectral magnitudes is performed by the Synchronized Noise Generation unit 96, the Hamming Windowing unit 97, the FFT unit 98, and the Unvoiced Magnitudes Evaluation unit 99 as shown in FIG. 5. The Synchronized Noise Generation unit 96 produces a white noise signal with an amplitude range of a unit. Similar to the process of obtaining the voiced magnitude values, the noise is processed in an identical manner at the encoder and the decoder side. Moreover, at the encoding side, a synchronizing property is provided which allows a better approximation of the unvoiced speech spectrum.

[0092] The signal obtained from the Synchronized Noise Generation unit 96 is windowed by the Hamming Windowing unit 97 and is processed by the FFT unit 98. In the Unvoiced Magnitudes Evaluation unit 99, the spectral magnitudes are calculated for every unvoiced band using the Least Square Method. The obtained set of unvoiced spectral magnitudes M^(uv)is combined with the set of the voiced magnitudes M^(v)to obtain the spectral magnitude vector M.

[0093] Referring back to FIG. 1, an encoding of the speech model parameters according to the present invention includes three parts. The encoding of the pitch frequency is performed by a Scalar Quantization unit 100. The pitch frequency value is restricted to the frequency range, for example f₀[50,400] and quantized into 256 levels (8 bits). The maximum error of the pitch frequency representation for this case is 0.684 Hz. The determined quantized value is passed to the Multiplexer unit 130. Also, the vector V of the group voiced/unvoiced decisions is simply passed to the Multiplexer unit 130.

[0094] The vector M of the spectral magnitudes is encoded in two stages. First, a reduction of the spectral magnitudes vector is performed by a Spectral Magnitudes Wavelet Reduction unit 110. Second, a hybrid encoding of the spectral magnitudes vector reduced is carried out by a Spectral Magnitudes Hybrid Encoding unit 120. The reduction of the spectral magnitudes vector according to the present invention will be described in details.

[0095] First, a logarithm of the elements of the vector M is taken as expressed in the following formula:

[0096] M_i=log₁₀M_i, 0<i<m

[0097] Here, m defines dimension of the vector M. The value m depends on the pitch frequency and varies in time. Afterwards, a transformation of the vector {tilde over (M)} of the dimension m to a vector {tilde over (M)}^rof a fixed dimension r is performed. Taking into consideration a further usage of WT for encoding of the vector {tilde over (M)}^r, the number r can be chosen such that r=l·2ⁿ, where l is a positive integer number, and n is a number of prospective steps of the above mentioned WT. In the preferred embodiment, the dimensionality of the vector {tilde over (M)} is reduced to a value r=16 with n=3 and l=2.

[0098] The reduction operation is irreversible, but the described implementation provides a reconstruction of vector {tilde over (M)} with high precision. If the dimension of vector {tilde over (M)} is equal to r, there is no need in the reduction operation. For other case, a procedure comprising the following steps is performed:

[0099] a cubic spline based on the elements of vector {tilde over (M)} is built;

[0100] a minimal number S is calculated such that s=r·2k≧m, k=0, 1, 2, . . . ;

[0101] a new uniform grid with S nodes is built and the values of cubic spline are calculated in these nodes;

[0102] k steps of Wavelet Transform are applied to the obtained set of s values;

[0103] the resultant r low-pass wavelet coefficients are elements of vector {tilde over (M)}^r, while the high-pass coefficients are discarded.

[0104] The number k of WT steps at this stage is not fixed and can differ for different signal frames.

[0105] Referring to FIG. 6, a hybrid encoding of the spectral magnitudes vector by the hybrid encoding unit 120 according to present invention will be described in details. The vector {tilde over (M)}^ris subjected to two encoding, namely a wavelet scheme in a Wavelet Encoder unit 121 and an inter-frame prediction scheme in an Inter-frame Prediction Encoder unit 122. During the two encoding processes, the effectiveness of each scheme is simultaneously estimated using a NSR criterion and the best scheme is selected as a base encoding for the vector {tilde over (M)}^rby a Comparator unit 123.

[0106] In the Wavelet Encoder unit 121, n steps of WT are applied to the vector {tilde over (M)}^r. Both l low-pass and r-l high-pass wavelet coefficients are subjected to quantization. A lattice quantization technique is used for encoding of the low-pass wavelet coefficients, while an adaptive scalar quantization is applied for the high-pass wavelet coefficients. A scalar quantizer symmetrical relatively to zero is built due to the nature of the high-pass wavelet coefficients. In the preferred embodiment, the number of WT steps n=3 and the number of the low-pass wavelet coefficients l=2. The biorthogonal (5,3) filters are used as the WT filters both at the reduction stage and at the encoding stage.

[0107] In the Inter-frame Prediction Encoder unit 122, an inter-frame prediction for encoding of the spectral magnitudes is used as a competing encoding scheme. The inter-frame prediction exploits a similarity of the spectral magnitudes in the neighbor frames and has a high effectiveness in the case of stationary signals. A prediction error is encoded using an adaptive scalar quantization.

[0108] Simultaneously with encoding, the decoding process takes place, which is necessary both for inter-frame prediction scheme operation and for quality estimation of test encoding schemes. The joint usage of competed encoding schemes such as Wavelet and Inter-Frame Prediction provides high effectiveness of the invented method. Thus, the Comparator unit 123 compares the effectiveness of the both schemes and dispatches a decision bit and data corresponding to the best scheme to the Multiplexer unit 130. The Multiplexer unit 130 combines the coded values of all parameters into an output plurality of bits and forms a bitstream.

[0109] FIG. 7 shows a block diagram of a decoder, which decodes the input bits and synthesizes a synthetic digital speech. The Demultiplexer unit 140 separates the input plurality of bits according to an accepted data structure. The Model Parameters Decoding unit 150 performs decoding of parameters, which determine the output speech. The Model Parameters Decoding unit 150 operates in an opposite manner to the model parameters encoding units (see the Scalar Quantization unit 100, the Spectral Magnitudes Wavelet Reduction unit 110, and the Spectral Magnitudes Hybrid Encoding unit 120).

[0110] A Bands' Correspondence Map Building unit 160 constructs the map, which forms the couples of the laced frequency bands by using the values of the pitch frequency for the current and previous frames. A voiced speech part is generated by a Voiced Synthesis unit 170 and the unvoiced speech part is generated by an Unvoiced Synthesis unit 180. A Summing unit 190 produces the synthetic digital speech by summing of the outputs the Voiced and Unvoiced Synthesis units 170 and 180.

[0111] The voiced part of a synthesized signal S^v(n) is produced as a sum of the appropriate harmonic components expressed by $\begin{array}{cc}{S}^v\left(n\right)=\sum _{m\in I^v}{S}_m^v\left(n\right),& \left(7\right)\end{array}$

[0112] where S_m^v(n), n=0, . . . , L−1 is a harmonic component signal corresponding to the m-th frequency band, L is a length of the non-overlapped part of the speech frame, and I^vis a set of frequency bands determined as the voiced bands.

[0113] Also, the harmonic component signal S_m^v(n) can be expressed as follows using the time index (sample number) n within the frame:

[0114] S_m^v(n)=A_m(n)COS(θ_m(n)), n=0, . . . , L−1,  (8)

[0115] where A_m(n) indicates the amplitude of the m-th harmonic interpolated between the beginning and the end of the frame, and θ_m(n) denotes the phase of the harmonic signal.

[0116] Although there are key problems in speech synthesis such as in the interpolation of the harmonic amplitudes, the interpolation of the harmonic angular frequencies, and providing continuity of the harmonic phases, one of the most critical problems may arise by the interaction of the inter-frame frequency bands. In vocoders similar to the MBE vocoders, the harmonic components of the current frame are laced with the harmonic components of the previous frame for the synthesis implementation. In the related art, the harmonics relating to the same frequency band number of the neighboring speech frames were laced.

[0117] In the preferred embodiment of the voiced synthesis according to the present invention, the harmonics relating to nearly the same frequencies are laced on the basis of a built map of the frequency bands correspondence. A detailed block diagram of the voiced speech synthesis 170 according to the present invention is shown in FIG. 8.

[0118] The input parameter set for voiced speech synthesis includes a pitch frequency f_o^c, a voicing decision vector V^c, and a spectral magnitude vector M^cfor the current frame, and a bands' correspondence map built by the Bands' Correspondence Map Building unit 160. A set of parameters f_o^p, V^p, M^pof the previous frame, which is stored in a Previous Frame Parameters Accumulator unit 171, is also used for the speech synthesis. A Lacing Controller unit 172 regulates the operation of a Phase Interpolator unit 173, an Angular Frequency Interpolator unit 174 and an Amplitude Interpolator unit 175 by choosing the approximation type depending on the voicing states of the laced bands. A Bank of Controlled Oscillators unit 176 provides the voiced speech synthesis using equation (7).

[0119] The significant distinction of the present invention lies in the presence of the Bands' Correspondence Map Building unit 160, which determines the way for the harmonic lacing. In the related art, the harmonics relating to the same frequency band number of the neighboring speech frames are laced. An example of a band/frequency correspondence under the harmonic synthesis in the related art is shown in FIG. 9. The pitch frequency of the previous frame is equal to 100 Hz while the pitch frequency of the current frame is equal to 83.7 Hz and the number of bands of the previous frame is equal to 39 while the number of bands of the current frame is equal to 47. As shown, a small pitch frequency change leads to a large frequency variation, especially for large harmonic numbers.

[0120] In FIG. 10, the frequency responses of the 7th, 18th and 33rd harmonic bands according to the related art are shown. These bands are voiced for both the current and previous frames. Under above-mentioned correspondence of harmonic bands, the frequency difference of the laced harmonics (for example, the 7th, 18th, 33rd bands in FIG. 10) causes a difference in amplitude and width frequency responses. This leads to an interaction of different frequency band responses and to a distorted shape of the excitation spectrum as shown in FIG. 11. Moreover, if the pitch jumps, annoying artifacts appear in the decoded speech.

[0121] An example of a band/frequency correspondence under the harmonic synthesis according to the present invention is shown in FIG. 12. The harmonic synthesis is performed on the base of direct and inverse maps, which give the correspondence between the frequency bands of the current and previous frames. As shown, the numbers of bands for correspondence may be different, but the bands' frequencies differ little both in the beginning and in the end of a frequency range (see Δf for the 33 band in FIG. 12).

[0122] The frequency responses for the 7th, 18th and 33rd harmonic bands according to the present invention are shown in FIG. 13 and as shown the harmonic bands have the same amplitude and width. The little hillocks near the main peaks correspond to the fading of the harmonics of the previous frame. The frequency response of the excitation signal is given in FIG. 14, which has a regular structure. It is important to note that the different bands are not overlapped and do not interact under the constructing of the excitation signal. This leads to a more correct and reliable evaluation of the amplitude without dramatic consequences due to a change in the pitch frequency.

[0123] Thus, couples of harmonics with the closest frequencies in the current and previous frames are selected and laced. The harmonics of previous frame which are not laced are smoothly decreased up to a zero amplitude and the harmonics of the current frame not laced are smoothly increased up to the determined amplitude.

[0124] The following notation will be used for the description of the present invention below. For the frequency bands m_cand m_pin the current and previous frames, m_c=φ(m_p) or m_p=φ¹(m_c). If a frequency band m_cin the current frame (or m_pin previous frame) is determined as voiced, m_cεI_c^vor m_pεI_p^v. If a frequency band m_cor m_pis determined as unvoiced, m_c∉I_c^vor m_c∉I_p^v. Let f₀^cbe a pitch frequency for the current frame (ω₀^c=2f₀^c); and let N_cbe a number of frequency bands (N_c=f_d/(2f₀^c), where f_dis a value of sampling frequency). Then, {M_mc}, m_c=0, . . . , N_c−1 is a set of magnitudes for every frequency band and I_c^vis a set of frequency bands which are determined as the voiced bands. Similarly for the previous frame, f₀^p; N_p; {M_mp}, m_p=0, . . . , N_p−1; and I_p^vare the pitch frequency, the number of frequency bands, the set of magnitudes and the set of voiced frequency bands.

[0125] The voiced speech synthesis is performed by the Bank of Controlled Oscillator unit 176 as shown in FIG. 8. The operation of the Bank of Controlled Oscillator unit 176 may be expressed by the following formulas. If the pitch frequency is increasing, f_o^c>f_o^p, i.e. N_c<N_p, the voiced part of the synthesized signal S(in) is calculated by summing along all the appropriate bank couples M=0, . . . , N_p−1 as follows: $\begin{array}{cc}{S}^v\left(n\right)=\sum _{m=0}^{N_p-1}{S}_m^v\left(n\right),& \left(9\right)\end{array}$

[0126] where m is a band couple number. The m-th band couple <m_p, m_c>consists of the m_pband and the m_cband, where m_p=m and m_c=φ(m_p). Here, φ(.) is a direct map which gives the correspondence between the frequency bands of the previous and current frames.

[0127] If the pitch frequency is decreasing, f_o^c<f_o^p, i.e. N_c>N_p, the voiced part of the synthesized signal S^v(n) is calculated by summing along all the appropriate band couples m=0, . . . , N_c−1 and is written as equation (10) below. $\begin{array}{cc}{S}^v\left(n\right)=\sum _{m=0}^{N_c-1}{S}_m^v\left(n\right),& \left(10\right)\end{array}$

[0128] The m-th band couple <m_p, m_c> consists of the m_pband and the m_cband, where m_c=m, m_p=φ⁻¹(m_c). The function φ⁻¹(.) is an inverse map, which gives the correspondence between the frequency bands of the current and previous frames.

[0129] If the pitch frequency is steady, f₀^c=f₀^p, i.e. N_c=N_p=N, the voiced part of the synthesized signal S^v(n) may be calculated without any map as follows: $\begin{array}{cc}{S}^v\left(n\right)=\sum _{m=0}^{N-1}{S}_m^v\left(n\right),& \left(11\right)\end{array}$

[0130] The Lacing Controller unit 172 regulates the operation of the Phase Interpolator unit 173, the Angular Frequency Interpolator unit 174, and the Amplitude Interpolator unit 175. There are three possible modes of interpolation depending on the voicing state of the laced bands. If the conditions m_cεI_c^vand m_pεI_p^vare satisfied for the m-th band couple <m_p, m_c>, a continuous harmonic is generated. The amplitude interpolation is carried out by the following formula: $\begin{array}{cc}{A}_m\left(n\right)=\{\begin{array}{cc}{M}_{m_p}+n·\left(M_{m_c}-M_{m_p}\right)/R,& \mathrm{if}n<R\\ M_{m_c},& \mathrm{otherwise}\end{array}& \left(12\right)\end{array}$

[0131] Here, M_mpand M_mcare magnitude values for the previous and current frames related to the m_pand m_cbands; n=0, . . . , L−1 is a sample number; L is the length of the non—overlapped part of the speech frame; and R is a length of a racing interval (0<R<L).

[0132] The interpolation of a phase and an angular frequency is carried out according to formulas:

[0133] θ_m(n)=nω_m+nΔω/2+φ_m(0),  (13)

[0134] where

[0135] ω_m=(m_p+1)·2π·f₀^p/f_d;

[0136] ω_m=(m_c+1)·2π·f₀^c/f_d;

[0137] Δω=(ω_m−ω_m)/L; and

[0138] where φ_mc(0) denotes the phase of the m_c-th harmonic at the beginning of the current frame which is equal to the phase of the corresponding harmonic at the end of the non-overlapped part of the previous frame, i.e. θ_mc(0)=0_mp(L).

[0139] If m_cεI_c^vand m_pεI_p^vfor the m-th band couple <m_p, m_c>, a fading harmonic is generated and the interpolation is carried out by equations (14), (15) below.

[0140] $\begin{array}{cc}{A}_m\left(n\right)=\{\begin{array}{cc}{M}_{m_p}-M_{m_p}·n/R,& \mathrm{if}n<R\\ 0,& \mathrm{otherwise}\end{array}& \left(14\right)\end{array}$  θ_m(n)=nω_m+θ_m(L)  (15)

[0141] If m_cεI_c^vand m_p∉Ip^vfor the m-th band couple <m_p, m_c>, a rising harmonic is generated and the interpolation of the harmonic amplitudes is carried out by equations (16), (17) below.

[0142] $\begin{array}{cc}{A}_m\left(n\right)=\{\begin{array}{cc}{M}_{m_c}·n/R,& \mathrm{if}n<R\\ M_{m_c},& \mathrm{otherwise}\end{array}& \left(16\right)\end{array}$  θ_m(n)=nω_m+φ_m(0),  (17)

[0143] where φ_mc(0) denotes the phase of the m_c-th harmonic at the beginning of the current frame which is equal to an initial phase value φ₀.

[0144] FIG. 15 shows a block diagram of the unvoiced speech synthesis 180 which according to the present invention includes a Synchronized Noise Generator unit 181 at the decoder side synchronized with the same unit 96 at the encoder side. Thus, the noise used for synthesis by the decoder is identical to the noise used for analysis by the encoder. A white noise signal waveform on the time axis, which was obtained from a white noise generator, is windowed by the Hamming Windowing unit 182. The result is processed by the FFT unit 183. The spectrum of the noise signal is multiplied by magnitudes Mm of the bands determined as unvoiced, whereas the amplitude of the voiced bands are set to zero.

[0145] The spectrum transformation is performed by the Noise Spectrum Transformation unit 184. The transformed spectrum is subjected to an inverse fast Fourier transform by an IFFT unit 185 using the phase values of the original noise signal. Afterwards, in an Add and Overlap unit 186, the obtained noise signal is overlapped with the noise signal of the previous frame stored by a Buffer 187 to produce an unvoiced speech part. In the Summing unit 190, the synthetic digital speech is produced by summing of the voiced and unvoiced speech parts.

[0146] The foregoing embodiments are merely exemplary and are not to be construed as limiting the present invention. The present teachings can be readily applied to other types of apparatuses. The description of the present invention is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art.