Split Matrix Quantization

1. Field of the Invention

The present invention relates to a speech synthesis quantization system.

2. Related Art

Speech coding systems have a wide range of potential applications, including telephony, mobile radio and speech storage. The primary objective of speech coding is to enable speech to be represented in digital form such that intelligible speech of acceptable quality can be generated from the representation, but it is very important to minimise the number of bits required by the representation so as to maximise system capacity.

In an efficient digital speech communication system, an input acoustic signal is converted to an electrical signal, and the electrical signal is converted into computed sequences of numeric measurements which effectively define the parameters of an "excitation source--vocal tract" speech synthesis model. Parameters which define the vocal tract part of the model determine an "envelope" component of the speech short-term magnitude spectrum, which in turn can be estimated using the Discrete Fourier Transform (DFT) or using Linear Predictive Coding (LPC) techniques. The vocal tract parameters of the system are extracted periodically from successive speech frames, the parameters are quantized, and the quantized parameters are transmitted, together with excitation source parameters, to a receiver for the subsequent reconstruction (synthesis) of the required speech signal. The present invention is concerned with the efficient quantization of vocal tract parameters.

There is a requirement for high speech quality coding systems which are capable of operating in the region of for example 1.2 to 3.2 kbits/sec. In this context of low bit rate coding, the efficient quantization of coefficients is important in order to maximise the number of bits which can be allocated to other components of the transmitted signals.

Scalar quantization of LPC filter coefficients typically requires 38 to 40 bits per analysis frame if the quantization process is to be "transparent", which term refers to the case where, despite noise being introduced by quantizing the LPC coefficients, no audible distortion can be detected in the output speech signal. It is known to exploit interframe correlation using differential coding and frequency delayed coding techniques to reduce the bit requirements to about 30 bits per frame. Still lower bit rates can be achieved using known vector quantization (VQ) techniques. Split-VQ or single stage VQ offer acceptable performance with realistic storage and codebook search characteristics at 24 and 20 bits per frame respectively. Further compression can be obtained in principle by exploiting interframe correlation between sets of LPC coefficients. Adaptive codebook VQ systems have been proposed and combined in certain cases with differential coding and fixed codebooks, and switched adaptive interframe vector prediction can be employed which offers high LPC coefficient quantization performance at 19 to 21 bits per frame.

Whereas the above schemes attempt to reduce interframe correlation in a backwards manner using past information, it is known to use matrix quantization to allow the introduction of delay into the process and simultaneous operation on sets of filter coefficients obtained from successive frames using VQ principals. Matrix quantization has been applied to coding systems operating at or below 800 bits per second where "transparency" in LPC parameter quantization is not required. Excessive codebook storage and search requirements have been identified however as being associated with this technique. High complexity and large storage requirements are also a factor in systems which optimally combine a variable bit rate (segmentation) operation and matrix quantization. This method offers reasonable filter coefficient quantization performance at about 200 bits per second, but although this approach in theory performs better than matrix quantization, matrix quantization continues to be of interest because it results in a fixed bit rate system.

Details of the known vector quantization systems referred to above can be derived from the paper: "Efficient coding of LSP parameters using split matrix quantisation by C. S. Xydeas and C. Papanastasiou, Proc. ICASSP-95, pp. 740-743, 1995.

It is an object of the present invention to provide an improved quantization system in which the complexity and storage requirements associated with known matrix quantization systems can be overcome.

According to the present invention there is provided a speech synthesis system in which coefficients of a speech synthesis filter are quantized, wherein a slowly evolving with time filter representation of p coefficients is generated for each of a series of N input speech frames to define a p by N matrix, with each row of the matrix containing N coefficients and each coefficient of one row being related to a respective one of the N frames, the matrix is split into a series of submatrices each made up from one or more of the said rows, and each sub-matrix is vector quantized independently of the other sub-matrices, using a weighting function, to produce a codebook index which is transmitted and used at the receiver to address a receiver codebook.

The weighting function may be a composite time/spectral function selected for example to emphasise i) distortion associated with high energy regions of the spectrum of each of the N input speech frames and ii) distortion in high energy voiced frames. The representation may be a line spectrum pair (LSP) filter coefficient representation. LSP is widely used in speech coding. Relevant background information can be obtained from the paper "Line spectrum pair (LSP) and speech data compression" by Frank K. Soong and Biing-Hwang Juang, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, San Diego, Mar. 19-21, 1984, and from references listed in that paper. The weighting function may be proportional to the value of the short term power spectrum measured at each frequency associated with the LSP elements of the sub-matrices.

A further weighting function may be applied to all the filter coefficients of the N input speech frames, the further weighting function being proportional to the energy and the degree of voicing of that frame.

First, second and third codebooks may be provided, the first codebook being selected when all N frames are voiced, the second codebook being selected when all N frames are unvoiced, and the third codebook being selected when the N frames include both voiced and unvoiced frames.

An embodiment of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which;

FIG. 1 is a representative speech waveform;

FIG. 2 illustrates LSP trajectories corresponding to the speech waveform of FIG. 1;

FIG. 3 is a schematic illustration of a subjective valuation system;

FIG. 3A is a more detailed "block diagram of the exemplary LPC analyzer and quantizer subsystems shown in FIG. 3; and

FIG. 4 illustrates the variation with bits per frame of a parameter used to evaluate the performance of the quantization process;

FIG. 5 plots relationships similar to those of FIG. 4 but with a variety of codebook configurations; and

FIG. 6 schematically represents storage requirements for different high quality LPC quantization schemes.

The invention proposes splitting a matrix representing a series of speech frames into sub-matrices which are then quantized independently with a view to overcoming the inherent drawbacks of known matrix quantization schemes, that is the drawbacks of high complexity and large storage requirements. In this context, four separate issues are discussed below;

I. Representations of the matrix elements as derived from LSP coefficients;

II. Distortion measures and associated time/spectral domain weighting functions used in codebook design and quantization processors;

III. Objective performance evaluation metrics which correlate well with subjective experiments performed using synthesised speech; and

IV. Complexity and codebook storage characteristics.

FIG. 3A depicts an exemplary LPC analyzer and quantizer subsystem for the speech synthesis system shown in FIG. 3. As those in the art will appreciate, the depicted signal processing for such a system typically is carried out by a suitably programmed digital signal processor or other suitable digital signal processing hardware/firmware/software.

The starting point for split matrix quantization is a digital electrical signal 12 (output from A/D converter 11) representing an input acoustic signal 10. The digital signal is divided at 14 into a series of speech frames 16 each of M msec duration. A slowly evolving with time filter coefficient representation 18 must then be produced, for example an LSP representation. LSP coefficients could be derived directly by analysis of the speech signal, or alternatively as described below a conventional LPC analysis may be applied to each of the series of speech frames at 20 to yield a series of coefficient vectors:

where p is the order of the LPC filter and n is the current frame. The LPC coefficients may be generated in a number of ways, for example using the Autocorrelation, Covariance or Lattice methods. Such methods are well known and are described in standard textbooks. The nth frame LPC coefficient vector a(n) is then transformed to an LSP representation:

This transformation process at 22 is performed over N consecutive speech frames to provide an p.times.N LSP matrix; ##EQU1##

The above matrix can be split up at 24 into K submatrices: ##EQU2##

Each row (or set of m(k) rows) in X corresponds to a "trajectory" in time of spectral coefficients over N successive frames, and these trajectories can be vector quantized independently at 26. These trajectories form the basis for codebooks at 28 provided at both the transmitter and receiver, the codebooks being identical and storing a series of trajectories each of which is associated with a codeword index. Having selected a trajectory from the transmitter codebook, the associated codebook index is transmitted at 3 D to the receiver and used at the receiver to retrieve the appropriate trajectory from the receiver codebook. In designing the corresponding k=1,2 . . . K trajectory codebooks, sequences of {L.sub.k } are obtained by sliding a N-frame window, one frame at a time, along the entire training sequence of LPC speech frames. This sliding block technique maximises the number of vectors employed in the codebook design process and ensures that all phoneme transitions present in the input training sequences are captured. Furthermore, in order to maximise efficiency, different codebook training sequences are generated and hence different codebooks are designed for each of the following three cases a) All N LPC frames are voiced, b) all N LPC frames are unvoiced and c) the N LPC frames segment includes both voiced and unvoiced frames.

In order to exploit interframe correlation, the p.times.N matrix elements should reflect the characteristics of the speech short-term magnitude spectral envelope which change slowly with time. Thus it is possible to employ a formant-bandwidth LSP based representation. Using statistical observations LSPs may be related to formants and bandwidths by means of a centre frequency (i.e. the mean frequency of an LPC pair) and an offset frequency (i.e. half the difference frequency of an LSP pair). However, formant/bandwidth information will not always provide smooth trajectories over time and can be therefore difficult to quantize within the split matrix quantization framework. On the other hand, LSPs offer an efficient LPC representation due to their monotonicity property and their relatively smooth evolution over time.

FIG. 1 shows a representative speech waveform in terms of amplitude versus time, and FIG. 2 shows the corresponding LSP trajectories. The time axis in both FIG. 1 and FIG. 2 is in terms of units of 20 msec each, each unit corresponding to one frame. Thus these figures represent waveforms over a period of 1.5 secs. The "smooth" LSP trajectories obtained during voiced speech are apparent. Both direct LSP and mean-difference LSP representations may be employed, but it is believed that superior results can be achieved with schemes based directly on LSP parameters.

Direct LSP based codebook design and search processes which have been put into effect have relied upon a weighted Euclidean distortion measure. This is defined as: ##EQU3##

where L'.sub.k represents the kth quantized submatrix and LSP".sub.S(k-1)+s are its elements.

The above equation includes a weighting factor w.sub.t (t) which is proportional to the energy and the degree of voicing in each LPC speech frame and is assigned to all the LSP spectral parameters of that frame. The weighting factor W.sub.t (t) is defined as follows: ##EQU4##

when the N LPC frames consist of both voiced and unvoiced frames

otherwise

where Er(t) is the normalised energy of the prediction error of frame t, En(t) is the RMS value of speech frame t and Aver(En) is the average RMS value of the N LPC frames. The values of the constants .alpha. and .alpha.1 are set to 0.2 and 0.15 respectively.

A further weighting factor W.sub.s (s,t) is also used which is proportional to the value of the short term power spectrum measured at each frequency associated with the LSP element of the m(k).times.N L.sub.k submatrix. The weighting factor W.sub.s (s,t) is defined as follows:

where P(LSP'.sub.S(k-1)+s) is the value of the power envelope spectrum of the speech frame t at the LSP'.sub.S(k-1)+s frequency. .beta. is equal to 0.15.

The weighting factor w.sub.s (s,t) ensures that distortion associated with high energy spectral regions is emphasised, as compared to low energy spectral regions. In a similar way, the weighting factor w.sub.t (t) ensures that distortion associated with voiced frames is emphasised and thus quantization accuracy increases in the case of voiced speech segments.

The performance of an LPC/LSP quantization process can be measured in terms of subjective tests and/or objective distortion related measures. Subjective tests are often performed using an arrangement as represented in FIG. 3. Here, the actual residual signal is used to excite the corresponding LPC filter whose coefficients are quantized. The term "transparent" LPC quantization refers to the case where, as a result of the noise introduced by quantizing the LPC coefficients, no audible distortion can be detected on the x.sub.n (i) output signal. Traditionally, objective measures that are used to assess the performance of quantization schemes operating on LPC parameters, are Spectral Distortion Measure (SDM) variants. SDM is defined as the root mean square difference formed between the original log-power LPC spectrum and the corresponding quantized log-power LPC spectrum. However, these SDM based measures focus on the accuracy of the quantization process to represent individual LPC frames, and thus fail to capture the perceptually important smooth evolution of LSP parameters across frames. The latter is exploited by split matrix quantization in accordance with the invention and as a consequence SDM measures do not relate well to subjective tests of the present invention.

A more accurate measure may be achieved by employing a time domain Segmental SNR metric, that is formed using the original X.sub.n (i) and synthesised X.sub.n (i) signals (see FIG. 3). However, the X.sub.n (i) and X.sub.n (i) signals are logarithmically (.mu.-law) processed. This effectively provides a 3.5 dB amplification of high frequency spectral components. Furthermore, a weighting factor Weig(n) is also used in the Logarithmic Segmental SNR (LogSegSNR) averaging process, which increases the "contribution" of voiced speech frames:

where En(n) is the energy of the nth frame and C=1 for a voiced frame or C=0.01 in the case of an unvoiced frame.

Extensive objective/subjective tests that have been conducted highlighted clearly the perceptual relevance of the LogSegSNR metric. However, it is advantageous to combine both the LogSegSNR and average SDM measures to establish accurate objective performance rules for "transparent" and "high quality" quantization of LPC parameters. The term "high quality" LPC quantization is used to indicated that, although a small difference can be perceived between the input and synthesised signals, nevertheless the effect of LPC quantization on the subjective quality of the output signal is negligible. In this context, "transparent" LPC quantization may be considered to be achieved when LogSegSNR>10 dB and AverSDM measured (using the weighting factor Weig.sub.t (n)) in the frequency range of 2.4 to 3.4 Khz is below 1.75 dB. The corresponding values for "high quality" LPC quantization are 10 dB.gtoreq.LogSegSNR.gtoreq.9.5 dB and 2 dB.gtoreq.AverSDM.gtoreq.1.75 dB.

The proposed split matrix quantization (SMQ) scheme described above has been simulated for different values of K (the number of submatrices in the system), m(k) (the number of rows in the kth submatrix) and N (the number of columns in the matrix, that is the number of successive LPC frames used to form the matrix). Corresponding codebooks have been designed using, for training, 150 min duration of multi-speaker, multi-language speech material. In addition, several minutes of "out of training" speech from two male and two female speakers was used to evaluate the performance of various SMQ configurations, and a conventional 3-way {3,3,4} Split-VQ scheme has been employed as a benchmark in these experiments. In all cases the number p of LSP's in a frame was 10. The simulations included examples for K=10 and K=5. In the latter case each submatrix had two rows, i.e. m(k)=2 for k=1, 2 . . . 5. These two cases are referred to below as "single track" (m(k)=1, k=1, 2 . . . 10) and "double track" (m(k)=2, k=1, 2 . . . 5). Results obtained are represented in FIGS. 4, 5 and 6.

The inability of SDM to adequately reflect subjective performance was apparent from the fact that a 3-way Split-VQ scheme operating at 22 bits/frame provided the same AverSDM value of 1.67 dB with that obtained from a 18 bits/frame Single Track (K=10, N=4) SMQ quantizer (ST-SMQ, N=4). Subjectively however, ST-SMQ, N=4 produced considerably better speech quality.

The crucial role of the weighting functions used in Equation 3 is highlighted in FIG. 4, where LogSegSMR values are plotted using different numbers of bits/frame for ST-SMQ, N=4 with or without weighting in the distortion measure. The 0.65 dB difference in the two curves corresponds to a net gain of 2 bits/frame.

FIG. 5 illustrates the LogSegSNR performance of several systems, as a function of bits/frame. An increase of N from 3 to 4 provides a 2 bits/frame advantage whereas a further increase to N=5 provides a smaller gain of 0.5 bits/frame. Thus with N=4 and a basic LPC frame of 20 msec duration, the system operates effectively at a rate of 12.5 segments/sec. This is comparable to the average phoneme rate and seems to be the segment length that exploits most of the existing interframe LPC correlation. Results are also included in FIG. 5 for Double Track SMQ (DT-SMQ) systems. These offer improved performance, as compared to ST-SMQ schemes. ST-SMQ quantizers can deliver an advantage of 12 bits/frame as compared to conventional Split-VQ.

Tables 1a to 1f below set out the bit allocations used to produce the results shown in FIG. 5:

FIG. 6 illustrates storage requirements in terms of the number of codebook elements required for different SMQ configurations.

Thus the present invention may be implemented in any one of a number of possible ways to achieve different performance/complexity characteristics.