US 6,161,089 AGrant
Multi-Subframe Quantization of Spectral Parameters
Issue Date:2000-12-12
•54 Claims
•7 Drawing Sheets
Abstract
Speech is encoded into a frame of bits. A speech signal is digitized into a sequence of digital speech samples that are then divided into a sequence of subframes. A set of model parameters is estimated for each subframe. The model parameters include a set of spectral magnitude parameters that represent spectral information for the subframe. Two or more consecutive subframes from the sequence of subframes may be combined into a frame. The spectral magnitude parameters from both of the subframes within the frame may be jointly quantized. The joint quantization includes forming predicted spectral magnitude parameters from the quantized spectral magnitude parameters from the previous frame, computing the residual parameters as the difference between the spectral magnitude parameters and the predicted spectral magnitude parameters, combining the residual parameters from both of the subframes within the frame, and quantizing the combined residual parameters into a set of encoded spectral bits which are included in the frame of bits.
Metadata
Assignee
- Digital Voice Systems, Inc.
Inventor
- John C. Hardwick
Application Information
Application Number:US 8181306
Filing Date:1997-03-14
Priority Date:1997-03-14
Art Unit:271
Classifications
IPC:
G10L 2100
Field of Search:
704203;204;206;208;230;219;222
Patent Drawings (7 sheets)
Description
Background
The invention is directed to encoding and decoding speech.
Speech encoding and decoding have a large number of applications and have been studied extensively. In general, one type of speech coding, referred to as speech compression, seeks to reduce the data rate needed to represent a speech signal without substantially reducing the quality or intelligibility of the speech. Speech compression techniques may be implemented by a speech coder.
A speech coder is generally viewed as including an encoder and a decoder. The encoder produces a compressed stream of bits from a digital representation of speech, such as may be generated by converting an analog signal produced by a microphone using an analog-to-digital converter. The decoder converts the compressed bit stream into a digital representation of speech that is suitable for playback through a digital-to-analog converter and a speaker. In many applications, the encoder and decoder are physically separated, and the bit stream is transmitted between them using a communication channel.
A key parameter of a speech coder is the amount of compression the coder achieves, which is measured by the bit rate of the stream of bits produced by the encoder. The bit rate of the encoder is generally a function of the desired fidelity (i.e., speech quality) and the type of speech coder employed. Different types of speech coders have been designed to operate at high rates (greater than 8 kbs), mid-rates (3-8 kbs) and low rates (less than 3 kbs). Recently, mid-rate and low-rate speech coders have received attention with respect to a wide range of mobile communication applications (e.g., cellular telephony, satellite telephony, land mobile radio, and in-flight telephony). These applications typically require high quality speech and robustness to artifacts caused by acoustic noise and channel noise (e.g., bit errors).
Vocoders are a class of speech coders that have been shown to be highly applicable to mobile communications. A vocoder models speech as the response of a system to excitation over short time intervals. Examples of vocoder systems include linear prediction vocoders, homomorphic vocoders, channel vocoders, sinusoidal transform coders ("STC"), multiband excitation ("MBE") vocoders, and improved multiband excitation ("IMBE.TM.") vocoders. In these vocoders, speech is divided into short segments (typically 10-40 ms) with each segment being characterized by a set of model parameters. These parameters typically represent a few basic elements of each speech segment, such as the segment's pitch, voicing state, and spectral envelope. A vocoder may use one of a number of known representations for each of these parameters. For example the pitch may be represented as a pitch period, a fundamental frequency, or a long-term prediction delay. Similarly the voicing state may be represented by one or more voiced/unvoiced decisions, by a voicing probability measure, or by a ratio of periodic to stochastic energy. The spectral envelope is often represented by an all-pole filter response, but also may be represented by a set of spectral magnitudes or other spectral measurements.
Since they permit a speech segment to be represented using only a small number of parameters, model-based speech coders, such as vocoders, typically are able to operate at medium to low data rates. However, the quality of a model-based system is dependent on the accuracy of the underlying model. Accordingly, a high fidelity model must be used if these speech coders are to achieve high speech quality.
One speech model which has been shown to provide high quality speech and to work well at medium to low bit rates is the Multi-Band Excitation (MBE) speech model developed by Griffin and Lim. This model uses a flexible voicing structure that allows it to produce more natural sounding speech, and which makes it more robust to the presence of acoustic background noise. These properties have caused the MBE speech model to be employed in a number of commercial mobile communication applications.
The MBE speech model represents segments of speech using a fundamental frequency, a set of binary voiced/unvoiced (V/UV) metrics, and a set of spectral magnitudes. A primary advantage of the MBE model over more traditional models is in the voicing representation. The MBE model generalizes the traditional single V/UV decision per segment into a set of decisions, each representing the voicing state within a particular frequency band. This added flexibility in the voicing model allows the MBE model to better accommodate mixed voicing sounds, such as some voiced fricatives. In addition this added flexibility allows a more accurate representation of speech that has been corrupted by acoustic background noise. Extensive testing has shown that this generalization results in improved voice quality and intelligibility.
The encoder of an MBE-based speech coder estimates the set of model parameters for each speech segment. The MBE model parameters include a fundamental frequency (the reciprocal of the pitch period); a set of V/UV metrics or decisions that characterize the voicing state; and a set of spectral magnitudes that characterize the spectral envelope. After estimating the MBE model parameters for each segment, the encoder quantizes the parameters to produce a frame of bits. The encoder optionally may protect these bits with error correction/detection codes before interleaving and transmitting the resulting bit stream to a corresponding decoder.
The decoder converts the received bit stream back into individual frames. As part of this conversion, the decoder may perform deinterleaving and error control decoding to correct or detect bit errors. The decoder then uses the frames of bits to reconstruct the MBE model parameters, which the decoder uses to synthesize a speech signal that perceptually resembles the original speech to a high degree. The decoder may synthesize separate voiced and unvoiced components, and then may add the voiced and unvoiced components to produce the final speech signal.
In MBE-based systems, the encoder uses a spectral magnitude to represent the spectral envelope at each harmonic of the estimated fundamental frequency. Typically each harmonic is labeled as being either voiced or unvoiced, depending upon whether the frequency band containing the corresponding harmonic has been declared voiced or unvoiced. The encoder then estimates a spectral magnitude for each harmonic frequency. When a harmonic frequency has been labeled as being voiced, the encoder may use a magnitude estimator that differs from the magnitude estimator used when a harmonic frequency has been labeled as being unvoiced. At the decoder, the voiced and unvoiced harmonics are identified, and separate voiced and unvoiced components are synthesized using different procedures. The unvoiced component may be synthesized using a weighted overlap-add method to filter a white noise signal. The filter is set to zero all frequency regions declared voiced while otherwise matching the spectral magnitudes labeled unvoiced. The voiced component is synthesized using a tuned oscillator bank, with one oscillator assigned to each harmonic that has been labeled as being voiced. The instantaneous amplitude, frequency and phase are interpolated to match the corresponding parameters at neighboring segments.
MBE-based speech coders include the IMBE.TM. speech coder and the AMBE.RTM. speech coder. The AMBE.RTM. speech coder was developed as an improvement on earlier MBE-based techniques. It includes a more robust method of estimating the excitation parameters (fundamental frequency and V/UV decisions) which is better able to track the variations and noise found in actual speech. The AMBE.RTM. speech coder uses a filterbank that typically includes sixteen channels and a non-linearity to produce a set of channel outputs from which the excitation parameters can be reliably estimated. The channel outputs are combined and processed to estimate the fundamental frequency and then the channels within each of several (e.g., eight) voicing bands are processed to estimate a V/UV decision (or other voicing metric) for each voicing band.
The AMBE.RTM. speech coder also may estimate the spectral magnitudes independently of the voicing decisions. To do this, the speech coder computes a fast Fourier transform ("FFT") for each windowed subframe of speech and then averages the energy over frequency regions that are multiples of the estimated fundamental frequency. This approach may further include compensation to remove from the estimated spectral magnitudes artifacts introduced by the FFT sampling grid.
The AMBE.RTM. speech coder also may include a phase synthesis component that regenerates the phase information used in the synthesis of voiced speech without explicitly transmitting the phase information from the encoder to the decoder. Random phase synthesis based upon the V/UV decisions may be applied, as in the case of the IMBE.TM. speech coder. Alternatively, the decoder may apply a smoothing kernel to the reconstructed spectral magnitudes to produce phase information that may be perceptually closer to that of the original speech than is the randomly-produced phase information.
The techniques noted above are described, for example, in Flanagan, Speech Analysis, Synthesis and Perception, Springer-Verlag, 1972, pages 378-386 (describing a frequency-based speech analysis-synthesis system); Jayant et al., Digital Coding of Waveforms, Prentice-Hall, 1984 (describing speech coding in general); U.S. Pat. No. 4,885,790 (describing a sinusoidal processing method); U.S. Pat. No. 5,054,072 (describing a sinusoidal coding method); Almeida et al., "Nonstationary Modeling of Voiced Speech", IEEE TASSP, Vol. ASSP-31, No. 3, June 1983, pages 664-677 (describing harmonic modeling and an associated coder); Almeida et al., "Variable-Frequency Synthesis: An Improved Harmonic Coding Scheme", IEEE Proc. ICASSP 84, pages 27.5.1-27.5.4 (describing a polynomial voiced synthesis method); Quatieri et al., "Speech Transformations Based on a Sinusoidal Representation", IEEE TASSP, Vol, ASSP34, No. 6, Dec. 1986, pages 1449-1986 (describing an analysis-synthesis technique based on a sinusoidal representation); McAulay et al., "Mid-Rate Coding Based on a Sinusoidal Representation of Speech", Proc. ICASSP 85, pages 945-948, Tampa, Fla., March 26-29, 1985 (describing a sinusoidal transform speech coder); Griffin, "Multiband Excitation Vocoder", Ph.D. Thesis, M.I.T, 1987 (describing the Multi-Band Excitation (MBE) speech model and an 8000 bps MBE speech coder); Hardwick, "A 4.8 kbps Multi-Band Excitation Speech Coder", SM. Thesis, M.I.T, May 1988 (describing a 4800 bps Multi-Band Excitation speech coder); Telecommunications Industry Association (TIA), "APCO Project 25 Vocoder Description", Version 1.3, Jul. 15, 1993, IS102BABA (describing a 7.2 kbps IMBE.TM. speech coder for APCO Project 25 standard); U.S. Pat. No. 5,081,681 (describing IMBE.TM. random phase synthesis); U.S. Pat. No. 5,247,579 (describing a channel error mitigation method and format enhancement method for MBE-based speech coders); U.S. Pat. No. 5,226,084 (describing quantization and error mitigation methods for MBE-based speech coders); U.S. Pat. No. 5,517,511 (describing bit prioritization and FEC error control methods for MBE-based speech coders).
Summary
The invention features a new AMBE.RTM. speech coder for use, for example, in a wireless communication system to produce high quality speech from a bit stream transmitted across a wireless communication channel at a low data rate. The speech coder combines low data rate, high voice quality, and robustness to background noise and channel errors. This promises to advance the state of the art in speech coding for mobile communications. The new speech coder achieves high performance through a new multi-subframe spectral magnitude quantizer that jointly quantizes spectral magnitudes estimated from two or more consecutive subframes. The quantizer achieves fidelity comparable to prior art systems while using fewer bits to quantize the spectral magnitude parameters. AMBE.RTM. speech coders are described generally in U.S. application Ser. No. 08/222,119, filed Apr. 4, 1994 and entitled "ESTIMATION OF EXCITATION PARAMETERS"; U.S. application Ser. No. 08/392,188, filed Feb. 22, 1995 and entitled "SPECTRAL REPRESENTATIONS FOR MULTI-BAND EXCITATION SPEECH CODERS"; and U.S. Application No. 08/392,099, filed Feb. 22, 1995 and entitled "SYNTHESIS OF SPEECH USING REGENERATED PHASE INFORMATION", all of which are incorporated by reference.
In one aspect, generally, the invention features encoding speech into a frame of bits. A speech signal is digitized into a sequence of digital speech samples that are divided into a sequence of subframes, each of which includes multiple digital speech samples. A set of speech model parameters is estimated for each subframe, the parameters including a set of spectral magnitude parameters that represent spectral information for the subframe. Consecutive subframes then are combined into a frame, and the spectral magnitude parameters from the subframes of the frame are jointly quantized to produce a set of encoder spectral bits that are included in a frame of bits for transmission or storage. The joint quantization includes forming predicted spectral magnitude parameters from quantized spectral magnitude parameters from a previous frame.
Embodiments of the invention may include one or more of the following features. The joint quantization may include computing residual parameters as the difference between the spectral magnitude parameters and the predicted spectral magnitude parameters. The residual parameters from the subframes of the frame may be combined and quantized into a set of encoder spectral bits.
The residual parameters may be combined by dividing the residual parameters from each subframe into frequency blocks and performing a linear transformation on the residual parameters within each frequency block to produce a set of transformed residual coefficients for each subframe. A minority of the transformed residual coefficients from the frequency blocks for each subframe may be grouped into a PRBA vector for the subframe, and the remaining transformed residual coefficients for each frequency block of each subframe may be grouped into a higher order coefficient (HOC) vector for the frequency block. The prediction residual block average (PRBA) vectors may be transformed to produce a transformed PRBA vector for each subframe, and the transformed PRBA vectors for the subframes of the frame may be combined by computing generalized sum and difference vectors from the transformed PRBA vectors, and combining the HOC vectors within each frequency block for the subframes of the frame by computing generalized sum and difference vectors from the HOC vectors for each frequency block.
The predicted spectral magnitude parameters may be formed by applying a gain of less than unity to a linear interpolation of quantized spectral magnitudes from a last subframe in a previous frame. The transformed residual coefficients may be computed for each frequency block using a Discrete Cosine Transform (DCT) followed by a linear two by two transform on two lowest order DCT coefficients. The length of each frequency block may be approximately proportional to a number of spectral magnitude parameters within the subframe.
The combined residual parameters may be quantized using a vector quantizer. Vector quantization may be applied to all or part of the generalized sum and difference vectors computed from the transformed PRBA vectors, and may be applied to all or part of the generalized sum and difference vectors computed from the HOC vectors.
Additional encoder bits may be produced by quantizing additional speech model parameters other than the spectral magnitude parameters. The additional speech model parameters may include parameters representative of a fundamental frequency and parameters representative of a voicing state. The frame of bits also may include redundant error control bits that protect at least some of the encoder spectral bits. The spectral magnitude parameters may represent log spectral magnitudes estimated for a Multi-Band Excitation (MBE) speech model, and may be estimated from a computed spectrum in a manner which is independent of a voicing state.
In another aspect, generally, the invention features decoding speech from a frame of bits. Decoder spectral bits are extracted from the frame of bits, and are used to jointly reconstruct spectral magnitude parameters for consecutive subframes within a frame of speech. The joint reconstruction includes inverse quantizing the decoder spectral bits to reconstruct a set of combined residual parameters for the frame from which separate residual parameters for each of the subframes are computed. Predicted spectral magnitude parameters are formed from reconstructed spectral magnitude parameters from a previous frame. The separate residual parameters are added to the predicted spectral magnitude parameters to form the reconstructed spectral magnitude parameters for each subframe within the frame. Digital speech samples are synthesized for each subframe using speech model parameters that include some or all of the reconstructed spectral magnitude parameters for the subframe.
Embodiments of this aspect of the invention may include one or more of the following features. The separate residual parameters may be computed by dividing each subframe into frequency blocks. The combined residual parameters for the frame may be separated into generalized sum and difference vectors representing transformed PRBA vectors combined across the subframes of the frame, and into generalized sum and difference vectors representing HOC vectors for the frequency blocks combined across the subframes of the frame. PRBA vectors may be computed for each subframe from the generalized sum and difference vectors representing the transformed PRBA vectors. HOC vectors may be computed for each subframe from the generalized sum and difference vectors representing the HOC vectors for each of the frequency blocks. The PRBA vector and the HOC vectors for each of the frequency blocks may be combined to form transformed residual coefficients for each of the subframes, and an inverse transformation may be performed on the transformed residual coefficients to produce the separate residual parameters for each subframe of the frame.
The predicted spectral magnitude parameters may be formed by applying a gain of less than unity to a linear interpolation of quantized spectral magnitudes from a last subframe of a previous frame. The separate residual parameters may be computed from the transformed residual coefficients by performing on each of the frequency blocks an inverse linear two by two transform on the two lowest order transformed residual coefficients within the frequency block and then performing an Inverse Discrete Cosine Transform (IDCT) over all the transformed residual coefficients within the frequency block.
Four of the frequency blocks may be used per subframe, and the length of each frequency block may be approximately proportional to a number of spectral magnitude parameters within the subframe. Inverse quantization to reconstruct a set of combined residual parameters for the frame may include using inverse vector quantization applied to one or more vectors.
The frame of bits may include other decoder bits in addition to the decoder spectral bits. These bits may be representative of speech model parameters other than the spectral magnitude parameters, such as a fundamental frequency and parameters representative of a voicing state. The frame of bits also may include redundant error control bits protecting at least some of the decoder spectral bits.
The reconstructed spectral magnitude parameters may represent log spectral magnitudes used in a Multi-Band Excitation (MBE) speech model. Synthesizing of speech for each subframe may include computing a set of phase parameters from the reconstructed spectral magnitude parameters.
In another aspect, the invention features encoding a level of speech into a frame of bits by digitizing a speech signal into a sequence of digital speech samples and dividing the digital speech samples into a sequence of subframes that each include multiple digital speech samples. A speech level parameter is estimated for each subframe. The speech level parameter is representative of the amplitude of the digital speech samples of the subframe. Consecutive subframes are combined into a frame, and the speech level parameters from the subframes within the frame are jointly quantized. This quantization includes computing and quantizing an average level parameter by combining the speech level parameters over the subframes within the frame, and computing and quantizing a difference level vector between the speech level parameters for each subframe within the frame and the average level parameter. Quantized bits representative of the average level parameter and the difference level vector are included in a frame of bits.
Embodiments of this aspect of the invention may include one or more of the following features. The speech level parameter for each subframe may be estimated as a mean of a set of spectral magnitude parameters computed for each subframe plus an offset. The spectral magnitude parameters may represent log spectral magnitudes estimated for a Multi-Band Excitation (MBE) speech model. The offset may be dependent on a number of spectral magnitude parameters in the frame.
The difference level vector may be quantized using vector quantization, and the frame of bits may include error control bits used to protect some or all of the quantized bits representative of the average level parameter and the difference level vector.
Other features and advantages of the invention will be apparent from the following description, including the drawings, and from the claims.
Brief Description of the Drawing
FIG. 1 is a simplified block diagram of a wireless communications system.
FIG. 2 is a block diagram of a communication link of the system of FIG. 1.
FIGS. 3 and 4 are block diagrams of an encoder and a decoder of the system of FIG. 1.
FIG. 5 is a general block diagram of components of the encoder of FIG. 3.
FIG. 6 is a flowchart of voice and tone detection functions of the encoder.
FIG. 7 is a block diagram of a multi-subframe magnitude quantizer of the encoder of FIG. 5.
FIG. 8 is a block diagram of a mean vector quantizer of the magnitude quantizer of FIG. 7.
Description
An embodiment of the invention is described in the context of a new AMBE.RTM. speech coder, or vocoder, which is widely applicable to the problems of wireless communications such as cellular or satellite telephony, mobile radio, airphones, voice pagers, and digital storage of speech such as in telephone answering machines and dictation equipment. Referring to FIG. 1, a mobile terminal or telephone 40 is connected across a wireless communication channel 42 to a mobile gateway or base station 44 which is connected to the public switched telephone network (PSTN) 46. The speech coder in the mobile telephone 40 and in the mobile base station 44 allows conventional telephones 48 to be bridged into the wireless network.
The described vocoder has a 40 ms frame size and operates at a data rate of 3900 bps (156 bits per frame). These bits are divided between speech coding and forward error control ("FEC") coding to increase the robustness of the system to bit errors that normally occur across a wireless communication channel. The vocoder is designed to operate most efficiently at low to medium data rates in which speech is coded and transmitted at rates of 1500 bps to 8000 bps, ignoring bits associated with FEC coding. However, appropriate modifications can be made to the vocoder to enable it to work at other data rates. The vocoder also may be adapted to other frame sizes, such as, for example, 30-60 ms frames. In one implementation, a dual-rate embodiment using a 45 ms frame size has been operated at data rates of 3467 bps and 6933 bps.
Referring to FIG. 2, the mobile telephone at the transmitting end achieves voice communication by digitizing speech 50 received through a microphone 60 using an analog-to-digital (A/D) converter 70 that samples the speech at a frequency of 8 kHz. The digitized speech signal passes through a speech encoder 80, where it is processed as described below. The signal is then transmitted across the communication link by a transmitter 90. At the other end of the communication link, a receiver 100 receives the signal and passes it to a decoder 110. The decoder converts the signal into a synthetic digital speech signal. A digital-to-analog (D/A) converter 120 then converts the synthetic digital speech signal into an analog speech signal that is converted into audible speech 140 by a speaker 130.
The speech coder in each terminal includes an encoder 80 and a decoder 110. As shown in FIG. 3, the encoder includes three main functional blocks: speech analysis 200, parameter quantization 210, and FEC encoding 220. FEC encoding typically includes bit prioritization and interleaving. As shown in FIG. 4, the decoder is similarly divided into FEC decoding 230, which may include deinterleaving and inverse bit prioritization, parameter reconstruction 240 (i.e., inverse quantization) and speech synthesis 250.
The speech coder may be designed to operate at multiple data rates. However, the described embodiment is a 3900 bps vocoder using 156 bits per 40 ms frame. These bits are divided into 103 bits used for the voice (i.e. source) coding plus 53 bits used for forward error correction (FEC) coding. Each 40 ms frame is divided into two 20 ms subframes, and speech analysis and synthesis are performed on a subframe basis while quantization and FEC coding are performed on a frame basis.
The FEC typically includes one or more short block codes and/or convolution codes. In the described embodiment, one [24,12] extended Golay code, three [23,12] Golay codes and two [15,11] Hamming codes are employed for each frame. The codes possessing more redundancy (i.e., the Golay codes) are used on the most sensitive voice bits while the codes with less redundancy (i.e., the Hamming codes) are used on less sensitive voice bits and the least sensitive voice bits are not protected with any code.
The data rate may be varied by changing either the number of voice bits or the number of FEC bits. There is a gradual effect on performance as the data rate is changed. Changes in the number of voice bits may be accommodated by reallocating the number of bits used to quantize the model parameters. In the event of a significantly higher data rate, where a corresponding increase in the number of bits used for vector quantization of the magnitude parameters would result in excessive complexity, scalar quantization, or a hierarchical approach that combines vector quantization as featured in the described embodiment with an error quantizer that quantizes the difference between the unquantized spectral magnitudes and the reconstructed result from vector quantization, may be used. An error quantizer using scalar quantization has been implemented in the context of a dual-rate system. The error quantizer reduces quantization distortion and increases perceived quality while adding only minimal complexity.
Referring to FIG. 3, the encoder first performs speech analysis 200. The first step in speech analysis is filterbank processing on each subframe followed by estimation of the MBE model parameters for each subframe. This involves dividing the input signal into overlapping subframes using an analysis window. For each 20 ms subframe, a MBE subframe parameter estimator estimates a set of model parameters that include a fundamental frequency (inverse of the pitch period), a set of voiced/unvoiced (V/UV) metrics and a set of spectral magnitudes. These parameters are generated using AMBE techniques. The speech parameters fully describe the speech signal and are passed to the encoder's quantization 210 block for further processing. Speech analysis techniques for AMBE.RTM. speech coders are described generally in U.S. Application No. 08/222,119, filed Apr. 4, 1994 and entitled "ESTIMATION OF EXCITATION PARAMETERS"; U.S. Application No. 08/392,188, filed Feb. 22, 1995 and entitled "SPECTRAL REPRESENTATIONS FOR MULTI-BAND EXCITATION SPEECH CODERS"; and U.S. Application No. 08/392,099, filed Feb. 22, 1995 and entitled "SYNTHESIS OF SPEECH USING REGENERATED PHASE INFORMATION", all of which are incorporated by reference.
Referring to FIG. 5, once the subframe model parameters 500 and 505 are estimated for the two subframes of a frame, a fundamental frequency quantizer 510 receives the estimated fundamental frequency parameters from both subframes, quantizes these parameters, and produces a set of bits encoding the fundamental frequencies for both subframes. A voicing quantizer 515 receives estimated voicing metrics for both subframes, and then quantizes these parameters into a set of encoded bits representing the voicing state within the frame. The encoded fundamental frequency bits and voicing bits are fed to a combiner 520 along with encoded spectral bits from a multi-subframe spectral magnitude quantizer 525. FEC encoding 530 is applied to the output of the combiner 520 and the resulting frame of bits 535 is suitable for transmission or storage.
As shown in FIG. 6, the encoder may incorporate an adaptive Voice Activity Detector (VAD) that classifies each subframe as either voice, background noise or a tone according to a procedure 600. The VAD algorithm uses local information to distinguish voice subframes from background noise (step 605). If both subframes within a frame are classified as noise (step 610), then the encoder quantizes the background noise that is present as a special Noise frame (step 615). When a frame is a noise frame, the system may choose not to transmit the frame to the decoder and the decoder will use previously received noise data in place of the missing frame. This voice activated transmission technique increases performance of the system by only requiring voice frames and occasional noise frames to be transmitted.
The encoder also may feature tone detection and transmission in support of DTMF, call progress (e.g., dial, busy and ringback) and single tones. The encoder checks each subframe to determine whether the current subframe contains a valid tone signal. If a tone is detected in a subframe (step 620), then the encoder quantizes the detected tone parameters (magnitude and index) in a special Tone frame as shown in Table 1 (step 625) and applies FEC coding prior to transmitting the frame to the decoder for subsequent synthesis. If a tone is not detected, then a standard voice frame is quantized as described below (step 630).
The vocoder includes VAD and Tone detection to classify each frame as either a standard Voice frame, a special Tone frame, or a special Noise frame. In the event that a frame is not classified as a special Tone frame, then the voice or noise information (as determined by the VAD) is quantized for the pair of subframes. The 156 available bits are allocated over the model parameters and FEC coding as shown in Table 2. After reserving bits for the excitation parameters (fundamental frequency and voicing metrics) and FEC coding, there are 85 bits available for the spectral magnitudes.
The multi-subframe quantizer quantizes the spectral magnitudes. The quantizer combines logarithmic companding, spectral prediction, discrete cosine transforms (DCTs) and vector and scalar quantization to achieve high efficiency, measured in terms of fidelity per bit, with reasonable complexity. The quantizer can be viewed as a two-dimensional (time-frequency) predictive transform coder. The quantizer jointly encodes the spectral magnitudes from all of the subframes (typically two) of the current frame. As a first step, the quantizer computes the logarithm of the estimated spectral magnitudes for each subframe to convert them into a domain that is better for quantization. The quantizer then may apply a low-frequency boost to the log spectral magnitudes to compensate for missing low-frequency energy which may have been removed through filtering in the telephone system or elsewhere. The magnitude quantizer then computes predicted spectral parameters for each subframe using quantized and reconstructed log spectral magnitudes from the last subframe of the prior frame. These prior magnitudes are linearly interpolated and resampled to compensate for the possible difference between the number of magnitudes in the prior subframe and the number of magnitudes in each of the subframes in the current frame. In addition to interpolation and resampling, the computation of the predicted spectral parameters removes the mean value of the parameters and applies a multiplicative "leakage factor" that is less than one (e.g., 0.8) to ensure that any error in previous magnitudes caused by bit errors decays away over a few frames.
FIG. 7 illustrates a dual-frame magnitude quantizer that receives inputs 1a and 1b from the MBE parameter estimators for two consecutive subframes. Input 1a represents the spectral magnitudes for odd numbered subframes and is given an index of 1. The number of magnitudes for subframe number 1 is designated by L.sub.1. Input 1b represents the spectral magnitudes for the even numbered subframes and is given the index of 0. The number of magnitudes for subframe number 0 is a variable, designated by L.sub.o.
Input la passes through a logarithmic compander 2a, which performs a log base 2 operation on each of the L.sub.1 magnitudes contained in input la and generates another vector with L.sub.1 elements in the following manner :
where y[i] represents signal 3a. Compander 2b performs the log base 2 operation on each of the L.sub.0 magnitudes contained in input 1b and generates another vector with L.sub.0 elements in a similar manner:
where y[i] represents signal 3b.
Mean calculators 4a and 4b following the companders 2a and 2b calculate means 5a and 5b for each subframe. The mean, or gain value, represents the average speech level for the subframe. Within each frame, two gain values 5a, 5b are determined by computing the mean of the log spectral magnitudes for each of the two subframes and then adding an offset dependent on the number of harmonics within the subframe.
The mean computation of the log spectral magnitudes 3a is calculated as: ##EQU1## where the output, y, represents the mean signal 5a.
The mean computation 4b of the log spectral magnitudes 3b is calculated in a similar manner: ##EQU2## where the output, y, represents the mean signal 5b.
The mean signals 5a and 5b are quantized by a quantizer 6 that is further illustrated in FIG. 8, where the mean signals 5a and 5b are referenced, respectively, as mean1 and mean2. First, an averager 810 averages the mean signals. The output of the averager is 0.5*(mean1+mean2). The average is then quantized by a five-bit uniform scalar quantizer 820. The output of the quantizer 820 forms the first five bits of the output of the quantizer 6. The quantizer output bits are then inverse-quantized by a five-bit uniform inverse scalar quantizer 830. Subtracters 835 then subtract the output of the inverse quantizer 830 from the input values mean1 and mean2 to produce inputs to a five-bit vector quantizer 840. The two inputs constitute a two-dimensional vector (z1 and z2) to be quantized. The vector is compared to each two-dimensional vector consisting of x1(n) and x2(n)) in the table contained in Table A ("Gain VQ Codebook (5-bit)"). The comparison is based on the square distance, e, which is calculated as follows:
for n=0, 1, . . . 31. The vector from Table A that minimizes the square distance, e, is selected to produce the last five bits of the output of block 6. The five bits from the output of the vector quantizer 840 are combined with the five bits from the output of the five-bit uniform scalar quantizer 820 by a combiner 850. The output of the combiner 850 is ten bits constituting the output of block 6 which is labeled 21c and is used as an input to the combiner 22 in FIG. 7.
Referring further to the main signal path of the quantizer, the log companded input signals 3a and 3b pass through combiners 7a and 7b that subtract predictor values 33a and 33b from the feedback portion of the quantizer to produce a D.sub.1 (l) signal 8a and a D.sub.1 (0) signal 8b.
Next, the signals 8a and 8b are divided into four frequency blocks using the look-up table in Table O. The table provides the number of magnitudes to be allocated to each of the four frequency blocks based on the total number of magnitudes for the subframe being divided. Since the number of magnitudes contained in any subframe ranges from a minimum of 9 to a maximum of 56, the table contains values for this same range. The length of each frequency block is adjusted such that they are approximately in a ratio of 0.2:0.225:0.275:0.3 to each other and the sum of the lengths equals the number of spectral magnitudes in the current subframe.
Each frequency block is then passed through a discrete cosine transform (DCT) 9a or 9b to efficiently decorrelate the data within each frequency block. The first two DCT coefficients 10a or 10b from each frequency block are then separated out and passed through a 2.times.2 rotation operation 12a or 12b to produce transformed coefficients 13a or 13b. An eight-point DCT 14a or 14b is then performed on the transformed coefficients 13a or 13b to produce a prediction residual block average (PRBA) vector 15a or 15b. The remaining DCT coefficients 11a and 11b from each frequency block form a set of four variable length higher order coefficient (HOC) vectors.
As described above, following the frequency division, each block is processed by the discrete cosine transform blocks 9a or 9b. The DCT blocks use the number of input bins, W, and the values for each of the bins, x(0), x(1), . . . , x(W-1) in the following manner:
The values y(0) and y(1) (identified as 10a) are separated from the other outputs y(2) through y(W-1) (identified as ##EQU3## 11a).
A 2.times.2 rotation operation 12a and 12b is then performed to transform the 2-element input vector 10a and 10b, (x(0),x(1)), into a 2-element output vector 13a and 13b, (y(0),y(1)) by the following rotation procedure :
An 8-point DCT is then performed on the four, 2-element vectors, (x(0),x(1), . . . ,x(7) ) from 13a or 13b according to the following equation: ##EQU4## The output, y(k), is an 8-element PRBA vector 15a or 15b.
Once the prediction and DCT transformation of the individual subframe magnitudes have been completed, both PRBA vectors are quantized. The two eight-element vectors are first combined using a sum-difference transformation 16 into a sum vector and a difference vector. In particular, sum/difference operation 16 is performed on the two 8-element PRBA vectors 15a and 15b, which are represented by x and y respectively, to produce a 16-element vector 17, represented by z, in the following manner:
for i =0, 1, ... , 7.
These vectors are then quantized using a split vector quantizer 20a where 8, 6, and 7 bits are used for elements 1-2, 3-4, and 5-7 of the sum vector, respectively, and 8 and 6 bits are used for elements 1-3 and 4-7 of the difference vector, respectively. Element 0 of each vector is ignored since it is functionally equivalent to the gain value that is quantized separately.
The quantization of the PRBA sum and difference vectors 17 is performed by the PRBA split-vector quantizer 20a to produce a quantized vector 21a. The two elements z(1) and z(2) constitute a two-dimensional vector to be quantized. The vector is compared to each two-dimensional vector (consisting of x1(n) and x2(n) in the table contained in Table B ("PRBA Sum[1,2] VQ Codebook (8-bit)"). The comparison is based on the square distance, e, which is calculated as follows:
for n=0,1, ..., 255. The vector from Table B that minimizes the square distance, e, is selected to produce the first 8 bits of the output vector 21a.
Next, the two elements z(3) and z(4) constitute a two-dimensional vector to be quantized. The vector is compared to each two-dimensional vector (consisting of x1(n)) and x2(n) in the table contained in Table C ("PRBA Sum[3,4] VQ Codebook (6-bit)"). The comparison is based on the square distance, e, which is calculated as follows:
for n=0,1, . . . , 63. The vector from Table C which minimizes the square distance, e, is selected to produce the next 6 bits of the output vector 21a.
Next, the three elements z(5), z(6) and z(7) constitute a three-dimensional vector to be quantized. The vector is compared to each three-dimensional vector (consisting of x1(n), x2(n) and x3(n) in the table contained in Appendix D ("PRBA Sum[5,7] VQ Codebook (7bit)"). The comparison is based on the square distance, e, which is calculated as follows:
for n =0, 1, . . . , 127. The vector from Table D which minimizes the square distance, e, is selected to produce the next 7 bits of the output vector 21a.
Next, the three elements z(9), z(10) and z(11) constitute a three-dimensional vector to be quantized. The vector is compared to each three-dimensional vector (consisting of x1(n), x2(n) and x3(n) in the table contained in Appendix E ("PRBA Dif[1,3] VQ Codebook (8-bit)"). The comparison is based on the square distance, e, which is calculated as follows:
for n=0,1, . . . , 255. The vector from Table E which minimizes the square distance, e, is selected to produce the next 8 bits of the output vector 21a.
Finally, the four elements z(12), z(13), z(14) and z(15) constitute a four-dimensional vector to be quantized. The vector is compared to each four-dimensional vector (consisting of x1(n), x2(n), x3(n) and x4(n) in the table contained in Table F ("PRBA Dif[4,7] VQ Codebook (6-bit)"). The comparison is based on the square distance, e, which is calculated as: ##EQU5## for n=0,1, . . . , 63. The vector from Table F which minimizes the square distance, e, is selected to produce the last 6 bits of the output vector 21a.
The HOC vectors are quantized similarly to the PRBA vectors. First, for each of the four frequency blocks, the corresponding pair of HOC vectors from the two subframes are combined using a sum-difference transformation 18 that produces a sum and difference vector 19 for each frequency block.
The sum/difference operation is performed separately for each frequency block on the two HOC vectors 11a and 11b, referred to as x and y respectively, to produce a vector, Z.sub.m : ##EQU6## where B.sub.m0 and B.sub.m1 are the lengths of the mth frequency block for, respectively, subframes zero and one, as set forth in Table O, and z is determined for each frequency block (i.e., m equals 0 to 3). The J+K element sum and difference vectors z.sub.m are combined for all four frequency blocks (m equals 0 to 3) to form the HOC sum/difference vector 19.
Due to the variable size of each HOC vector, the sum and difference vectors also have variable, and possibly different, lengths. This is handled in the vector quantization step by ignoring any elements beyond the first four elements of each vector. The remaining elements are vector quantized using seven bits for the sum vector and three bits for the difference vector. After vector quantization is performed, the original sum-difference transformation is reversed on the quantized sum and difference vectors. Since this process is applied to all four frequency blocks a total of forty (4* (7+3)) bits are used to vector quantize the HOC vectors corresponding to both subframes.
The quantization of the HOC sum and difference vectors 19 is performed separately on all four frequency blocks by the HOC split-vector quantizer 20b. First, the vector z.sub.m representing the mth frequency block is separated and compared against each candidate vector in the corresponding sum and difference codebooks contained in the Appendices. A codebook is identified based on the frequency block to which it corresponds and whether it is a sum or difference code. Thus, the "HOC Sum0 VQ Codebook (7-bit)" of Table G represents the sum codebook for frequency block 0. The other codebooks are Table H ("HOC Dif0 VQ Codebook (3-bit)"), Table I ("HOC Sum1 VQ Codebook (7-bit)"), Table J ("HOC Dif1 VQ Codebook (3-bit)"), Table K ("HOC Sum2 VQ Codebook (7-bit)"), Table L ("HOC Dif2 VQ Codebook (3-bit)"), Table M ("HOC Sum2 VQ Codebook (7-bit)"), and Table N ("HOC Dif3 VQ Codebook (3-bit)"). The comparison of the vector z.sub.m for each frequency block with each candidate vector from the corresponding sum codebooks is based upon the square distance, e1.sub.n for each candidate sum vector (consisting of x1(n), x2(n), x3(n) and x4(n)) which is calculated as: ##EQU7## and the square distance e2.sub.m for each candidate difference vector (consisting of x1(n), x2(n), x3(n) and x4(n)), which is calculated as: ##EQU8## where J and K are computed as described above.
The index n of the candidate sum vector from the corresponding sum notebook which minimizes the square distance e1.sub.n is represented with seven bits and the index m of the candidate difference vector which minimizes the square distance e2.sub.m is represented with three bits. These ten bits are combined from all four frequency blocks to form the 40 HOC output bits 21b.
Block 22 multiplexes the quantized PRBA vectors 21a, the quantized mean 21b, and the quantized mean bits 21c to produce output bits 23. These bits 23 are the final output bits of the dual-subframe magnitude quantizer and are also supplied to the feedback portion of the quantizer.
Block 24 of the feedback portion of the dual-subframe quantizer represents the inverse of the functions performed in the superblock labeled Q in the drawing. Block 24 produces estimated values 25a and 25b of D.sub.1 (1) and D.sub.1 (0) (8a and 8b) in response to the quantized bits 23. These estimates would equal D.sub.1 (1) and D.sub.1 (0) in the absence of quantization error in the superblock labeled Q.
Block 26 adds a scaled prediction value 33a, which equals 0.8* P.sub.1 (l), to the estimate of D.sub.1 (l) 25a to produce an estimate M.sub.1 (1) 27. Block 28 time-delays the estimate M.sub.1 (1) 27 by one frame (40 ms) to produce the estimate M.sub.1 (-1) 29.
A predictor block 30 then interpolates the estimated magnitudes and resamples them to produce L.sub.1 estimated magnitudes after which the mean value of the estimated magnitudes is subtracted from each of the L.sub.1 estimated magnitudes to produce the P.sub.1 (1) output 31a. Next, the input estimated magnitudes are interpolated and resampled to produce L.sub.0 estimated magnitudes after which the mean value of the estimated magnitudes is subtracted from each of the L.sub.0 estimated magnitudes to produce the P.sub.1 (0) output 31b.
Block 32a multiplies each magnitude in P.sub.1 (l) 31a by 0.8 to produce the output vector 33a which is used in the feedback element combiner block 7a. Likewise, block 32b multiplies each magnitude in P.sub.1 (1) 31b by 0.8 to produce the output vector 33b which is used in the feedback element combiner block 7b. The output of this process is the quantized magnitude output bits 23, which form the encoder spectral bits for the current frame.
Experimentation has shown that the PRBA and HOC sum vectors are typically more sensitive to bit errors than the corresponding difference vectors. In addition, the PRBA sum vector is typically more sensitive than the HOC sum vector. These relative sensitivities are employed in a prioritization scheme which orders the bits according to their relative sensitivity to bit errors. Generally, the most significant fundamental bits and average gain bits are followed by the PRBA sum bits and the HOC sum bits, and these are followed by the PRBA difference bits and HOC difference bits, followed by any remaining bits. Prioritization is followed by FEC encoding and interleaving to form the encoder output bit stream. FEC encoding may employ block codes or convolution codes. However, in the described embodiment, one [24,12] extended Golay code protects the 12 highest priority (i.e., the most sensitive) bits, three [23,12] Golay codes protect the 36 next highest priority bits and two [14,11] Hamming codes protect the 22 next highest priority bits. The remaining 33 bits per frame are unprotected.
The corresponding decoder is designed to reproduce high quality speech from the encoded bit stream after it is transmitted and received across the channel. The decoder first deinterleaves each frame and performs error correction decoding to correct and/or detect certain likely bit error patterns. To achieve adequate performance over the mobile communications channel, all error correction codes are typically decoded up to their full error correction capability. Next, the FEC decoded bits are used by the decoder to reassemble the quantization bits for the frame from which the model parameters representing the two subframes within the frame are reconstructed.
The AMBE.RTM. decoder uses the reconstructed log spectral magnitudes to synthesize a set of phases which are used by the voiced synthesizer to produce natural sounding speech. The use of synthesized phase information significantly lowers the transmitted data rate, relative to a system which directly transmits this information or its equivalent between the encoder and decoder. The decoder then applies spectral enhancement to the reconstructed spectral magnitudes in order to improve the perceived quality of the speech signal. The decoder further checks for bit errors and smooths the reconstructed parameters if the local estimated channel conditions indicate the presence of possible uncorrectable bit errors. The enhanced and smoothed model parameters (fundamental frequency, V/UV decisions, spectral magnitudes and synthesized phases) are used in speech synthesis. In general, the decoder performs the procedures illustrated in FIGS. 5 and 7, but in reverse.
The reconstructed parameters form the input to the decoder's speech synthesis algorithm which interpolates successive frames of model parameters into smooth segments of speech. The synthesis algorithm uses a set of harmonic oscillators (or an FFT equivalent at high frequencies) to synthesize the voiced speech. This is added to the output of a weighted overlap-add algorithm to synthesize the unvoiced speech. The sums form the synthesized speech signal which is output to a D-to-A converter for playback over a speaker. While this synthesized speech signal may not be close to the original on a sample-by-sample basis, it is perceived as the same by a human listener.
Other embodiments are within the scope of the following claims.
Claims
Certificate of Correction present
The claims shown below may not include correction changes. Use the corrected PDF for authoritative text.
What is claimed is:
1. A method of encoding speech into a frame of bits, the method including: digitizing a speech signal into a sequence of digital speech samples; dividing the digital speech samples into a sequence of subframes, each of the subframes including multiple digital speech samples; estimating a set of speech model parameters for each subframe, wherein the speech model parameters include a set of spectral magnitude parameters that represent spectral magnitude information for the subframe; combining consecutive subframes from the sequence of subframes into a frame; jointly quantizing the spectral magnitude parameters from the consecutive subframes of the frame to produce a set of encoder spectral bits, wherein: the joint quantization includes forming predicted spectral magnitude parameters from quantized spectral magnitude parameters from a previous subframe; a subframe of the frame includes a number of spectral magnitude parameters that may vary from a number of spectral magnitude parameters in the previous subframe; and the joint quantization accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe; and including the encoder spectral bits in a frame of bits.
2. The method of claim 1, wherein the joint quantization comprises: computing residual parameters as the difference between the spectral magnitude parameters and the predicted spectral magnitude parameters; combining the residual parameters from the consecutive subframes within the frame; and quantizing the combined residual parameters into a set of encoder spectral bits.
3. The method of claim 1, wherein the spectral magnitude parameters correspond to a frequency-domain representation of a spectral envelope of the subframe.
4. The method of claim 1, wherein the number of spectral magnitude parameters in the subframe of the frame may vary from a number of spectral magnitude parameters in a second subframe of the frame; and the joint quantization accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the second subframe of the frame.
5. The method of claim 4, wherein the joint quantization accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint quantization.
6. The method of claim 1, wherein the joint quantization accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe by interpolating and resampling spectral magnitude parameters for the previous subframe and using the interpolated and resampled spectral magnitude parameters in forming the predicted spectral magnitude parameters.
7. The method of claim 1, wherein the joint quantization accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint quantization.
8. A method of encoding speech into a frame of bits, the method including: digitizing a speech signal into a sequence of digital speech samples; dividing the digital speech samples into a sequence of subframes, each of the subframes including multiple digital speech samples; estimating a set of speech model parameters for each subframe, wherein the speech model parameters include a set of spectral magnitude parameters that represent spectral information for the subframe; combining consecutive subframes from the sequence of subframes into a frame; jointly quantizing the spectral magnitude parameters from the consecutive subframes of the frame to produce a set of encoder spectral bits, wherein the joint quantization includes forming predicted spectral magnitude parameters from quantized spectral magnitude parameters from a previous frame; and including the encoder spectral bits in a frame of bits; wherein the joint quantization comprises: computing residual parameters as the difference between the spectral magnitude parameters and the predicted spectral magnitude parameters; combining the residual parameters from the consecutive subframes within the frame; and quantizing the combined residual parameters into a set of encoder spectral bits; and combining the residual parameters from the consecutive subframes within the frame comprises: dividing the residual parameters from each of the subframes into frequency blocks; performing a linear transformation on the residual parameters within each frequency block to produce a set of transformed residual coefficients for each subframe; grouping a minority of the transformed residual coefficients from the frequency blocks for each subframe into a prediction residual block average (PRBA) vector for the subframe; grouping the remaining transformed residual coefficients for each frequency block of each subframe into a higher order coefficient (HOC) vector for the frequency block; transforming the PRBA vectors to produce a transformed PRBA vector for each subframe; combining the transformed PRBA vectors for the subframes of the frame by computing generalized sum and difference vectors from the transformed PRBA vectors; and combining the HOC vectors within each frequency block for the subframes of the frame by computing generalized sum and difference vectors from the HOC vectors for each frequency block.
9. The method of claim 1, 2 or 8, further comprising producing additional encoder bits by quantizing additional speech model parameters other than the spectral magnitude parameters.
10. The method of claim 9, wherein the additional speech model parameters include parameters representative of a fundamental frequency and parameters representative of a voicing state.
11. The method of claim 1, 2 or 8, wherein the frame of bits includes redundant error control bits protecting at least some of the encoder spectral bits.
12. The method of claim 1, 2 or 8, wherein the spectral magnitude parameters represent log spectral magnitudes estimated for a Multi-Band Excitation (MBE) speech model.
13. The method of claim 12, wherein the spectral magnitude parameters are estimated from a computed spectrum in a manner which is independent of a voicing state.
14. The method of claim 2 or 8, wherein the predicted spectral magnitude parameters are formed by applying a gain of less than unity to a linear interpolation of quantized spectral magnitudes from a last subframe in a previous frame.
15. The method of claim 8, wherein the transformed residual coefficients are computed for each of the frequency blocks using a Discrete Cosine Transform (DCT) followed by a linear two by two transform on two lowest order DCT coefficients.
16. The method of claim 15, wherein the length of each frequency block is approximately proportional to a number of spectral magnitude parameters within the subframe.
17. The method of claim 2 or 8, wherein quantizing the combined residual parameters includes using at least one vector quantizer.
18. The method of claim 8, wherein quantizing the combined residual parameters includes applying vector quantization to all or part of the generalized sum and difference vectors computed from the transformed PRBA vectors and applying vector quantization to all or part of the generalized sum and difference vectors computed from the HOC vectors.
19. The method of claim 18, wherein the frame includes two consecutive subframes from the sequence of subframes.
20. A speech encoder for encoding speech into a frame of bits, the encoder including: means for digitizing a speech signal into a sequence of digital speech samples; means for dividing the digital speech samples into a sequence of subframes, each of the subframes including multiple digital speech samples; means for estimating a set of speech model parameters for each subframe, wherein the speech model parameters include a set of spectral magnitude parameters that represent spectral magnitude information for the subframe; means for combining consecutive subframes from the sequence of subframes into a frame; means for jointly quantizing the spectral magnitude parameters from the consecutive subframes of the frame to produce a set of encoder spectral bits, wherein: the means for jointly quantizing forms predicted spectral magnitude parameters from quantized spectral magnitude parameters from a previous subframe; a subframe of the frame includes a number of spectral magnitude parameters that may vary from a number of spectral magnitude parameters in the previous subframe; and the means for jointly quantizing accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe; and means for forming a frame of bits including the encoder spectral bits.
21. The speech encoder of claim 20, wherein the spectral magnitude parameters correspond to a frequency-domain representation of a spectral envelope of the subframe.
22. The speech encoder of claim 20, wherein the number of spectral magnitude parameters in the subframe of the frame may vary from a number of spectral magnitude parameters in a second subframe of the frame; and the means for jointly quantizing accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the second subframe of the frame.
23. The speech encoder of claim 22, wherein the means for jointly quantizing accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint quantization.
24. The speech encoder of claim 20, wherein the means for jointly quantizing accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe by interpolating and resampling spectral magnitude parameters for the previous subframe and using the interpolated and resampled spectral magnitude parameters in forming the predicted spectral magnitude parameters.
25. The speech encoder of claim 20, wherein the means for jointly quantizing accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint quantization.
26. A method of decoding speech from a frame of bits, the method comprising: extracting decoder spectral bits from the frame of bits; using the decoder spectral bits to jointly reconstruct spectral magnitude parameters for consecutive subframes within a frame of speech, wherein the joint reconstruction includes: inverse quantizing the decoder spectral bits to reconstruct a set of combined residual parameters for the frame from which separate residual parameters for each of the subframes are computed; forming predicted spectral magnitude parameters from reconstructed spectral magnitude parameters from a previous subframe; and adding the separate residual parameters to the predicted spectral magnitude parameters to form the reconstructed spectral magnitude parameters for each subframe within the frame; wherein a subframe of the frame includes a number of spectral magnitude parameters that may vary from a number of spectral magnitude parameters in the previous subframe; and the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe; and synthesizing digital speech samples for each subframe within the frame of speech using speech model parameters which include some or all of the reconstructed voiced/unvoiced metrics and some or all of the reconstructed spectral magnitude parameters for the subframe.
27. The method of claim 26, wherein the spectral magnitude parameters correspond to a frequency-domain representation of a spectral envelope of the subframe.
28. The method of claim 26, wherein the number of spectral magnitude parameters in the subframe of the frame may vary from a number of spectral magnitude parameters in a second subframe of the frame; and the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the second subframe of the frame.
29. The method of claim 28, wherein the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint reconstruction.
30. The method of claim 26, wherein the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe by interpolating and resampling spectral magnitude parameters for the previous subframe and using the interpolated and resampled spectral magnitude parameters in forming the predicted spectral magnitude parameters.
31. The method of claim 26, wherein the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint reconstruction.
32. A method of decoding speech from a frame of bits, the method comprising: extracting decoder spectral bits from the frame of bits; using the decoder spectral bits to jointly reconstruct spectral magnitude parameters for consecutive subframes within a frame of speech, wherein the joint reconstruction includes; inverse quantizing the decoder spectral bits to reconstruct a set of combined residual parameters for the frame from which separate residual parameters for each of the subframes are computed; forming predicted spectral magnitude parameters from reconstructed spectral magnitude parameters from a previous frame; and adding the separate residual parameters to the predicted spectral magnitude parameters to form the reconstructed spectral magnitude parameters for each subframe within the frame; and synthesizing digital speech samples for each subframe within the frame of speech using speech model parameters which include some or all of the reconstructed spectral magnitude parameters for the subframe; wherein the computing of the separate residual parameters for each subframe from the combined residual parameters for the frame comprises: dividing each subframe into frequency blocks; separating the combined residual parameters for the frame into generalized sum and difference vectors representing transformed PRBA vectors combined across the subframes of the frame, and into generalized sum and difference vectors representing HOC vectors for the frequency blocks combined across the subframes of the frame; computing PRBA vectors for each subframe from the generalized sum and difference vectors representing the transformed PRBA vectors; computing HOC vectors for each subframe from the generalized sum and difference vectors representing the HOC vectors for each of the frequency blocks; combining the PRBA vector and the HOC vectors for each of the frequency blocks to form transformed residual coefficients for each of the subframes; and performing an inverse transformation on the transformed residual coefficients to produce the separate residual parameters for each subframe of the frame.
33. The method of claim 26, or 32, wherein the frame of bits includes other decoder bits in addition to the decoder spectral bits, wherein the other decoder bits are representative of speech model parameters other than the spectral magnitude parameters.
34. The method of claim 33, wherein the speech model parameters include parameters representative of a fundamental frequency and parameters representative of a voicing state.
35. The method of claim 26 or 32, wherein the reconstructed spectral magnitude parameters represent log spectral magnitudes used in a Multi-Band Excitation (MBE) speech model.
36. The method of claim 26 or 32, wherein the frame of bits includes redundant error control bits protecting at least some of the decoder spectral bits.
37. The method of claim 26 or 32, wherein the synthesizing of speech for each subframe includes computing a set of phase parameters from the reconstructed spectral magnitude parameters.
38. The method of claim 26 or 32, wherein the predicted spectral magnitude parameters are formed by applying a gain of less than unity to a linear interpolation of quantized spectral magnitudes from a last subframe of a previous frame.
39. The method of claim 32, wherein the separate residual parameters are computed from the transformed residual coefficients by performing on each of the frequency blocks an inverse linear two by two transform on the two lowest order transformed residual coefficients within the frequency block and then performing an Inverse Discrete Cosine Transform (IDCT) over all the transformed residual coefficients within the frequency block.
40. The method of claim 39, wherein four of the frequency blocks are used per subframe and wherein the length of each frequency block is approximately proportional to a number of spectral magnitude parameters within the subframe.
41. The method of claim 26 or 32, wherein the inverse quantization to reconstruct a set of combined residual parameters for the frame includes using inverse vector quantization applied to one or more vectors.
42. A decoder for decoding speech from a frame of bits, the decoder including: means for extracting decoder spectral bits from the frame of bits; means for using the decoder spectral bits to jointly reconstruct spectral magnitude parameters for consecutive subframes within a frame of speech, wherein the joint reconstruction includes: inverse quantizing the decoder spectral bits to reconstruct a set of combined residual parameters for the frame from which separate residual parameters for each of the subframes are computed; forming predicted spectral magnitude parameters from reconstructed spectral magnitude parameters from a previous subframe; and adding the separate residual parameters to the predicted spectral magnitude parameters to form the reconstructed spectral magnitude parameters for each subframe within the frame; wherein a subframe of the frame includes a number of spectral magnitude parameters that may vary from a number of spectral magnitude parameters in the previous subframe; and the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe; and means for synthesizing digital speech samples for each subframe within the frame of speech using speech model parameters which include some or all of the reconstructed spectral magnitude parameters for the subframe.
43. The method of claim 42, wherein the speech level parameter for each subframe is estimated as a mean of a set of spectral magnitude parameters computed for each subframe plus an offset.
44. The method of claim 43, wherein the spectral magnitude parameters represent log spectral magnitudes estimated for a Multi-Band Excitation (MBE) speech model.
45. The method of claim 43, wherein the offset is dependent on a number of spectral magnitude parameters in the frame.
46. The decoder of claim 42, wherein the spectral magnitude parameters correspond to a frequency-domain representation of a spectral envelope of the subframe.
47. The decoder of claim 42, wherein the number of spectral magnitude parameters in the subframe of the frame may vary from a number of spectral magnitude parameters in a second subframe of the frame; and the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the second subframe of the frame.
48. The decoder of claim 47, wherein the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint reconstruction.
49. The decoder of claim 42, wherein the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in the subframe of the frame and the number of spectral magnitude parameters in the previous subframe by interpolating and resampling spectral magnitude parameters for the previous subframe and using the interpolated and resampled spectral magnitude parameters in forming the predicted spectral magnitude parameters.
50. The decoder of claim 42, wherein the joint reconstruction accounts for any variation between the number of spectral magnitude parameters in a subframe of the frame and the number of spectral magnitude parameters in a second subframe of the frame by transforming the spectral magnitude parameters for the two subframes to produce one or more output vectors and limiting the number of elements within each output vector that are used in the joint reconstruction.
51. A method of encoding a level of speech into a frame of bits, the method comprising: digitizing a speech signal into a sequence of digital speech samples; dividing the digital speech samples into a sequence of subframes, each of the subframes including multiple digital speech samples; estimating a speech level parameter for each of the subframes, wherein the speech level parameter is representative of the amplitude of the digital speech samples comprising the subframe; combining a plurality of consecutive subframes from the sequence of subframes into a frame; jointly quantizing the speech level parameters from the plurality of consecutive subframes within the frame, characterized in that the joint quantization includes computing and quantizing an average level parameter by combining the speech level parameters over the subframes within the frame, and computing and quantizing a difference level vector between the speech level parameters for each subframe within the frame and the average level parameter; and including quantized bits representative of the average level parameter and the difference level vector in a frame of bits.
52. The method of claim 51 or 43, wherein the difference level vector is quantized using vector quantization.
53. The method of claim 51 or 43, wherein the frame of bits includes error control bits used to protect some or all of the quantized bits representative of the average level parameter and the difference level vector.
54. The method of claim 51, wherein the spectral magnitude parameters correspond to a frequency-domain representation of a spectral envelope of the subframe.
Patent Citations (45)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US3706929 | 1972-12-01 | Robinson et al. | |
| US3975587 | 1976-08-01 | Dunn et al. | |
| US3982070 | 1976-09-01 | Flanagan | |
| US4091237 | 1978-05-01 | Wolnowsky et al. | |
| US4422459 | 1983-12-01 | Simson | |
| US4583549 | 1986-04-01 | Manoli | |
| US4618982 | 1986-10-01 | Horvath et al. | |
| US4622680 | 1986-11-01 | Zinser | |
| US4720861 | 1988-01-01 | Bertrand | |
| US4797926 | 1989-01-01 | Bronson et al. | |
| US4821119 | 1989-04-01 | Gharavi | |
| US4879748 | 1989-11-01 | Picone et al. | |
| US4885790 | 1989-12-01 | McAulay et al. | |
| US4905288 | 1990-02-01 | Gerson et al. | |
| US4979110 | 1990-12-01 | Albrecht et al. | |
| US5023910 | 1991-06-01 | Thomson | |
| US5036515 | 1991-07-01 | Freeburg | |
| US5054072 | 1991-10-01 | McAulay et al. | |
| US5067158 | 1991-11-01 | Arjmad | |
| US5081681 | 1992-01-01 | Hardwick et al. | |
| US5091944 | 1992-02-01 | Takahasi | |
| US5095392 | 1992-03-01 | Shimazaki et al. | |
| US5113448 | 1992-05-01 | Nomura et al. | |
| US5195166 | 1993-03-01 | Hardwick et al. | |
| US5216747 | 1993-06-01 | Hardwick et al. | |
| US5226084 | 1993-07-01 | Hardwick et al. | |
| US5226108 | 1993-07-01 | Hardwick et al. | |
| US5247579 | 1993-09-01 | Hardwick et al. | |
| US5265167 | 1993-11-01 | Akamine et al. | |
| US5307441 | 1994-04-01 | Tzeng | |
| US5517511 | 1996-05-01 | Hardwick et al. | |
| US5596659 | 1997-01-01 | Normile et al. | |
| US5630011 | 1997-05-01 | Lim et al. | |
| US5664053 | 1997-09-01 | Laflamme et al. | |
| US5696873 | 1997-12-01 | Bartkowiak | |
| US5704003 | 1997-12-01 | Kleijn et al. | |
| EPX123456 | 1984-10-01 | ||
| EPX154381 | 1985-09-01 | ||
| EPX422232A1 | 1991-04-01 | ||
| EPX577488A1 | 1994-01-01 | ||
| WOX92/05539 | 1992-04-01 | ||
| WOXWO92/10830 | 1992-06-01 | ||
| WOX92/10830 | 1992-06-01 | ||
| WOXWO94/12932 | 1994-06-01 | ||
| WOXWO94/12972 | 1994-06-01 |
Non-Patent Literature (38)
- Digital Speech Processing, Synthesis, and Recognition by Sadaoki Furui, p62, p135, 1989
- Almeida et al., "Harmonic Coding: A Low Bit-Rate, Good-Quality Speech Coding Technique," IEEE (1982), pp. 1664-1667.
- Almeida, et al. "Variable-Frequency Synthesis: An Improved Harmonic Coding Sheme", ICASSP (1984), pp. 27.5.1-27.5.4.
- Atungsiri et al., "Error Detection and Control for the Parametric Information in CELP Coders", IEEE (1990), pp. 229-232.
- Brandstein et al., "A Real-Time Implementation of the Improved MBE Speech Coder", IEEE (1900), pp. 5-8
- Campbell et al., "The New 4800 bps Voice Coding Standard", Mil Speeh Tech Conference (Nov. 1989), pp. 64-70.
- Chen et al., "Real-Time Vector APC Speech Coding at 4800 bps with Adaptive Postifiltering", Proc. ICASSP (1987), pp. 2185-2188.
- Cox et al., "Subband Speech Coding and Matched Convolutional Channel Coding for Mobile Radio Channels," IEEE Trans. Signal Proc., vol. 39, No. 8 (Aug. 1991), pp. 1717-1731.
- Digital Voice Systems, Inc., "INMARSAT-M Voice Codec", Version 1.9 (Nov. 18, 1992), pp. 1-145.
- Digital Voice Systems, Inc., "The DVSI IMBE Speech Compression System," advertising brochure (May 12, 1993).
- Digital Voice Systems, Inc., "The DVSI IMBE Speech Coder," advertising brochure (May 12, 1993).
- Flanagan, J.L., Speech Analysis Synthesis and Perception, Springer-Verlag (1982), pp. 378-386.
- Fujimura, "An Approximation to Voice Aperiodicity", IEEE Transactions on Audio and Electroacoutics, vol. AU-16, No. 1 (Mar. 1968), pp. 68-72.
- Griffin, et al., "A High Quality 9.6 Kbps Speech Coding System", Proc. ICASSP 86, Tokyo, Japan, (Apr. 13-20, 1986), pp. 125-128.
- Griffin et al., "A New Model-Based Speech Analysis/Synthesis System", Proc. ICASSP 85, Tampa, FL (Mar. 26-29, 1985), pp. 513-516.
- Griffin, et al. "A New Pitch Detection Algorithm", Digital Signal Processing, No. 84, Elsevier Science Publishers (1984), pp. 395-399.
- Griffin et al., "Multiband Excitation Vocoder" IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 36, No. 8 (1988), pp. 1223-1235.
- Griffin, "The Multiband Excitation Vocoder", Ph.D. Thesis, M.I.T., 1987.
- Griffin et al. "Signal Estimation from Modified Short-Time Fourier Transform", IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP-32, No. 2 (Apr. 1984), pp. 236-243.
- Hardwick et al. "A 4.8 Kbps Multi-band Excitation Speech Coder, " Proceedings from ICASSP, International Conference on Acoustics, Speech and Signal Processing, New York, N.Y. (Apr. 11-14, 1988), pp. 374-377.
- Hardwick et al. "A 4.8 Kbps Multi-Band Excitation Speech Coder," Master's Thesis, M.I.T., 1988.
- Hardwick et al. "The Application of the IMBE Speech Coder to Mobile Communications," IEEE (1991), pp. 249-252.
- Heron, "A 32-Band Sub-band/Transform Coder Incorporating Vector Quantization for Dynamic Bit Allocation", IEEE (1983), pp. 1276-1279.
- Levesque et al., "A Proposed Federal Standard for Narrowband Digital Land Mobile Radio", IEEE (1990), pp. 497-501.
- Makhoul, "A Mixed-Source Model For Speech Compression and Synthesis", IEEE (1978), p. 163-166.
- Makhoul et al., "Vector Quantization in Speech Coding", Proc. IEEE (1985), pp. 1551-1588.
- Maragos et al., "Speech Nonlinearities, Modulations, and Energy Operators", IEEE (1991), pp. 421-424.
- Mazor et al., "Transform Subbands Coding With Channel Error Control", IEEE (1989), pp. 172-175.
- McAulay et al., "Mid-Rate Coding Based on a Sinusoidal Representation of Speech", Proc. IEEE (1985), pp. 945-948.
- McAulay et al., Multirate Sinusoidal Transform Coding at Rates from 2.4 Kbps to 8 Kbps., IEEE (1987), pp. 1645-1648.
- McAulay et al., "Speech Analysis/Synthesis Based on a Sinusoidal Representation," IEEE Transactions on Acoustics, Speech and Signal Processing V. 34, No. 4, (Aug. 1986), pp. 744-754.
- McCree et al., "A New Mixed Excitation LPC Vocoder", IEEE (1991), pp. 593-595.
- McCree et al., "Improving the Performance of a Mixed Excitation LPC Vocoder in Acoustic Noise", IEEE (1992), pp. 137-139.
- Rahikka et al., "CELP Coding for Land Mobile Radio Applications," Proc. ICASSP 90, Albuquerque, New Mexico, Apr. 3-6, 1990, pp. 465-468.
- Rowe et al., "A robust 2400bit/s MBE-LPC Speech Coder Incorporating Joint Source and Channel Coding," IEEE (1992), pp. 141-144.
- Secrest, et al., "Postprocessing Techniques for Voice Pitch Trackers", ICASSP, vol. 1 (1982), pp. 172-175.
- Tribolet et al., Frequency Domain Coding of Speech, IEEE Transactions on Acoustics, Speech and Signal Processing, V. ASSP-27, No. 5, pp 512-530 (Oct. 1979).
- Yu et al., "Discriminant Analysis and Supervised Vector Quantization for Continuous Speech Recognition", IEEE (1990), pp. 685-688.