Device for generating multiple quality level bit-rates in a video encoder

[0004] This application is a continuation in part of Ser. No. 09/103,405 filed Jun. 24, 1998, now abandoned and assigned to the same assignee as that of the present invention.

[0005] The present invention relates to the video encoding standard H.263 developed by the International Telecommunication Union (ITU) for very low bit-rate multimedia telecommunication and particularly to a device for generating multiple quality level bit-rates in a H.263 video encoder.

[0006] The H.263 standard developed by the ITU (International Telecommunication Union) is a part of its H.324/H.323 recommendations for very low bit-rate multimedia telecommunication. The H.263 coding scheme which is described in “Video Coding for Very Low bit-rate Communication”, Draft ITU-Recommendation H.263, May 1996, is based on earlier schemes used in H.261 and MPEG-1/2 standards, and using a Hybrid-DPCM concept comprising a motion estimation/compensation mechanism, transform coding and quantization. Each image is divided into blocks of size 16×16 pixels (called macroblocks) and the macroblock in the current picture is predicted from the previous picture using motion estimation techniques. After the prediction, the macroblock is divided into four blocks of size 8×8 pixels. The prediction error is then transformed using the Discrete Cosine Transform (DCT) and the resulted coefficients are quantized and stored in the bitstream along with the motion parameters and other side information. The H.263 standard contains several improvements compared to earlier standards which allow a substantial reduction in the bit-rate while maintaining the same image quality. These improvements make it most suitable for very low bit-rate communication (but do not exclude it from being used in high bit-rate compression as well).

[0007] The H.263 bit-stream syntax defines the structure of the coded data from the basic block data to the entire image. The quality of the reconstructed video sequence can be controlled by changing the quantization step in the encoding process according to a pre-defined rate control mechanism. This allows a flexibility in generating a video sequence according to a desired bit-rate or image quality.

[0008] When a system is designed to transmit video content to a wide range of communication channels (video servers or video-on-demand), it is highly desired to be able to use one compressed video sequence which can accommodate all needs as opposed to keeping several versions of the compressed video sequence each compressed to a different bit-rate according to the target communication channel.

[0009] Therefore, the main object of the invention is to provide a device incorporated in a video encoder for accommodating a wide range of communication channels without storing several video bitstreams.

[0010] Another object of the invention is to provide a device used in a video encoder for supplying different quality level video bitstreams by using the same stored video sequence with different quantization step sizes.

[0011] Accordingly, the device according to the invention is used in a video encoder comprising motion estimation means providing a predicted block for each predefined block based upon estimating the motion between the predefined block of the current image and the corresponding block in the previous image, transform means for transforming a prediction error resulting from the difference between the predicted block and the predefined block into the frequency domain, and quantizing means for quantizing the coefficients of the prediction error in the frequency domain and providing the quantized coefficients to a video multiplex coding unit, wherein the quantized coefficients are de-quantized and inverse transformed to give back the prediction error and add it to the predicted block whereby the result is provided to the motion estimation means in order to get a new current predicted block. Such a device which generates, from one video sequence, multiple video bitstreams of different bit-rates and corresponding to different quality levels comprises a number n of stages corresponding each to a quality level i=1 to n, each stage comprising computing means for reducing the prediction error in accordance with the quality level i and a corresponding quantized prediction error residual QDi, and summing means for obtaining a cumulative prediction error: ${\mathrm{QD}}_{\mathrm{TOTAL}}=\sum _{j=0}^{j=i}{\mathrm{QD}}_j$

[0012] corresponding to quality level i.

[0013] wherein QD_jis the dequantized value of the quantized prediction error residual QD_j.

[0014] The invention further provides a way to efficiently store up to four compressed video sequences corresponding to up to four different quality level compression of the same video sequence.

[0015] The major advantage of the invention is to save storage space or transmission bits when this data is respectively stored as a data file or sent on a transmission line because the motion vector information are only stored once for the four different compressed video sequences. In addition, only one file (for storage) or stream (for transmission) is used as opposed to the normal case where each compressed video sequence is handled independently. One other advantage of the solution is that video decoders able to process H.263 standard bitstreams but not modified according to the new storing of data of the invention, will be however able, with no change, to reconstruct the base layer of the compressed data while discarding the rest of the sub-bitstreams it cannot decode.

[0016] The objects, features and other characteristics of the invention will become more apparent from the following detailed description with reference to the accompanying drawings in which:

[0017] FIG. 1 represents a block-diagram of a H.263 encoder of the prior art wherein a device according to the invention is used.

[0018] FIG. 2 is a schematic block-diagram illustrating the way of generating the quantized prediction error residuals corresponding to increased quality levels in the device according to the invention, and

[0019] FIG. 3 is a schematic block-diagram representing the summing means of the device according to the invention for getting a cumulative prediction error.

[0020] FIG. 4 shows the bitstream data structure according to the H.263 May 1996 draft being the support for storing four quality levels of the same video sequence according to the invention.

[0021] The H.263 encoder in which the device according to the invention is used, is based on the hybrid-DPCM scheme which is used in most of the standard video coders today. When the encoding function has to be implemented, Coding Control Unit 10 controls switching circuits 12 and 14 as illustrated in FIG. 1 which consists in a video encoder of the prior art same as FIG. 3 of the Draft ITU-Recommendation H.263, May 1996.

[0022] The encoder is composed of a transform coder 16 which transforms a prediction error found by subtracting in subtractor 18 the current macroblock received as in input from a predicted corresponding macroblock into the frequency domain by using the Discrete Cosine Transform (DCT) in which the information is represented in a compact way suitable for compression. The DCT coefficients are then quantized in quantizer 20 with many fewer bits. This quantization, which provides a quantizing index q for transform coefficients, introduces the lossy aspect of the video encoder.

[0023] The predicted macroblock used to determine the prediction error is provided by a motion estimation unit 22 which provides motion vectors v pointing to the chosen macroblock in the previous image.

[0024] The prediction error, and the motion vectors v, form the information needed for the reconstruction process in the decoder. Indeed, the prediction of the current macroblock is performed with respect to the previous reconstructed image in a similar way as is done in the decoder to avoid any mismatch. To achieve this, a complete decoder is actually implemented in the encoder loop. All the information sent by the encoder to the decoder is coded using Hufman coding which represents the bits in a compact and efficient way. The reconstruction is performed by taking the quantized and transformed prediction error and performing inverse quantization in inverse quantizer 24 and performing inverse DCT (IDCT) in inverse Transform Coder 26. Then, the macroblock predicted from the previous reconstructed image is added to the prediction error in adder 28 to form the current reconstructed block provided to motion estimation unit 22. Note that this mechanism is performed to each macroblock of the image.

[0025] The control information from Coding Control Unit 10, quantizing index q for transform coefficients and motions vectors v are then provided to the Video Multiplex Coding Unit 30.

[0026] In the encoder of FIG. 1, the quantization performed by quantizer 20 is a process in which the DCT coefficients of the prediction error PE which can have many values in a specific range are converted into other values that are chosen from a much smaller subset in that range. The number of possible values is determined by the quantization step size. This parameter determines the number of levels in the range and hence the number of possible values that the coefficients can have after quantization.

[0027] The inverse quantization performed by inverse quantizer 24 is the inverse process in which the coefficients are transformed back to the original domain with values from that domain. The reconstructed value can be different from the original due to the quantization effect. The quantization error is computed by subtracting the reconstructed coefficients from the original coefficient values, the difference representing the error introduced by the quantization process. Note that, to be able to get back to the original domain of values, the inverse quantization step size must be identical to the quantization step size.

[0028] An essential feature of the invention is to reduce the quantization error by using several quantization stages wherein the error in each stage is reduced by decreasing the quantization step size (since the coefficients are divided by this step size). This is illustrated in FIG. 2.

[0029] In FIG. 2, the coefficients of the prediction error PE (after DCT transformation) are quantized in quantizer 40 with the quantization step value Q₀. Therefore, QD₀is a coarse representation of the DCT coefficients which would normally be used in the encoder without the invention or the quantized prediction error in the base level.

[0030] At each level i of the mechanism according to the invention, E_irepresents the prediction error residual as being the difference between the prediction error residual E_i−1of level i−1 and the de-quantized value of the quantized prediction error residual QD_i−1by inverse quantizer IQ_i−1. Thus, the de-quantized value of QD₀obtained from inverse quantizer 42 (having an inverse quantization step size IQ₀equal to quantization step size Q₀) is subtracted from prediction error PE in subtractor 44 to get prediction error residual E₁. Then, residual E₁is quantized in quantizer 46 having a quantization step size Q₁, smaller than Q₀, in order to obtain QD₁which is the quantized prediction error residual in level i=1. The de-quantized value of QD₁obtained from inverse quantized 48 (with an inverse quantization step size IQ₁equal to Q₁) is subtracted from residual E₁in subtractor 50 to get residual E₂. E₂is then quantized in quantizer 52 having a quantization step Q₂smaller than Q₁in order to obtain QD₂. The de-quantized value of QD₂obtained at the output of inverse quantizer 54 (having an inverse quantization step size IQ₂equal to Q₂) is subtracted from residual E₂in subtractor 56 to get residual E₃. Finally, residual E₃is quantized in quantizer 58 (having a quantization step value Q₃smaller than Q₂to obtain QD₃.

[0031] As can be seen in FIG. 2, at each stage, the quantized prediction error residual is computed from the prediction error residual of the previous stage. No information is lost between stages since, each time, the error between the input signal and the quantized signal is completely transferred to the next stage. The lossy nature of the compression (quantization error) is introduced only at the output of the final stage where the smallest quantization step is used and so, the error is minimal. Note that since the more stages there are, the smaller the prediction error will be, more than four stages could be used to be closer to the original prediction error.

[0032] Since the residual from the previous stage is quantized using a finer quantizer, the DCT coefficients can be rebuilt in an accumulative mechanism illustrated in FIG. 3. In such a mechanism, the de-quantized values of QD₀to QD₃by respectively inverse quantizers IQ₀, IQ₁, IQ₂, IQ₃, are summed in summing circuit 60 to get a value QD_TOTALaccording to ${\mathrm{QD}}_{\mathrm{TOTAL}}=\sum _{j=0}^{j=i}{\mathrm{QD}}_j$

[0033] wherein i can take any value 0, 1, 2, or 3. The quantized value QD_TOTALis then de-quantized in inverse transform coder 26 as usual.

[0034] Thus, the rebuilding mechanism can stop at any level i if the desired bit-rate or picture quality has been achieved (according to the restrictions of the communication channel or the decoder at the other end). In the present case, QD_TOTALcan be obtained by using QD₀alone, or QD₀+QD₁, or QD₀+QD₁+QD₂or QD₀+QD₁+QD₂+QD₃. In this way, up to four versions of the same compressed video sequence can be produced from one bitstream. It must be noted that QD₀being the base prediction error and QD₁, QD₂, QD₃being components of the errors, when these components are added to QD₀, the result is closer to the original value before quantization. In other words, the greater is i in the above formula giving QD_TOTAL, the better will be the quality of the video image.

[0035] In the preferred embodiment of the invention, up to four different compressed video sequences corresponding to up to four different quantization step sizes can be stored as a unique video sequence. This is done by using the existing bitstream and sub-bitstream structure of a compressed video sequence as defined by the H.263 standard but applied to specific data. This data structure can be used to store efficiently this multi-level video sequence as a data file or to transfer it on a transmission line. With this embodiment, one can use the ability of the H.263 coder to handle up to four sub-bitstreams without the need to change the bit-stream structure. Such a structure is described in “Video Coding for very Low bit-rate Communication”, ITU-T Recommendation H.263, May 1996,

[0036] The data structure defined by H.263 standard is illustrated in FIG. 4. The picture layer is the upper layer the other data structures ‘Group Of Blocks layer’, ‘Macroblock Layer’ and ‘Block layer’ are embedded one in the other for providing all the video pixel block information for all the blocks of a picture. One bit (CPM) indicates the usage of sub-bitstreams. Then, two bits which are present if the CPM bit if activated, allow up to four independent sub-bitstreams to be defined within the total H.263 bitstream. Picture Sub Bitstream Indicator (PSBI) are two bits which are present only if CPM is indicated. They indicate that the picture header and all following information until the next picture of Group of Blocks (GOB) headers belong to the same sub-bitsream. Group Sub Bitstream Indicator (GSBI) are two bits which are present only if CPM is activated. They indicate that the GOB header and all following information until the next picture or GOB start code belongs to the same sub-bitstream. This mode is provided to transfer up to four independent bitstreams: annexe C in H.263 standard draft, May 1996, states that ‘The information in each individual bitstream is also completely independent from the information in the other bitstreams’.

[0037] The video encoder of the preferred embodiment of the invention generates the multi-quality level information and stores up to four quality level encoded video sequences of the same video sequence as four sub-bitstreams in a unique H.263 bitstream. For the preferred embodiment, the use of CPM bit in the H.263 bitstream means that there are multiple quality level information included as sub-bitstreams. PSBI qualifies which quality-level it is. The GSBI field will indicate to which quality-level the following group of blocks belong. The Macroblock Layer and Block Layer have the same use than with the usual H.263 bitstream, these are the macroblock and block encoded information.

[0038] Note that the motion vectors information which accompany each macroblock should be stored only in the base bitstream. This information is used by all the other sub-bitstreams. Quantization information is stored in the usual way for each sub bit-stream (if needed, according to the rate-control mechanism used by the coder). The only macroblock information included in the sub-bitstreams apart for the base one are the quantized residual values, so the overhead of using the sub-bitstream structure and the proposed scaleability mechanism is minimal.

[0039] The decoder of the preferred embodiment receives the bitstream as described and is able to use some or all of the bitstreams to reconstruct the video sequence at a quality (or bit rate) which matches the decoder needs. If, according to the bandwidth of the communication channel used, only a subset of the bitstream can be sent by the video encoder through the communication channel, the video decoder will reconstruct the video sequence at a quality which will match the channel bandwidth.