Resolution-Variable Adaptive Picture Compression

The present invention relates to a resolution-variable adaptive picture compression method and the apparatus therefor, and more particularly, to a resolution-variable adaptive picture compression method in which not only the resolution of the picture being compressed is varied, but also the frame structure in compression can be controlled according to the picture element's condition, so as to enhance the picture quality for a given compression ratio.

In general, in the compression of a digitized motion picture, it is determined whether the subsequent frame in a succession of frames should be coded by an intra-mode or by an inter-mode, so that even if a complex frame is being input, the bit amount is fixed according to buffer capacity. Therefore, picture quality is degraded.

Also, since it is assumed that the picture being input in a succession is always moving continuously, even a frame sequence showing a small amount of motion change or a simple construction should be furnished with a fixed amount of bits, which is often considered unnecessary.

Moreover, since a method for compressing or enlarging the picture is not used, the picture quality may be degraded for the given bit ratio.

Accordingly, it is an object of the present invention to provide a resolution-variable adaptive picture compression method for varying the resolution of the picture being compressed and controlling the frame layout according to the picture condition, so as to enhance picture quality for a given compression ratio.

It is another object of the present invention to provide an apparatus most suitable to the resolution-variable adaptive picture compression method.

To accomplish the above object, the present invention provides a resolution-variable adaptive picture compression method comprising the steps of:

(a) changing the resolution of a motion picture, determining the difference picture between the current frame and preceding frame, and varying the frame structure according to the degree of motion in the motion picture;

(b) controlling the quantization step value when quantizing with respect to the varied structure frame, using the gradient operation result by which picture complexity can be determined; and

(c) transmitting an error compensating signal to a transmitting channel along with the difference picture quantized by the quantization step in order to compensate for the error generated while converting the difference picture signal which is quantized according to the quantization step value into the original signal and while enlarging the original signal by a predetermined ratio, to thereby perform error compensation during picture enlargement.

To accomplish another object, the present invention provides a resolution-variable adaptive picture compression apparatus comprising an encoder including a frame memory for sequentially storing the picture sequence of the applied motion pictures, a variable-length coder for compressing the picture signal read out from the frame memory and converting the same into a bit stream, and a transmission buffer for buffering the output signal of the variable-length coder, the apparatus further comprising:

difference picture operation means for scaling down the motion picture stored in the frame memory by frame at a predetermined ratio to thereby obtain the difference picture between the previously transferred picture and the scaled down picture;

mode selection means for selecting whether the current frame is compressed according to an intra-mode or inter-mode, from the difference picture;

compressing means for compressing the motion picture or the difference picture which is compressed in the mode selected by the mode selection means; and

error picture transferring means for enlarging the compressed motion picture to the original picture and obtaining and transferring the error between the compressed motion picture and the enlarged picture.

To accomplish still another object, the present invention provides a resolution-variable adaptive picture compression apparatus having a decoder including a data buffer for buffering the signal which is compressed and transferred via transfer channel and a variable-length decoder for variable-length-decoding the output signal of the data buffer, the apparatus comprising:

expansion means for expanding the output signal of the variable-length decoder according to the compression mode in the signal applied at the data buffer;

enlarging means for enlarging the output signal of the expansion means in a predetermined ratio; and

error compensation means for compensating for the error of the enlarged picture of the enlarging means.

The above object and other advantages of the present invention will become more apparent by describing in detail a preferred embodiment of the present invention with reference to the attached drawings in which:

FIG. 1 is a block diagram of an encoder for use in a resolution-variable adaptive picture compression apparatus according to the present invention;

FIG. 2 is a block diagram of a decoder for use in the resolution-variable adaptive picture compression apparatus according to the present invention;

FIG. 3 shows the variable frame structure;

FIG. 4 shows the form of subblock division for use in the operation for intra-mode or inter-mode selection;

FIG. 5 illustrates pixel positions on the complexity of an 8.times.8 block for performing the gradient operation in order to determine the quantization step value in an intra-mode coding; and

FIG. 6 is a diagram showing the relationship of the active picture elements used for picture expansion.

FIG. 1 is a block diagram showing an encoder for use in a resolution-variable adaptive picture compression apparatus.

Referring to FIG. 1, the encoder includes a first frame memory 2 for storing an applied picture sequence 1; a first picture compressor 102 for compressing the output signal of first frame memory 2 into one half its original size; a first subtracter 115 for subtracting the output signal of a second frame memory 110 (to be described later) from the output signal of first picture compressor 102; a difference picture memory 103 for storing the output signal of first subtracter 115; a mode selector 104 for selecting a mode according to the complexity of the picture output signal from difference picture memory 103 and the degree of motion between frames to thereby selectively output the output signal of first picture compressor 102 or that of difference picture memory 103; a transform coder 105 for transforming the output signal of mode selector 104 into a frequency component; a first quantizer 106 for quantizing the output signal of transform coder 105; a quantization step controller 101 for controlling the quantization step in first quantizer 106 according to the output signals of first picture compressor 102 and a transmission buffer 4 (described below); a first inverse-quantizer 107 for inversely quantizing the output signal of first quantizer 106; a first inverse-transform coder 108 for restoring the output signal of first inverse-quantizer 107 to the original signal; a mode compensator 109 for compensating the output signal of first inverse-transform coder 108; a second frame memory 110 for storing the output signal of mode compensator 109; a first picture enlarger 111 for doubling the output signal of second frame memory 110; a second subtracter 116 for subtracting the output signal of first picture enlarger 111 from the output signal of first frame memory 2; second picture compressor 112 for compressing the output signal of second subtracter 116 to one half its original size in both horizontal and vertical directions; a second quantizer 113 for quantizing the output signal of second picture compressor 112; a synchronizer 114 for synchronizing the output signal of first quantizer 106 to the output signal of second quantizer 113; a variable-length coder (VLC) 3 for variable-length-coding the output signal of synchronizer 114; and a transmission buffer 4 for buffering the output signal of VLC 3 to be output to the transmitting channel and simultaneously feeding back the same to quantization step controller 101.

FIG. 2 is a block diagram showing a decoder for use in the resolution-variable adaptive picture compression apparatus according to the present invention.

Referring to FIG. 2, the decoder includes a variable-length decoder (VLD) 6 for variable-length-decoding the output signal of a data buffer 5; a synchronization separator 202 for separating the synchronization signal from the output signal of VLD 6; a second inverse quantizer 203 for inversely quantizing the output signal of synchronization separator 202; a second inverse-transform coder 204 for restoring the output signal of second inverse quantizer 203 back to the original signal; a first adder 211 for adding the output signal of second inverse-transform coder 204 to that of a compression picture frame memory 206 (described below); a flag setter 201 for setting a flag in the output signal of synchronization separator 202; a multiplexer 205 for selectively outputting the output signal of second inverse-transform coder 204 or the output of first adder 211 according to the output signal of flag setter 201; a compression picture frame memory 206 for storing the output signal of multiplexer 205; a second picture enlarger 207 for doubling the output signal of compression picture frame 206; an enlargement error memory 208 for storing the output signal of synchronization separator 202; a third inverse-quantizer 209 for inversely quantizing the output signal of enlargement error memory 208; a third picture enlarger 210 for doubling the output signal of third inverse-quantizer 209; and a second adder 212 for adding the output signal of second picture enlarger 207 to the output signal of third picture enlarger 210 and then storing or displaying the summed result.

FIG. 3 shows the variable-frame structure. Here, reference symbol I represents an intra frame of the intra-mode, and P is a predictive frame of the inter-mode.

FIG. 4 shows how the picture block is divided into the subblocks for use in the operation for intra-mode or inter-mode selection. Here, reference symbols M and N represent the horizontal and vertical resolutions of a frame, respectively. Also, reference numerals p and q are the horizontal and vertical resolutions of each subblock, respectively.

FIG. 5 illustrates pixel positions on the complexity of an 8.times.8 block for performing the gradient operation in order to determine the quantization step value in an intra-mode coding.

FIG. 6 illustrates the relationship for each pixel that is active during picture enlargement by using the weight value thereof.

The operation of the resolution-variable adaptive picture compression apparatus will be described with reference to FIGS. 1-6.

Two basic concepts supporting the present invention are as follows:

Firstly, among the successive frames of a motion picture, those frames showing little motion should use the previously coded frame to thereby reduce the required amount of bits. Here, since it is unnecessary to obtain picture motion vectors, the apparatus can be more simplified while varying the frame structure.

Secondly, the picture size is reduced to one half its original length and width. Here, only the error component for enlargement is additionally coded and transferred for a decrease of the data amount (i.e., the total number of bits is decreased), so that the picture quality can be improved.

The resolution-variable adaptive picture compression apparatus according to the present invention comprises generally a picture compressing portion (i.e., an encoder) and a picture restoring portion (i.e., a decoder).

The operation of the encoder and decoder will now be described. Referring the encoder shown in FIG. 1, picture sequence 1 of a motion picture is stored in first frame memory 2 by frames. First picture compressor 102 compresses each frame to thereby scale down the length and width to one half the original size. At this point, to remove the interference between pictures (frames), low-pass-filtering is performed before the subsampling is performed.

First subtracter 115 obtains the difference between the output signal of first picture compressor 102 and the previously coded frame supplied by second frame memory 110, and stores the difference in difference picture memory 103.

Mode selector 104 selects which mode (among the intra-mode and inter-mode) will be used for coding the difference picture sequentially read out from difference picture memory 103, depending upon the picture complexity and the degree of motion between frames.

At this time, as for the successive intra-modes shown in FIG. 3, the minimum repetition period of the intra-mode frame is two frames, and its maximum repetition period is preferably restricted to less than 30 frames, to prevent picture quality degradation. (This period can be set arbitrarily by the system designer.) Mode selector 104 receives the output signal of first picture compressor 102 (i.e., the original picture) and the output signal of difference picture memory 103, that is, the difference picture. Here, as shown in FIG. 4, an M.times.N frame is partitioned into 64 p.times.q subblocks, and the variance of each block is obtained as follows: ##EQU1## where X(m,n) is a pixel value in a subblock, X is an average value of the subblock, and i is an integer from 1 to 64.

Mode selector 104 determines the frame to be one having little motion if the number of blocks, among the 64 blocks, having a variance obtained by Equation (1) exceeding a given threshold value is less than five. Accordingly, as to such a frame, mode selector 104 encodes the output signal of difference picture memory 103, i.e., a difference picture, in an inter-mode, and otherwise encodes the output signal of first picture compressor 102, i.e., the original picture, in an intra-mode.

Transform coder 105 is for transforming the space picture value into frequency component, generally, by a discrete cosine transform (DCT) operation.

First quantizer 106 quantizes the frequency component output from transform coder 105, wherein the quantization step size is controlled by quantization step controller 101. In the case of inter-mode coding the difference picture, the quantization step value does not change. However, in the case of intra-mode coding the original picture, the gradient operation result for the original 8.times.8 block frame is utilized for coding the original picture, as shown in FIG. 5. As for the 8.times.8 block coordinates, the gradient operation is expressed as follows:

For an 8.times.8 block, g.sub.1 and g.sub.2 are calculated by Equations (2) and (3) based on a 2.times.2 subblock. Next, horizontal and vertical gradients G.sub.H and G.sub.V, and diagonal gradients G.sub.C and G.sub.D are obtained using this result. All gradients are summed so that total gradient G is obtained for an 8.times.8 block. The G value, generally of a fixed ratio (e.g., 0.5 to 3), is multiplied by the quantization step value which is fed back to transmission buffer 4 so that human visual perception can be best complimented. The quantized signal of first quantizer 106 is supplied to synchronizer 114 for producing a bit-stream and is sent to first inverse-quantizer 107 for producing a reference signal for the next frame calculation. The output of first inverse quantizer 107 is next sent to first inverse-transform coder 108, and further to mode compensator 109, so that it is finally recorded in second frame memory 110 as a restored preceding frame signal. The preceding frame signal may be used for producing an error signal for use in compensating for the error in the picture enlargement.

The preceding frame signal is applied at first adder 115 to be used for the calculation of the succeeding frame in the manner described above. The preceding frame signal is also provided to first picture enlarger 111 to aid in the doubling of the width and length of the picture. Next, second subtracter 116 subtracts the output signal of first picture enlarger ill from the original picture signal used in encoding. The difference value from subtracter 116 is sent to second picture compressor 112 which compresses the picture into one half its original size. Next, the output signal of second picture compressor 112 is sent to and quantized by second quantizer 113. The output signal of second quantizer 113 is sent to synchronizer 114, together with the quantized signal of first quantizer 106. In synchronizer 114, the output signals of first quantizer 106 and second quantizer 113 are sequentially arranged and sent to variable-length coder 3.

Variable-length coder 3 converts the output signal of synchronizer 114 to a certain bit stream by using the Huffman coding method, so that the converted signal is output to transmission buffer 4.

Transmission buffer 4 calculates the amount of bit stream which is sent from variable-length coder 3, and determines the proper quantization step for subsequently coding the next frame, which is then, on the one hand, fed back to quantization step controller 101 and, on the other hand, sent to the transfer channel and recording medium at a fixed bit rate. Therefore, a series of encoding operations is completed.

The decoding of the encoded and transferred signal is performed as shown in FIG. 2, wherein the job sequence is opposite to the encoding.

Firstly, the bit stream coming into data buffer 5 is sent to variable-length decoder 6 which converts the bit stream into picture data that is subsequently applied at synchronization separator 202.

Synchronization separator 202 separates the picture data into the control and picture signals. The separated original picture signal comes through second inverse-quantizer 203 and second inverse-transform coder 204 and is converted into a space pixel value, which is, on the one hand, applied directly at multiplexer 205 and, on the other hand, applied at first adder 211 so as to be added to the previously decoded frame. The summed result is then sent to multiplexer 205.

Multiplexer 205 selects the output signal of second inverse-transform coder 204 or the output signal of first adder 211 according to the control signal generated by flag setter 201, so as to record the selected signal in compression picture frame memory 206.

Second picture enlarger 207 doubles the data read out from compression picture frame memory 206, with respect to picture width and length. Then, second adder 212 adds the doubled data to error data enlarged by synchronization separator 202, enlarging error memory 208, third inverse-quantizer 209 and third picture enlarger 210. Therefore, the summed result is recorded or displayed for the desired purpose, thus completing the entire decoding process.

The picture enlarging operation method just used in the present invention will be described, hereinafter, in more detail. Referring to FIG. 6, the pixel positions in the compressed picture are denoted by the letters A, B, C, D, E, F, G, H and I. Here, the pixel positions where interpolation should be performed by enlargement are denoted by reference numbers. In FIG. 6, "1, "2" and "3" pixels values are determined as follows: ##EQU3##

Once the "1," "2" and "3" values are determined, interpolation is performed repetitively while moving to the right and downward. To calculate the values of pixels "1", "2" and "3", the values of nine pixels in the compressed picture are averaged with their own weights as expressed in Equations (9) through (11) so that the nearest pixels are heavily weighted. That is, such operation is repetitively performed throughout the entire frame until the picture enlarging operation is completed.

In the resolution-variable adaptive picture compression method and apparatus as described above, it is unnecessary to calculate the motion vectors, so that the operation can be simplified, thereby attaining cost savings. Also, since the bit rate is increased with the varying of resolution, the picture quality is enhanced. Accordingly, this apparatus and method can be utilized in low-resolution application systems.

Also, the apparatus and method use a variable frame structure so as to improve picture quality in regard to a variety of pictures. Human visual perception is considered in the gradient calculation and the error signal is utilized for compensating for the error in picture enlargement, which thereby improves picture quality.