Video signal encoding apparatus

[0002] The present invention relates to a video signal encoding apparatus for enlarging a video signal through a coding process.

[0003] In encoding images, such situations frequently occur in which it is required to enlarge moving pictures. When an input moving picture is encoded after it is enlarged, we consider that the picture is first enlarged in a spatial domain and then encoded. In this case, such processes as discrete cosine transform (DCT) are performed subsequent to the enlargement of the picture. Accordingly, the volume of calculation required for encoding increases as compared with that where the image is not enlarged.

[0004] A video signal encoding apparatus capable of encoding an input moving picture after enlarging it, while avoiding the problem just described of increase in the volume of calculation, is disclosed in Japanese Non-examined Publication No. H10-308946. An example of the video signal encoding apparatus performing encoding after enlargement of an input moving picture is shown in FIG. 5. This apparatus is described in the case of doubling an input picture in the vertical direction.

[0005] Block converter 502 divides input image 501 into 8×4 blocks formed of eight horizontal pixels and four vertical lines. Inter-frame prediction unit 503 performs inter-frame prediction with the use of the divided 8×4 blocks and reference picture generated by reference picture replacing unit 508 and generates 8×4 differential blocks and motion vectors. When inter-frame prediction is not performed, 8×4 blocks generated in block converter 502 comes to be 8×4 differential blocks.

[0006] Then, 8×4 DCT coefficient blocks are generated from 8×4 differential blocks by DCT unit 504. Quantizer 505 quantizes 8×4 DCT coefficient blocks to generate 8×4 quantization coefficient blocks. Inverse quantizer 506 inverse-quantizes 8×4 quantization coefficient blocks to generate 8×4 inverse quantization coefficients blocks.

[0007] Further, inverse discrete cosine transform (IDCT) unit 508 performs IDCT processing on 8×4 inverse quantization coefficient blocks to generate 8×4 decoded blocks. Reference picture replacing unit 508 updates the reference picture with the use of 8×4 decoded blocks and motion vectors generated in inter-frame prediction unit 503.

[0008] Vertical doubling unit 509 enlarges 8×4 quantization coefficient blocks, obtained through quantization processing, in a frequency domain to generate 8×8 quantization coefficient blocks. Thus, the size in the vertical direction of the block becomes double that of the input picture.

[0009] Variable length coding (VLC) unit 510 performs variable length coding processing on 8×8 quantization coefficient blocks to generate compressed data 511.

[0010] By subjecting data first to DCT and quantization processing and then encoding the obtained data in a frequency domain as described in the above example, the amount of data calculation required for the picture coding is reduced from that where the input picture is first enlarged and then encoded.

[0011] In the example described above, the size of the block subjected to IDCT processing in the local decoder is 8×4. On the other hand, when a decoding apparatus, which has received the output of the compressed data from this apparatus, decodes the compressed data, IDCT processing is performed on 8×8 blocks as a unit. Namely, the size of the blocks subjected to IDCT processing in the former apparatus, i.e., the encoding apparatus for encoding an image, is different from the size of the blocks subjected to IDCT processing in the latter apparatus, the decoding apparatus receiving the encoded data and decoding the received data. Due to the difference in the size of blocks, errors occur in the frame subjected to the inter-frame predictive processing between the image encoded and the image decoded. Especially, errors accumulate at the edge portions of the image and this causes distortion of the image or color registration error.

[0012] The object of the present invention is the provision of a video signal encoding apparatus enlarging the image in a frequency domain when performing encoding of a motion picture thereby reducing the amount of data calculation required for the encoding and suppressing deterioration in image quality.

[0013] The video signal encoding apparatus of the present invention performs DCT calculation and quantization with the use of m×n block (m, n: natural number) as the unit block for coding a motion picture and, then, converting the block into s×t block (s, t: natural number, s≧m, t≧n), performs variable length coding. Thus, the apparatus generates compressed data of an enlarged motion picture and delivers the enlarged output.

[0014] In the local decoder of the apparatus, it is selected by a switcher whether m×n inverse quantization coefficients, obtained by inverse quantization, are to be decoded by an s×t IDCT unit or by an m×n IDCT unit, in accordance with the accuracy in motion compensation.

[0015] Further, the apparatus has a resizing unit for adjusting a motion-compensated reference picture to the picture size of an input image.

[0016] In another embodiment of the invention, the apparatus has the same configuration as that of the above described embodiment except that the local decoder in this apparatus decodes m×n inverse quantization coefficients, obtained by inverse quantization, in an s×t IDCT unit.

[0017] Having the described configuration, the video signal encoding apparatus performs the encoding by enlarging a motion picture in a frequency domain so that the amount of data calculation in the encoding is decreased and deterioration in picture quality is suppressed.

[0018] FIG. 1 is a block diagram showing a video signal encoding apparatus according to embodiment 1 of the invention.

[0019] FIG. 2 is a diagram showing a method of producing pixel data when motion compensation is made.

[0020] FIG. 3A and FIG. 3B are graphs showing difference between four-point IDCT and eight-point IDCT.

[0021] FIG. 4 is a block diagram showing a video signal encoding apparatus according to embodiment 2 of the invention.

[0022] FIG. 5 is a block diagram showing a video signal encoding apparatus of a prior art.

[0023] Exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

[0024] Referring to FIG. 1, the video signal encoding apparatus of the invention comprises frame arranger 102, motion detector 103, subtractor 104, m×n DCT unit 105, quantizer 106, VLC unit 107, and local decoder 120. Local decoder 120 comprises inverse quantizer 109, switcher 110, m×n IDCT unit 111, s×t IDCT unit 112, adder 113, frame memory 114, motion compensator 115, and resizing unit 116. The video signal encoding apparatus processes input image 101 to output compressed data 108. Operation of the apparatus will be described below.

[0025] Here, the case where the format of compressed data 108 as the output is that of MPEG and an input image is doubled in both vertical and horizontal directions (m=n=4, s=t=8) will be described. Frame arranger 102 rearranges input image 101 in the order of coding.

[0026] Motion detector 103 obtains motion vectors by using the rearranged image only when the image as the object of encoding is subjected to motion-compensation by inter-frame prediction.

[0027] The size of the output image is double the size of the input image both in the vertical and horizontal directions. Accordingly, the value of the motion vector in compressed data 108 is adapted to become double the value of the motion vector obtained as above both in vertical and horizontal directions.

[0028] Subtractor 104 obtains the differential between the picture subjected to inter-frame prediction and the reference picture decoded by local decoder 120 to generate differential data.

[0029] m×n DCT unit 105 divides intra-frame predictive coding picture, or differential data, into 4×4 blocks formed of four horizontal pixels and four vertical lines and, thereupon, performs two-dimensional four-point DCT calculation on the same to generate 4×4 DCT coefficients.

[0030] Quantizer 106 quantizes the 4×4 DCT coefficients to obtain 4×4 quantization coefficients. For the quantization at this time, a matrix of 4×4 elements on the lower-order components of the 8×8 quantization matrix is used.

[0031] VLC unit 107 converts 4×4 quantization coefficients into 8×8 quantization coefficients and applies variable length coding to the 8×8 quantization coefficients and such information as motion vectors required for coding, and thereby outputs compressed data 108. Here, 4×4 quantization coefficients are used as data of lower-order components of 8×8 quantization coefficients, that is, data of higher-order components of 8×8 quantization coefficients are set to zeros so that 4×4 quantization coefficients are converted into 8×8 quantization coefficients.

[0032] Thus, the input image is doubled both vertically and horizontally in compressed data 108.

[0033] Inverse quantizer 109 inverse-quantizes 4×4 quantization coefficients to obtain 4×4 inverse quantization coefficients. Here, in the same way as in quantizer 106, inverse quantization is performed by using 4×4 matrix on the lower-order components of 8×8 quantization matrix.

[0034] Switcher 110 outputs data of 4×4 inverse quantization coefficients to m×n IDCT unit 111 or s×t IDCT unit 112 depending on the condition of motion detection and the like. Description will be given later as to the switching condition.

[0035] m×n IDCT unit 111 performs two-dimensional four-point IDCT calculation on the 4×4 inverse quantization coefficients to generate 4×4 block data.

[0036] s×t IDCT unit 112, treating the 4×4 inverse quantization coefficients as data on the lower-order components, performs two-dimensional eight-point IDCT calculation on the same to generate 8×8 block data.

[0037] Switcher 110 switches the output of inverse quantizer 109 over to m×n IDCT unit 111 or s×t IDCT unit 112 in the manner as described below.

[0038] The switching condition of switcher 110 will now be described.

[0039] The description will be made taking the case where the picture format, for example, is 4:2:0.

[0040] (1) Switcher 110, when the motion vector obtained in motion detector 103 is of one pel accuracy, outputs the inverse quantized data for luminance component to mxn IDCT unit 111 (so that the data is inverse-converted to 4×4 block data) and outputs the inverse quantized data for color difference component to s×t IDCT unit 112 (so that the data is inverse-converted to 8×8 block data).

[0041] (2) Switcher 110, when the motion vector obtained in motion detector 103 is of half pel accuracy, outputs the inverse quantized data for both luminance component and color difference component to s×t IDCT unit 112, and s×t IDCT unit 112 in turn generates 8×8 block data.

[0042] The switching condition of switcher 110 will be decried below in more detail.

[0043] (1) When the motion vector is to be obtained at one pel accuracy, since motion vectors doubled in both vertical and horizontal directions are used in compressed data 108, motion compensation at two pel accuracy is required.

[0044] When luminance component is motion-compensated at two pel accuracy, color difference component is motion-compensated at one pel accuracy by scaling up the motion vector for luminance component.

[0045] At this time, luminance component requires motion compensation at two pel accuracy in 8×8 block. This is virtually equivalent to motion compensation at one pel accuracy in 4×4 block.

[0046] On the other hand, since color difference component requires motion compensation at one pel accuracy in 8×8 block, the local decoder also generates 8×8 block and, by using the same, performs motion compensation at one pel accuracy.

[0047] (2) When the motion vector is to be obtained at half pel accuracy, luminance component in compressed data 108 requires motion compensation at one pel accuracy. When luminance component is motion-compensated at one pel accuracy, color difference component is motion-compensated at half pel accuracy. At this time, luminance component requires motion compensation at one pel accuracy in 8×8 block and color difference component requires motion compensation at half pel accuracy in 8×8 block. Accordingly, the local recorder also generates 8×8 block and performs motion compensation by using the same.

[0048] (3) When the motion vector is to be obtained at ¼ pel accuracy, motion compensation at half pel accuracy is required in compressed data 108. At this time, since half pel accuracy is required in 8×8 block for both luminance signal and color difference signal, it is required that the local decoder also generate 8×8 block and make motion compensation by using the same.

[0049] Below, errors of decoded image will be considered that are produced between, for example, an encoding apparatus and a decoding apparatus, receiving the encoded data therefrom and decoding the same, when the size of the block in which IDCT is performed is different between the encoding apparatus and the decoding apparatus.

[0050] Here, as an example, the difference between the case where motion compensation is performed at half pel accuracy in 4×4 block data and the case where motion compensation is made at one pel accuracy in 8×8 block data will be described.

[0051] First, description about motion compensation will be made with reference to FIG. 2. When compensation at one pel accuracy is made, a reference picture for motion compensation is generated with the use of pixels of one pel accuracy shown in FIG. 2 (indicated by 0). When compensation at half pel accuracy is made, the mean value of two horizontally adjoining circles (0) is taken as the pixel at half pel accuracy (indicated by ×). Although compensation at half pel accuracy in the horizontal direction is illustrated in FIG. 2, motion compensation is made also in vertical or oblique direction with the use of mean, values of two points or four points of pixels.

[0052] Taking an example where, for simplicity, one-dimensional four-point data (given by 20, 20, 20, 150) is used as input data, the reason why errors accumulate due to difference in number of orders in IDCT will be described by using FIG. 3A and FIG. 3B.

[0053] (1) As described above, when motion compensation at half pel accuracy is made for 4×4 block data, motion compensation at one pel accuracy is required to be made for 8×8 block data. Referring to FIG. 3A, four-point DCT is performed on four point input data indicated by (20, 20, 20, 150) and, thereby, four-point DCT coefficients are generated. Four-point IDCT is performed on the same to generate four-point DCT coefficients. The data are indicated as four-point IDCT data 0 in the output data shown in FIG. 3B. The mean values of adjoining two points of four-point IDCT data are indicated by output data Δ in FIG. 3B. Then, eight-point IDCT is performed by having zeros added to the higher-order components of the four-point DCT coefficients and eight-point IDCT coefficients are obtained by conversion. The thus generated eight-point IDCT data are indicated by output data×in FIG. 3B.

[0054] (2) Then, on the basis of the output data shown in FIG. 3B, four-point IDCT calculation 0 and eight-point IDCT calculation ×will be compared and what is known therefrom will be considered.

[0055] It is known that eight-point IDCT data×seem to have such values as obtained by applying predictive filtering to data of four-point IDCT calculation 0. When four-point DCT calculation is performed on step-formed data as shown in the example and eight-point IDCT calculation with zeros added to the higher-order components is performed, it is known that the data takes on a form causing an overshoot. Further, when the mean value of each combination of two points from the left is taken, each mean value has virtually equal value to each of four-point IDCT data.

[0056] (3) The reason why errors accumulate will now be described. As illustrated in FIG. 3B, when motion compensation at half pel accuracy is performed by obtaining the mean value with the use of four-point IDCT data, the data indicated by Δ at the right-hand end of FIG. 3B is the mean value of the data at the left-hand end of four-point IDCT data of the block (not shown) adjoining the four-point block shown in FIG. 3B and the data of the four-point IDCT data indicated by 0 at the right-hand end of FIG. 3B. In this case, no overshoot is produced as in eight-point IDCT data.

[0057] In this example, the difference between the data at the right-hand end of the four-point IDCT data, in FIG. 3B, and the data at the right-hand end of the eight-point IDCT data, or the difference between the second data from the right of the four-point IDCT data and the fourth data from the right of the eight-point IDCT data constitutes the error between the four-point IDCT data and the eight-point IDCT data.

[0058] The phenomena as described above can occur also in the case of two-dimensional data.

[0059] When IDCT in local decoder 120 is two-dimensional four-point IDCT and IDCT in the system decoding compressed data 108 is two-dimensional eight-point IDCT, errors occur as with the above described case between IDCT data in local decoder 120 and IDCT data in the decoding apparatus decoding compressed data received thereby.

[0060] Description as to what effects these phenomena cause on the picture will now be made. When there is no motion in the picture, the errors as described above are not conspicuous in the decoded output by the decoding apparatus. However, when an edge portion of luminance data has moved, an error occurs at the edge portion of the IDCT data in a first predictive coded picture (P picture) due to the difference between IDCT data in the coding apparatus and IDCT data in the decoding apparatus as described above. The error causes the picture to appear as if it has double edges. Further, on the next P picture, another error accumulates at the double-edge portion so that the picture is caused to appear as if it has triple edges. Such a thing is repeated and eventually a state is brought about in which the edge has errors corresponding to the number of the P pictures. The same phenomena occur also with color registration, namely, an edge portion in the color component comes to have doubled edges, tripled edges, and so on. This causes the picture to look that it has produced a chromatic blur or a color registration error.

[0061] Then, if 8×8 IDCT calculation is performed in s×t IDCT unit 112 on luminance data or color difference data subjected to motion compensation at half pel accuracy, the operation of IDCT in the local decoder and the operation of IDCT in decoding compressed data 108 agree with each other. As a result, IDCT data in the local decoder and IDCT data in the decoder match with each other and, therefore, a chromatic blur or a color registration error as described above can be eliminated.

[0062] Adder 113 adds 4×4 block data or 8×8 block data of one picture when an inter-frame predictive picture is subjected to local decoding and a reference picture compensated for motion to generate a new reference picture.

[0063] Frame memory 114 stores 4×4 block data or 8×8 block data of one picture to be subjected to intra-frame predictive coding or a new reference picture generated in adder 113.

[0064] Motion compensator 115 makes motion compensation for the reference picture of a picture to be encoded, by using motion vectors detected by motion detector 103. At this time, motion compensation for 4×4 block data is made by using the motion vectors detected by motion detector 103 as they are. For 8×8 block data, motion compensation is made by using motion vectors doubled in both vertical and horizontal directions.

[0065] Resizing unit 116 reduces the motion-compensated reference picture from 8×8 block data to 4×4 block data in conformity with the size of input image 101. To reduce the size at this time, 8×8 block data are divided into divisions each thereof being 2×2 block and the mean value of the four points is taken.

[0066] While the format in compressed data 108 was set to be MPEG in the foregoing description, similar arrangement is applicable to H. 261, H. 263, or other coding system including intra-frame predictive coding and inter-frame predictive coding in combination.

[0067] Although the case where m=n=4 and s=t=8 was described above, similar encoding of an input image and enlargement of the size is possible even if other combinations of m, n, s, and t are used.

[0068] Further, as described above, when 4×4 quantization coefficients are converted into 8×8 quantization coefficients in VLC unit 107, the conversion can be attained by setting 4×4 quantization coefficients as the data on the lower-order components of 8×8 quantization coefficients and forming the data on the higher-order components by zeros.

[0069] Further, instead of allowing it to be made in VLC unit 107, the conversion of 4×4 quantization coefficients into 8×8 quantization coefficients by setting 4×4 quantization coefficients as the data on the lower-order components of 8×8 quantization coefficients and forming the data on the higher-order side by zeros may be made in quantizer 106.

[0070] Further, instead of allowing it to be made in VLC unit 107, setting of 4×4 quantization coefficients to be the data on the lower-order components of 8×8 quantization coefficients and forming the data on the higher-order components by zeros thereby producing 8×8 DCT coefficients may be made at the output of m×n DCT unit 105.

[0071] Further, fast processing algorithm may be used for m×n DCT unit 105, m×n IDCT unit 111, and s×t IDCT unit 112.

[0072] Referring to FIG. 4, the video signal encoding apparatus of the present invention comprises frame arranger 102, motion detector 103, subtractor 104, m×n DCT unit 105, quantizer 106, VLC unit 107, and local decoder 122. Local decoder 122 comprises inverse quantizer 109, s×t IDCT unit 112, adder 113, frame memory 114, motion compensator 115, and resizing unit 116. The video signal encoding apparatus processes input image 101 and outputs compressed data 108. Except for switcher 110 and m×n IDCT unit 111, the configuration of embodiment 2 is the same as the configuration of embodiment 1. Hence, points which are different from the configuration of embodiment 1 will be mainly described.

[0073] The same as in embodiment 1, the case where the format of compressed data 108 as the output is MPEG and the input image is turned into an image doubled in both vertical and horizontal directions (m=n=4, s=t=8) will be described.

[0074] Here, s×t IDCT unit 112 generates 8×8 block data from 4×4 quantization coefficients irrespective of the accuracy in motion detection by motion detector 103 and for both luminance data and color difference data.

[0075] Motion compensator 115 performs motion compensation with the use of vectors obtained by doubling the motion vectors detected in motion detector 103 both in vertical and horizontal directions.

[0076] Resizing unit 116 reduces, the same as in embodiment 1, the size of the reference picture compensated for motion from 8×8 block data to 4×4 block data in conformity with the size of input image 101. To reduce the size at this time, 8×8 block data are divided into divisions each thereof being 2×2 block and the mean value of the four points is taken.

[0077] Operations of other components of the apparatus also are the same as in embodiment 1.

[0078] While the format in compressed data 108 was set to be MPEG in the foregoing description, similar arrangement is applicable to H. 261, H. 263, or other coding system including intra-frame predictive coding and inter-frame predictive coding in combination.

[0079] Although the case where m=n=4 and s=t=8 was described above, similar coding of an input image and enlargement of the size is possible even if other combinations of m, n, s, and t are used.

[0080] Further, fast algorithm may be used for m×n DCT unit 105 and s×t IDCT unit 112.

[0081] According to the video signal encoding apparatus of embodiment 1 of the invention, the amount of data calculation in the coding process can be decreased by enlarging an input image in a frequency domain than in coding the image after it is enlarged. By changing the system in the local decoder in accordance with the accuracy of motion compensation, picture quality can be prevented from being deteriorated and the amount of data calculation in the coding process can be decreased further.

[0082] According to the video signal encoding apparatus of embodiment 2 of the invention, the amount of data calculation in the coding process can be decreased by enlarging an input image in a frequency domain and, further, picture quality can be prevented from being deteriorated irrespective of the accuracy in the motion compensation.