US 6,792,046 B2Grant
Encoding system and encoding method
Issue Date:2004-09-14
•13 Claims
•19 Drawing Sheets
Abstract
An encoding system capable of preventing the overflow of a buffer and performing the encoding of a VOP within a predetermined time period even where the processing time period necessary for encoding each macroblock is not constant. In order to provide the encoding system, the maximum number of bits Tmax is set for each VOP, and the output of encoding unit and the output of fixed code output unit is switched and outputted so that the number of bits Sc of the VOP may not exceed the Tmax.
Metadata
Assignee
- Mitsubishi Denki Kabushiki Kaisha
Inventors
- Yoshiko Hatano
- Takashi Nakao
- Junko Kijima
- Mamoru Inamura
- Kazuhiro Sugiyama
Application Information
Application Number:US 09/842,772
Filing Date:2001-04-27
Priority Date:2000-04-27
Art Unit:7
Classifications
IPC:
H04N 712H04N 1104
Field of Search:
3752400537524007
Patent Drawings (19 sheets)
Description
Background of the Invention
[0002] 1. Field of the Invention
[0003] This invention relates to an encoding system and an encoding method for a mobile telephone or a TV telephone system, for example, to encode video signals in real time.
[0004] 2. Description of the Related Art
[0005] FIG. 1 is a block diagram of an encoding system of the background art, as disclosed on pp. 39 to pp. 40 of “All of MPEG-4” (Association of Industrial Search), for example; FIG. 2 is an explanatory view showing an input signal of this encoding system of the background art; FIGS. 3ato 3dare explanatory diagrams showing constructions of bitstreams; and FIG. 4 is an explanatory diagram showing positions (arrangements) on a screen (in a displayed state) of a video packet.
[0006] In FIG. 1, reference numeral 1 designates a subtracter for receiving an external input signal (e.g., a luminance signal and two chrominance signals in the shown example) as its first input. The output of the subtracter 1 is inputted through DCT (Discrete Cosine Transformer) 2 and quantizer 3 to a DC/AC predictor 4 for predicting the quantized values of a DC component and an AC component, and dequantizer 6. The output of the DC/AC predictor 4 is fed to the first input of variable length coder 5, which outputs a bitstream.
[0007] On the other hand, the output of the dequantizer 6, to which the output of the quantizer 3 is inputted, is fed through IDCT 7 (IDCT: Inverse DCT) to the first input of an adder 8. The output of this adder 8 is fed to a memory 9, the output of which is fed to the first input of predicted image former 10 and the first output of a motion detector 11.
[0008] An external input signal is fed to the second input of the motion detector 11, the output of which is fed to the second input of the predicted image former 10 and a motion vector predictor 12.
[0009] The output of the motion vector predictor 12 is fed to the second input of variable length coder 5. On the other hand, the output of the predicted image former 10 is fed to the second input of the subtracter 1 and the second input of the adder 8.
[0010] Here will be described the operations. First of all, the video signals are divided into macroblocks or basic processing units, as shown in FIG. 2, and are inputted as external input signals (where the external input signals are basically inputted as the macroblocks, which may be directly inputted or may be converted thereinto by preprocessing unit for generating the macroblocks).
[0011] Where the video signals inputted are 4:2:0, 16 pixels×16 lines of a luminance signal (Y) are as large in the screen as 8 pixels×8 lines of two chrominance signals (Cb, Cr). Therefore, six blocks (i.e., four blocks for the luminance signal and two blocks for the chrominance signals) of 8 pixels×8 lines construct one macroblock.
[0012] Here, it is premised that the video object plane (VOP: a unit image) to be inputted as an external input has a rectangular shape identical to the frame.
[0013] Each block is quantized in the quantizer 3 after subjected to the discrete cosine transform (DCT). The DCT coefficients thus quantized are transformed together with additional informations such as a quantizing parameter into the variable length codes after the coefficients of the individual DC and AC components were predicted in the DC/AC predictor 4.
[0014] This is the intra coding (as also called the “in-frame encoding”). The VOP for coding all the macroblocks as intra coding will be called the “I-VOP (Intra-VOP)”.
[0015] On the other hand, the quantized DCT coefficients are dequantized in the dequantizer 6 and are decoded by the IDCT 7 so that the decoded image is stored in the memory 9. The decoded signal stored in this memory 9 is used at an inter coding (which may be called the “inter-frame encoding”).
[0016] In the inter coding case, the motion detector 11 detects the motion vectors indicating the motions of the macroblocks which are inputted as the external input signals. This motion vector indicates such a position in the decoded image stored in the memory 9 as takes the minimum difference from the macroblock inputted.
[0017] The predicted image former 10 forms a predicted image on the basis of the motion vector which is detected in the motion detector 11.
[0018] Subsequently, a differential signal is determined between the macroblock inputted and the predicted image formed in the predicted image former 10 and is subjected to the DCT in the DCT 2 so that it is quantized in the quantizer 3.
[0019] The DCT coefficients quantized are converted together with the additional information such as the predicted motion vector and the quantized parameter into the variable length codes. On the other hand, the quantized DCT coefficients are dequantized in the dequantizer 6 and subjected to the IDCT in the IDCT 7. The output of the IDCT7 is added to the predicted image by the adder 8 so that the sum is stored in the memory 9.
[0020] For the inter coding, there are two types of prediction. One type is a forward prediction, which is made in the display order of the images only from the VOP preceding in time, and the other type is a bidirectional prediction, which is made from both the preceding VOP and the succeeding VOP. The VOP to be encoded by the forward prediction is called the “P-VOP (Predictive VOP), and the VOP to be encoded by the bidirectional prediction is called the “B-VOP (Bidirectionally Predictive VOP)”.
[0021] With reference to FIG. 3, here will be described the construction of the bitstream to be outputted from the variable length coder 5. The bitstream of 1 VOP is constructed of one or more video packets, as shown in FIG. 3a.
[0022] Here, one video packet is composed of encoding data of one or more macroblocks, and the first video packet of the VOP is assigned the VOP header to its head and stuffing bits for byte alignment to its tail (as shown in FIG. 3b).
[0023] The second and subsequent video packets are assigned a Resync Marker for detecting the leading video packet and the video packet header to its head and the stuffing bits to its tail (as shown in FIG. 3c).
[0024] Here, the stuffing bits are added at the unit of 1 to 8 bits to the terminal end (cut) of the video packet for adjusting the byte alignment, and is discriminated in its meaning from the stuffing data, as will be described in the following.
[0025] On the other hand, the stuffing data can be introduced in an arbitrary number into the video packet, as shown in FIG. 3d. In the case of MPEG4 Video, for example, the stuffing data is called the “stuffing macroblock”, which can be introduced like the macroblock into an arbitrary video packet. This stuffing data is abandoned (not substantially used) on the side of the decoding system.
[0026] The stuffing data, as defined herein, is used as words of 9 bits or 10 bits for increasing the number of bits but. independently of the byte alignment (for adjusting the terminal. end of the video packet, for example) and is used between the macroblocks so that its meaning is discriminated from the aforementioned stuffing bits.
[0027] The number of macroblocks to be inserted into one video packet is arbitrary but may be generally so constructed, if an error propagation is considered, that each video packet may have a substantially constant number of bits. Where the number of bits in the video packet is thus substantially constant, the area to be occupied by each video packet in one VOP is not constant, as shown in FIG. 4.
[0028] With reference to FIG. 5, here will be detailed the operations of the DC/AC predictor 4 (i.e., on the luminance signal Y-component of the macroblock).
[0029] As described above, the DC/AC predictor 4 predicts the coefficients of the DC component and the AC components of the quantized DCT coefficients which are outputted from the quantizer 3 in the intra coding case. In the inter coding case, the DC component and the AC components are not predicted, but the quantized DCT coefficients, as outputted from the quantizer 3, are outputted as they are to the variable length coder 5. In this case, the luminance signal Y and the chrominance signals Cb, Cr are separately subjected to the DC/AC prediction.
[0030] Here will be described the predictions of the DC component and the AC components of the intra coding case.
[0031] If the quantized DCT coefficients of the block being coded are designated by Fx (i, j) (0≦i≦7 and 0≦j≦7), if the quantized DCT coefficients of the lefthand adjacent block are designated by Fa (i, j) (0≦i≦7 and 0≦j≦7), if the quantized DCT coefficients of the upper adjacent block are designated by Fc (i, j) (0≦i≦7 and 0≦j≦7), and if the quantized DCT coefficients of the lefthand upper block are designated by Fb (i, j) (0≦i≦7 and 0≦j≦7), the prediction direction is determined at first from the quantized DC component Fb (0, 0) of the lefthand upper block, the quantized DC component Fa (0, 0) of the lefthand adjacent block and the DC component Fc (0, 0) of the upper adjacent block.
[0032] If the quantization step size of the DC component of the lefthand adjacent block is designated by Qda, if tile quantization step size of the DC component of the lefthand upper block is designated by Qdb and if the quantization step size of the DC component of the upper adjacent block is designated by Qdc, for example, the dequantized DC components fa (0, 0), fb (0, 0) and fc (0, 0) are determined by the following relations:
[0033] fa(0, 0)=Fa(0, 0)×Qda;
[0034] fb(0, 0)=Fb(0, 0)×Qdb;
[0035] and
[0036] fc(0, 0)=Fc(0, 0)×Qdc.
[0037] If the following relation holds, it is conceived that the correlations are intense in the vertical direction, so that the predictions are made from the dequantized DC components fc (0, 0) of the upper adjacent block:
[0038] |fa(0,0)−fb(0,0)|<|fb(0, 0)−fc(0,0)|.
[0039] If the aforementioned relation does not hold, it is conceived that the correlations are intense in the horizontal direction, so that the predictions are made from the dequantized DC components fa (0, 0) of the lefthand adjacent block.
[0040] In the case of predicting the DC components from the upper adjacent block, the predicted DC component Px (0, 0) is determined by setting the following relation:
[0041] Px(0, 0)=Fx(0, 0)−fc(0, 0)/Qdx.
[0042] In the case of predicting the DC components from the lefthand adjacent block, the predicted DC component Px (0, 0) is determined by setting the following relation:
[0043] Px(0, 0)=Fx(0, 0)−fa(0, 0)/Qdx.
[0044] Here, Qdx is the quantization step size of the DC component of the current block, and the aforementioned divisions are calculated by the rounding method, for example.
[0045] Subsequently, the AC components are predicted by using the prediction direction of the DC components. In the following, Qpa denotes the quantization parameter of the lefthand adjacent block, Qpc denotes the quantization parameter of the upper adjacent block and Qpx denotes the quantization parameter of the current block. If the DC component is predicted from the upper adjacent block, the first row of the quantized AC components are predicted as follows:
[0046] Px(i, 0)=Fx(i, 0)−(Fc(i, 0)×Qpc)/Qpx(i=1 to 7).
[0047] On the other hand, if the DC component is predicted from the lefthand adjacent block, the first column of the quantized AC components are predicted as follows:
[0048] Px(0, j)=Fx(0, j)−(Fa(0, j)×Qpa)/Qpx(j=1 to 7).
[0049] Thus, the predicted AC components Px (i, 0) or Px (0, j) are determined. The aforementioned divisions are calculated by the rounding method, for example.
[0050] After the aforementioned predictions of the AC components were independently made for the six blocks composing one macroblock, it is determined as follows on a macroblock basis whether or not the AC components are to be predicted.
[0051] Here, an AC prediction decision index SB of the block is determined in the following manner as the index for deciding whether the original video signals are left as they are (without the prediction of the AC component) or predicted. Where the prediction is made from the upper adjacent block, the AC prediction decision index SB is determined from the following formula:
[0052] SB=Σ|Fx(i, 0)|−Σ|Px(i, 0)| (i=1 to 7). [Formula 1]
[0053] Where the prediction is made from the lefthand adjacent block, the AC prediction decision index SB is determined from the following Formula:
[0054] SB=|Σ(0, j)|−Σ|Px(0, j)| (j=1 to 7). [Formula 2]
[0055] After calculating the AC prediction decision indexes for all blocks in the current macroblock, the sum of these indexes is calculated, that is SBS=ΣSB. If the sum SBS is in the following relation, the AC components are predicted, but otherwise not predicted:
[0056] SBS≧0.
[0057] Here in the case of predicting the AC components, ac_pred_flag=1, but otherwise, ac_pred_flag=0. With this additional information ac_pred_flag, each macroblock is encoded by the variable length coder 5.
[0058] In the case of the macroblock of ac_pred_flag=1, for each block predicted from the upper adjacent block, the value Ox (i, j) is determined from the following relations:
[0059] For each block predicted from the lefthand adjacent block, the value Ox (i, j) is determined from the following relations:
[0060] For the block belonging to the macroblock of ac_pred_flag=0, the value Ox (i, j) is determined from the following relations:
[0061] This value ox (i, j) is outputted as the output of the DC/AC predictor 4 to the variable length coder 5.
[0062] In the predictions thus far described, where the current block is on the left end of the VOP, there is neither the lefthand adjacent block nor the lefthand upper block for the current block. Therefore, a predetermined constant β is used as the values of the dequantized DC components fa (0, 0) and fb (0, 0) to be used in the aforementioned predictions. In this case, the AC components Fa (i, j) and Fb (i, j) ((i, j)≠(0, 0)) to be used in the aforementioned predictions are set to 0.
[0063] The constant β is an intermediate value of the range of the value of the DC component of the DCT coefficients to be outputted from the DCT 2. Specifically, β=1,024 where the DC component to be outputted from the DCT 2 is 11 bits and takes a value from 0 to 2,047.
[0064] Similarly, where the current block is on the upper end of the VOP, there is neither the upper adjacent block nor the lefthand upper block for the current block. Therefore, the aforementioned constant β is used as the values of the dequantized DC components fc (0, 0) and fb (0, 0) to be used in the aforementioned predictions, and the AC components Fc (i, j) and Fb (i, j) ((i, j)≠(0, 0)) are set to 0.
[0065] In the aforementioned predictions, moreover, where the block lefthand adjacent to the current block belongs to a video packet different from that of the current block, the dequantized DC component fa (0, 0) to be used in the aforementioned predictions is assumed to take a value of the aforementioned constant β, and the AC components Fa (i, j) ((i, j)≠(0, 0)) are 0.
[0066] In the aforementioned predictions, likewise, where the block upper adjacent to the current block belongs to a video packet different from that of the current block, the dequantized DC component fc (0, 0) to be used in the aforementioned predictions is assumed to take a value of the aforementioned constant β, and the AC components Fc (i, j) ((i, j)≠(0, 0)) are 0.
[0067] In the aforementioned predictions, on the other hand, where the block lefthand upper to the current block belongs to a video packet different from that of the current block, the dequantized DC component fb (0, 0) to be used in the aforementioned predictions is assumed to take a value of the aforementioned constant , and the AC components Fb (i, j) ((i, j)≠(0, 0)) are 0.
[0068] Thus, the DC/AC predictor 4 is so constructed by not referring the DC component and the AC components between the blocks belonging to the different video packets that the propagation of a transmission error in the DC/AC predictions may be confined in the video packet.
[0069] The encoding system of the background art thus far described has not sufficiently considered the processing for avoiding the overflow of the transmission buffer or the underflow of the VBV buffer or a virtual buffer on the receiver side.
[0070] Moreover, the number of bits for a macroblock is usually increased/decreased by adjusting the quantization parameters to be used in the quantizer 3, but there is no explicit way for processing the case in which overflow of the transmission buffer occurs even with the maximum quantization parameter (while supressing the number of bits with most coarse quantization).
[0071] Processing time for encoding a VOP is also an issue. Where the rate of the VOP to be inputted is F (1/sec.), it is requested that all the macroblocks composing one VOP be encoded for a time period of 1/F (sec.) or shorter.
[0072] Where the motion detector 11 is so constructed as to change the search range of the motion vector adaptively in response to the motion of the object in the VOP, however, the time period necessary for the motion detector 11 to detect the motion vector of each macroblock changes for the individual macroblocks so that the time period for processing one VOP is not constant. Therefore, new control is necessary for encoding all the macroblocks in a VOP within a predetermined time period.
Summary of the Invention
[0073] An object of this invention is to provide an encoding method for avoiding the overflow of a transmission buffer and the underflow of a VBV buffer effectively.
[0074] This invention also contemplates to propose an encoding system and an encoding method for ending the encoding of one VOP within a predetermined time period even where the time period necessary for encoding one macroblock is not constant.
Brief Description of the Drawings
[0075] FIG. 1 is a block diagram showing an encoding system of the background art;
[0076] FIG. 2 is an explanatory diagram showing an input signal to the encoding system of the background art;
[0077] FIGS. 3ato 3dare explanatory diagrams showing structure of bitstreams in the encoding system of the background art;
[0078] FIG. 4 is an explanatory diagram showing the position of video packets on the screen in the encoding system of the background art;
[0079] FIG. 5 is an explanatory diagram showing a DC/AC prediction in the encoding system of the background art;
[0080] FIG. 6 is a block diagram showing Embodiment 1 of the invention;
[0081] FIGS. 7aand 7bare explanatory diagrams showing the states (for the case of an I-VOP) of a temporary buffer and a transmission buffer in Embodiment 1 of the invention;
[0082] FIGS. 8aand 8bare explanatory diagrams showing the states (for the case of a P-VOP) of the temporary buffer and the transmission buffer in Embodiment 1 of the invention;
[0083] FIG. 9 is a flow chart (for the case of the P-VOP) showing Embodiment 1 of the invention;
[0084] FIG. 10 is a flowchart (for the case of the I-VOP) showing Embodiment 1 of the invention;
[0085] FIG. 11 is a flowchart (for the case of the P-VOP) showing Embodiment 2 of the invention;
[0086] FIG. 12 is a flowchart (for the case of the I-VOP) showing Embodiment 2 of the invention;
[0087] FIG. 13 is a flow chart showing Embodiment 3 of the invention;
[0088] FIG. 14 is a flow chart showing Embodiment 4 of the invention;
[0089] FIG. 15 is a block diagram showing Embodiment 5 of the invention;
[0090] FIGS. 16aand 16bare explanatory diagrams showing the states of a temporary buffer in Embodiment 5 of the invention;
[0091] FIG. 17 is a block diagram showing Embodiment 6 of the invention;
[0092] FIG. 18 is an explanatory diagram showing the state of a transmission buffer in Embodiment 7 of the invention; and
[0093] FIG. 19 is a block diagram showing Embodiment 7 of the invention.
Detailed Description of the Preferred Embodiments
[0094] This invention will be specifically described in its embodiments with reference to the accompanying drawings.
[0095] Embodiment 1
[0096] FIG. 6 shows an encoding system according to Embodiment 1 of this invention. In FIG. 6, reference numeral 1 designates a subtracter for receiving an external input signal as its first input. The output of the subtracter 1 is inputted through DCT 2 and quantizer 3 to a DC/AC predictor 4 and dequantizer 6. The output of the DC/AC predictor 4 is fed to the first input of variable length coder 5a.
[0097] On the other hand, the output of the dequantizer 6 is fed through IDCT 7 to the first input of an adder 8. The output of this adder 8 is fed to a first input of a memory 9, the output of which is fed to the first input of predicted image former 10 and the first input of a motion detector 11.
[0098] An external input signal is fed to the second input of the motion detector 11, the output of which is fed to the second input of the predicted image former 10 and a motion vector predictor 12. The output of the predicted image former 10 is fed to the second input of the subtracter 1 and the second input of the adder 8.
[0099] The output of the motion vector predictor 12 is fed to the second input of variable length coder 5a. Here, the encoding unit is constructed from the aforementioned components, from the subtracter 1 for receiving the external input signal to the variable length coder 5afor outputting the variable length code corresponding to the external input signal (Of course, the construction disclosed herein is only one example and could be exemplified by the known one capable of encoding the external input signal in a corresponding manner).
[0100] The first output of the variable length coder 5ais fed to the first input of a temporary buffer 101, and the second output of the variable length coder 5ais fed to rate control unit 102.
[0101] The temporary buffer 101 is fed at its second input with the output of fixed code output unit 104 and at its third input with the first output of the rate control unit 102. The output of the temporary buffer 101 is fed to the first input of a transmission buffer 103 (Here, the temporary buffer 101 or the transmission buffer 103 corresponds to storage unit.).
[0102] The second output of the rate control unit 102 is fed to the second input of the memory 9, and the third output of the rate control unit 102 is fed to the second input of the transmission buffer 103. The output of the transmission buffer 103 is outputted (transmitted) as the bitstream.
[0103] The bitstream outputted (transmitted) is received and decoded on the side of the decoding system.
[0104] Here will be described the operations.
[0105] First of all, the video signals are divided into macroblocks or basic processing units, as shown in FIG. 2, and are inputted as input macroblocks to the subtracter 1 and the motion detector 11. Where the video signals inputted are 4:2:0, for example, 16 pixels×16 lines of a luminance signal (Y) are as large in the screen as 8 pixels×8 lines of two chrominance signals (Cb, Cr), so that six blocks of 8 pixels×8 lines construct one macroblock.
[0106] In the intra coding case, each block is subjected to the DCT and then quantized. This quantized DCT coefficients are predicted in the DC/AC predictor 4 and is then encoded together with the additional information such as the quantization parameter. The quantized DCT coefficients are dequantized by the dequantizer 6 and subjected to the IDCT and decoded. The output of the IDCT 7, i.e., the decoded image is stored in the memory 9.
[0107] In the inter coding case, the motion detector 11 detects the motion vector indicating the motions of the inputted macroblocks. This motion vector indicates such a position in the decoded image stored in the memory 9 as takes the minimum difference from the input macroblock.
[0108] The predicted image former 10 forms a predicted image on the basis of the motion vector which is detected in the motion detector 11. Next, a difference is determined between the input macroblock and the predicted image and is subjected to the DCT by the DCT 2 and quantized by the quantizer 3.
[0109] The quantized DCT coefficients are encoded together with the additional information such as the motion vector predicted by the motion vector predictor 12 and the quantization parameter, by the variable length coder 5a. On the other hand, the quantized DCT coefficients are dequantized in the dequantizer 6. And the output of the dequantizer 6 is subjected to the IDCT in the IDCT 7. And the output of the IDCT 7 is added to the predicted image outputted by the predicted image former 10 so that the sum is stored in the memory 9.
[0110] Here will be specifically described the operations of the variable length coder 5a.
[0111] For each macroblock, the variable length coder 5aencodes the quantized DCT coefficients and additional information, stores them in the temporary buffer 101, and outputs the number of bits used for the current macroblock to the rate control unit 102.
[0112] In the case of the I-VOP of MPEG4, for example, the AC components of the DCT coefficients in each block, as outputted from the DC/AC predictor 4, is one-dimensionally scanned at first by the zigzag scanning method or the like to perform the run-length encoding of the combination between the number of zero and the coefficient of non-zero. The coded data of AC components in each block are stored in the temporary buffer 101.
[0113] After the coefficient data of each block, as shown in FIG. 7a, there are sequentially encoded and stored in the temporary buffer 101: mcbpc which is the code for MTYPE (macroblock type such as intra/inter mode) and cbpc indicating whether or not each block of the chrominance signal has the non-zero AC components; dquant indicating the quantization parameter; the DC components of the DCT coefficients in each block; ac_pred_flag indicating whether or not the AC prediction has been made; and cbpy indicating whether or not each block of Y has the non-zero AC coefficients.
[0114] Here, the number of bits for each macroblock is outputted to the rate control unit 102.
[0115] Similarly, in the case of P-VOP of MPEG4, the encoded data, ordered as shown in FIG. 8a, are stored in the temporary buffer 101.
[0116] On the basis of the number of bits in each macroblock outputted from the variable length coder 5a, the rate control unit 102 arranges the macroblocks so that the length of each video packet may not exceed a predetermined value, and transfers the arranged macroblocks from the temporary buffer 101 to the transmission buffer 103.
[0117] In the case of MPEG4, for example, the header is added to the head of each video packet, which is rearranged in the order of the specified bitstream and is transferred, as shown in FIGS. 7band 8b.
[0118] On the other hand, the rate control unit 102 sets the maximum number of bits Tmax for each VOP so that the transmission buffer 103 may not overflow or so that the VBV (Video Buffering Verifier) buffer may not underflow. (The VEV buffer is a buffer model of a decoder. It is required in order to bound memory requirements for the bitstream buffer needed by a decoder.)
[0119] The rate control unit 102 selects output of the variable length coder 5aor the output of the fixed code output unit 104 so that the number of bits for the current VOP may not be greater than value Tmax and stores it in the temporary buffer 101.
[0120] Here, Tmax is the upper limit of the number of bits for the VOP to prevent the transmission buffer 103 from overflowing and the VBV buffer from underflowing.
[0121] Here will be detailed the operations.
[0122] The rate control unit 102 determines the maximum number of bits Tmax for each VOP before encoding it. For example, if the transmission buffer 103 has a capacity Bs (bits), and if the current occupancy of the transmission buffer 103 is B (bits), it is sufficient for preventing the transmission buffer 103 from overflowing that the number of bits for the VOP is Bs−B or less. It is, therefore, sufficient that Tmax is set as follow:
[0123] Tmax≦Bs−B.
[0124] In the case of administrating the VBV buffer, the number of bits to be received for one VOP period by VBV buffer is Rp:
[0125] Rp=R/F.
[0126] Where R(bits/sec) is the bit rate and F(1/sec) is the VOP rate(frame rate).
[0127] If the occupancy of the VBV buffer at the decoding time of the previous VOP is expressed by vbv_bits (bits), it is sufficient for preventing the VBV buffer from underflowing that the number of bits used for the current VOP does not exceed vbv_bits+Rp. In short, it is sufficient to set Tmax in the following relations:
[0128] Tmax≦vbv_bits+Rp.
[0129] Before encoding each VOP, therefore, the rate control unit 102 sets Tmax as follow:
[0130] Tmax=min(vbv_bits+Rp, Bs−B).
[0131] Where min(a,b) takes the lesser of the values a and b.
[0132] Here, the occupancy vbv_bits of the VBV buffer is the estimated occupancy of the buffer on the decoder side. If the decoder is so configured that the decoding time is to be delayed when its buffer becomes empty, the VBV buffer is allowed to underflow. In such cases, it is unnecessary to check the underflow of the VBV buffer. So Tmax can be set as follow:
[0133] Tmax=Bs−B.
[0134] Since the occupancy B of the transmission buffer 103 changes with time, Tmax also changes with time and is calculated for each VOP.
[0135] Next, for every macroblock, the rate control unit 102 calculates the total number of bits used for the current VOP. In accordance with the flow charts shown in FIGS. 9 and 10, the rate control unit 102 selects either the codes outputted from the variable length coder 5aor the fixed codes outputted from the fixed code output unit 104 for the current macroblock, and stores them in the temporary buffer 101.
[0136] Here, FIG. 9 shows a flow chart of the case in which the current VOP is the P-VOP (in which the encoding type is the inter), and FIG. 10 shows a flow chart of the case in which the current VOP is the I-VOP (in which the encoding type is the intra).
[0137] Rate Control for P-VOP
[0138] Here will be described the operations of the rate control unit 102 of the P-VOP case.
[0139] In this P-VOP case, the variable length coder 5aoutputs the coded data of coefficients, not_coded, mcbpc, the motion vectors, cbpy and dquant, as shown in FIG. 8a, for each macroblock. However, all of these codes are not always present. For example, where all the coefficients are 0 and the motion vector is (0, 0), only 1 bit for not_coded=1 is present. This is the case of the minimum code length for the macroblock in P-VOP.
[0140] In P-VOP case, therefore, the fixed code output unit 104 outputs only 1 bit of not_coded=1 as the fixed code. In short, the fixed code output unit 104 outputs the fixed code of the macroblock having the minimum code length for the encoding type of the current VOP.
[0141] For each macroblock, the rate control unit 102 calculates the number of bits used for the current VOP Then it replaces the codes of the current macroblock by the fixed code outputted from the fixed code output unit 104, when the total number of bits used for all macroblocks in the current VOP will exceed Tmax, even the fixed code is selected for all the remaining macroblocks.
[0142] Supposing that the total number of macroblocks in a VOP is designated by A and the current macroblock number is designated by K (0≦K≦A−1), the number of remaining macroblocks to be encoded is expressed by:
[0143] M=A−K−1.
[0144] If the fixed code of the fixed code output unit 104 is selected for the remaining M macroblocks, the total number of bits used for the current VOP will be expressed by:
[0145] Sc+mb_bit+M×L+α,
[0146] where Sc denotes the number of bits used for the macroblocks with the macroblock number 0 through K−1 in the current VOP, mb_bit denotes the number of bits of the codes outputting by the variable length coder 5afor the current macroblock (No. K), and L denotes the code length of the fixed code, and α denotes the number of bits used for the “additional codes” to be generated in the macroblocks of number K through A−1.
[0147] Here, the “additional codes” means the codes generated on the video packet basis, such as Resync Marker, the video packet header, the stuffing bits and motion_marker. And, α≧0.
[0148] Where the following relation holds for the current macroblock (having the macroblock number K):
[0149] Sc+mb_bit+M×L+α>Tmax,
[0150] the fixed code outputted by the fixed code output unit 104 is stored in the temporary buffer 101. Otherwise, the code outputted by the variable length coder 5ais stored in the temporary buffer 101 (as shown in FIG. 9).
[0151] Here, α is determined in the following manner. The number of bits used for the macroblocks with macroblock number K through A−1 is at least:
[0152] (M+1)×L,
[0153] and the number of video packets for them can be the following number:
[0154] (M+1)×L/VPlen+1,
[0155] where VPlen denotes the predetermined video packet length. Therefore, the number of bits for the additional codes will be:
[0156] α=((M+1)×L/VPlen+1)×Cp,
[0157] where Cp denotes the sum of the number of bits for Resync Marker, the video packet header and motion_marker in the P-VOP.
[0158] In order to simplify the calculation, using the following relation:
[0159] M+1≦A,
[0160] α may be set as a fixed value for P-VOP, as expressed by:
[0161] α=(A×L/VPlen+1)×Cp.
[0162] If the fixed code is selected and stored in the temporary buffer 101 for the current macroblock, the decoded image of the current macroblock stored in the memory 9 is replaced by the decoded image of the same macroblock in the previous VOP which is also stored in the memory 9, because the current macroblock is forced to be not coded macroblock.
[0163] In detail, the decoded image of the macroblock with macroblock number K in the previous VOP is copied to the image data area for the current macroblock with macroblock number K in the current VOP. This operation is executed in the memory 9.
[0164] In the case of the P-VOP, the fixed code not_coded=1 is output from the fixed code output unit 104. And the decoded image of the current macroblock can be obtained by copying the decoded image of the same macroblock in the previous VOP.
[0165] Rate Control for I-VOP
[0166] Here will be described the operations of the rate control unit 102 of the I-VOP case.
[0167] In this I-VOP case, the variable length coder 5aoutputs the coded data of the AC components, mcbpc, dquant, the DC component, ac_pred_flag and cbpy, as shown in FIG. 7a, for each macroblock.
[0168] However, all of these codes are not always present. For example, if both of cbpc included in mcbpc and cbpy are 0, coefficients of the coded data are not present. And if macroblock type MTYPE, which is included in mcbpc, doesn't have dquant, this dquant is not present also.
[0169] In I-VOP case, therefore, the fixed code output unit 104 outputs the fixed coded data which represents that all of the DC and AC component data are 0, dquant=0 and ac_pred_flag=0. This is the case of the minimum code length for the macroblock in I-VOP, for such as MPEG2 or MPEG4 coding system.
[0170] As in case P-VOP case, for each macroblock, the rate control unit 102 calculates the number of bits used for the current VOP. Then it replaces the codes of the current macroblock by the fixed code outputtted from the fixed code output unit 104, when the total number of bits used for all macroblocks in the current VOP will exceed Tmax, even the fixed code is selected for all the remaining macroblocks.
[0171] The replacement operation control is based on following relation:
[0172] Sc+mb_bit+M×L+α>Tmax
[0173] (Note that M=A−K−1),
[0174] where Sc denotes the number of bits used for the macroblocks with the macroblock number 0 through K−1 in the current VOP, mb_bit denotes the number of bits of the codes outputting by the variable length coder 5afor the current macroblock (No. K), and L denotes the code length of the fixed code. As shown in FIG. 10, if the above relation is holds, fixed code output of fixed code output unit 104 is stored in the temporary buffer 101. Otherwise, code output of variable length coder 5ais stored in the temporary buffer 101.
[0175] Here, α denotes the number of bits to be generated per video packet for the macroblock with macroblock number K through A−1. And a includes data, such as Resync Marker, the video packet header, the stuffing bits or dc_marker. Note that α≧0.
[0176] Here, α is determined in the following manner. The number of bits used for the macroblocks with macroblock number K thruough A−1 is at least:
[0177] (M+1)×L,
[0178] and the number of video packets for them can be the following number:
[0179] (M+1)×L/VPlen+1,
[0180] where VPlen denotes the predetermined video packet length.
[0181] Therefore, the number of bits for the additional codes will be:
[0182] α=((M+1)×L/VPlen+1)×Ci,
[0183] where Ci denotes the sum of the number of bits for Resync Marker, the video packet header, stuffing bits and dc_marker in the I-VOP. In order to simplify the calculation, using the following relation:
[0184] M+1≦A,
[0185] α may be set as a fixed value for I-VOP, as expressed by:
[0186] α=(A×L/VPlen+1)×Ci.
[0187] If the fixed code is selected and stored in the temporary buffer 101 for the current macroblock, and the code output of variable length coder 5ais selected for the previous macroblock, new video packet is generated from the current macroblock, as shown in FIG. 10.
[0188] In I-VOP case, even if ac pred_flag=0, DC components prediction is executed. Therefore, if the DC components in the temporary buffer 101 are 0, it is not indicated that DC component Fx(0,0) output of the quantizer 3 is 0, but the predicted DC component O×(0,0) output of the DC/AC predictor 4 is 0.
[0189] Therefore, in general, the decoded image from the fixed code is not fixed, even if the fixed code output unit 104 outputs the fixed code to the current macroblock.
[0190] However, in practical manner, DC/AC predictor 4 does not refer to coefficients of the DC components between blocks of deferent video packet. But DC/AC predictor 4 refers to a constant β, which is an intermediate value in the range of the values of the DC components. Therefore, when the fixed code is selected and stored in the temporary buffer 101, if new video packet is generated from the current macroblock, as aforementioned, dequantized DC component fx(0,0) is:
[0191] fx(0, 0)=β.
[0192] Therefore, if the fixed codes outputted from the fixed code output unit 104 are decoded, each pixel of the image has a constant value γ. Here, γ is an intermediate value in the range of the values of the image signal level. For example, if the range of the image signal level can be expressed by 8 bit, that is 0 through 255, the constant value γ is set at 128.
[0193] In this manner, the DC components after the dequantization of each block of the current macroblock (No. K) is equal to constant β. And if the fixed code is selected for a next macroblock (No. K+1), the DC components are equal to constant β, and each pixel of the image has a constant image signal level γ.
[0194] If the fixed code is selected and stored in the temporary buffer 101 for the current macroblock, the decoded image of the current macroblock stored in the memory 9 is replaced by the constant image signal level γ. Therefore, the constant image signal level γ is stored in the decoded image area for the current macroblock of the current VOP in memory 9.
[0195] As described above, this embodiment 1 enables to prevent the number of bits for each VOP from exceeding the maximum number of bits Tmax by selecting either the codes outputted from the variable length coder 5aor the fixed codes outputted from the fixed code output unit 104 on the basis of the flow charts of FIGS. 9 and 10.
[0196] Furthermore, this embodiment 1 enables to get the decoded image corresponding to the fixed code outputted from the fixed code output unit 104 without any new calculation even in I-VOP, because it is determined whether or not the new video packet is to be generated from the current macroblock on the basis of the flow charts of FIG. 10.
[0197] Embodiment 2
[0198] In the foregoing Embodiment 1, the rate control unit 102 selects the output of the variable length coder 5aor the fixed code output unit 104, on the basis of the flow charts shown in FIGS. 9 and 10.
[0199] On the other hand, in the Embodiment 2, the rate control unit 102 selects the output of the variable length coder 5aor the fixed code output unit 104, on the basis of flow charts shown in FIGS. 11 and 12.
[0200] Here, FIG. 11 shows a flow chart of the case that the current VOP is the P-VOP (in which the coding type is the inter). And FIG. 12 shows a flow chart of the case that the current VOP is the I-VOP (in which the coding type is the intra).
[0201] Rate Control for P-VOP
[0202] First of all, the case of P-VOP will be described with reference to FIG. 11.
[0203] As Embodiment 1, the rate control unit 102 controls for the current macroblock (No. K) that the fixed code outputted by the fixed code output unit 104 is stored in the temporary buffer 101 when the relation of the total number of bits used for the current VOP and Tmax is as follows:
[0204] Sc+mb_bit+M×L+α>Tmax (1),
[0205] where Sc denotes the number of bits used for the macroblocks with the macroblock number 0 through K−1 in the current VOP, mb_bit denotes the number of bits of the codes outputted by the variable length coder 5afor the current macroblock (No. K), M denotes the number of macroblocks at and after macroblock number K+1 of the current VOP, and L denotes the code length of the fixed code.
[0206] α denotes the number of bits used for the “additional codes” to be generated in the macroblocks of number K through A−1. Here, the “additional codes” means the codes generated on the video packet basis, such as Resync Marker, the video packet header, the stuffing bits and motion_marker. And, α≧0. As described in Embodiment 1, α either may be calculated for each macroblock or may be fixed for each coding of the VOP.
[0207] If aforementioned Relation (1) holds for the current macroblock, it is highly probable that Relation (1) holds for the next macroblock. Because the number of bits are accumulated.
[0208] For example, a number of bits Sc′ of the macroblock of number 0 through K is the following sum of the code length L of the fixed code, as outputted by the fixed code output unit 104, and the number of bits Sc up to the macroblock number K−1:
[0209] Sc′=Sc+L.
[0210] Here, mb_bit is the number of bits of the code outputted by the variable length coder 5afor the macroblock number of K, and mb_bit′ is the number of bits of the code outputted by the variable length coder 5afor the macroblock number of K+1. If Relation (1) holds for the macroblock number K, if mb_bit is equal to mb_bit′, and if the aforementioned a value is equal for the two macroblocks, the following relations hold:
[0211] Sc′+mb_bit′+(M−1)×L+α=Sc+mb_bit+M×L+α>Tmax, and
[0212] Relation (1) also holds for the macroblock No. K+1.
[0213] Where Relation (1) holds for the macroblock No. K, therefore, the calculations can be omitted while assuming that Relation (1) also holds for the macroblock at and after the macroblock number K.
[0214] As shown in FIG. 11, it is judged at first whether or not the fixed code of the fixed code output unit 104 has been selected for the previous macroblock with macroblock number K−1. If the fixed code of the fixed code output unit 104 has been selected for the previous macroblock, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101, for the current macroblock.
[0215] On the other hand, if the fixed code of the fixed code output unit 104 has not been selected for the previous macroblock, the aforementioned Relation (1) is evaluated. If Relation (1) holds, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101. Otherwise, the output of the variable length coder 5ais stored in the temporary buffer 101.
[0216] Rate Control for I-VOP
[0217] In I-VOP case, as shown in FIG. 12, it is judged at first whether or not the fixed code of the fixed code output unit 104 has been selected for the previous macroblock with macroblock number K−1. If the fixed code has been selected, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101 for the current macroblock with macroblock number K. In this case, it is unnecessary to generate a new video packet from the current macroblock, as Embodiment 1.
[0218] On the other hand, if the fixed code of the fixed code output unit 104 has not been selected for the previous macroblock, the rate control unit 102 judges that the aforementioned Relation (1) is satisfied or not. If Relation (1) holds, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101, and a new video packet is generated from the current macroblock. Otherwise, the output of the variable length coder 5ais stored in the temporary buffer 101.
[0219] Here in Embodiment 2, if Relation (1) holds for a macroblock in one VOP, for at and after the current macroblock, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101. Therefore, after the current macroblock, no operation is needed for the coding unit which includes the subtracter 1, the DCT 2, the quantizer 3, the DC/AC predictor 4, the variable length coder 5a, the dequantizer 6, the IDCT 7, the adder 8, the predicted image former 10, the motion detector 11 and the motion vector predictor 12.
[0220] Therefore, volume of processing and power consumption can be reduced by stopping the operation of the coding unit.
[0221] Embodiment 3
[0222] In Embodiment 3, the rate control unit 102 selects the output of the variable length coder 5aor the fixed code output unit 104, on the basis of the flow charts shown in FIG. 13.
[0223] Here, FIG. 13 shows a flow chart of the case that the current VOP is the P-VOP (in which the coding type is the inter).
[0224] Rate Control for P-VOP
[0225] For example, if the motion detector 11 operates adaptively to change the search range of the motion vector according to the motion of an object in the VOP, necessary time for the motion detector 11 to detect the motion vector of each macroblock changes every macroblock and the processing time of one VOP is not constant.
[0226] Under such conditions, if the processing time, including the necessary time for motion vector detecting, is insufficient for coding all the macroblocks within the predetermined processing time for one VOP, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101. And the operations of the subtracter 1, the DCT 2, the quantizer 3, the DC/AC predictor 4, the variable length coder 5a, the dequantizer 6, the IDCT 7, the adder 8, the predicted image former 10, the motion detector 11 and the motion vector predictor 12 are not made.
[0227] The rate control unit 102 measures the elapsed time from the instant when the leading macroblock (macroblock number 0) constructing the current VOP was inputted. If the elapsed time exceeds a predetermined processing time Tp, the fixed code of the fixed code output unit 104 is always stored in the temporary buffer 101 as shown in FIG. 13. Otherwise, on the basis of the aforementioned Relation (1), the output (the fixed code) of the fixed code output unit 104 and the output of the variable length coder 5aare selected and stored in the temporary buffer 101.
[0228] Here, the predetermined processing time Tp of this case is set to one VOP period at the maximum, because the processing for one VOP has to be completed within the VOP period. Therefore, if another processing is incorporated into this VOP period, the difference of one VOP period and the necessary time for another processing is the maximum value for the processing time Tp.
[0229] Rate Control for I-VOP
[0230] Where the current VOP is I-VOP, the rate control unit 102 selects the code to be stored in the temporary buffer 101 in accordance with the flow chart of FIG. 10 or 12, as in Embodiment 1 or Embodiment 2.
[0231] Embodiment 4
[0232] In Embodiment 4, the rate control unit 102 selects the output of variable length coder 5aor the fixed code output unit 104, on the basis of the flow charts shown in FIG. 14.
[0233] Here, FIG. 14 shows a flow chart of the case that the current VOP is the P-VOP (in which the coding type is the inter).
[0234] Rate Control for P-VOP
[0235] As in embodiment 2, it is judged at first whether or not the fixed code of the fixed code output unit 104 has been selected for the previous macroblock with macroblock number K−1. If the fixed code of the fixed code output unit 104 has been selected for the previous macroblock, the fixed code of the fixed code output unit 104 is stored in the temporary buffer 101, for the current macroblock.
[0236] Next, as in embodiment 2, The rate control unit 102 measures the elapsed time from the instant when the leading macroblock (macroblock number 0) constructing the current VOP was inputted. If the elapsed time exceeds a predetermined processing time Tp, the fixed code of the fixed code output unit 104 is always stored in the temporary buffer 101. Otherwise, on the basis of the aforementioned Relation (1), the output (the fixed code) of the fixed code output unit 104 and the output of the variable length coder 5aare selected and stored in the temporary buffer 101.
[0237] Rate Control for I-VOP
[0238] Where the current VOP is I-VOP, the rate control unit 102 selects the code to be stored in the temporary buffer 101 in accordance with the flow chart of FIG. 10 or 12, as in Embodiment 1 or Embodiment 2.
[0239] Embodiment 5
[0240] The fixed code output unit 104 shown in Embodiment 1, is likely to be independent of the variable length coder 5a. However, if these elements are constructed by the software, the ROM table can be provided for common use for both the fixed code output unit 104 and the variable length coder 5a.
[0241] As described in Embodiment 1, the fixed code of the fixed code output unit 104 is one of the I-VOP/P-VOP macroblock codes. Therefore, the ROM table for common use can be realized by integrating the fixed code output unit 104 and the variable length coder 5a.
[0242] FIG. 15 shows an coding apparatus according to Embodiment 5 of this invention. In FIG. 15, numeral 1 designates a subtracter for receiving the external input signals as its first input. The output of the subtracter 1 is inputted through the DCT 2 and the quantizer 3 to the DC/AC predictor 4 and the dequantizer 6. The output of the DC/AC predictor 4 is fed to the first input of a variable length coder 5b.
[0243] On the other hand, the output of the dequantizer 6 is fed through the IDCT 7 to the first input of the adder 8. The output of this adder 8 is fed to the first input of the memory 9, the output of which is fed to the first input of the predicted image former 10 and the first input of the motion detector 11.
[0244] The external input signal is fed to the second input of the motion detector 11, the output of which is fed to the second input of the predicted image former 10 and to the motion vector predictor 12. The output of the predicted image former 10 is fed to the second input of the subtracter 1 and the second input of the adder 8.
[0245] The output of the motion vector predictor 12 is fed to the second input of the variable length coder 5b.
[0246] The first output of the variable length coder 5bis fed to the first input of the temporary buffer 101, and the second output of the variable length coder 5bis fed to the rate control unit 102.
[0247] The temporary buffer 101 is fed at its second input with the first output of the rate control unit 102. The output of the temporary buffer 101 is fed to the first input of the transmission buffer 103.
[0248] The second output of the rate control unit 102 is fed to the second input of the memory 9 and the third input of the variable length coder 5b, and the third output of the rate control unit 102 is fed to the second input of the transmission buffer 103. The output of the transmission buffer 103 is outputted (transmitted) as the bitstream.
[0249] This bitstream outputted (transmitted) is received and decoded on the side of the decoding system.
[0250] Here will be described the operations.
[0251] Embodiment 5 is different from Embodiment 1 in the operations of the variable length coder 5band the temporary buffer 101. The remaining portions are similar to those of Embodiment 1, so that their description will be omitted.
[0252] For each macroblock, the variable length coder Sb encodes the data of each macroblocks, as in Embodiment 1, and stores the codes in the temporary buffer 101, as shown in FIG. 16a. And then, the variable length coder 5bstores a leading address Ak of the codes which are stored for the current macroblock with the macroblock number K. And the number of bits mb_bit of the codes, generated for the current macroblock, is outputted to the rate control unit 102.
[0253] Next, the rate control unit 102 judges that the foregoing Relation (1) is satisfied or not. If Relation (1) holds, the rate control unit 102 controls the write address of the temporary buffer 101 to be Ak, and output the signal indicating the selection of the fixed code to the memory 9 and the variable length coder 5b.
[0254] In response to the signal indicating the selection of the fixed code, the variable length coder 5boutputs the fixed code, as predetermined f or each encoding type of the VOP, to the temporary buffer 101. At this time, the write address of the temporary buffer 101 has already been returned to Ak, so that the code of the macroblock number K is overwritten with the fixed code. As in the data construction of the temporary buffer shown in FIG. 16b, therefore, the fixed code is stored next to the code of the macroblock number K−1.
[0255] In response to the flag indicating the selection of the fixed code, following operation is executed as described in Embodiment 1. That is, the memory 9 stores the constant γ in the decoded image data area of the macroblock number K, if in the I-VOP case. In memory 9, the decoded image data of the macroblock with macroblock number K in the previous VOP is copied to the decoded image area for the macroblock with macroblock number K in the current VOP, in the P-VOP case.
[0256] With the construction thus far described, in Embodiment 5, the variable length coder 5bcan be given the functions of the following two elements: the elements for encoding each macroblock; and the elements for outputting the fixed code prepared for each encoding type of the VOP, thereby to reduce the size of the circuit.
[0257] Embodiment 6
[0258] FIG. 17 shows an encoding system according to Embodiment 6 of the invention. In FIG. 17, numeral 1 designates a subtracter for receiving the external input signals as its first input. The output of the subtracter 1 is fed through the DCT 2 and the quantizer 3 to the first input of a variable length coder 5cand the dequantizer 6.
[0259] The output of the dequantizer 6 is fed through the IDCT 7 to the first input of the adder 8. The output of this adder 8 is fed to the first input of the memory 9, the output of which is fed to the first input of the predicted image former 10 and the first input of the motion detector 11.
[0260] The external input signal is fed to the second input of the motion detector 11, the output of which is fed to the second input of the predicted image former 10 and to the motion vector predictor 12. The output of the predicted image former 10 is fed to the second input of the subtracter 1 and the second input of the adder 8.
[0261] On the other hand, the output of the motion vector predictor 12 is fed to the second input of the variable length coder 5c. Here, the encoding unit is constructed to include the aforementioned elements, from the subtracter 1 for receiving the external input signal to the variable length coder 5cfor outputting the variable length code corresponding to the external input signal (although the construction disclosed herein is only one example so that it could be exemplified by the known one capable of encoding the external input signal in a corresponding manner).
[0262] The first output of the variable length coder 5cis fed to the first input of the temporary buffer 101, and the second output of the variable length coder 5cis fed to the input of the rate control unit 102.
[0263] The temporary buffer 101 is fed at its second input with the output of the fixed code output unit 104 and at its third input with the first output of the rate control unit 102. The output of the temporary buffer 101 is fed to the first input of the transmission buffer 103.
[0264] The second output of the rate control unit 102 is fed to the second input of the memory 9, and the third output of the rate control unit 102 is fed to the second input of the transmission buffer 103. The output of the transmission buffer 103 is outputted (transmitted) as the bitstream.
[0265] The bitstream thus outputted (transmitted) is received and decoded on the side of the decoding system.
[0266] Here will be described the operations.
[0267] Embodiment 6 is different from Embodiment 1 in that the DC/AC prediction is not made where the encoding type is the intra. Specifically, the variable length coder 5cencodes the quantized DCT coefficients which are outputted from the quantizer 3. In the case of the encoding system conforming to the international standard of ITU-T H.263, for example, the DC component is always encoded at 8 bits in the I-VOP.
[0268] Therefore, the fixed code output unit 104 outputs the fixed codes of the macroblock for the I-VOP, in which: the DC component of each block is 128; all the AC components are 0; and dquant=0.
[0269] Without the DC prediction of this case, therefore, a new video packet need not be constructed from the current macroblock even where the fixed code of the fixed code output unit 104 is selected. In I-VOP case, therefore, the rate control unit 102 selects either the fixed code of the fixed code output unit 104 or the output of the variable length coder 5c, and stores it in the temporary buffer 101 in accordance with the flow charts any of FIG. 9, 11, 13 or 14.
[0270] In P-VOP case, as in Embodiments 1 to 4, either the output (the fixed code) of the fixed code output unit 104 or the output of the variable length coder 5cis selected and stored in the temporary buffer 101 in accordance with the flow charts any of FIG. 9, 11, 13 or 14.
[0271] Embodiment 7
[0272] In Embodiment 6, either the output of the variable length coder 5cor the output (the fixed code) of the fixed code output unit 104 is stored in the temporary buffer 101, and is transferred from the temporary buffer 101 to the transmission buffer 103. Where the data structure is not the data partition type or where the data rearrangement does not need, the temporary buffer 101 can be omitted (in this modification, the storage unit corresponds to the transmission buffer 103). For example, the construction may be modified such that either the output of the variable length coder 5cor the output (the fixed code) of the fixed code output unit 104 is inputted directly to the transmission buffer 103.
[0273] In the case of the encoding system conforming to the international standard of ITU-T H.263 in the case of no data partition, for example, the bitstream to be outputted from the transmission buffer 103 has a construction shown in FIG. 18. In the case of the data partition type of the MPEG4 as shown in FIG. 7bor FIG. 8b, the codes of each macroblock are divided into following categories (1) to (3):
[0274] (1) mcbpc, dquant and the DC component,
[0275] (2) ac_pred_flag and cbpy,
[0276] (3) the coefficient data of each block.
[0277] But, in H.263, the codes of a plurality of macroblocks are not collectively constructed for the every category. Accordingly, it is unnecessary to rearrange the generated codes for the every macroblocks.
[0278] Therefore, if the variable length coder 5coutputs the codes of the macroblocks in accordance with the format shown in FIG. 18, the temporary buffer 101 for the rearrangement of. the codes can be eliminated.
[0279] FIG. 19 shows an encoding system according to Embodiment. 7 of the invention. In FIG. 19, numeral 1 designates a subtracter for inputting the external input signals as its first input. The output of the subtracter 1 is fed through the DCT 2 and the quantizer 3 to the first input of the variable length coder 5cand to the dequantizer 6.
[0280] The output of the dequantizer 6 is fed through the IDCT 7 to the first input of the adder 8. The output of the adder 8 is fed to the first input of the memory 9, the output of which is fed to the first input of the predicted image former 10 and to the first input of the motion detector 11.
[0281] The external input signal is fed to the second input of the motion detector 11, the output of which is fed to the second input of the predicted image former 10 and to the motion vector predictor 12. The output of the predicted image former 10 is fed to the second input of the subtracter 1 and the second input of the adder 8.
[0282] The output of the motion vector predictor 12 is fed to the second input of the variable length coder 5c.
[0283] The first output of the variable length coder 5cis fed to the first input of the transmission buffer 103, and the second output of the variable length coder 5cis fed to the rate control unit 102.
[0284] The output (the fixed code) of the fixed code output unit 104 is fed to the second input of the transmission buffer 103 and the first output of the rate control unit 102 is fed to third input of the transmission buffer 103. On the other hand, the second output of the rate control unit 102 is fed to the second input of the memory 9.
[0285] The output of the transmission buffer 103 is outputted (transmitted) as the bitstream. This bitstream outputted (transmitted) is received and decoded on the side of the decoding system.
[0286] Here will be described the operations.
[0287] Embodiment 7 is different from Embodiment 6 in that the variable length coder 5cand the fixed code output unit 104 output the fixed codes to the transmission buffer 103. The rate control unit 102 judges for each macroblock which of the output of the variable length coder 5cor the output (fixed code) of the fixed code output unit 104 is to be selected, on the basis of the number of bits for the codes outputted from the variable length coder 5c, as in Embodiment 6. And the rate control unit 102 controls such that the selected output is stored in the transmission buffer 103.
[0288] Here in the foregoing Embodiments 1 to 7, in the setting of the maximum number of bits Tmax for each VOP, the reading rate R of the transmission buffer 103 is constant rate. Even if the reading rate is variable rate, the maximum number of bits Tmax can be set to prevent the overflow of the transmission buffer 103 or the underflow of the VBV buffer.
[0289] The case in which the reading rate R of the transmission buffer 103 is variable, corresponds to the case in which the maximum transmission rate is determined and the transmission rate is shared within the maximum transmission rate by the kinds (e.g., the video or the audio) of transmission information.
[0290] In this case, too, on the basis of the flow charts of FIGS. 9 to 14, by selectively storing either the output of the encoding unit for encoding each macroblock or the fixed code fixed for each encoding type of the VOP, the control can be so made that the number of bits for each VOP may not exceed the maximum value Tmax.
[0291] In the foregoing Embodiments 1 to 7, there have been described the case of the data partition type of MPEG4 and the case of the international standard of ITU-T H.263. In the case of the non data partition type or in the case of MPEG2, however, the number of bits can be controlled with a construction similar to the aforementioned ones.
[0292] Moreover, this invention or aforementioned embodiments of this invention could also be applied to the case that the input signal is not 4:2:0 video format signal or to the case in which the VOP (i.e., the unit image) is not rectangular.
[0293] The invention may be embodied in other specific forms without departing from the spirit or essential parts thereof. The above embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
[0294] The entire disclosure of Japanese Patent Application No. 2000-127291 filed on Apr. 27, 2000 including specification, claims, drawings and summary are incorporated herein by reference in its entirety.
Claims
What is claimed is:
1. An encoding system comprising:
Tmax≦vbv_bits+Rp, and Rp=R/F
wherein Tmax designates the maximum number of bits , Rp designates a number of bits to be read from the storage unit for the unit image, R designates a bit rate for reading bits from said storage unit, F designates a rate for encoding the unit image, and vbv_bits designates an occupied space of the VBV buffer in the previous unit image.
an encoding unit operable to encode external input signals of a macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output unit operable to output a fixed code predetermined for an encoding type of said unit image;
a storage unit operable to store either said coded data outputted from said encoding unit or said fixed code outputted from said fixed code output unit for said macroblock; and
a rate control unit operable to determine which of said coded data and said fixed code is stored in said storage unit in accordance with a number of bits encoded by said encoding unit for said coded data,
wherein said rate control unit determines a maximum number of bits, as can be generated by encoding the unit image, for each macroblock of said unit image, and determines which of the coded data outputted from said encoding unit and the fixed code outputted from said fixed code output unit on the basis of said maximum number of bits, and
wherein said rate control unit determines the maximum number of bits on the basis of the following:
2. An encoding system comprising:
Tmax≦min (vbv_bits+Rp, Bs−B), and Rp=R/F,
wherein Tmax designates the maximum number of bits, Rp designates a number of bits to be read from the storage unit for the unit image, R designates a bit rate for reading bits from said storage unit, F designates a rate for encoding unit image, vbv_bits designates an occupied space of the VBV buffer in the previous unit image, Bs designates a capacity of said storage unit, and B designates an occupied space of said storage unit.
an encoding unit operable to encode external input signals of a macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output unit operable to output a fixed code predetermined for an encoding type of said unit image;
a storage unit operable to store either said coded data outputted from said encoding unit or said fixed code outputted from said fixed code output unit for said macroblock; and
a rate control unit operable to determine which of said coded data and said fixed code is stored in said storage unit in accordance with a number of bits encoded by said encoding unit for said coded data,
wherein said rate control unit determines a maximum number of bits, as can be generated by encoding the unit image, for each macroblock of said unit image, and determines which of the coded data outputted from said encoding unit and the fixed code outputted from said fixed code output unit on the basis of said maximum number of bits, and
wherein said rate control unit determines the maximum number of bits on the basis of either the following relationships:
4. An encoding system comprising:
Sc+mb_bit+M×L+α>Tmax
an encoding unit operable to encode external input signals of a macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output unit operable to output a fixed code predetermined for an encoding type of said unit image;
a storage unit operable to store either said coded data outputted from said encoding unit or said fixed code outputted from said fixed code output unit for said macroblock; and
a rate control unit operable to determine which of said coded data and said fixed code is stored in said storage unit in accordance with a number of bits encoded by said encoding unit for said coded data,
wherein said rate control unit determines a maximum number of bits, as can be generated by encoding the unit image, for each macroblock of said unit image, and determines which of the coded data outputted from said encoding unit and the fixed code outputted from said fixed code output unit on the basis of said maximum number of bits, and
wherein said rate control unit determines to store in said storage unit said fixed code, as outputted by said fixed code output unit, for a current macroblock of said unit image,
wherein a relationship among a number of bits mb_bit of the codes outputted by the encoding unit for the current macroblock, a number of bits Sc for a previous macroblock of said unit image, the maximum number of bits Tmax of said unit image, a number M of the macroblocks to be encoded subsequent to the current macroblock of said unit image, a code length L of the fixed code to be outputted by said fixed code output unit for said unit image, and a number of bits α of additional codes to be generated at the unit of a video packet of said unit image is expressed by:
(wherein α≧0).
5. An encoding system comprising:
Sc+mb_bit+M×L+α>Tmax
an encoding unit operable to encode external input signals of a macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output unit operable to output a fixed code predetermined for an encoding type of said unit image;
a storage unit operable to store either said coded data outputted from said encoding unit or said fixed code outputted from said fixed code output unit for said macroblock; and
a rate control unit operable to determine which of said coded data and said fixed code is stored in said storage unit in accordance with a number of bits encoded by said encoding unit for said coded data,
wherein said rate control unit determines a maximum number of bits, as can be generated by encoding the unit image, for each macroblock of said unit image, and determines which of the coded data outputted from said encoding unit and the fixed code outputted from said fixed code output unit on the basis of said maximum number of bits, and
wherein said rate control unit makes a control to store in said storage unit said fixed code, as outputted by said fixed code output unit, not only for a current macroblock but also for an M number of macroblocks to be encoded subsequent to said current macroblock of said unit image, and
the relationship among a number of bits mb_bit of the codes outputted by the encoding unit for said current macroblock, a number of bits Sc of a previous macroblock constructing said unit image, the maximum number of bits Tmax of said unit image, a number M of the macroblocks to be encoded subsequent to the current macroblock of said unit image, a code length L of the fixed code to be outputted by said fixed code output unit for said unit image, and the number of bits α of additional codes to be generated at the unit of a video packet constructing said unit image is expressed by:
(wherein α≧0).
6. An encoding system according to claim 5, wherein
said rate control unit constructs a new video packet from the current macroblock, and
the encoding type of the unit image is intra coding.
7. An encoding method comprising:
Tmax≦vbv_bits+Rp, and Rp=R/F
wherein Tmax designates the maximum number of bits, Rp designates a number of stored bits to be read for the unit image, R designates a bit rate for reading stored bits, F designates a rate for encoding the unit image, and vbv_bits designates an occupied space of a VBV buffer for a previous unit image.
an encoding step for encoding external input signals of each macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output step for outputting a fixed code predetermined for an encoding type of said unit image;
a rate control step for determining one of said coded data and said fixed code in accordance with a number of bits encoded at said encoding step for said coded data; and
a storage step for storing determined one of said coded data and said fixed code for said macroblock,
wherein said rate control step determines a maximum number of bits, as can be generated by encoding the unit image, and determines said one of said coded data and said fixed code on the basis of said maximum number of bits, and
wherein said rate control step determines the maximum number of bits Tmax on the basis of the following relationship:
8. An encoding method comprising:
Tmax≦min (vbv_bits+Rp, Bs−B), and Rp=R/F
wherein Tmax designates the maximum number of bits, Rp designates a number of stored bits to be read for the unit image, R designates a bit rate for reading stored bits, F designates a rate for encoding the unit image, vbv_bits designates an occupied space of a VBV buffer for a previous unit image, Bs designates an allowed storage capacity, and B designates an occupied storage space.
an encoding step for encoding external input signals of each macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output step for outputting a fixed code predetermined for an encoding type of said unit image;
a rate control step for determining one of said coded data and said fixed code in accordance with a number of bits encoded at said encoding step for said coded data; and
a storage step for storing determined one of said coded data and said fixed code for said macroblock,
wherein said rate control step determines a maximum number of bits, as can be generated by encoding the unit image, and determines said one of said coded data and said fixed code on the basis of said maximum number of bits, and
wherein said rate control step determines the maximum number of bits substantially on the basis of either the following relationships:
10. An encoding method comprising:
Sc+mb_bit+M×L+α>Tmax
an encoding step for encoding external input signals of each macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output step for outputting a fixed code predetermined for an encoding type of said unit image;
a rate control step for determining one of said coded data and said fixed code in accordance with a number of bits encoded at said encoding step for said coded data; and
a storage step for storing determined one of said coded data and said fixed code for said macroblock,
wherein said rate control step determines a maximum number of bits, as can be generated by encoding the unit image, and determines said one of said coded data and said fixed code on the basis of said maximum number of bits, and
wherein said rate control step makes a control to store in said storage step said fixed code, as given at said fixed code output step, for a current macroblock of said unit image,
wherein a relationship among a number of bits mb_bit of the codes given at the encoding step for the current macroblock, the number of bits Sc for a previous macroblock of said unit image, the maximum number of bits Tmax of said unit image, a number M of the macroblocks to be encoded subsequent to the current macroblock, a code length L of the fixed code to be given at said fixed code output step for said unit image, and a number of bits a of additional codes to be generated at the unit of a video packet constructing said unit image is expressed by:
(wherein α≧0).
11. An encoding method comprising:
Sc+mb_bit+M×L+α>Tmax
an encoding step for encoding external input signals of each macroblock of a unit image, to output coded data corresponding to said external input signals;
a fixed code output step for outputting a fixed code predetermined for an encoding type of said unit image;
a rate control step for determining one of said coded data and said fixed code in accordance with a number of bits encoded at said encoding step for said coded data; and
a storage step for storing determined one of said coded data and said fixed code for said macroblock,
wherein said rate control step determines a maximum number of bits, as can be generated by encoding the unit image, and determines said one of said coded data and said fixed code on the basis of said maximum number of bits, and
wherein said rate control step determines said fixed code for storage in association with the current macroblock, in addition to a number M of macroblocks to be encoded subsequent to said current macroblock for said unit image, and
the relationship among a number of bits mb_bit of the codes given at the encoding step for said current macroblock, the number of bits Sc of a previous macroblock constructing said unit image, the maximum number of bits Tmax of said unit image, a number M of the macroblocks to be encoded subsequent to the current macroblock constructing said unit image, a code length L of the fixed codes to be given at said fixed code output step for said unit image, and the number of bits α of additional codes to be generated at the unit of a video packet constructing said unit image is expressed by:
(wherein α≧0).
12. An encoding method according to claim 11, wherein
said rate control step constructs a new video packet from the current macroblock, and
the encoding type of the unit image is intra coding.
13. A method for encoding an input signal according to an encoding type, comprising:
generating a fixed code associated with a minimum number of bits at which a processing unit of the input signal can be encoded based on the encoding type;
generating a variable length code for a current processing unit of the input signal based on an encoding type; and
selecting one of the fixed code and the variable length code to encode the current processing unit,
wherein the selecting step includes determining whether using the variable length code for encoding the current processing unit would create at least one of an overflow and underflow condition in a buffer,
wherein the selecting step selects the fixed code when the determining step determines that the overflow condition would be created, and
wherein the selecting step selects the fixed code for a subsequent processing unit if the fixed code was selected for the current processing unit of the input signal.
Patent Citations (11)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US5631644(A) | 1997-05-01 | Katata et al. | Examiner |
| US5995151(A) | 1999-11-01 | Naveen et al. | Examiner |
| US6044396(A) | 2000-03-01 | Adams | Examiner |
| US6094455(A) | 2000-07-01 | Katta | Examiner |
| US6188700(B1) | 2001-02-01 | Kato et al. | Examiner |
| US6263020(B1) | 2001-07-01 | Gardos et al. | Examiner |
| US6366614(B1) | 2002-04-01 | Pian et al. | Examiner |
| US6366704(B1) | 2002-04-01 | Ribas-Corbera et al. | Examiner |
| EP660612(A1) | 1995-06-01 | Applicant | |
| EP660612(A2) | 1995-06-01 | Applicant | |
| EP836329(A1) | 1998-04-01 | Applicant |
Non-Patent Literature (4)
- Office Action dated Mar. 12, 2004 from the Chinese Patent Office, with translation.Applicant
- Vetro, A. et al.: “MPEG-4 Rate Control for Multiple Video Objects,” IEEE Transactions on Circuits and Systems for Video Technology, IEEE, Inc. New York vol. 9, No. 1, Feb. 1999, pp. 186-199.Applicant
- Wiegand, T. et al.: “Rate-Distortion Optimized Mode Selection for Very Low Bit Rate Video Coding . . . ” IEEE Transactions on Circuits and Systems for Video Technology, IEEE Inc. New York, vol. 6, No. 2, Apr. 1, 1996.Applicant
- John D. Villasenor and Ya-Qin Zhang “Robust Video Coding Algorithms and Systems”—Proceedings of the IEEE, vol. 87, No. 10. Oct. 1999.Applicant