Picture Signal Coding Apparatus

This invention relates to a picture signal coding apparatus which is of particular utility when employed in the coding of still pictures.

For the reduction of the cost of picture signal transmission, there have heretofore been proposed DPCM, .DELTA.M and like coding systems, for example, in A. Habihi and G. S. Robinson "A Survey of Digital Picture Coding" Computer, 7,5, pp. 22-34 (May 1974). These known systems are called prediction coding systems, in which a prediction signal is produced based on a sample preceding a reference point and a prediction error signal which is a difference between a sample to be coded and the prediction signal is quantized for transmission. Due to the property of the picture signal, the prediction error signal occurs frequently in the range of which its amplitude is small, so that even if the prediction error signal is quantized roughly, the picture quality is not significantly deteriorated; therefore the number of bits necessary for coding can be reduced as compared with that in ordinary PCM in which a sample is coded as it is.

With these conventional prediction coding systems, however, the prediction error signal must be transmitted for every sample point, and the number of prediction error signal quantizing levels cannot be reduced significantly because of contributing factors to deterioration of the picture quality such as slope over load, granular noise, false contour, etc. For these reasons, the number of bits used is relatively large as a whole.

As a result of novelty search conducted by the European Patent Office, there was not found any particular prior literature corresponding to this invention but U.S. Pat. Nos. 3,403,226, 3,940,555 and 3,984,626 were cited as reference literatures. Now, a description will be given of U.S. Pat. No. 3,403,226 of the type in which a picture frame is divided into blocks. In this patent, one of the picture elements is selected as a typical picture element for each block and coded by the PCM technique so that it can represent, by itself, its level, and the difference between the typical picture element and each of the other picture elements of the same block is coded into a PCM code.

Accordingly, though the number of quantizing levels for the latter PCM code is smaller than that for the typical picture element, transmission of the PCM codes requires a plurality of bits, resulting in an appreciably large number of bits being needed as a whole.

An object of this invention is to provide a picture signal coding apparatus which enables coding with less bits than those used in the prior art.

According to this invention, a frame of a gray-scaled picture is divided into a plurality of blocks, and based on the statistical property of the luminance levels of picture elements in each block, the picture elements of the block are separated into gray components and resolution components, thereafter being coded. For instance, a threshold value is set in each block based on the luminance distribution therein, and the luminance level of each picture element is checked to determine whether it is higher or lower than the threshold value. From the check results and the luminance levels of the picture elements are obtained two typical luminance levels. For each block, the two typical luminance levels and the check results are transmitted in the form of codes. Letting a.sub.0 and a.sub.1 represent the typical luminance levels, x.sub.t represent the threshold value, x.sub.i represent the luminance level of each picture element, and y.sub.i represent the luminance level of the picture element after being converted by coding, decoding is achieved by the following conversion:

where when x.sub.i .gtoreq.x.sub.t, .phi..sub.i =1, where when x.sub.i <x.sub.t, .phi..sub.i =0, and where .phi..sub.i is the complement of .phi..sub.i. The typical luminance levels a.sub.0 and a.sub.1 are gray components, and .phi..sub.i is a resolution component.

The typical luminance levels a.sub.0 and a.sub.1 and the threshold value x.sub.t are determined, for example, in the following manner.

In accordance with a first method, a condition x=y that mean luminance levels x and y of the picture elements in each block remain unchanged before and after coding, is given, and the mean luminance level of the picture elements in each block before coding is used as the threshold value x.sub.t, namely, x.sub.t =x, and the following mean square errors before and after conversion in each block ##EQU1## are minimized, whereby to obtain the typical luminance levels a.sub.0 and a.sub.1 and the threshold value x.sub.t. That is, letting x.sub.i represent the luminance level of each picture element is each block, N.sub.0 represent the number of picture elements in the block, x represent the means luminance level of the picture elements in the block, x.sub.L represent the mean luminance level of the group of those picture elements whose luminance levels are lower than the mean luminance level x, N.sub.1 represent the number of such picture elements, x.sub.H represent the mean luminance level of the group of those picture elements whose luminance levels are higher than the mean luminance level x and N.sub.2 represent the number of such picture elements, the mean luminance levels x, x.sub.L and x.sub.H are given as follows: ##EQU2## obtaining the condition that the mean square error powers .epsilon..sub.2 for the typical luminance levels a.sub.0 and a.sub.1 are minimized, namely, (.delta./.delta.a.sub.0).epsilon..sup.2 =0 and (.delta./.delta.a.sub.1).epsilon..sup.2 =0, the typical luminance levels a.sub.0 and a.sub.1 are given as follows:

This method is referred to as the two average value method.

A second method is to obtain the typical luminance levels a.sub.0 and a.sub.1 and the threshold value x.sub.t from the abovesaid condition x=y, a condition x.sup.2 =y.sup.2 that luminance powers x.sup.2 and y.sup.2 of the picture elements in the blocks before and after conversion remain unchanged before and after coding, and a condition (.delta./.delta.x.sub.t).epsilon..sup.2 =0 that the mean error power becomes minimized in respect of the threshold value x.sub.t. That is, letting x.sub.L' represent the mean luminance level of the group of those picture elements whose luminance levels are lower than the threshold value, N.sub.1 represents the number of such picture elements, x.sub.H' represent the mean luminance level of the group of those picture elements whose luminance levels are higher than the threshold value, N.sub.2 represent the number of such picture elements, x represent the mean luminance level of picture elements in each block before coding and .delta..sup.2 represent variance of luminance levels of the picture elements, the means luminance levels x.sub.L', x.sub.H' and variance .delta..sup.2 are given as follows: ##EQU3##

The threshold value x.sub.t is given as follows: ##EQU4##

The typical luminance levels a.sub.0 and a.sub.1 are given as follows: ##EQU5##

This method is referred to as the variance method.

In the case of this method, if it were supposed x.sub.t =x so as to simplify the process of obtaining the threshold value x.sub.t, the mean square error .epsilon..sup.2 is not always minimized, but the typical luminance levels a.sub.0 and a.sub.1 are the same as those obtainable in the above.

A third method is to simultaneously satisfy the conditions ##EQU6## utilized in the first and second methods. In this instance, the threshold value x.sub.t is expressed as follows: ##EQU7## and the typical luminance levels a.sub.0 and a.sub.1 are as follows: ##EQU8## This method is referred to as the variable threshold method.

Thus, in this invention, there are transmitted the typical luminance levels a.sub.0 and a.sub.1 in each block and the resolution component indicative of the level a.sub.0 or a.sub.1 at which each picture element is to be decoded. In other words, for each picture element, information "1" or "0" is transmitted, and for each block, the typical luminance levels a.sub.0 and a.sub.1 of the block are transmitted. Accordingly, the number of bits used is markedly reduced as a whole, and in addition, since the picture frame is divided into blocks, excellent picture reconstruction can be achieved.

FIG. 1 is a block diagram illustrating an embodiment of the picture signal coding apparatus according to this invention;

FIG. 2 is a diagram showing an example of a picture divided into blocks;

FIG. 3 is a block diagram illustrating an example of a coding apparatus 14 utilized in FIG. 1;

FIG. 4 is a block diagram showing an example of a threshold value calculator 17 used in FIG. 3;

FIG. 5 is a block diagram showing an example of an a.sub.0 calculator 19 utilized in FIG. 3;

FIG. 6 is a block diagram illustrating an example of a decoding apparatus for decoding a coded signal from the coding apparatus of this invention;

FIG. 7 is a block diagram showing an example of a decoder 42 used in FIG. 6;

FIG. 8 is a block diagram showing another embodiment of coding apparatus of this invention which employs a microprocessor;

FIG. 9 is a flowchart showing the basic operations for coding;

FIG. 10 is a flowchart showing a coded block read-in routine;

FIG. 11 is a flowchart showing a threshold value calculating routine;

FIG. 12 is a flowchart showing a resolution component discriminating routine and a typical luminance level calculating routine; and

FIG. 13 is a flowchart showing the basic operation for decoding.

Referring now to FIG. 1, a picture signal picked up by a TV camera 11 is quantized by an analog-to-digital converter 12 for each picture element, thereafter being stored in a video memory 13. The picture picked up by the TV camera 11 is divided into, for instance, N.times.M blocks, as depicted in FIG. 2, and each block is composed of n.times.m picture elements. The picture signal stored in the video memory 13 is read in a coding apparatus 14 for each block.

Letting x.sub.i represent the luminance level of each picture element in the block, the luminance level x.sub.i is converted by the coding apparatus 14 to y.sub.i in the following process:

where when x.sub.i .gtoreq.x.sub.t, .phi..sub.i =1 and .phi..sub.i =0, and when x.sub.i <x.sub.t, .phi..sub.i =0 and .phi..sub.i =1. The values x.sub.t, a.sub.0 and a.sub.1 are obtained from the distribution of luminance levels of the picture elements in the block by one of the three methods described previously. By the coding apparatus 14, picture information in the block is converted to gray components a.sub.0 and a.sub.1 and resolution components (.phi..sub.0, .phi..sub.1, . . . .phi..sub.q) (q=m.times.n). These converted codes are stored in a buffer memory 15, from which they are read out thereof at a constant speed suitable to a transmission line and provided thereon via a modem 16.

The coding apparatus 14 is constructed, for example, as illustrated in FIG. 3. The output from the video memory 13 is applied to a threshold value calculator 17, one input terminal of a comparator 18, an a.sub.0 calculator 19 and an a.sub.1 calculator 21, respectively. In the threshold value calculator 17, for example, as shown in FIG. 4, the picture element signals applied via a terminal 22 from the video memory 13 are sequentially accumulated in an adder 23, the initial value of which is reset at zero. Upon completion of addition of all the picture elements of the block, the output from the adder 23 is provided to a dividend input terminal of a divider 24, in which the input thereto is divided by a divisor stored in a divisor register 25. The value of the divisor is the number of all picture elements of the block m.times.n, and is preset. In this manner, the mean luminance level value x of the picture elements in the block is derived as the threshold value x.sub.t at a terminal 27.

The threshold value x.sub.t thus obtained is applied to the other input terminal of the comparator 18 in FIG. 3. The comparator 18 compares the luminance level x.sub.i of each picture element with the threshold value x.sub.t to provide the resolution component (.phi..sub.i) to the buffer memory 15. The compared output .phi..sub.i and its inverted output .phi..sub.i from an inverter 28 is applied to the a.sub.1 calculator 21 and the a.sub.0 calculator 19, respectively, to obtain the gray components a.sub.0 and a.sub.1.

In the a.sub.0 calculator 19, for example, as depicted in FIG. 5, the picture element signals x.sub.i from the video memory 13 are supplied via a terminal 31 to a gate 29. The resolution component .phi..sub.i from the inverter 28 is also applied as a control signal to the gate 29 via a terminal 32. The gate 29 is opened or closed depending upon whether the resolution component .phi..sub.i is 1 or 0. Accordingly, in the case where x.sub.i <x.sub.t, namely, .phi..sub.i =1, the picture element signals are accumulated in an adder 33 via the gate 29. Thus, all picture elements in the block that x.sub.i <x.sub.t are added together in the adder 33.

The number of the gate control signals .phi..sub.i at the terminal 32 in this case is counted by a divisor counter 34 to obtain the number N.sub.1 of the picture elements that x.sub.i <x.sub.t for each block. The divider 35 divides the added result ##EQU9## of the adder 33 by the picture element number N.sub.1 stored in the divisor counter 34, providing the gray component a.sub.0 at a terminal 36. The a.sub.1 calculator 21 in FIG. 3 is identical in construction with the a.sub.0 calculator 19 and produces the gray component a.sub.1 at a terminal 37. The gray components a.sub.0 and a.sub.1 thus obtained are stored in the buffer memory 15 at suitable timings.

The picture signal coded, as described above, is decoded in the following manner. For instance, as shown in FIG. 6, the coded picture signal applied via a terminal 38 from the transmission line is received and demodulated by a modem 39. The demodulated signal is once stored in a buffer memory 41 and then supplied to a decoder 42, in which each picture element signal is decoded, thereafter being written in a frame memory 43. The content of the frame memory 43 is read out thereof and converted by a digital-to-analog converter 44 to an analog signal, which is reconstructed by a video monitor 45 into a picture.

The decoder 42 is constructed, as shown in FIG. 7. A demultiplexer 46' separates the signal from the buffer memory 41 into the gray components a.sub.0 and a.sub.1 and the resolution component .phi..sub.i. The components a.sub.0, a.sub.1 and .phi..sub.i are transferred to buffer registers 48', 49' and 51, respectively. By the information .phi..sub.i from the buffer register 51, gates 52 and 53 are controlled to open or closed to permit the passage therethrough of either one of the information a.sub.0 and a.sub.1 stored in the buffer registers 48' and 49', providing at a terminal 54 the information as each picture element signal in the block. That is, when .phi..sub.i =0, i.e. x.sub.i <x.sub.t on the transmitting side, the gate 52 is opened by the inverted output .phi..sub.i from an inverter 55 to pass on the information a.sub.0 in the buffer register 48' to the terminal 54, whereas when .phi..sub.i =1 i.e. x.sub.i .gtoreq.x.sub. t on the transmitting side, the gate 53 is opened by the information .phi..sub.i to pass on the information a.sub.1 in the buffer register 49' to the terminal 54. Though not shown, since the position of each picture element in each block is known, the output at the terminal 54 is written in the frame memory 43 at an address corresponding to the abovesaid position. When the picture element signals of all blocks have thus been decoded and decoded information of one frame has been stored in the frame memory 43, a still picture of one frame is reconstructed on the video monitor 45.

The coding and decoding described above can also be achieved by a stored program, using the so-called microcomputer. For example, as illustrated in FIG. 8, the output from the TV camera 11 is supplied to the analog-to-digital converter 12 for conversion into a digital signal, the output side of the analog-to-digital converter 12 being connected to a system bus 46. To the system bus 46 are also connected the video memory 13 and the digital-to-analog converter 44. Further, a central processing unit (CPU) 47, a read only memory (R0M) 48 and a random access memory (RAM) 49 are respectively connected to the system bus 46. The output side of the digital-to-analog converter 44 is connected to the video monitor 45.

The read only memory 48 stores therein a program for coding and decoding, and the central processing unit 47 sequentially reads out the program from the read only memory 48 and interprets and executes it, thereby to effect coding and decoding. The video memory 13 stores therein the output from the TV camera 11 in digital form or the decoded picture signal, and the stored content of the video memory 13 is read out thereof and converted by the digital-to-analog converter 44 to an analog signal for input to the video monitor 45. The random access memory 49 is utilized for temporarily storing data necessary for coding and decoding.

A basic flowchart for coding is illustrated in FIG. 9. Upon starting, the content N of a row block memory is made 1 in a step S.sub.1, and then the content M of a column block memory is made 1 in a step S.sub.2. In a step S.sub.3, picture element signals of the block designated by the contents N=1 and M=1 of the block memories are read in a temporary storage. Based on these picture element signals, the threshold value x.sub.t is calculated in a step S.sub.4. In a step S.sub.5, the threshold value x.sub.t is stored in a threshold memory corresponding to each picture element in the block. Next, in a step S.sub.6, a resolution component .phi..sub.IJ is discriminated from each picture element signal x.sub.IJ and the threshold value x.sub.t corresponding thereto, and the gray components, that is, the typical luminance levels a.sub.0 and a.sub.1 are calculated. Each block is composed of picture elements of n rows and m columns, I=1, 2, . . . m and J=1, 2, 3, . . . n.

In a step S.sub.7, the components a.sub.0, a.sub.1 and .phi..sub.IJ are transmitted, and then in a step S.sub.8, the content M of the column block memory is added with 1. In the next step S.sub.9, it is checked whether the memory content M has reached its maximum value Mm or not, and in the case where the content M is larger than the maximum value Mm, the content N of the row block memory is added with 1 in a step S.sub.10. In a step S.sub.11, it is checked whether the content N is in excess of its maximum value Nm, and if the content N is smaller than or equal to the maximum value Nm, the operation goes back to the step S.sub.2. Where the content M of the column block memory is smaller than or equal to the maximum value Mm in the step S.sub.9, the operation goes to the step S.sub.3. When the content N is in excess of the maximum value Nm in the step S.sub.11, it means completion of coding of the entire frame, and the operation is stopped.

A coded block read-in routine in the step S.sub.3 is shown, for example, in FIG. 10. At first, in a step S.sub.12 the content M of the column block memory is set to M of block numbers M and N to be read in, and the content N of the row block memory is made N. The content J of a block row address memory which indicates that addresses J and I in the block is made 1 in a step S.sub.13, and in a step S.sub.14 the content I of a block column address memory is made 1. Next, in a step S.sub.15, a calculation I+(M-1).times.m of a frame row address i and a calculation J+(N-1).times.n of a frame column address j are carried out. The luminance level x.sub.ij of picture element at the address (ij) is read out from the video memory 13 in a step S.sub.16 and then transferred to the random access memory 49. Next, in a step S.sub.17 the content I of the block column address memory is added with 1, and it is checked in a step S.sub.18 whether the content I is smaller than, equal to or larger than m. Where I.ltoreq.m, the operation goes back to the step S.sub.15, in which the luminance of the next picture element is read out and transferred. When I>m in step S.sub.18, the content J of the block row address memory is added with 1 in a step S.sub.19, and the result of the addition is checked in a step S.sub.20. Where J.gtoreq.n, the operation goes back to the step S.sub.14, whereas when J>n the block read-in operation is completed. In this manner, the addresses of the picture elements in each block on the frame are sequentially designated, and the picture element signals are read in a temporary storage.

A threshold value x.sub.t calculation routine in the step S.sub.5 in FIG. 9 is illustrated in FIG. 11. This routine starts with making the content SUM of a summing memory zero in a step S.sub.21. In the next step S.sub.22, the content J of the block column address memory is made 1, and in a step S.sub.23, the content I of the block row address memory is made 1. In a step S.sub.24, the picture element signal x.sub.IJ designated by the contents I and J of the address memories is added to the content SUM of the summing memory and the added result is stored therein. Thereafter, in a step S.sub.25, the content I of the block column address memory is added with 1 and the added result is checked in a step S.sub.26. In the case of I.ltoreq.m, the operation goes back to the step S.sub.24, in which luminance levels of the picture elements in the block are accumulated one after another. In the case of I>m, the operation proceeds to a step S.sub.27, in which the content J of the block row address memory is added with 1, and the result of addition is checked in a step S.sub.28. Where J.ltoreq.n, the operation goes back to the step S.sub.23, whereas where J>n the operation proceeds to a step S.sub.29, in which the accumulated value SUM in the summing memory is divided by the number of picture elements in the block, m.times.n, to obtain the threshold value x.sub.t, which is stored in the threshold value memory.

In FIG. 12 is shown the calculation routine of the components .phi..sub.IJ, a.sub.0 and a.sub.1 in the step S.sub.6 in FIG. 9. At first, in a step S.sub.30 contents S.sub.1 and S.sub.2 of first and second summing memories and contents N.sub.1 and N.sub.2 of first and second counter memories are respectively made zero. Then, in a step S.sub.31 the content J of the block row address memory is made 1, and in a step S.sub.32 the content I of the block column address memory is made 1. In a step S.sub.33 the luminance x.sub.IJ of the picture element designated by the contents I and J of these address memories and the threshold value x.sub.t are compared with each other. In the case of x.sub.IJ <x.sub.t, the operation proceeds to a step S.sub.34, in which the content .phi..sub.IJ of a resolution memory at the corresponding address is made 0, and in a step S.sub.35 the content N.sub.1 of the first counter memory is added with 1 and the picture element signal x.sub.IJ is accumulated to the content S.sub.1 of the first summing memory in a step S.sub.36. Where x.sub.IJ .gtoreq.x.sub.t in the step S.sub.33, the operation goes to a step S.sub.37, in which the content .phi..sub.IJ of the resolution memory at the corresponding address is made 1, and in a step S.sub.38 the content N.sub.2 of the second counter memory is added with 1 and the content S.sub.2 of the second summing memory is accumulatively added with x.sub.IJ in a step S.sub.39. The step S.sub.36 or S.sub.39 is followed by a step S.sub.40, in which the content I of the block column address memory is added with 1, and the result of addition is checked in a step S.sub.41. In the case of I.ltoreq.m, the operation goes back to the step S.sub.33, in which the next picture element signal is compared with the threshold value. In the case of I>m, the operation goes to a step S.sub.42, in which the content J of the block row address memory is added with 1. The result of addition is checked in a step S.sub.43, and if J.ltoreq.n, the operation goes back to the step S.sub.32, and if J>n, the content S.sub.2 of the second summing memory is divided by the content N.sub.2 of the second counter memory to obtain the typical luminance level a.sub.1 in a step S.sub.44. Next, in a step S.sub.45 the content N.sub.0 of the first counter memory is checked, and when the content N is not zero, the content S.sub.1 of the first summing memory is divided by the content N.sub.1 of the first counter memory to obtain the typical luminance level a.sub.0 in a step S.sub.46. If N.sub.1 =0 in the step S.sub.45, the typical luminance level a.sub.0 is made equal to a.sub.1 in a step S.sub.47.

Turning next to FIG. 13, the decoding operation will be described. Let it be assumed that the video memory 13 in FIG. 8 has stored therein the components a.sub.0, a.sub.1 and .phi..sub.IJ of one frame. At first, in a step S.sub.48 the content N of the row block memory is made 1, and in a step S.sub.49 the content M of the column block memory is made 1. Then, based on the contents M and N of these block memories, the components a.sub.0, a.sub.1 and .phi..sub.IJ of each picture element in the block are read in the random access memory 49 in a step S.sub.50. Next, in a step S.sub.51 the components a.sub.0 and a.sub.1 are written as an output y.sub.IJ in the video memory 13 at the corresponding address depending upon whether the component .phi..sub.IJ is 0 or 1. In the next step S.sub.52, the content M of the column block memory is added with 1 and its result is checked in a step S.sub.53. Where M.ltoreq.Mm, the operation goes back to the step S.sub.50, in which a code of the next block is read in a temporary storage. Where M>Mm, the content N of the row block memory is added with 1 in a step S.sub.54. Then, the content N is checked in a step S.sub.55, and if N.ltoreq.Nm, the operation goes back to the step S.sub.49, and if N>Nm, it means completion of decoding of all picture elements and the decoding operation is stopped.

The decoding operation may also be achieved in the following manner: At first, a picture element address (ij) on the frame is determined, the components a.sub.0, a.sub.1 and .phi..sub.ij of the block to which the address (ij) belong are read out of the video memory 13, decoding of y.sub.ij =.phi..sub.ij a.sub.0 +.phi..sub.ij a.sub.1 is effected and the decoded output y.sub.ij is written in the video memory 13 at the address (ij); such a decoded output is similarly obtained in connection with each of the other addresses.

As has been described in the foregoing, according to the picture signal coding apparatus of this invention, the picture frame is divided into a plurality of blocks and, for each block, at least two gray components and resolution components for each picture element are obtained, so that the minuteness and the tone wedge property of the picture are not impaired, and in addition, since a plurality of bits are not coded for each picture element, the number of bits used is small as a whole and the coding can be made highly efficient. Further, since the coding can be achieved by addition for each picture element or division for each block, the coding can be easily executed by a stored program, using a microprocessor, and the structure therefor can be made relatively simple as compared with that formed as a wired logic. As the resolution component is a binary signal and corresponds to each picture element, coding errors do not markedly degrade the picture quality. Decoding is achieved by controlling a gate circuit with the resolution component and the structure therefor can be formed easily.

In the foregoing, the blocks are each described to be square but may also be of a triangular, rhombic or like configuration. Moreover, the foregoing embodiment uses one threshold value and two typical luminance levels a.sub.0 and a.sub.1, but a plurality of threshold values may also be employed. In such a case, the picture element signals of one block are each classified into any one of a plurality of level ranges defined by the plurality of threshold values to obtain the resolution component .phi..sub.i. Consequently, this resolution component .phi..sub.i is represented by a plurality of bits. From this resolution component .phi..sub.i, each picture element signal and the threshold values, the typical luminance levels are calculated in the respective level ranges.