Apparatus and Method for Processing Signals of an Imaging Device

This invention relates to imaging devices, and particularly to an apparatus and method for processing signals of an imaging device having a most suitable digital signal processing function for a color video camera or the like.

An example of the digital signal processing for a video camera in association with this invention is described in Japanese Patent Publication Gazette No. 63-45153.

The solid-state imaging devices include various different types, and can be roughly divided into the MOS type and the CCD type. In general, the MOS type image sensor has multiple outputs, and the CCD type image sensor has a single output. From the viewpoint of digital signal processing, the CCD type sensor having a single output needs only one A/D converter, and is thus more advantageous than the MOS type sensor which requires a large number of A/D converters. The CCD type sensor includes various different types, and at present the most common one is the picture element mixture read type CCD sensor which is described in The Institute of Television Engineers of Japan, Technical report, TEBS101-1, ED836, pages 1 to 6. This CCD sensor is different in construction from that disclosed in Japanese Patent Publication Gazette No. 63-45153, but it is able to make digital signal processing using the same system.

The digital signal processing in the video camera using the picture element mixture read type CCD sensor has many problems to be solved, one of which is the quantize bit number problem with the A/D converter.

The A/D converter always converts an input voltage within a certain range into a constant digital value. At this time, since the input analog value is not converted into a digital value in one-to-one correspondence, error occurs. This error enters into the signal component as quantize noise.

It is generally known that the quantize bit number, under which the S/N ratio in the video signal processing is not seriously affected by the quantize noise, is 8 bits in the luminance signal and 6 bits in the chrominance signal at the final output stage of the camera.

In the video camera using the above CCD sensor for the digital signal processing, the following processings are performed for producing the luminance signal and chrominance signal, which processing increases the quantize noise.

For the luminance signal:

Gamma processing: The signal is raised to the power of 1/2.2 for the correction of the 2.2-power characteristic of the Braun tube. For this reason, the gain of the luminance signal of low level is increased, thus emphasizing the quantize noise.

For chrominance signals:

Gamma processing: Same as above.

Matrix: The chrominance signal such as RGB is produced by arithmetic processing of the sensor output signals associated with complementary color filters of magenta (Mg), cyan (Cy), yellow (ye) and green (G). At this time, since the chrominance signals of RGB are produced by subtraction processing, the produced RGB signal has a smaller level than the original signal level. Since the signal component is small, the noise component becomes apparently large so that the S/N ratio is decreased by quantize noise.

White balance: The level of the RGB signal produced by the above matrix is changed by the color temperature of the light source. For correcting this, the gains of the R and B signals are increased or decreased. At this time, since the quantize noise of the chrominance signals which are increased in gain is also emphasized at the same gain, the S/N ratio is decreased.

The quantize noise in the above processings is increased particularly for the chrominance signal. At the final output, in order that the quantize noise level included in the chrominance signal is made equal to that in the case of the 6-bit quantization, it is necessary that the quantize bit number of the A/D converter for converting the output signal from the sensor into a digital signal is selected to be at least about 10 bits. However, at present it is difficult to realize the A/D converter of 10 bits or above for video, and thus it is difficult to digitize the signal processing in the camera.

It is an object of the present invention to provide an apparatus and method for processing signals of an imaging device wherein the quantize noise in the signal processing side can be reduced and the problem with the A/D converter can be solved.

The above object can be achieved by providing a color temperature detecting circuit for detecting the color temperature of a light source which is illuminating an object to be picked up, gamma .gamma. correction circuits (called variable .gamma. correction circuits) of which the .gamma. characteristics are changed by a control signal fed from the outside, and a .gamma. control circuit for controlling the .gamma. correction characteristics of the variable .gamma. correction circuits in accordance with the detected color temperature information produced from the color temperature detection circuit.

The semiconductor used for the present solid-state imaging device is silicon (Si). The silicon Si has the photoelectric conversion characteristic of a high sensitivity to long wavelengths (red side) but of a low sensitivity to short wavelengths (blue side). To balance the sensitivities to red and blue components, a cyan filter or the like is used to suppress the red component of light incident on the sensor. Excessive reduction of the red component, however, will reduce the brightness sensitivity, and thus there is a limit to the correction for the balance of sensitivities to the red and blue components. Thus, since the filter characteristic is generally determined to increase the sensitivity the red component, the blue signal level is lower than the red signal level. On the other hand, when the color temperature of the light source is changed, the levels of the red and blue signals are changed. FIG. 9 shows the change of the red and blue signals. From FIG. 9 it will be seen that the red signal decreases with the increase of color temperature but increases with the decrease of color temperature, and that the blue signal increases with the increase of color temperature but decreases with the decrease of color temperature.

The .gamma. correction is to reversely correct the 2.2-power characteristic of the Braun tube. In other words, as shown in FIG. 2 by the correction curve 4, the gain is high for the low level region of the input signal (the .gamma. correction curve gradient is steep) but low for the high level region of the input signal (the .gamma. correction curve gradient is gentle).

Therefore, the increase of the quantize noise of the chrominance signal becomes adverse at a low color temperature at which the blue signal originally having a low signal level is further reduced, and when the blue signal is at a low level. In this case, in order to achieve white balance it is necessary to increase the total gain of the blue signal component. As a result, since the quantize noise is also increased at a high gain, the total S/N ratio of the signal is decreased.

When the detected color temperature is a low color temperature at which the level of the blue component is low, the .gamma. control circuit controls the variable .gamma. correction circuits to change the .gamma. characteristic in accordance with the detected color temperature so that the gain for low level is decreased. As a result, at a low temperature at which the quantize noise included in the blue signal increases, the gain for the low level region is decreased, and thus the quantize noise due to the .gamma. characteristic can be suppressed from increasing, and finally the quantize noise increase is prevented. Although the reproduced color is changed by the change of the .gamma. correction characteristics, the change of the reproduced color occurs at a considerably low color temperature and thus the picture quality deterioration is small as compared with the increase of the quantize noise.

Similarly, at a high color temperature at which the red component level is small, the gain of the red signal component is increased for white balance. This results in the increase of the quantize noise included in the red signal. In this case, the variable .gamma. correction circuits are controlled in accordance with the detected color temperature to change the .gamma. characteristics so that the gain of the input red signal is decreased in the low level region.

As a result, the quantize noise can be finally suppressed from increasing.

FIG. 1 is a block diagram of one embodiment of an apparatus for processing signals of an imaging device of this invention.

FIG. 2 shows the input/output characteristic of the variable .gamma. correction circuit.

FIGS. 3 and 4 are explanatory diagrams for explaining examples of the arrangement of the variable .gamma. correction circuit.

FIG. 5 is a block diagram of a second embodiment of the invention.

FIG. 6 is a block diagram of a third embodiment of the invention.

FIG. 7 is a block diagram of a fourth embodiment of the invention.

FIG. 8 is a block diagram of a fifth embodiment of the invention.

FIG. 9 shows the color temperature dependence characteristics of red and blue signal level.

FIG. 1 is a block diagram of the first embodiment of an apparatus for processing signals of an imaging device of the invention. This embodiment is the basic construction of the invention.

In FIG. 1, there are shown variable .gamma. correction circuits 1a, 1b and 1c of which the .gamma. correction characteristics can be changed, a color temperature detection circuit 2 for detecting a color temperature, and a .gamma. control circuit for controlling the .gamma. correction characteristics.

The operation of this embodiment is as follows.

First, the color temperature detection circuit 2 detects the temperature of a light source illuminating an object from which the image is picked up, and supplies the detected temperature value to the .gamma. control circuit 3. The .gamma. control circuit 3 decides from the supplied detected color temperature value if the present temperature is the low color temperature or high color temperature at which the quantize noise of chrominance signals increases to cause problems (generally the quantize noise of blue signal increases only at low color temperatures as described above). If the detected temperature is in the region in which the quantize noise increases, the .gamma. control circuit controls the variable .gamma. correction circuits 1a, 1b and 1c to change the .gamma. correction characteristics so that the gains of input signals are low in the low level region. The variable .gamma. correction circuits 1a, 1b and 1c make .gamma. correction to the input red (R) signal, green (G) signal and blue (B) signal as set by the .gamma. control circuit. In this case, at the low temperature or the high temperature, the gain of the B signal or R signal is increased for white balance, and thus the quantize noise increases, but the gains at the low level at which the quantize noise increases to cause problems are decreased upon .gamma. correction as described above, so that finally the quantize noise can be prevented from increasing.

Thus, according to this embodiment, it is possible to prevent the quantize noise of chrominance signals from increasing due to the change of the color temperature of the light source, and it is possible to achieve the object of the invention.

FIG. 2 shows one example of the .gamma. correction characteristics of the variable .gamma. correction circuits in FIG. 1. The .gamma. correction is normally approximate to a polygonal line for simplicity of the circuits. The .gamma. correction characteristics at both input and output are normalized by "1".

In FIG. 2, one example of the normal .gamma. correction of the chrominance signal is shown at 4. As to the characteristic 4, the gain gradient relative to a low-level input is large, thus increasing the amplification degree to the quantize noise. If, now, this input/output characteristic is represented by .gamma., the input signal by x, and the output signal by .gamma.(x), then the .gamma. characteristic can be expressed as

.gamma..sub.1 (x) is the result of .gamma. correction based on a certain .gamma. characteristic. Normally, K=1. When K approaches 0, the .gamma. characteristic becomes close to the characteristic 5 shown in FIG. 2, and thus the gain of the input signal can be decreased in the low level region.

FIGS. 3 and 4 are explanatory diagrams for explaining examples of the arrangement of the variable .gamma. correction circuit in FIG. 1.

In FIG. 3, there are shown a .gamma. correction circuit 1 having a fixed .gamma. characteristic, a subtracting circuit 5, an addition circuit 6, and a coefficient multiplying circuit 7. In this embodiment, the input signal x is first supplied to the .gamma. correction circuit 1, the subtraction circuit 5 and the addition circuit 6. The .gamma. correction circuit 1 applies a .gamma. correction to the input signal x (for example, in accordance with the characteristic 4 shown in FIG. 2) and supplies the output .gamma..sub.1 (x) after .gamma. correction to the subtraction circuit 5. The subtraction circuit 5 subtracts the signal x from the supplied signal .gamma..sub.l (x), and supplies the obtained signal {.gamma..sub.1 (x)-x} to the coefficient multiplier 7. The coefficient multiplier multiplies the input signal {.gamma..sub.1 (x)-x} by K, and supplies a signal {.gamma..sub.1 (x)-x}.multidot.K to the addition circuit 6. The addition circuit 6 adds the signal {.gamma..sub.1 (x)-x}.multidot.K to the supplied signal x, and produces

which is equal to the above equation (1).

Thus, the .gamma. characteristic can be changed by varying the value of the coefficient K.

An example of the construction shown in FIG. 4 will be mentioned. In FIG. 4, there are shown an arithmetic circuit 8, the coefficient multiplying circuit 7 and the adding circuit 6. In this embodiment, the input signal to the arithmetic circuit 8 is processed according to the characteristic (corresponding to the subtraction of characteristic 5 from characteristic 4 in FIG. 2) shown in FIG. 4 at 9 directly into the {.gamma..sub.1 (x)-x} signal, and this signal is multiplied by K and then added to the input signal x, thus the expression (1) of {.gamma..sub.1 (x)-x}.multidot.K+x being produced.

FIG. 5 is a block diagram of a second embodiment of the invention. In FIG. 5, there are shown variable gain amplifiers 9a and 9b for white balance, a tracking circuit 10, and a color difference matrix circuit 11. In FIG. 5, like elements corresponding to those in the first embodiment are identified by the same reference numerals.

At present, the color temperature method (system for white balance) is divided roughly into (1) internal light measurement system and (2) external light measurement system. The embodiment in FIG. 5 employs the white balance of the internal light measurement system.

The operation of this embodiment will be described below.

The tracking circuit 10 is a control circuit for keeping the gain constant when the R, B signals are changed as shown in FIG. 9. This tracking circuit produces R-signal and B-signal gain adjusting signals in accordance with the signal from the color temperature detection circuit 2, and supplies them to the variable gain amplifiers 9a and 9b, respectively. The variable gain amplifiers 9a and 9b are formed of multiplying circuits and multiply the input R-signal and B-signal by the R-signal and B-signal adjusting signals fed from the tracking circuit, respectively. Thus, the variable gain amplifiers are adjusted in their gains, and supply the amplified R-signal and B-signal after gain adjustment to the variable .gamma. correction circuits 1a, 1b and 1c. The color difference matrix 11 arithmetically processes the red (R') signal, green (G') signal and Blue (B') signal which are .gamma.-corrected by the variable .gamma. correction circuits 1a, 1b and 1c, and produces difference signals (R-Y) and (B-Y). The arithmetic processing is, in the NTSC system, effected according to

The color temperature detection circuit 2 arithmetically processes the color difference signals (B-Y) and (R-Y) to produce an average value, and supplies the average value to the .gamma. control circuit 3 and the tracking circuit 10. The variable gain amplifiers 9a and 9b, the .gamma. correction circuits 1a and 1c, the color difference matrix 11, the color temperature detection circuit 2 and the tracking circuit 10 constitute a so-called control loop, and operate to provide

for an achromatic color portion of white or the like. When the above conditions are satisfied, the control loop is in the balanced state. In the balanced state, the color temperature detection circuit 2 produces a signal corresponding to the color temperature of the light source. Thus, the .gamma. control circuit 3 can discriminate the color temperature from the signal supplied from the color temperature detection circuit 2, and thus controls the variable .gamma. correction circuits 1a, 1b and 1c in accordance with the discriminated color temperature as in the first embodiment so as to achieve the same effect as in the first embodiment.

FIG. 6 is a block diagram of a third embodiment of the invention. This embodiment employs the white balance of the external light measurement system.

In FIG. 6, there is shown a sensor 12 for measuring the color temperature of the light source (generally a photodiode with a color filter of red or blue). The other portions have the same functions as in the above embodiments. Here, like elements corresponding to those in the above embodiments are identified by the same reference numerals and will not be described.

In this embodiment, the color temperature detection circuit 2 detects the color temperature from the signal (for example, blue level signal, red level signal and so on) from the sensor 12, and supplies the detected signal to the tracking circuit 10 and the .gamma. control circuit 3, which then perform white balance and control of .gamma. correction characteristics as in the first embodiment. As a result, in this embodiment the quantize noise of chrominance signals can be inhibited from increasing due to the change of color temperature as in the above embodiments.

FIG. 7 is a block diagram of a fourth embodiment of the invention. This embodiment employs one variable .gamma. correction circuit for effecting .gamma. correction of the R signal, G signal and B signal in a dot sequence. When the variable .gamma. correction circuit 1 processes at a satisfactorily high speed, a single variable .gamma. correction circuit suffices, resulting in realization of a small circuit scale arrangement.

FIG. 8 is a block diagram of a fifth embodiment of the invention. In FIG. 8, there is shown a low illumination detecting circuit 13. In this embodiment, the low illumination detecting circuit 13 is added to the arrangement of the first embodiment, so that the .gamma. correction characteristic can be controlled not only against the high color temperature and low color temperature, but also against low illumination. At a low illumination, an AGC circuit is operative, so that the gain of each color signal is increased. The result is the increase of quantize noise. Thus, the .gamma. characteristics are changed to suppress the final quantize noise. The low illumination detecting circuit 13 averages, for example, the luminance signal though not shown. If the average value is lower than a constant value, the low illumination detecting circuit detects that the illumination is low, and supplies this detected signal to the .gamma. control circuit 3. The .gamma. control circuit 3 controls the variable .gamma. correction circuits 1a, 1b and 1c in accordance with the signal fed from the low illumination detecting circuit 13 so that at a low illumination the .gamma. correction characteristics can be changed to reduce the gains for low levels. Thus, according to this embodiment, the quantize noise can be prevented from increasing at low color temperatures or high color temperatures, and also the quantize noise at a low illumination can be similarly prevented from increasing.

All the above embodiments have been described from the viewpoint of the prevention of quantize noise increase on the basis of digital signal processing. However, these embodiments are also able to reduce the normal noise (for example, random noise) in addition to the quantize noise. Thus, by analog processing it is possible to prevent noise from increasing at high color temperatures or low color temperatures.

Moreover, the .gamma. correction circuit for calculation of Eq. (1) can be used for not only color signals but also for the luminance signal. The arrangement itself of the .gamma. correction circuit having the characteristic of Eq. (1) is a noble feature which could not be seen in the prior art, and is not limited to the application to these embodiments.