Color imaging by independently controlling gains of each of R, Gr, Gb, and B signals

[0001] 1. Field of the Invention

[0002] The present invention relates to a single-chip color imaging apparatus and method.

[0003] 2. Description of the Related Art

[0004] In the conventional single-chip color imaging apparatus using a CCD (charge-coupled device) or the like as a solid-state image sensor, filters for three primary colors R (red), G (green) and B (blue), respectively, are disposed in a positions corresponding to pixels of the CCD. In this color imaging apparatus, a luminance signal is produced based on a light incident upon the CCD, and color signals are produced correspondingly to the light incident upon the CCD through the three primary color filters. Namely, R, G and B color signals are produced correspondingly to a light incident upon the CCD pixels on which the three primary color filters are disposed.

[0005] In the CCD used in the color imaging apparatus, R, G and B color filters are provided for pixels, respectively, and disposed horizontally in a sequence of R, G, R, G, . . . , R and G, for example. Color signals are produced correspondingly to the three primary color filters on the pixels. Therefore, in such a CCD, a G color signal will not be produced from a pixel on which an R color filter is provided and it should be produced by interpolation of color signals.

[0006] That is, for a horizontal interpolation of color signals from pixels to acquire an interpolated image, color signals from pixels disposed successively in the horizontal direction are added together and averaged. Similarly, for a vertical interpolation, color signals from pixels disposed successively in the vertical direction are added together and averaged. For acquisition of a G color signal from a CCD pixel on which the R color filter is disposed, the above-mentioned interpolation is effected. For produce a pixel data through an interpolation, values indicative of vertical and horizontal correlations are detected. For this detection, signals from pixels surrounding the pixel in consideration are filtered for calculation. Further in this color imaging apparatus, interpolated pixel data acquired by interpolation are weighted by the correlation values.

[0007] However, since the photosensors for pixels in the CCD are different in sensitivity from each other, there will exist a difference in amplitude between a G signal from a G pixel included in a horizontal line of R, G, R, G, . . . , R and G color filters and a G signal from a G pixel included in a horizontal line of G, B, G, B, . . . , G and B color filters. Therefore, in a single-chip color imaging apparatus, when a luminance signal is produced from color signals from pixels in each horizontal line, the above difference in sensitivity from one photosensor to the other will cause a difference in amplitude between luminance signals from the pixels in different horizontal lines and it will appear as a horizontal stripe-like noise in a monitoring image and captured image.

[0008] It is therefore an object of the present invention to overcome the above-mentioned drawbacks of the prior art by providing a single-chip solid-state color imaging apparatus and method adapted such that no amplitude difference caused by the sensitivity difference will exist between luminance signals from pixels included in horizontal lines and thus the horizontal stripe-like noise appearing in a monitoring image and captured image will be reduced.

[0009] According to the present invention, there is provided a color imaging apparatus including:

• • [0010] a solid-state image sensor having photosensors color-coded with three primary color filters formed like a matrix correspondingly to pixels of the solid-state image sensor, to provide three primary color signals acquired as captured image signals;
  • [0011] a four-channel signal detecting means for detecting, from the three primary color signals provided from the solid-state image sensor, an R signal acquired from R pixels in a horizontal line of R, G, R, G, . . . , R and G color filters, a Gr signal acquired from the G pixels in the same horizontal line, a Gb signal acquired from G pixels in a horizontal line of G, B, G, B, . . . , G and B color filters, and a B signal acquired from the B pixels in the same horizontal line;
  • [0012] a four-channel variable-gain amplifying means whose channels are controllable in gain independently of each other to amplify the R, Gr, Gb and B signals; and
  • [0013] a gain controlling means for controlling, based on an output from the signal detecting means, the gain of each channel of the variable-gain amplifying means so that the R, Gr, Gb and B signals amplified by the variable-gain amplifying means are equal in level to one another for an achromatic color image.

[0014] According to the present invention, there is also provided a color imaging method including the steps of:

• • [0015] detecting, from the three primary color signals provided from the solid-state image sensor having photosensors color-coded with three primary color filters formed like a matrix correspondingly to pixels of the solid-state image sensor, an R signal acquired from R pixels in a horizontal line of R, G, R, G, . . . , R and G color filters, a Gr signal acquired from the G pixels in the same horizontal line, a Gb signal acquired from G pixels in a horizontal line of G, B, G, B, . . . , G and B color filters, and a B signal acquired from the B pixels in the same horizontal line;
  • [0016] amplifying the R, Gr, Gb and B signals by a four-channel variable-gain amplifying means whose channels are controllable in gain independently of each other; and
  • [0017] controlling, by a gain controlling means, based on an output from the signal detecting means, the gain of each channel of the variable-gain amplifying means so that the R, Gr, Gb and B signals amplified by the variable-gain amplifying means are equal in level to one another for an achromatic color image.

[0018] According to the present invention, an output from a detecting means used typically in this field of art is used to calculate an amplitude difference between the Gr and Gb signals and set gains multiplied by compensation factors, respectively, with which the Gr and Gb signals after subjected to white balance processing will have an equal amplitude, thereby permitting to eliminate a horizontal stripe in a monitoring image and reduce a compression noise in a still image during image capturing. This can be attained only by modifying the internal configuration of a CPU, without any change of the conventional system and with no costs.

[0019] Also, with the present invention, it is possible to eliminate an amplitude difference between luminance signals from the pixels in the horizontal lines of color filters, which would be caused by the difference in sensitivity from one photosensor to another in the solid-state image sensor.

[0020] Therefore, according to the present invention, there is provided a single-chip solid-state color imaging apparatus and method in which the horizontal stripe-like noise appearing in a monitoring image and captured image can be reduced.

[0021] These objects and other objects, features and advantages of the present intention will become more apparent from the following detailed description of the preferred embodiments of the present invention when taken in conjunction with the accompanying drawings.

[0031] Referring now to FIG. 1, there is illustrated in the form of a block circuit diagram the solid-state color imaging apparatus according to the present invention. The solid-state color imaging apparatus is generally indicated with a reference 10.

[0032] The solid-state color imaging apparatus 10 is a single-chip solid-state color imaging apparatus including an imaging optical system 11, a CCD (Charge Coupled Device) 12 having an image sensing layer on which an image of an object is formed by the imaging optical system 11, an S/H•AGC circuit 13, an A/D converter 14 supplied with a captured image signal taken from the CCD 12 via the S/H•AGC circuit 13, a white balance processor 15 supplied with a digital captured image signal from the A/D converter 14, a captured image signal processor 16 supplied with the captured image signal adjusted in white balance by the white balance processor 15, and a compression/conversion circuit 17 supplied with a luminance signal Y and chrominance signals Cr and Cb generated by the captured image signal processor 16.

[0033] As schematically illustrated in FIG. 2, the CCD 12 is an interline transfer type CCD including a plurality of photosensors S for pixels arranged in the form of a matrix, a vertical transfer register VR disposed along each vertical line extending along one side of each trains of photosensors S, and a horizontal transfer register HR disposed along a line connecting the ends of the vertical transfer registers VR. This CCD is coded in color with filters of three primary colors formed from areas penneable to R (red) color light, areas penneable to G (green) color light, and areas penneable to B (blue) color light and arranged in the form of a matrix correspondingly to the photosensors S, that is, pixels. In the filters of three primary colors, areas permeable to rays of color light are disposed horizontally in a sequence of R, G, R, G, . . . , R and G or G, B, G, B, . . . , G and B, and there are alternately disposed horizontal lines along each of which R, G, R, G, . . . , R and G color filters are disposed (will be referred to as “RG line” hereinafter) and horizontal lines along each of which G, B, G, B, . . . , G and B color filters are disposed (will be referred to as “GB line” hereinafter). Pixels corresponding to photosensors color-coded with the three primary color filters, respectively, will be referred to as R, G and B pixels in the context of the color coding, and in order to provide a distinction between G pixels included in the RG and GB lines, respectively, the G pixel in the RG line will be referred to as Gr pixel while the G pixel in the GB line will be referred to as Gb pixel, hereinafter.

[0034] In the CCD 12, a capturing charge provided from each photosensor S correspondingly to an amount of incident light upon the photosensor via the imaging optical system 11 is transferred to the vertical transfer register VR for each vertical line, and read from each vertical transfer register VR via the horizontal transfer register HR for a horizontal line at each time.

[0035] The S/H•AGC circuit 13 samples and holds for each pixel the capturing charge read from the CCD 12 line-sequentially and adjusts the gains, and supplies the capturing charge as a capturing signal line-sequentially to the A/D converter 14 as shown in FIG. 3.

[0036] The A/D converter 14 digitizes pixel by pixel, namely, spot by spot, the next capturing signal in the order of the lines and supplies it to the while balance processor 15.

[0037] As shown in FIG. 4, the white balance processor 15 includes a signal selector 151 to extract captured image signals in the order of spots, digitized by the A/D converter 14 as the R, B, Gr and Gb signals, a WB (white balance) amplifier 152 and WB detector 153, supplied with the R, B, Gr and Gb signals extracted by the signal selector 151, and an arithmetic circuit 154 supplied with an output from the WB detector 153.

[0038] The WB amplifier 152 is a four-channel amplifier consisting of a variable-gain amplifier 152r for the R signal, variable-gain amplifier 152b for the B signal, variable-gain amplifier 152gr for the Gr signal, and a variable-gain amplifier 152gb for the Gb signal. The gains Rgain, Bgain, Grgain and Gbgain of the variable-gain amplifiers 152r, 152b, 152gr and 152gb are controlled independently by the arithmetic circuit 154.

[0039] The WB detector 153 is a four-channel detector consisting of a detector 153r for the R signal, detector 153b for the B signal, detector 153gr for the Gr signal, and a detector 153gb for the Gb signal. Outputs from these detectors 153r, 153b, 153gr and 153gb are supplied to the arithmetic circuit 154.

[0040] The arithmetic circuit 154 includes a mean calculator 154A supplied with the outputs from the detectors 153gr and 153gb of the WB detector 153, and a WB calculator 154B supplied from the outputs from the detectors 153r and 153b of the WB detector 153 and from the output of mean calculator 154A. The arithmetic circuit 154 is formed from a CPU (central processing unit).

[0041] The arithmetic circuit 154 determines the gains Rgain, Bgain, Grgain and Gbgain of the variable-gain amplifiers 152r, 152b, 152gr and 152gb of the WB amplifier 152 so that the R, B, Gr and Gb signals extracted by the signal selector 151 are equal in level to one another for an achromatic color image. More particularly, the mean calculator 154A is used to average, for levels R, B, Gr and Gb of the R, B, Gr and Gb signals, the levels Gr and Gb of the Gr and Gb signals among the R, B, Gr and Gb signals supplied to the variable-gain amplifiers 152r, 152b, 152gr and 152gb, respectively, and a white balance is attained by controlling values R_amp, B_amp, Gr_amp and Gb_amp of the gains Rgain, Bgain, Grgain and Gbgain of the variable-gain amplifiers 152r, 152b, 152gr and 152gb by means of the WB calculator 154B so that the variable-gain amplifiers 152r, 152b, 152gr and 152gb will provide R, B, Gr and Gb signal outputs having the following signal levels R′, B′, Gr′ and Gb′, respectively.

• • [0042] R′=R×R_amp
  • [0043] B′=B×B_amp
  • [0044] Gr′=Gr×Gr_amp=Gr×1×Grcomp
  • [0045] Gb′=Gb×Gb_amp=Gb×1×Gbcomp
    where Grcomp and Gbcomp are set to have a value of Grcomp+Gbcomp=2.

[0046] As shown in FIG. 5, the captured image signal processor 16 includes a gamma-correction circuit 16A supplied with the R, B, Gr and Gb signals adjusted in white balance by the white balance processor 15, and a luminance signal generator 16B supplied with the R, B, Gr and Gb signals gamma-corrected by the gamma-correction circuit 16A, and chrominance matrix circuit 16C.

[0047] In the captured image signal processor 16, the R, B, Gr and Gb signals adjusted in white balance by the white balance processor 15 are gamma-corrected by the gamma-correction circuit 16A, and a luminance signal Y and chrominance signals Cr and Cb are produced from the gamma-corrected R, B, Gr and Gb signals supplied from the gamma-correction circuit 16A by the luminance signal generator 16B and chrominance matrix circuit 16C, respectively.

[0048] The compression/conversion processor 17 compresses the luminance signal Y and chrominance signals Cr and Cb from the captured image signal processor 16 for write into a memory and converts the same signals for provision as a video signal.

[0049] In the solid-state color imaging apparatus 10, for a monitoring image (through image) used for monitoring an image being captured and still image during the capture, the luminance signal generator 16B of the captured image signal processor 16 produces the luminance signal Y for the RG line is produced from the R and Gr signals and the luminance signal Y for the GB line from the B and Gr signals. The monitoring image signal is provided as an NTSC signal output, for example, from the compression/conversion processor 17.

[0050] Since there is generally a sensitivity difference from one photosensor S to another in the CCD 12, the sensitivity difference between the photosensors S will cause an amplitude difference between the Gr signal obtainable from the Gr pixel in the RG line and the Gb signal obtainable from the Gb pixel in the GB line. Therefore, an amplitude difference caused by the sensitivity difference from one photosensor S to another of the CCD 12 will exist between the luminance signal Y for the RG line produced from the R and Gr signals and the luminance signal Y for the GB line produced from the B and Gr signals. The sensitivity difference will possibly appear as a horizontal stripe-like noise in the monitoring image and captured image. In this solid-state color imaging apparatus 10, however, the values R_amp, B_amp, Gr_amp and Gb_amp of the gains Rgain, Bgain, Grgain and Gbgain of the four-channel amplifiers 152r, 152b, 152gr and 152gb can be controlled by the white balance processor 15 to make the Gr′ and Gb′ signals coincide in level with each other and reduce the horizontal stripe-like noise in the monitoring image and captured image.

[0051] In the solid-state color imaging apparatus 10, to eliminate the amplitude difference between the luminance signal Y for the RG line produced from the R and Gr signal and the luminance signal Y for the GB line produced from the B and Gr signals, the values Gr_amp and Gb_amp of the gains Grgain and Gbgain, respectively, of the variable-gain amplifier 152gr for the Gr signal in the white balance processor 15 and the variable-gain amplifier 152gb for the Gb signal, respectively, are set by the arithmetic circuit 154 following the procedure shown in FIG. 6 for example.

[0052] In the gain setting procedure shown in the flow chart of FIG. 6, compensation factors for the gains Grgain and Gbgain are set to be fixed values in order to eliminate the amplitude difference between the Gr and Gb signals. The amplitude difference between the Gr and Gb signals is pre-calculated in the gain setting procedure, and compensation factors for the gains Grgain and Gbgain, with which no amplitude difference will exist between the Gr and Gb signal after subjected to the while balance processing, are set as fixed values and the gains Grgain and Gbgain are compensated with the factors.

[0053] More particularly, the arithmetic circuit 154 of the while balance processor 15 acquires an output from the WB detector 153 at step S1, and judges the mode and incident light at step S2.

[0054] Then at step S3, the arithmetic circuit 154 determines, according to the results of mode judgment and incident light judgment effected at step S2, values R_amp and B_amp of the gains Rgain and Bgain of the variable-gain amplifiers 152r and 152b of the WB amplifier 152.

[0055] Further at step S4, the arithmetic circuit 154 calculates a difference in amplitude between the Gr and Gb signals in advance, and takes as compensation factors Grcomp and Gbcomp fixed values c_Gr and c_Gb having been set so that the levels Gr′ and Gb′ of the white balance-processed Gr and Gb signals provided from the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152, namely, amplitudes thereof, will be equal to each other.

[0056] Moreover at step S5, the arithmetic circuit 154 takes as the values Gr_amp and Gb_amp the values (×1 for example), respectively, normally used as the values Gr_amp and Gb_amp of the gains Grgain and Gbgain of the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152, which are multiplied by the compensation factors Grcomp and Gbcomp, respectively.

[0057] The gain setting procedure in the flow chart shown in FIG. 6 can advantageously be effected with only a very little modification for the common white balance control, and thus the modification can be done very easily. In this gain setting procedure, however, the amplitude difference has to be pre-calculated for determination of compensation factors, and if there is a large variation from one CCD to another and the pre-calculated amplitude difference differs from an presumed one, it is not possible to compensate the amplitude difference between the Gr and Gb signals after subjected to the while balance processing. If there is a large variation between the Gr and Gb signals depending upon the status of an object to be imaged, there is a likelihood that the variation cannot be compensated. However, since the gain setting procedure can easily be modified for the common white balance control when it is known that the variation from one CCD to another is small or when the difference between the Gr and Gb signals is small depending upon the object to be imaged, it can be said to be a very effective procedure.

[0058] Also, for elimination of the amplitude difference between the luminance signal Y for the RG line formed from the R and Gr signals and the luminance signal Y for the GB line formed from the B and Gr signals, the solid-state color imaging apparatus 10 according to the present invention may be adapted to set by the arithmetic circuit 154 the values Gr_amp and Gb_amp of the gains Grgain and Gbgain, respectively, of the variable-gain amplifier 152gr for the Gr signal and variable-gain amplifier 152gb for the Gb signal, both in the white balance processor 15 following the gain setting procedure in the flow chart shown in FIG. 7.

[0059] To eliminate the amplitude difference between the Gr and Gb signals, the gain setting procedure shown in the flow chart of FIG. 7 is designed to calculate the compensation factors for the gains Grgain and Gbgain for accommodation of even CCDs varying largely from each other. In this gain setting procedure, compensation factors for the gains Grgain and Gbgain, with which there will exist no amplitude difference between the Gr and Gb signals, are pre-calculated for each solid-state color imaging apparatus during manufacture, and are written into a memory such as an EEPROM or the like. The compensation factors for the gains Grgain and Gbgain, with which the white balance-processed Gr and Gb signals are equal in level to each other, are recalled from the memory and set for compensation of the gains.

[0060] In the gain setting procedure, the arithmetic circuit 154 of the white balance processor 15 acquire the output from the WB detector 153 at step 111, and judges the mode and incident light at step S12.

[0061] Then at step S13, the arithmetic circuit 154 determines, according to the results of mode judgment and incident light judgment effected at step S12, values R_amp and B_amp of the gains Rgain and Bgain of the variable-gain amplifiers 152r and 152b of the WB amplifier 152.

[0062] Further at step S14, the arithmetic circuit 154 calculates a difference in amplitude between the Gr and Gb signals in advance during manufacture of this solid-state color imaging apparatus 10 (digital camera) and takes as compensation factors Grcomp and Gbcomp values m_Gr and m_Gb having been calculated so that the levels Gr′ and Gb′ of the white balance-processed Gr and Gb signals provided from the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152, namely, amplitudes thereof, will be equal to each other, and having been taken into the memory.

[0063] Moreover at step S15, the arithmetic circuit 154 takes as the values Gr_amp and Gb_amp the values (×1 for example), respectively, normally used as the values Gr_amp and Gb_amp of the gains Grgain and Gbgain of the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152, which are multiplied by the compensation factors Grcomp and Gbcomp, respectively.

[0064] FIG. 8 shows a Grgain/Gbgain compensation factor calculation routine effected for each of the digital cameras during manufacture.

[0065] That is, to execute the compensation factor calculation routine shown in FIG. 8, the arithmetic circuit 154 acquires at S21 the result of detection from the WB detector 153 after an object is imaged, and determines at step S22 the values m_Gr and m_Gb of the compensation factors Grcomp and Gbcomp for the gains Grgain and Gbgain of the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152, respectively, so that the levels Gr′ and Gb′, namely, amplitudes, of the white balance-processed Gr and Gb signals provided from the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152 are G (mean value of the levels Gr and Gb of the Gr and Gb signals before subjected to the white balance processing).

[0066] Then at step S23 the arithmetic circuit 154 writes to the memory the values m_Gr and m_Gb of the compensation factors Grcomp and Gbcomp having been determined at step S22 so that they can be used any time in the common routine shown in FIG. 7.

[0067] The gain setting procedure shown in the flow chart in FIG. 7 can advantageously absorb a variation from one CCD to another by acquiring data on each of the digital cameras during manufacture and writing the values m_Gr and m_Gb of the compensation factors Grcomp and Gbcomp when the variation is so large that the amount of compensation cannot be fixed. When the difference between the Gr and Gb signals varies largely depending upon the status of the object, however, the gain setting procedure in FIG. 7 cannot completely compensate the variation. However, when the difference between the Gr and Gb signals is small depending upon the status of the object, it suffices to calculate compensation factors only once during manufacture of the digital cameras without any modification of the common routine. Also, the gain setting procedure in FIG. 7 can easily be modified. Thus, this procedure can be said to be an effective one.

[0068] Also, to eliminate the difference in amplitude between the luminance signal Y for the RG line formed from the R and Gr signals and the luminance signal Y for the GB line formed from the B and Gr signals, the solid-state color imaging apparatus 10 may be adapted to set the values Gr_amp and Gb_ap of the gains Grgain and Gbgain of the variable-gain amplifier 152gr for the Gr signal and the variable-gain amplifiers 152gb for the Gb signal, respectively, of the white balance processor 15 by the arithmetic circuit 154 following the gain setting procedure in the flow chart shown in FIG. 9 for example.

[0069] In the gain setting procedure in the flow chart shown in FIG. 9, the arithmetic circuit 154 calculates an amplitude difference between Gr and Gb signals from the output from the WB detector 153 to determine compensation factors for the gains Grgain and Gbgain, respectively, in order to eliminate the amplitude difference between the Gr and Gb signals. Thus, this gain setting procedure permits to positively compensate any amplitude difference.

[0070] That is, the arithmetic circuit 154 of the white balance processor 15 acquires first at step S31 the output from the WB detector 153, and then judges the mode and incident light at step S32.

[0071] Then the arithmetic circuit 154 writes at step S23 to the memory the values m_Gr and m_Gb of the compensation factors Grcomp and Gbcomp having been determined at step S32 so that they can be used at any time in the common routine shown in FIG. 7.

[0072] Next at step S34, the arithmetic circuit 154 determines the compensation factors Grcomp and Gbcomp for the gains Grgain and Gbgain of the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152 so that the levels Gr′ and Gb′, namely, amplitudes, of the signals Gr and Gb provided from the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152 are G (mean value of the levels Gr and Gb of the signals Gr and Gb signals before subjected to the white balance processing).

[0073] At step S35, the arithmetic circuit 154 applies LPF (low pass filtering) to the compensation factors Grcomp and Gbcomp calculated at step S34. Thus, the influence on the compensation factors Grcomp and Gbcomp of a fluctuation of the result of the detection of the Gr and Gb signals by the WB detector 153 is reduced.

[0074] Further at step S36, the arithmetic circuit 154 limits the compensation factors Grcomp and Gbcomp having been subjected to LPF at step S35 so that the gains Grgain and Gbgain of the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152 will not take any values Gr_amp and Gb_amp which are not expected.

[0075] At step S37, the arithmetic circuit 154 takes as Gr_amp and Gb_amp the values (×1 for example), respectively, normally used as the values Gr_amp and Gb_amp of the gains Grgain and Gbgain of the variable-gain amplifiers 152gr and 152gb of the WB amplifier 152, which are multiplied by the compensation factors Grcomp and Gbcomp.

[0076] The gain setting procedure in the flow chart shown in FIG. 9 may not be varied so much from the common gain setting procedure for the white balance control and can advantageously be effected with no problem also when the variation from one CCD to another is large or when the amplitude difference between the Gr and Gb signals varies depending upon the status of the object.

[0077] In the foregoing, the present invention has been described concerning three kinds of gain setting procedures in which the values Gr_amp and Gb_amp of the gains Grgain and Gbgain of the variable-gain amplifier 152gr for the Gr signal and variable-gain amplifier 152gb for the Gb signal, respectively, of the white balance processor 15 are set by the arithmetic circuit 154 to eliminate the amplitude difference between the luminance signal Y for the RG line formed from the R and Gr signals and the luminance signal Y for the GB line formed from the B and Gr signals. In addition, any of the procedures with only a little modification/addition of the program in the CPU in the arithmetic circuit 154 permits to reduce the horizontal stripe-like noise in the monitoring image and noise in the still image during image capturing. All these methods are advantageous since the conventional system has not to be modified and thus no costs are taken.