Image sensing apparatus which determines white balance correction information before photographing

[0001] This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2004-017450, filed Jan. 26, 2004, the entire contents of which are incorporated herein by reference.

[0002] 1. Field of the Invention

[0003] The present invention relates to a technique for adjusting white balance of an image sensing apparatus comprising an image sensing element.

[0004] 2. Description of the Related Art

[0005] For example, as shown in FIG. 11, many of image sensing elements have heretofore comprised a micro lens 100, a color filter 101 and the like for each pixel in order to enhance a condensing ratio per pixel. A refractive index of the micro lens 100 depends on a wavelength of incident light (dependence on the wavelength).

[0006] Especially, as shown in FIG. 11, when the light strikes from an oblique direction of the micro lens 100, the above-described dependence on the wavelength becomes remarkable, and light having a wavelength that is not condensed on a light receiving section 102 is also generated (oblique incidence characteristic).

[0007] An object of one mode of the present invention is to obtain images having an appropriate white balance even in any combination in an image sensing apparatus capable of enhancing precision of white balance adjustment and changing a photographing lens or an image sensing element.

[0008] To achieve this object, according to one mode of the present invention, there is provided an image sensing apparatus comprising: an image sensing section having an image sensing element which picks up a subject image passed through a photographing lens having a diaphragm regulating a quantity of light to thereby output an image signal; a storage section which stores a white balance modification coefficient to further modify a white balance correction coefficient in accordance with an optical characteristic of the image sensing section, the white balance correction coefficient performing white balance correction with respect to the image signal output from the image sensing section in accordance with an incidence characteristic upon the image sensing element based on optical characteristics including setting of the diaphragm of the photographing lens; and an adjustment control section which performs white balance adjustment with respect to the image signal output from the image sensing section based on the white balance correction coefficient modified by the white balance modification coefficient.

[0009] Advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.

[0022] First, a constitution of an image sensing apparatus according to a first embodiment of the present invention will be described with reference to FIG. 1.

[0023] Here, a camera system in which a camera main body is combined with a replaceable photographing lens (hereinafter referred to as a replaceable lens) will be described as an example of this image sensing apparatus.

[0024] As shown in FIG. 1, the camera system which is the image sensing apparatus according to the first embodiment of the present invention has a lens module 1, an image sensing module 2, a RAM 3, an image signal processing circuit 4, a microcomputer 5, a ROM 6, an EEPROM 7, a mode setting SW 8, a recording medium 9 and the like. The lens module 1 is disposed on the side of the replaceable lens. Other constituting elements are disposed on the side of a camera main body to which the replaceable lens is attachable.

[0025] In the lens module 1 on the replaceable lens side, an optical system 1a including a photographing lens, zoom lens, diaphragm and the like, a microcomputer 1b which controls the whole lens module, a ROM 1c in which various data and the like are stored, a communication section 1d and the like are disposed. At least a first table is stored in this ROM 1c as described later in detail. The lens module 1 is detachably connected to the image sensing module 2 via a connecter, and is electrically connected to the same via a communication section 1d and 2d. The communication sections 1d and 2d are constituted as connectors.

[0026] Moreover, the image sensing module 2 on the camera main body side is provided with an image sensing element 2a such as a CCD, an interface (hereinafter referred to as I/F) circuit 2b, and a ROM 2c for temporarily storing various data. The RAM 3 interposed between the image sensing module 2 and the image signal processing circuit 4 stores an image signal sent from the I/F circuit 2b. The image sensing module 2 is detachably connected to the camera main body side via a connecter, and is electrically connected to the same via a communication section 10.

[0027] In more detail, the image signal processing circuit 4 has a digital gain circuit 4a, a synchronization circuit 4b, a color conversion circuit 4c, and a gamma conversion circuit 4d.

[0028] The digital gain circuit 4a amplifies an red (R) signal, a green (Gr) signal, a green (Gb) signal, and a blue (B) signal which are image signals for colors, temporarily stored in the RAM 3 with digital gains calculated by the microcomputer 5 in a method described later in detail, and accordingly outputs an R′ signal, a Gr′ signal, a Gb′ signal, and a B′ signal as level-adjusted signals.

[0029] That is, the digital gains amplify the signal for each color, included in the image signals output from the image sensing element 2a.

[0030] Moreover, the synchronization circuit 4b includes a sample holding circuit (not shown) and the like, performs a synchronization process of the R′, Gr′, Gb′, and B′ signals, and outputs the R, G, and B signals. The color conversion circuit 4c subjects these R, G, and B signals to color conversion, and outputs R′, G′, and B′ signals. The gamma conversion circuit 4d γ-converts these R′, G′, and B′ signals, and records the results in the recording medium 9.

[0031] Additionally, the ROM 6 stores at least a second table described later in detail. The EEPROM 7 stores various data. The mode setting SW 8 switches various modes.

[0032] It is to be noted that image sensing means described in claims corresponds to the image sensing element 2a, image sensing module 2 including the element or the like, image signal processing means corresponds to the image signal processing circuit 4 or the like, storage means corresponds to the ROM 1c in the lens module 1, the ROM 6 on the camera main body side or the like, and control means corresponds to the microcomputer 5 or the like. Correction coefficient storage means described in claims corresponds to the ROM 1c of the lens module 1 or the like, modification coefficient storage means corresponds to the ROM 6 or the like, and white balance adjustment means corresponds to the image signal processing circuit 4 or the like. Moreover, communication means corresponds to the communication section 1d, 10 or the like. Additionally, the present invention is not limited to the above-described relations.

[0033] In this constitution, a subject image which has struck via the photographing lens, zoom lens, and diaphragm of the optical system 1a of the lens module 1 is picked up by the image sensing element 2a of the image sensing module 2, and an image signal is output, and stored in the RAM 3 via the I/F circuit 2b.

[0034] In the ROM 1c of the lens module 1 on the replaceable lens side, at least the first table is stored in which diaphragm values are associated with white balance (hereinafter referred to as WB) correction coefficients for performing correction corresponding to characteristics relating to the diaphragm of the optical system 1a.

[0035] Here, the WB correction coefficient is a coefficient for performing the correction corresponding to the characteristics relating to the diaphragm of the optical system 1a in the first embodiment.

[0036] On the other hand, in the ROM 6 on the camera main body side, the WB modification coefficient further modifies the WB correction coefficient. In the ROM 6, at least a second table is stored in which the correction coefficient is associated with the WB modification coefficient for performing the correction corresponding to optical characteristics of the image sensing element 2a.

[0037] Here, the WB modification coefficient further modifies the WB correction coefficient, and performs the correction corresponding to the optical characteristics of the image sensing element 2a for use.

[0038] This second table is predetermined based on the optical characteristics of the image sensing element 2a.

[0039] The storage place of each table is one example, and, needless to say, the present invention is not limited to this.

[0040] The microcomputer 5 calculates the digital gains based on a WB mode set by the mode setting SW 8. Here, two different WB modes will be described hereinafter as first and second WB modes.

[0041] For example, an operation of the first WB mode will be performed as follows. That is, the WB correction coefficient corresponding to the diaphragm value, and the WB modification coefficient are calculated based on the first and second tables. Moreover, values obtained by multiplying the WB gains by the WB correction and modification coefficients are digital gains for amplifying the signals for the respective colors included in the image signals output from the image sensing element 2a.
Digital gain=WB gain×WB correction coefficient×WB modification coefficient  (1)

[0042] WB adjustment is controlled to be performed based on the digital gain by the image signal processing circuit 4.

[0043] The image signal of the RAM 3 is read by the image signal processing circuit 4, and the image signal is subjected to white balance adjustment based on the digital gain calculated as described above.

[0044] On the other hand, an operation of a second WB mode will be performed as follows. That is, the microcomputer 5 calculates a first WB correction coefficient corresponding the diaphragm value at a prior photographing time based on the first table.

[0045] Here, the prior photographing refers to the photographing which is performed prior to actual photographing of the subject image in order to obtain predetermined data, and the photographing of the actual desired subject image will be referred to as the actual photographing.

[0046] Furthermore, a second WB correction coefficient corresponding the diaphragm value at the time of the actual photographing, and the WB modification coefficient are read based on the first and second tables.

[0047] Moreover, a value of the WB gain multiplied by a value obtained by dividing the second WB correction coefficient by the first WB correction coefficient, and the WB modification coefficient is obtained as the digital gain for amplifying the signal for each color included in the image signal output from the image sensing element 2a.
Digital gain=WB gain×(WB correction coefficient (F value at actual photographing time)/WB correction coefficient (F value at prior photographing time))×WB modification coefficient (WB correction coefficient at actual photographing time)  (2)

[0048] The WB adjustment by the image signal processing circuit 4 is controlled to be performed based on this digital gain. The image signal of the RAM 3 is read by the image signal processing circuit 4, and the WB adjustment is performed with respect to the image signal based on the digital gain calculated as described above.

[0049] Here, a spectral sensitivity distribution of the image sensing element will be described with reference to FIG. 2.

[0050] The image sensing element 2a has spectral sensitivity characteristics shown in FIG. 2. The characteristics of R, G, and B light components are as shown in FIG. 2. Sensitivities of human eyes with respect to the colors are strongest with respect to G light components, and the G light components are seen brightest.

[0051] It is to be noted that the image sensing module 2 including this image sensing element 2a can be replaced with another image sensing module including an image sensing element having different optical characteristics. Accordingly, image sensing modules suitable for photographing purposes are usable such as an image sensing module suitable for nighttime photographing, and an image sensing module suitable for a broad dynamic range.

[0052] Dependence of light receiving sensitivity of the image sensing element on an incidence angle will be described with reference to FIG. 3.

[0053] As shown in FIG. 3, the image sensing element 2a has dependence on the incidence angle, and a relative sensitivity fluctuates by the incidence angle of the light. That is, as the incidence angle of the light increases, a difference is generated in a drop of the relative sensitivity with respect to each of the R, G, and B light components. A drop degree represented as the relative sensitivity (R/G, B/G) especially with respect to the G light component increases. Furthermore, a degree of the fluctuation differs for each image sensing element by fluctuations in designing a micro lens or manufacturing the image sensing element 2a.

[0054] Dependence of an output of the image sensing element on an F value will be described with reference to FIG. 4.

[0055] As shown in FIG. 4, even when micro lenses disposed for pixels are prepared under the same design, a 1/F value increases in a relation between an inverse number (1/F) of an F value and relative sensitivity (R/G, B/G) (i.e., the diaphragm opens), and accordingly a drop is generated in the relative sensitivity with respect to R, G, and B light components. Moreover, a degree of the drop of the relative sensitivity largely differs with each light component.

[0056] In view of the above-described properties, in the image sensing apparatus according to the first embodiment of the present invention, the digital gain is calculated using a WB correction coefficient correspondence table which is the first table, and a WB modification coefficient correspondence table which is the second table, and the signal for each color is amplified with the digital gain.

[0057] This will be described hereinafter in detail.

[0058] First, the WB correction coefficient correspondence table will be described with reference to FIG. 5.

[0059] As shown in FIG. 5, in the WB correction coefficient correspondence table, diaphragm values are associated with WB correction coefficients of R, B signals. In this WB correction coefficient correspondence table, one selected image sensing element is positioned with an image sensing element which is a reference, and correction coefficients of gains of the R and B signals are determined in consideration of the dependence of the image sensing element constituting the reference on the incidence angle. As described above, as to the dependence of the image sensing element on the incidence angle, when the incidence angle increases, the drop of the relative sensitivity with respect to the R and B signals remarkably appears, and this WB correction coefficient compensates for this drop. It is to be noted that the WB correction coefficient correspondence table is stored, for example, in the ROM 1c in the lens module 1 of FIG. 1.

[0060] Next, a WB modification coefficient correspondence table will be described with reference to FIG. 6.

[0061] As shown in FIG. 6, in the WB modification coefficient correspondence table, the WB correction coefficients read from FIG. 5 are associated with WB modification coefficients of R and B signals.

[0062] For example, when this WB correction coefficient is 1.03, a WB modification coefficient_R corresponding to the optical characteristic of the image sensing element 2a is 1.01, and a WB modification coefficient_B is 0.99 from the WB modification coefficient correspondence table.

[0063] It is to be noted that this WB modification coefficient correspondence table is stored in the ROM 2c of the image sensing module.

[0064] In the image sensing apparatus according to this embodiment, the microcomputer 5 calculates the digital gain referring to the correspondence tables of FIGS. 5 and 6.

[0065] More concretely, the microcomputer 5 reads a WB correction coefficient_R and a WB correction coefficient_B corresponding to the diaphragm value of the optical system 1a from the ROM 1c referring to the correspondence table of FIG. 5.

[0066] Moreover, the microcomputer reads the WB modification coefficient_R corresponding to the WB correction coefficient_R, and the WB modification coefficient_B corresponding to the WB correction coefficient_B from the ROM 6 referring to the correspondence table of FIG. 6. Furthermore, the microcomputer calculates the digital gain based on them. For example, the microcomputer 5 calculates the digital gain based on equation (1) in the first WB mode, and based on equation (2) in the second WB mode.

[0067] An operation at the time of the prior photographing of the image sensing apparatus according to the first embodiment of the present invention will be described hereinafter in detail with reference to a flowchart of FIG. 5, and appropriately with reference to FIGS. 5, 6 here.

[0068] A release switch (not shown) is turned on to enter the operation at the time of the prior photographing, and then the microcomputer 5 first performs predetermined photometry calculation, exposure control, and image sensing control (steps S1 to S3). Moreover, the microcomputer stores image sensing data (image signal) from the image sensing element 2a in the RAM 3 (step S4).

[0069] Next, the microcomputer 5 detects the WB gain based on the image signal (step S5), and stores the WB gain in the EEPROM 7 (step S6). Furthermore, the microcomputer stores the F value at the prior photographing time in the EEPROM 7 (step S7), and ends the operation at the prior photographing time.

[0070] By the above-described process according to one example, the WB gain at the prior photographing time, for use in subsequent processes, is associated with the F value, and stored in the EEPROM 7.

[0071] Next, an operation at the time of the actual photographing of the image sensing apparatus according to the first embodiment of the present invention will be described in detail with reference to a flowchart of FIG. 8, and appropriately with reference to FIGS. 5, 6 here.

[0072] When entering the operation at the actual photographing time, first the microcomputer 5 performs predetermined photometry calculation, exposure control, and image sensing control (steps S11 to S13).

[0073] The image sensing element 2a stores image sensing data (image signal) in the RAM 3 (step S14).

[0074] Next, the microcomputer reads the WB correction coefficient_R and WB correction coefficient_B corresponding to the F value at the actual photographing time (from the ROM 1c) referring to the WB correction coefficient correspondence table of FIG. 5 (step S15).

[0075] Furthermore, the microcomputer reads the WB modification coefficient_R and WB modification coefficient_B corresponding to the WB correction coefficient_R and WB correction coefficient_B (from the ROM 6) referring to the WB modification coefficient correspondence table of FIG. 6 (step S16).

[0076] Next, the microcomputer 5 judges whether the WB mode is set to any of an automatic WB (hereinafter referred to as AWB), manual WB (hereinafter referred to as MWB), and one-touch WB (hereinafter referred to as OTWB) in the mode setting SW 8 (step S17).

[0077] Here, the AWB mode is a mode in which white balance adjustment suitable for a scene is automatically performed based on image sensing data. In the MWB mode, the white balance adjustment suitable for the scene is manually designated beforehand by a photographer. Moreover, in the OTWB mode, the white balance is adjusted based on a white balance adjustment gain obtained beforehand by picking up a subject image which is a white reference.

[0078] First, when judging that the AWB mode is set in the step S17, the microcomputer 5 analyzes the image sensing data, and performs judgment concerning the WB gain (step S18).

[0079] Moreover, the WB correction coefficient_R and WB correction coefficient_B read in the step S15, the WB modification coefficient_R and WB modification coefficient_B read in the step S16, and the WB gain obtained in the step S18 are substituted into the calculation equation (1), and accordingly the digital gains of the R and B signals are calculated (step S19).

[0080] Thereafter, the process advances to processes of and after step S25.

[0081] On the other hand, when judging that the MWB mode is set in the step S17, the microcomputer 5 reads the WB gain designated beforehand by the photographer from the ROM 6 (step S20).

[0082] Moreover, the process advances to an MWB side of the judgment of the step S21. The microcomputer substitutes the WB correction coefficient_R and WB correction coefficient_B read in the step S15, the WB modification coefficient_R and WB modification coefficient_B read in the step S16, and the WB gain at the prior photographing time obtained in the step S20 into the calculation equation (1), and calculates the digital gains of the R and B signals (step S19).

[0083] Thereafter, the process advances to the processes of and after the step S25. Here, the white balance adjustment gain designated in the MWB mode is stored beforehand in the ROM 6.

[0084] Furthermore, when judging that the OTWB mode is set in the step S17, the microcomputer 5 reads the WB gain obtained at the prior photographing time (from the EEPROM 7) (step S20).

[0085] Moreover, the process advances to an OTWB side of the judgment of the step S21. The microcomputer reads the F value at the prior photographing time (step S22), and reads the WB correction coefficient at the prior photographing time from the EEPROM 7 (step S23). Moreover, the microcomputer substitutes the WB correction coefficient_R and WB correction coefficient_B at the actual photographing time read in the step S15, the WB gain at the prior photographing time read in the step S20, and the F value at the prior photographing time, and the WB correction coefficient_R and WB correction coefficient_B read in the steps S22, 23 into the calculation equation (2), and calculates the digital gains of the R and B signals (step S24).

[0086] Thereafter, the process advances to the processes of and after the step S25.

[0087] Thus, the digital gain circuit 4a amplifies R, Gr, Gb, and B signals which are image signals temporarily stored in the RAM 3 with the digital gain calculated by the microcomputer 5 to thereby adjust levels, and outputs R′, Gr′, Gb′, and B′ signals (step S25).

[0088] Furthermore, the synchronization circuit 4b performs synchronization of the R′, Gr′, Gb′, and B′ signals, and outputs the R, G, and B signals.

[0089] Moreover, the color conversion circuit 4c subjects the R, G, and B signals to color conversion, and outputs R′, G′, and B′ signals. Furthermore, the gamma conversion circuit 4d subjects these R′, G′, and B′ signals to γ conversion (step S26).

[0090] Results obtained in this manner are recorded in the recording medium 9 (step S27).

[0091] An image sensing apparatus according to a second embodiment of the present invention will be described in a case where an image sensing module comprising a CCD having a WB modification coefficient correspondence table of FIG. 9 and characteristics is attached instead of the image sensing module comprising the CCD having the WB modification coefficient correspondence table of FIG. 6 and the characteristics. In this CCD of the image sensing module, a CCD (hereinafter referred to as a narrow pitch module) having an adjacent pixel pitch narrower than that of an image sensing element 2a is used.

[0092] Here, as shown in FIG. 1, the image sensing module is electrically connected to a lens module via a communication section 1d and 2d. The image sensing module is electrically connected to a microcomputer, RAM 3 and the like of the image sensing apparatus via a communication section 10. The communication sections 1d, 2d and 10 are constituted as connectors, and the image sensing module is accordingly set to be attachable/detachable in this example.

[0093] In the image sensing apparatus according to the second embodiment of the present invention, a digital gain is calculated using a WB correction coefficient correspondence table of FIG. 5 which is a first table and a WB modification coefficient correspondence table of FIG. 9 which is a second table, and a signal for each color is amplified with the digital gain. This will be described hereinafter in detail.

[0094] Since FIG. 5 has been described above in the embodiment, the description is omitted here. The WB modification coefficient correspondence table of FIG. 9 will be described.

[0095] As shown in FIG. 9, in the WB modification coefficient correspondence table, WB correction coefficients read from FIG. 5 are associated with WB modification coefficients of R and B signals.

[0096] Here, by the WB modification coefficients of FIG. 9, excessive/insufficient correction with respect to optical characteristics of an image sensing element which is a reference is further modified. When the WB correction coefficient is corrected by the WB modification coefficient having this narrow pitch module, for example, correction can be performed in the same manner as in appropriate WB correction performed in the first embodiment. In comparison of FIG. 9 with FIG. 6, through regulation, the WB modification coefficients of FIG. 9 corresponding to the narrow pitch module is more strongly influenced by incidence angles, and therefore the modification coefficients have increasing modification amounts.

[0097] It is to be noted that correspondence between the modification coefficient and each diaphragm, and resolution of precision may be varied in such a manner as to achieve optimum correction by the optical characteristics of the image sensing element of the attached image sensing module. For example, the correspondence between the modification coefficient and each diaphragm value may be adjusted into the resolution in accordance with a correction amount, or the correction coefficients with respect to zoom positions may be set with respect to not only TELE, WIDE positions but also different focal distances.

[0098] This WB modification coefficient correspondence table is stored in an ROM 2c of an image sensing module 2 in the same manner as in the first embodiment.

[0099] An image sensing apparatus according to a third embodiment of the present invention will be described in a case where a lens module (hereinafter referred to as a zoom lens module) comprising a zoom lens having a WB correction coefficient correspondence table of FIG. 10, instead of the WB correction coefficient correspondence table of FIG. 5 described above, and characteristics is attached.

[0100] Here, as shown in FIG. 1, an image sensing module is electrically connected to the zoom lens module via a communication section 1d and 2d. The communication section 1d and 2d are constituted as a connector, and accordingly set to be attachable/detachable in the same manner as in the second embodiment.

[0101] In the image sensing apparatus according to the third embodiment of the present invention, a digital gain is calculated using a WB correction coefficient correspondence table of FIG. 10 which is a first table and a WB modification coefficient correspondence table of FIG. 6 which is a second table, and a signal for each color is amplified with the digital gain. This will be described hereinafter in detail.

[0102] The WB correction coefficient correspondence table of FIG. 10 will be described hereinafter.

[0103] In the zoom lens, an influence of a diaphragm of a luminous flux which enters an image sensing element differs with a focal distance. Therefore, as shown in FIG. 5, in the WB correction coefficient correspondence table, diaphragm values are associated with WB correction coefficients of the R and B signals in focal distances on a wide angle side (WIDE) and a telescope side (TELE). It is to be noted that the WB correction coefficient correspondence table is stored, for example, in an ROM 1c in the zoom lens module.

[0104] Here, the WB correction coefficients of FIG. 10 indicate excess/shortage of correction with respect to optical characteristics of a photographing lens which is a reference. When the correction is performed by the WB correction coefficient of the zoom lens module and the WB modification coefficient of the image sensing module, for example, correction can be performed in the same manner as in appropriate WB correction performed in the first embodiment. As seen from FIG. 10, in a short focal distance (i.e., the wide side), the correction value of the WB correction coefficient is larger even in the same diaphragm.

[0105] It is to be noted that correspondence between the modification coefficient and each diaphragm, and resolution of precision may be varied in such a manner as to achieve optimum correction by the optical characteristics of the photographing lens of the attached lens module.

[0106] Even when the combination of the lens module with the image sensing module is changed as described above, or in any of the above-described embodiments, the appropriate WB correction can be performed.

[0107] As described above, according to the embodiments of the present invention, the precision of the WB adjustment is enhanced, so-called color fogging phenomenon is suppressed, and an appropriately exposed image is obtained.

[0108] Moreover, in the calculation of the digital gain relating to the WB adjustment, a preferable process can be performed in accordance with various photographing modes. Furthermore, the WB adjustment can be performed fully considering the fluctuations of the image sensing element at a manufacturing time, optical characteristics of the micro lens and the like.

[0109] The first to third embodiments of the present invention have been described above, but the present invention is not limited to these embodiments, and can be variously modified/changed within the scope of the present invention.

[0110] Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general invention concept as defined by the appended claims and their equivalents.