Method for processing a backlight image and device thereof

[0001] 1. Field of the Invention

[0002] The present invention relates to a method for processing a backlight image and a device thereof, and more specifically, to a method for compensating a darker region of a backlight image and a device thereof.

[0003] 2. Description of the Prior Art

[0004] With the development of technology, analog tools are gradually replaced by digital products. Digital camera is the most representative example. Compared with a traditional camera, images are digitalized into digital signals by the digital camera. Then the digital signals are saved into a file form in a memory device. A user can connect the digital camera to a computer system, meaning that the images can be saved in a hard disk. In such a manner, the images took by the digital camera can be displayed on a display device, or output by a printer immediately. Moreover, the user can also edit the saved images using an image editing tool to create images with special effects.

[0005] It is unavoidable for users to take photos of backlight image. Backlight image means an image having a dark object with a bright background. For improving this drawback, many methods for compensating a backlight image have been developed. The prior arts involve overcompensating the whole background based on the entire image, or compensating the backlight image in two stages through fuzzy technology. The methods mentioned above usually result injunction effect of the image. For example, please refer to FIG. 1. FIG. 1 is a γ correction curve of two-stage backlight image compensating method according to a prior art. The γ correction curve presents a corresponding relationship of brightness before and after being adjusted. The region that the unadjusted luminance value smaller than ‘A’ is the darker part of the backlight image. The two-stage compensating method according to the prior art involves only processing the darker part of the backlight image in a brightness-enhancement way. Therefore, there is a considerable discontinuity of color level at the border between two image-compensating stages (at the unadjusted luminance ‘A’). That affects smoothness of the color level and then lowers the quality of the image. For solving this problem, another backlight image compensating method has been disclosed in the paper, “Detection and Compensation Algorithm for Backlight Images with Fuzzy Logic and Adaptive Compensation Curve”, issued by C. L. Chin and C. T. Lin on International Journal of Pattern Recognition and Artificial Intelligence. However, the amount of calculations in the method is huge because of the extra compensation region detection. That will reduce the image processing efficiency. Therefore, how to develop a method to balance the image quality and calculation efficiency is the main subject of the present image processing technology.

[0006] It is therefore a primary objective of the claimed invention to provide a method for compensating a darker brightness of an image and a device thereof to solve the above-mentioned problem.

[0007] The claimed invention provides an image processing method comprising (a) inputting image data comprising a plurality of pixels; (b) setting a first luminance interval and a second luminance interval; (c) generating a first mean luminance value from a plurality of pixels of the image data within the first luminance interval; (d) generating a second mean luminance value from a plurality of pixels of the image data within the second luminance interval; (e) generating a first luminance threshold according to the first mean luminance value; (f) setting a compensation parameter; (g) generating a first compensation luminance value according to the first mean luminance value and the compensation parameter; (h) generating a second compensation luminance value according to the first luminance threshold, the first mean luminance value, the second mean luminance value and the compensation parameter; (i) obtaining a greater compensation luminance value according to a comparison between the first and second compensation luminance values; and (j) adjusting luminance values of image data between a smaller compensation luminance value and the greater compensation luminance value according to the smaller and greater compensation luminance values.

[0008] The claimed invention further provides an image processing system comprising an image inputting module for inputting an image data; a mean luminance value calculating device for generating a first mean luminance value according to a plurality of pixels within a first luminance interval of the image data, and generating a second mean luminance value according to a plurality of pixels within a second luminance interval of the image data; a compensation luminance value calculating device for generating a first compensation luminance value according to the first mean luminance value and a compensation parameter, and generating a second compensation luminance value according to a first luminance threshold, the first mean luminance value, the second mean luminance value, and the compensation parameter; a compensation luminance value comparing device for generating a greater compensation luminance value according to a comparison between the first and second compensation luminance values; and a processing device for adjusting luminance values of image data between a smaller compensation luminance value and the greater compensation luminance value according to the smaller and greater compensation luminance values.

[0009] These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

[0015] Please refer to FIG. 2. FIG. 2 is a functional block diagram of an image processing system 10 according to an embodiment of the present invention. The image processing system 10 can be a digital camera, a mobile phone, or a computer system. The image processing system 10 comprises an image inputting module 12 for inputting an image data. The image inputting module 12 can be an image capturing module for taking images. The image inputting module 12 comprises a lens 14 for receiving light, and an image sensor 16 for transforming light received by the lens 14 into electrical signals to generate the image data. The image sensor 16 comprises a plurality of CCD (Charge Coupled Devices) image sensors or a plurality of CMOS (Complementary Metal-oxide Semiconductor) image sensors. The image processing system 10 further comprises a determining device 18, a mean luminance value calculating device 20, a compensation luminance value calculating device 22, a compensation luminance value comparing device 24, a processing device 26, a display module 28, and an input module 32. The determining device 18 is used for determining if the image data input by the image inputting module 12 is a backlight image. The mean luminance value calculating device 20 is used for generating a first mean luminance value C₁ according to a plurality of pixels within a first luminance interval, and generating a second mean luminance value C₂ according to a plurality of pixels within a second luminance interval. The compensation luminance value calculating device 22 is used for generating a first compensation luminance value Y₁ according to the first mean luminance value C₁ and a compensation parameter C, and generating a second compensation luminance value Y₂ according to a first luminance threshold C₁+σ, the first mean luminance value C₁, the second mean luminance value C₂ and the compensation parameter C. The compensation luminance value comparing device 24 is used for generating a greater compensation luminance value B through comparing the first compensation luminance value Y₁ with the second offset luminance Y₂. The processing device 26 is used for generating an adjusted luminance value between a smaller compensation luminance value b and the greater compensation luminance value B according to the smaller compensation luminance value b and the greater compensation luminance value B. The display module 28 is used for displaying the adjusted image data. If the determining device 18, the mean luminance value calculating device 20, the compensation luminance value calculating device 22, and the compensation luminance value comparing device 24 are program codes, the image processing system 10 can comprise a memory module 30 for saving the determining device 18, the mean luminance value calculating device 20, the compensation luminance value calculating device 22, and the compensation luminance value comparing device 24. The input module 32 is used for inputting data corresponding to the compensation parameter C, the first luminance threshold C₁+σ, the smaller luminance b, and so on.

[0016] Please refer to FIG. 3. FIG. 3 is a flowchart of a method for processing the image data through the image processing system 10 according to the embodiment of the present invention. The method comprises the following steps.

[0017] Step 100: The image inputting module 12 inputs the image data comprising a plurality of pixels;

[0018] Step 102: The determining device 18 determines if the image data input by the image inputting module 12 is a backlight image. If so, go to Step 104; if not, go to Step 122 to end the procedure. That is to say, the backlight image compensating procedure for the image data will not be executed.

[0019] Step 104: The mean luminance value calculating device 20 generates the first mean luminance value C₁ according to the plurality of pixels within the first luminance interval;

[0020] Step 106: The mean luminance value calculating device 20 generates the second mean luminance value C₂ according to the plurality of pixels within the second luminance interval;

[0021] Step 108: Generate the first luminance threshold C₁+σ according to the first mean luminance value C₁;

[0022] Step 110: Set the compensation parameter C;

[0023] Step 112: The compensation luminance value calculating device 22 generates the first compensation luminance value Y₁ according to the first mean luminance value C₁ and the compensation parameter C;

[0024] Step 114: The compensation luminance value calculating device 22 generates the second compensation luminance value Y₂ according to the first luminance threshold C₁+σ, the first mean luminance value C₁, the second mean luminance value C₂, and the compensation parameter C;

[0025] Step 116: The compensation luminance value comparing device 24 generates a greater compensation luminance value B through comparing the first compensation luminance value Y₁ with the second compensation luminance value Y₂;

[0026] Step 118: The processing device 26 generates the adjusted luminance value Y_out within the image data between the smaller compensation luminance value b and the greater compensation luminance value B according to a relation degree r(i), the unadjusted luminance value Y_in, b, and B;

[0027] Step 120: The processing device 26 controls the display module 28 to display the adjusted image data according to the adjusted luminance value Y_out within the image data between the smaller compensation luminance value b and the greater compensation luminance value B;

[0028] Step 122: End.

[0029] More detailed descriptions for the steps mentioned above will be provided. First, the image inputting module 12 is utilized to input image data, such as photos. Light emitted or reflected by the object can be received by the lens 14 and then be transformed into electrical signals by the image sensor 16 to generate image data. Next, the determining device 18 determines whether the image data input by the image inputting module 12 is a backlight image. If the determining device 18 determines the image data is a backlight image, the backlight image compensating procedure for the image data will be executed. If not, the backlight image compensating procedure for the image data will not be executed. That is to say, the image data will be output directly or be edited by other image algorithm without image processing. Many methods for determining if the image data input by the image inputting module 12 is a backlight image have already been disclosed in the prior art. For example, please refer to FIG. 4. FIG. 4 is a histogram of the image data. The histogram of the image data is generated according to the luminance values (such as gray level) of the plurality of pixels and the corresponding pixel points. The distribution of the histogram can be divided into three luminance intervals. The first luminance interval is a dark range. The second luminance interval is a light range. And the third luminance interval is a middle luminance interval. The boundary ‘X’ of the first luminance interval and the third luminance interval and the boundary ‘Y’ of the second luminance interval and the third luminance interval can be defined by a user. In general, when the luminance interval of the image data lies in between 0 and 2^L (L is a positive integer, such as 8), X can be equal to 25%˜40% of 2^L and Y can be equal to 75%˜85% of 2^L. When the image data conforms to three conditions at the same time, the determining device 18 determines the image is a backlight image. The conditions are described as follows: (1) a ratio of the number of pixels within the first luminance interval to the number of pixels within the image data, and a ratio of the number of pixels within the second luminance interval to the number of pixels within the image data are greater than a specific ratio respectively, such as 3%; (2) a sum of the ratio of the number of pixels within the first luminance interval to the number of pixels within the image data and the ratio of the number of pixels within the second luminance interval to the number of pixels within the image data is greater than a specific ratio, such as 70%; (3) the number of pixels within the first luminance interval are not equal to the number of pixels within the second luminance interval. The aforementioned conditions are utilized to determine if the number difference of pixels between the light range and the dark range is too big. If so, the image data is determined as a backlight image. If not, the image data is not determined as a backlight image.

[0030] Next, the mean luminance value calculating device 20 generates the first mean luminance value C₁ according to the luminance value distribution of the plurality of pixels within the first luminance interval and generates the second mean luminance value C₂ according to the luminance value distribution of the plurality of pixels within the second luminance interval. The first mean luminance value C₁ is generated correspondingly according to half an accumulated number corresponding to luminance values within the first luminance interval. The accumulated number is accumulated from a number corresponding to the smallest luminance value to a number corresponding to the greatest luminance value within the first luminance interval. The first mean luminance value C₁ can also be generated through calculating a weighted average of the luminance values and the number corresponding to the luminance values within the first luminance interval. In a same manner, the second mean luminance value C₂ is generated through the said methods.

[0031] As for Step 108, the first luminance threshold C₁+σ can be generated through a sum of a value σ and the first mean luminance value C₁. The value σ can be input by the user via the input module 32. The value σ can also be generated when a ratio of the accumulated luminance within the range [C₁−σ, C] to the accumulated luminance within the first luminance interval is equal to a specific ratio, such as 90%. Mostly, the value σ is equal to C₁ because the first mean luminance value C₁ is very close to the origin (the smallest luminance value within the first luminance interval) and C₁−σ cannot be a negative value.

[0032] Next, there are definitions for a relation degree r(i) and a compensation luminance value Y. The definition of the relation degree r(i) is described as follows wherein i is a luminance value:

[0033] $r=\frac{d_2}{d_1+d_2}=\left\{\begin{array}{cc}\frac{d_2}{d_1+d_2},& \forall i\in \left[0,C_1-1\right]\\ \frac{d_2}{C_2-C_1}& \forall i\in \left[C_1,C_1+\sigma \right]\end{array}\right\}$
d_i=|C₁−i|,d₂=|C₂−i|

[0034] The definition of the compensation luminance value Y is described as follows wherein where C is the compensation parameter:
Y=i+r×C,∀iε[0,C₁+σ],Cε[0,2^L−1]

[0035] The value C can be input by the user through the input module 32 or can be the average of the first mean luminance value C₁ and the second mean luminance value C₂, wherein the greater the value C is, the larger the area of the compensation range is.

[0036] Next, the greatest value of the compensation luminance value Y within iε[0,C₁+σ] is needed to be an upper limit of the image compensation range. The greatest value of the compensation luminance value Y can be generated when i is equal to C₁ or C₁+σ. That will avoid unnecessary amount of the calculations. When i is equal to C₁, the first compensation luminance value Y₁ is generated by the following formula:
Y₁=C₁+C

[0037] When i is equal to C₁+σ, the second compensation luminance value Y₂ is generated by the following formula:

[0038] $Y_2=\left(C_1+\sigma \right)+\frac{C_2-\left(C_1+\sigma \right)}{C_2-C_1}\times C$

[0039] In Step 112 and Step 114, the first compensation luminance value Y₁ and the second compensation luminance value Y₂ are generated respectively by the compensation luminance value calculating device 22. As mentioned above, because the greatest value of the compensation luminance value Y can be generated when i is equal to C₁ or C₁+σ, the compensation luminance value comparing device 24 in Step 116 compares the first compensation luminance value Y₁ with the second compensation luminance value Y₂ to generate a greater compensation luminance value B wherein the greater compensation luminance value B is a greater value between the first compensation luminance value Y₁ and the second compensation luminance value Y₂.

[0040] Next, the adjusted luminance Y_out between the smaller compensation luminance value b (set by the user) and the greater compensation luminance value B can be generated through the following formula:

[0041] $Y_{\mathrm{out}}=B-\frac{{\left(B-Y_{\mathrm{in}}\right)}^{\gamma }}{{\left(B-b\right)}^{\gamma -1}},\gamma \ge 1,Y_{\mathrm{in}}\ge b$

[0042] And the smaller compensation luminance value b and the value γ can be input by the input module 32. The smaller compensation luminance value b is much smaller than the greater compensation luminance value B. For example, please refer to FIG. 5. FIG. 5 is a γ correction curve given L=8 after the image data compensation processing is executed according to the embodiment of the present invention. The continuous line is the γ correction curve after being compensated. The dashed line is the γ correction curve before being compensated. When b is set for 10, γ is set for 2, and the greater compensation luminance value B generated by the compensation luminance value comparing device 24 is equal to 200, the γ correction curve as shown in FIG. 5 can be generated. As shown in FIG. 5, the range of the image data compensation is limited between the smaller compensation luminance value b and the greater compensation luminance value B. The image data greater than the greater compensation luminance value B is not adjusted (the slope of the γ correction curve is equal to 1). The γ correction curve after being compensated according to the present invention can avoid a color level discontinuity at the border between two image compensating stages (the greater compensation luminance value B) and can still keep the color level smoothness of the image. That will increase the quality of the image. Finally, the processing device 26 can control the display module 28 to display the adjusted image data according to the adjusted luminance Y_out between the smaller compensation luminance value b and the greater compensation luminance value B.

[0043] Moreover, given fixed point implementation (gamma=2), there are only two division operations and 2(B−b)+1 multiplication operations in the backlight image compensating algorithm of the present invention. Compared with the prior art methods, the present invention can save the amount of calculations efficiently.

[0044] Compared with the prior art method, the image processing method and system thereof according the present invention can solve the discontinuity problem at the border between two image compensating stages and can still keep the color level smoothness and quality of the image. Furthermore, the algorithm according the present invention can save the amount of calculations efficiently. Therefore, the method of the present invention is a method to balance image quality and cost.

[0045] Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.