Method for compensating image color when adjusting the contrast of a digital color image

[0002] The invention relates generally to the field of digital color image processing and, more particularly, for compensating the image color differences for the applied luminance tone scale.

[0003] There exist in the prior art several methods of applying a tone scale function to a color image. The first method involves applying the tone scale function to each of the color signals of the original image. This may be implemented by applying a tone scale function Look-Up-Table to each of the red, green, and blue channels of an image independently. In general, tone scale application by this method has a color characteristic in addition to the modification of the overall image contrast. The image processed by a tone scale function in the manner described will effect each pixel's hue and saturation.

[0004] A second method of tone scale application involves rotating a color image into a space consisting of a luminance signal and several chrominance channels. The tone scale function is then applied only to the luminance channel. The processed image is obtained by applying an inverse matrix to produce transformed red, green, and blue pixel values. This method preserves the original chrominance signal at each pixel. However, because the luminance contrast was modified and the chrominance contrast was preserved, the color saturation of the resulting image may appear artificial and unnatural.

[0005] In an effort to address this problem, Viggiano and Wang proposed (1992 TAGA Proceedings: A Comparison of Algorithms for Mapping Color Between Media of Differing Luminance Ranges,pp. 959-974) a method of scaling the chroma signal of a digital color image by a factor midway between unity and the ratio of the L* range of the reproduction and the original. However, this solution does not anticipate tone scale functions that vary the level of compression dependent upon the intensity of the input. Thus, a single factor that scales the chroma signal of the input image is inappropriate for a wide range of possible tone scale functions.

[0006] In addition, in U.S. Pat. No. 5,446,504, Wada describes a method to correct color saturation when compressing the dynamic range of an image. However, this method once again determines the correction factor based upon the relationship of the input image dynamic range and the output image dynamic range.

[0007] In addition, in U.S. Pat. No. 5,638,138, Hickman describes a method of modifying the luminance signal of an input image, then “multiplying the color components by a transfer ratio of modified luminance to unmodified luminance to obtain modified color components.” However, this method again does not consider that a tone scale function may consist of several ranges of varying level of contrast compression or enhancement.

[0008] Thus, none of the methods described in the prior art have determined a specific color adjustment factor based upon both the intensity values of the pixel being processed and the given tone scale function. Consequently, there exists a need for a method of adjusting the color saturation of each image pixel by a factor related to the local slope of the given tone scale function.

[0009] The object of the present invention is directed to providing a superior method of compensating the color saturation characteristics of a digital color image for the application of a tone scale function. Briefly summarized, according to one aspect of the present invention, the invention resides in the steps of: (a) receiving a tone scale function; (b) calculating a local slope of the tone scale function for each pixel of the digital color image; (c) calculating a color saturation signal from the digital color image, and (d) adjusting the color saturation signal of the digital color image for each of the pixels based on the calculated local slope.

[0010] These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings.

[0011] FIG. 1 is a block diagram illustrating an overview of the present invention;

[0012] FIG. 2 is an exploded view of the color difference adjuster of FIG. 1;

[0013] FIG. 3 is an exploded view of an alternative embodiment of the color difference adjuster of FIG. 1;

[0014] FIG. 4 is an exploded view of an alternative embodiment of the color difference adjuster of FIG. 1; and,

[0015] FIG. 5 is a illustration of a two-dimensional LUT used to determine the color position weight K.

[0016] In the following description, the present invention will be described in the preferred embodiment as a software program. Those skilled in the art will readily recognize that the equivalent of such software may also be constructed in hardware.

[0017] Referring to FIG. 1, there is illustrated an overview of the present invention. It is instructive to note that the present invention utilizes a digital color image that is typically a two-dimensional array of red, green, and blue pixel values. The description of the present invention is applicable to an image of any resolution.

[0018] The purpose of the present invention is the modification of the color saturation characteristics of a color image in accordance with the luminance contrast modification. The color saturation characteristics of a color image may be represented by a color saturation signal. Those skilled in the art will recognize that color saturation signals may be calculated in a number of ways. For example, as one calculation of a color saturation signal, color difference signals may be calculated simply by implementing matrix multiplication on pixel values of the original image. Color difference signals refer to the difference of a color channel to a reference channel. A three channel color image consisting of a red, green, and blur channels may be converted to a three channel image consisting of a luminance channel and two color difference channels by matrix multiplication.

[0019] A single color saturation signal could be created by converting the two color difference signals to polar form by calculating an angle and a radius. The calculation of polar coordinates from rectangular coordinates is well known in the art and will not be further described. When such a conversion is performed, the radius of the polar representation of the color difference signals represents the color saturation signal. The method of the present invention could be performed with equal success on such a color saturation measure. However, because of the simplicity of calculating the color difference signals, the preferred embodiment of the present invention modifies the color difference signals as a means of modifying the color saturation characteristics.

[0020] As an alternative method of calculating a color saturation signal, in The Reproduction of Colour,Fifth Edition by Dr. R. W. G. Hunt, the calculation of CIE 1976 saturation is described to provide an objective measure of saturation for any color.

[0021] Those skilled in the art will readily recognize that although the present invention is described in regard to modification of color difference signals, similar operations performed on alternative image color saturation signals do not significantly deviate from the scope of the present invention.

[0022] The input image is passed into the color difference adjuster 10. The color difference adjuster adjusts the relative color difference of each pixel in the input image in accordance to a provided tone scale function, as will be further described. It is necessary to note that the provided tone scale function may be implemented as a LUT, or may be the implied tonal response of a device that resides in the imaging chain. The output of the color difference adjuster is a transformed color image with color difference signals modified to compensate for the tone scale function.

[0023] Referring to FIG. 2, therein is illustrated an exploded view of the color difference adjuster 10. The preferred embodiment shown in FIG. 2 assumes that the input image has not yet been modified by the given tone scale. An alternative embodiment will describe a method of application of the present invention on an image that has already had the tone scale applied to the luminance channel of the image at the time the image is passed to the color difference adjuster 10. The color difference signals as well as a luminance signal of the original color image are computed by the color difference transformer 20. The computation of the color difference signals by the color difference transformer may be performed by a matrix operation. For example, in the preferred embodiment, a luminance signal and two color difference signals may be determined for each pixel of the input image by the following matrix transform: $\left[\begin{array}{ccc}\frac{1}{3}& \frac{1}{3}& \frac{1}{3}\\ -\frac{1}{4}& \frac{1}{2}& -\frac{1}{4}\\ -\frac{1}{2}& 0& \frac{1}{2}\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{c}L\\ C_1\\ C_2\end{array}\right]$

[0024] Where:

[0025] R, G, and B represent the original red, green, and blue intensities at a particular pixel;

[0026] L represents the luminance signal; and,

[0027] C₁, C₂represent the color difference signals.

[0028] In an alternative embodiment, a luminance signal and three color difference signals may be determined for each signal, also with a matrix transform: $\left[\begin{array}{ccc}\frac{1}{3}& \frac{1}{3}& \frac{1}{3}\\ -\frac{2}{3}& \frac{1}{3}& \frac{1}{3}\\ \frac{1}{3}& -\frac{2}{3}& \frac{1}{3}\\ \frac{1}{3}& \frac{1}{3}& -\frac{2}{3}\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{c}L\\ C_1\\ C_2\\ C_3\end{array}\right]$

[0029] Where:

[0030] R, G, and B represent the original red, green, and blue intensities at a particular pixel;

[0031] L represents the luminance signal; and,

[0032] C₁, C₂, and C₃represent the color difference signals.

[0033] In general, the calculation of each color difference signal is a linear combination of the red, green, and blue intensities of a pixel of the original image. The coefficients are such that the sum of the weights of the red, green, and blue coefficients must sum to 0.

[0034] Again referring to FIG. 2, the given tone scale function is input to the slope calculator 30. The original red, green, and blue pixel values are also input to the slope calculator 30. The purpose of the slope calculator 30 is to compute the local slope of the given tone scale function, evaluated for the current pixel being processed. In the preferred embodiment, the equation used to calculate the local slope of the given tone scale function is: $f^{\prime }=\frac{f\left(\max \left(R,G,B\right)\right)-f(\min \left(R,G,B\right)}{\max \left(R,G,B\right)-\min \left(R,G,B\right)}$ $f^{\prime }=\frac{f\left(1/3\left(R+G+B\right)+\Delta \right)-f\left(1/3\left(R+G+B\right)-\Delta \right)}{2\Delta }$

[0035] If max(R,G,B)−min(R,G,B)>10 otherwise

[0036] Where:

[0037] Δ=a small constant. In the preferred embodiment, Δ=5;

[0038] f(x)=the value of the tone scale function, evaluated at x;

[0039] f′=local slope of the tone scale function;

[0040] max(a,b,c)=the maximum value of a, b, and c; and,

[0041] min(a,b,c)=the minimum value of a, b, and c.

[0042] As an alternative embodiment, the slope of the tone scale function may be calculated as: $f^{\prime }=\frac{f\left(1/3\left(R+G+B\right)+\Delta \right)-f\left(1/3\left(R+G+B\right)-\Delta \right)}{2\Delta }$

[0043] Where:

[0044] Δ=a small constant. In this embodiment, Δ=5.

[0045] In a further alternative method of calculating the local slope of the given tone scale, the following equation may also be implemented by the slope calculator 30: $f^{\prime }=\frac{1}{\max \left(R,G,B\right)-\min \left(R,G,B\right)+1}\left[\sum _{i=\min \left(R,G,B\right)}^{\max \left(R,G,B\right)}\frac{f\left(i+\Delta \right)-f\left(i-\Delta \right)}{2\Delta }\right]$

[0046] Where:

[0047] All terms have been previously defined.

[0048] Those skilled in the art will realize that many alternative methods of calculating the local slope of the given tone scale function may be formulated. However, the variance between these alternative methods is low and will not introduce significant alterations from this description of the present invention.

[0049] Still referring to FIG. 2, the output of the slope calculator 30 is passed to the adjustment factor calculator 40. The purpose of the adjustment factor calculator 40 is to determine a scaling factor for the color difference signals, based upon the local slope of the tone scale function as determined by the slope calculator 20.

[0050] The adjustment factor calculator 40 calculates for each color difference signal C_i(i=1,2, . . . ) an adjustment factor H_i(i=1,2, . . . ). In the preferred embodiment, the adjustment factor is calculated by the formula:

[0051] H_i=γ_i(f′−f′₀)+f′₀

[0052] Where:

[0053] γi=slope influence factor. This parameter typically ranges from 0 to 1.0. In the preferred embodiment, γi=0.45 for all i;

[0054] f′=calculated local slope of tone scale function. The output of the slope calculator 30; and,

[0055] f′₀=nominal slope constant. In the preferred embodiment, f′₀=1.0.

[0056] Those skilled in the art will realize that the exact formulation of the equation implemented by the adjustment factor calculator 40 for the purpose of calculating the adjustment factor may take on many forms without significantly deviating from the scope of the present invention.

[0057] In an alternative method for calculating the adjustment factor H_ian additional term K, the color position weight, is introduced.

[0058] H_i=Kγ_i(f′−f′₀)+f′₀

[0059] Where:

[0060] K=color position weight.

[0061] In general, the value of the color position weight K ranges from 0.0 to 1.0. The color position weight K enables the method of the present invention to allow the calculation of the adjustment factor H_ito vary based upon the color characteristics of the pixel. In general, K is a function of the luminance L and color difference C_ivalues. FIG. 5 illustrates the use of a two-dimensional LUT 90 as a method of determining the value of K for each pixel. As shown in FIG. 5, the value of the color position weight K is shown to be dependent upon the color differences C_i. The value of the color position weight K shown in FIG. 5 may also be derived analytically by the following mathematical formula: $K=\exp \left[\frac{-1}{2000}\left(0.23{\left(C_1+95\right)}^2-0.15\left(C_1+95\right)\left(C_2+170\right)+0.36{\left(C_2+170\right)}^2\right)\right]$

[0062] This formulation of the color position weight K allows the method of the present invention to adjust the color difference values of pixels that are colored similarly to human flesh (K˜1.0), but preserve the original color difference values for pixels that are colored differently than human flesh (K˜0.0). This embodiment has been found to be advantageous to avoid desaturation of saturated objects such as sky, while allowing for the required modifications to the color difference values of flesh color pixels.

[0063] Those skilled in the art will realize that the determination of a color position weight K may take many forms (such as introducing a luminance L value dependence into the determination of the color position weight K) without significantly deviating from the scope of the present invention.

[0064] Again referring to FIG. 2, the output of the adjustment factor calculator 40 is passed to the multiplier 50. The purpose of the multiplier 50 is to scale each of the color difference signals by the output of the adjustment factor calculator 40. The scaled color difference signals that are the output of the multiplier 50 are hereafter referred to as the compensated color difference signals. That is:

[0065] C_pi=C_i*H_iFor i=1,2,

[0066] Where:

[0067] C_pi=the output of multiplier 50. I.e. compensated color difference signal;

[0068] C_i=color difference signals; and,

[0069] H_i=output of adjustment factor calculator 40.

[0070] Again referring to FIG. 2, the luminance signal L calculated by the color difference transformer may be passed to the tone scale applicator 60. However, in regard to the present invention, the given tone scale function may be applied to the image as a processing step following the application of the present invention. The tone scale applicator 60 modifies the luminance signal with the application of the given tone scale function. A Look-Up-Table may be implemented in order to apply the tone scale function. As an alternative method of applying the given tone scale function to the luminance signal L, the method of U.S. Pat. No. 5,012,333 may be implemented. In either of these cases, the signal output of the tone scale applicator is a function of the input luminance signal.

[0071] L_p=f(L)

[0072] Where:

[0073] L_p=the transformed luminance signal. I.e. the output of the tone scale applicator 60; and,

[0074] f(x)=the value of the given tone scale function, evaluated at x. If the given tone scale represents the implied tonal response of a device in the image processing path, then L_p=L.

[0075] Again referring to FIG. 2, the compensated color difference signals and the luminance signal are passed to the RGB reconstructor 70. The RGB reconstructor 70 converts the image from a luminance signal and a plurality of color difference signals to an image containing at each pixel location red, green, and blue intensity values. This conversion is accomplished by the use of a matrix. If a three by three matrix was used by the color difference transformer 20, then the conversion performed by the RGB reconstructor is to simply multiply a vector of the luminance signal and two color difference signals by the inverse of the matrix used by the color difference transformer 20. In the preferred embodiment, $\left[\begin{array}{ccc}1& -\frac{2}{3}& -1\\ 1& \frac{4}{3}& 0\\ 1& -\frac{2}{3}& 1\end{array}\right]\left[\begin{array}{c}{L}_p\\ C_{\mathrm{p1}}\\ C_{\mathrm{p2}}\end{array}\right]=\left[\begin{array}{c}{R}_p\\ G_p\\ B_p\end{array}\right]$

[0076] Where:

[0077] R_p, G_p, and B_prefer to the processed red, green, and blue pixels respectively.

[0078] If the alternative embodiment of using a 4×3 matrix is implemented, then three color difference channels are created. In that case, the RGB reconstructor 70 performs the following operation in order to produce R_p, G_p, and B_p $\left[\begin{array}{cccc}1& -1& 0& 0\\ 1& 0& -1& 0\\ 1& 0& 0& -1\end{array}\right]\left[\begin{array}{c}L\\ C_{\mathrm{p1}}\\ C_{\mathrm{p2}}\\ C_{\mathrm{p3}}\end{array}\right]=\left[\begin{array}{c}{R}_p\\ G_p\\ B_p\end{array}\right]$

[0079] In general, the product of the matrix used by the RGB reconstructor 70 and the matrix used by the color difference transformer 20 is an identity matrix.

[0080] The R_p, G_p, and B_psignals are output from the RGB reconstructor 70. When considered together, these three processed color planes form a processed image in which the color difference signals have been modified in accordance with the luminance modification by a given tone scale function.

[0081] In an alternative embodiment to the present invention, it is possible that the image input to the color difference adjuster 10 may already have been modified by the given tone scale function. The given tone scale function may have been applied to either each of the color channels of the input image or to only the luminance channel of the given image.

[0082] In the case where the given tone scale function has been applied to each of the color records of the input image, the alternative embodiment is illustrated by FIG. 3.

[0083] As illustrated in FIG. 3, the image input to the color difference adjuster 10 is first passed to the color difference transformer 20 in order to generate a luminance signal and several color difference signals. The color difference signals output from the color difference transformer 20 are passed to the multiplier, which scales each of the color difference channels by a factor output by the adjustment factor calculator 40, as previously described in the preferred embodiment. The modified color difference channels output from the multiplier 50 are, along with the luminance channel output from the color difference transformer 20, passed to the RGB reconstructor 70. The output of the RGB reconstructor 70 is a processed image with the color differences appropriately adjusted for the given tone scale function.

[0084] Again referring to FIG. 3, the tone scale function that has already been applied to each of the color channels of the input image is passed to the inverse function generator 80. The inverse function generator constructs the inverse of the tone scale function. Finding the inverse of a function is a well known technique in mathematics and will not be further discussed. The inverse tone scale function generated by the inverse function generator 80 is then passed to the slope calculator 30, the operation of which has previously been described. The output of the slope calculator is passed to the adjustment factor calculator 40, the operation of which has also been described.

[0085] If the luminance signal of the image input to the color difference adjuster 10 has been modified by the given tone scale function prior to the image being passed to the color difference adjuster 10, then the alternative embodiment illustrated by FIG. 4 may be implemented to modify the color difference signals of the input image in an appropriate manner.

[0086] The alternative embodiment illustrated by FIG. 4 is identical to the preferred embodiment with the exception that there is no option to apply the given tone scale function to the luminance signal of the input image by the tone scale applicator 60 shown in FIG. 2. In the case of operating the present invention on a digital color image with a luminance signal modified by the given tone scale function prior to the application of the present invention, an additional tone scale applicator 60 would be redundant and was thus omitted from the alternative embodiment as illustrated in FIG. 4.