Automatic contrast enhancement

[0001] This application claims priority from Singapore Application No. 200605450-6 of the same title filed Aug. 8, 2006, the disclosure of which is hereby incorporated by reference.

[0002] The present invention generally relates to image processing technologies and, more particularly, to video and image processing devices and processes for automatic contrast enhancement during video and/or image processing.

[0003] In the context of digital image processing, a clear image would generally mean that features of the image can clearly be identified or interpreted. Contrast enhancement is typically used for improving the interpretation of features of an image. However, as opposed to other imaging controls, such as exposure control, the control of contrast is difficult due to its nonlinear transform function characteristic.

[0004] Histogram modification is often called for in contrast enhancement of an image. A histogram of an image provides pixel intensity of brightness for analyzing the lightness or darkness characteristic of the image. The histogram is a graph showing population of pixels at each grayscale value within the image. For an 8-bit grayscale image, for example, there are 256 different possible intensities, and thus, the histogram of that 8-bit image may show pixel distributions amongst the 256 levels of grayscale values. Similarly, for color images, the histogram may be provided by three individual histograms of red, green and blue channels.

[0005] Depending upon the implementation, the output of a processed histogram may be a processed image of the histogram, or a data file representing the histogram statistics.

[0006] Before natural video from a broadcast, video CD and DVD source is displayed at the consumer end on a TV or other display device, the video can undergo several non-linear processes that include analog to digital and digital to analog conversions, sensor circuitry, transmission attenuation and amplification, and digital encoding and decoding. As a result, the dynamics of the video frames/streams are distorted and the visual quality of the picture in terms of contrast is reduced. Thus, there is a need to reverse the changes and restore visual attractiveness.

[0007] One common method is to provide an adjustable contrast feature on a display device, where the adjustable contrast feature allows manual adjustment of the contrast by stretching the dynamic range of the video with clamping at black level. Problems with this method are that the contrast adjustment is manual so as not be able to adapt to the nature of the source, and that the dynamic stretching characteristics are fixed so as not be suitable for all pictures.

[0008] Histogram equalization is a method employed in image processing to improve the contrast of images by flattening the distribution density in the histogram of an image. However, such a method targets an output at middle gray level regardless of the brightness of the input image, which makes dark images too bright with a white-wash effect and bright pictures too dark or overly contrasted, such that the enhancement makes the picture unnatural or the interpretation of image content is altered. This brings about serious consequences in motion images, i.e. video, when a day scene moves to a night scene and vice versa.

[0009] It is known to use a dual segment transfer function with a lower segment gain adjusted by dark sample distribution, an upper segment gain adjusted by frame or field peak, and an adaptive pivot point that separates the two segments adjusted by the image brightness.

[0010] Further, there is known a contrast enhancement transform made up of two independent transform functions, one with levels less or equal to mean level and the other with levels greater or equal to the mean. In addition, the individual transform functions are constructed in dependence on the distribution of samples in their respective regions.

[0011] The above two methods perform picture and histogram analysis and adjust contrast gain automatically according to parameters not limited to but including picture brightness, sample distribution, frame/field peak. However, these methods have a limitation on correcting over-contrasted pictures.

[0012] FIG. 11 shows an existing contrast improvement method by gathering pictures of sequence into histogram, equalizing histogram class peaks by redistributing histogram values above pre-determined limit to neighboring class, temporally filtering processed histogram recursively to preserve edges, and transforming the histogram to mapping function for altering picture contrast. See, Nenonen, EP 0856813.

[0013] There is yet another image enhancement method by obtaining the histogram of a picture with quantized input pixel values, calculating the cumulative density function, and interpolating the latter as a transform function. The method further describes the control of the transform function such that the mean level of the histogram can be mapped to itself.

[0014] These two methods retain the mean brightness of the picture by controlling the transform function such that the mean brightness mapped onto itself. The mean brightness is sometimes not the same as the perceived brightness. The change in overall brightness impression is acceptable in still pictures but in video sequences with fading transitions there might be an inversion in the lighting condition in consecutive pictures.

[0015] The histogram processing techniques suffer from over-compensating contrast such that some mid-contrasted pictures may become over-contrasted after contrast enhancement. The cumulative density function may result in steep slopes on the transfer curve that produces artifacts including a relatively unrealistic picture with the extremes of dark and bright and loss of desirable details as a result of that overstretched contrast, and unnatural picture with over-expansion of undesired details as a result of overstretching of certain gray levels.

[0016] In addition, in pictures with good contrast and a large portion of near black/white background, good contrasted features may be compressed. Picture blurring may occur as a result of reduced gray level difference between a dominant region and neighboring minor regions from the compression of certain gray level ranges.

[0017] Last but not least, temporal consistency can sometimes be a problem in contrast enhancement with automatic gain control. Slight changes in pictures may have different contrast enhancement effects and this may cause flicker in moving pictures. Solutions like recursive temporal filtering in one of the above methods reduce the flicker but do not respond to scene change in video sequences.

[0030] The present invention overcomes the above limitations and drawbacks of contrast enhancement techniques. It enhances the contrast of pictures, restores visual attractiveness of video and preserves the naturalness of scenes by providing a peak-preserving automatic contrast enhancement technique.

[0031] In one embodiment, there is provided, a process for enhancing contrast of an image having pixels in different brightness intensities, the process comprises generating a histogram in discrete bins, each bin representing a pixel population of at least one pixel brightness intensity; identifying a peak and a peak region of the histogram, wherein the peak region is a range of discrete bins around the peak; computing an average pixel population within the peak region; distributing the pixel populations of the discrete bins within the peak region that exceeds the average pixel population; and generating a transfer curve for mapping onto the image.

[0032] It is preferred that the process of contrast enhancement further comprises expanding the histogram to provide 2ⁿ discrete bins when the generated histogram has a number of discrete bins less than 2ⁿ, where n is number of bits of the image. The process may also filter the histogram with a smoothing filter for avoiding discontinuities of pixel population between successive discrete bins. An adaptive gain may be computed based on the peak, the peak region and the average pixel population during the contrast enhancement process.

[0033] In accordance with the above embodiment, the generated histogram may also include 2ⁿ discrete bins, where n is number of bits of the image and the histogram may be provided based on an active window of the image. Preferably, the peak, the peak region, and the peak density are identified based on the filtered histogram. Also, the transfer curve may comprise two opposing banded curves separated by a pivot point that corresponds to the peak. It is also preferred that, the peak region comprise continuous discrete bins having pixel population higher than a histogram average, wherein the histogram average is an average of a total pixel population over the number of discrete bins.

[0034] Alternatively, the smoothing filter may comprise smoothing factors, and wherein the smoothing filter provides relatively higher smoothing factors at mid level pixel intensities than that of high and low pixel intensities. It is also preferred that the smoothing filter further comprises a smearing parameter for de-emphasizing the histogram.

[0035] It is also preferred that, while distributing the pixel populations, the pixel populations which exceed the average pixel population are distributed along two sides of the peak. The distribution may occur within the peak region, and overflows when the distribution of the pixel population reaches boundaries of the peak region are truncated.

[0036] In accordance with an embodiment, the contrast enhancement may be used on a video comprising video frames of images. Accordingly, the process may further comprise computing a sum of absolute difference (SAD) of the histogram of a current image based on a previous image; and performing a SAD weighted recursive temporal filtering to the histogram. It is also preferred that the process comprise computing a SAD of a transfer curve; and performing a SAD weighted recursive temporal filtering on the transfer curve.

[0037] In another embodiment, there is provided an image processor for enhancing contrast of an image having pixels comprises a histogram generator for generating a histogram in discrete bins, each bin representing a pixel population of at least one pixel brightness intensity; a peak analyzer for identifying a peak, a peak region and an average pixel population of the histogram, wherein the peak region is a range of discrete bins around the peak, and the average pixel population is an average of the pixel population within the peak region; a pixel distributor for distributing the pixel populations of the discrete bins within the peak region that exceeds the average pixel population; and a transfer curve generator for generating a transfer curve for mapping onto the image.

[0038] It is preferred that the image processor further expands the histogram to provide 2ⁿ discrete bins when the generated histogram has a number of discrete bins less than 2ⁿ, where n is number of bits of the image. The image processor may further comprise smoothing filter for filtering the histogram to avoid discontinuities of pixel population between successive discrete bins. The image processor may also compute an adaptive gain based on the peak, the peak region and the average pixel population during the contrast enhancement process.

[0039] The image processor may generate the histogram with 2ⁿ discrete bins, where n is number of bits of the image and the histogram may be provided based on an active window of the image. Preferably, the peak, the peak region, and the peak density are identified based on the filtered histogram. Also, the transfer curve may comprise two opposing banded curves separated by a pivot point that corresponds to the peak. It is also preferred that, the peak region comprise continuous discrete bins having pixel population higher than a histogram average, wherein the histogram average is an average of a total pixel population over the number of discrete bins.

[0040] Alternatively, the smoothing filter may comprise smoothing factors, and wherein the smoothing filter provides relatively higher smoothing factors at mid level pixel intensities than that of high and low pixel intensities. It is also preferred that the smoothing filter further comprises a smearing parameter for de-emphasizing the histogram.

[0041] It is also preferred that, while distributing the pixel populations, the pixel populations that exceed the average pixel population are distributed along two sides of the peak. The distribution may occur within the peak region, and overflows when the distribution of the pixel population reaches boundaries of the peak region are truncated.

[0042] In accordance with an embodiment, the image processor may be used for processing a video comprising video frames of images. Accordingly, the process may further comprises computing a sum of absolute difference (SAD) of the histogram of a current image based on a previous image; and performing a SAD weight recursive temporal filtering to the histogram. It is also preferred that the process comprises computing a SAD of transfer curve; and performing a SAD weighted recursive temporal filtering on the transfer curve.

[0043] In yet another embodiment, a display comprises a receiver for receiving input image from image sources; an image processor in accordance with the aforesaid embodiment, a screen for outputting the processed image.

[0044] The embodiments may be understood more readily by reference to the following detailed description.

[0045] Throughout this application, where publications are referenced, the disclosures of these publications are hereby incorporated by reference, in their entireties, into this application in order to more fully describe the state of art to which this invention pertains.

[0046] It is to be appreciated that video and/image processing comprises more than one operation or process. Thus, before being outputted, an input video signal is subjected to a series of processes to enhance the visual qualities of the processed video and/or images.

[0047] Referring to FIG. 1, there is provided a schematic block diagram of contrast enhancement processing of an input video in accordance with one embodiment of the present invention. The Contrast Enhancement Process 100 comprises a Histogram Pre-Process 102, Peak Analysis 103, Histogram Redistribution 104, Adaptive Gain Computation 105, Transfer Curve Generation 106, and Transfer Curve Post-Process 107. Briefly, the Histogram Pre-Process 102 filters the histogram temporally to minimize variation in pre-analyzed histogram, and then smoothes the histogram along a gray level axis. Then the Peak Analysis 103 detects from the pre-processed histogram a peak region and extracts a peak density, peak location and peak distribution from the pre-processed histogram. Based on the detected region and the extracts, histogram values in excess of peak density are re-redistributed in the step of the Histogram Redistribution 104. The Adaptive Gain Computation process 105 calculates a contrast gain in accordance with extracted peak density, peak position and peak distribution. The Transfer Curve Generation 106 then computes cumulative density functions and outputs a transfer curve. If desired, the transfer curve may be computed with white and black boost. The Transfer Curve Post-Process 107 then filters the transfer curve temporally to ensure temporal stability in the contrast enhancement process 100.

[0048] As also shown in FIG. 1, the Input Video signal when received is channeled into two paths (A and B), where the Video signal in the A path goes through the Histogram Calculation 101 and the Contrast Enhancement Process 100 to generate a specific filtered transfer curve for the input video signal, and then the filtered transfer curve from the A path maps to the video input signal from the B path in the Contrast Mapping 108, resulting in a contrast enhanced video output. In the step of the Histogram Calculation 101, a histogram in a predetermined number of discrete bins is generated based on the input video. Methods of generating the histogram are well known in the art, and any suitable method would be applicable for an appropriate embodiment of the present invention.

[0049] Still referring to FIG. 1, the Histogram Calculation 101 and Contrast Mapping 108 are not included in the box of Contrast Enhancing Process 100. Because both Histogram Calculation 101 and Contrast Mapping 108 are pixel-based operations, they require higher operating power. In one embodiment, these operations are preferably operated on a customized hardware or high-speed processors. In contrast, the boxed Contrast Enhancement Process 100 requires less processing time; thus it can be implemented as software on general controllers or system processors. It is to be appreciated that the configuration of the operations is arbitrary, and in the following descriptions all steps 101-108 are considered a series of operations of the Contrast Enhancing Process 100.

[0050] Referring now to FIGS. 2-8, there are provided more detailed descriptions of the operations of the Contrast Enhancement Process 100.

[0051] In one embodiment, the Histogram Calculation 101 generates the histogram based an active window of the input video; thus any external boundaries of the input video, if present, such as black regions when a wide-screen video is viewing in a letterbox-sized display, are excluded. Each pixel in the active window is scanned and pixel count is performed based on their corresponding grayscale so as to form a histogram of discrete bins. Each discrete bin represents a pixel population with the same pixel brightness intensity. For example, an 8-bits grayscale image consists of 256 (2⁸) different possible intensities or tones. Accordingly, the histogram may consist of 256 discrete bins generally numbered from 0 to 255 showing the brightness, of which the 0^th bin represents the population of black pixels, and the 255^th bin represents the population of white pixels. However, as the Histogram Calculation 101 requires higher operating power, in an alternative embodiment, it is preferred that the Histogram Calculation 101 produces a histogram with a number of bins less than 2ⁿ, where n is the number of bits of that image. That could effectively reduce the resource load for storage and transmission, thereby improving the efficiency of the device carrying out the contrast enhancement process 100 in accordance with the present embodiment. Taking the earlier example, instead of having 256 bins for the 8-bit image, a 32-bin histogram may be generated by increasing the gray level width in each bin. Accordingly, each bin now represents 8 continuous gray levels instead of one. In accordance with a further alternative embodiment, instead of having uniform gray level width for each bin, the Histogram Calculation 101 generates a histogram with varying gray level width for each bin.

[0052] The Histogram Pre-Process 102 provides pre-processing to the histogram generated by the Histogram Calculator 101 with filters. At first, a SAD weighted temporal filter is applied to minimize variations in the histogram bin's value for temporal consistency. To achieve that, the number of bins, nbBin, of the current histogram (of the current input video frame) are compared with the previous histogram (of the previous input video frame) to compute a sum of absolute difference (SAD) histogram, SADHist, as provided by formula (1),

[0053] $\begin{array}{cc}\mathrm{SADHist}=\sum _{i=1}^{\mathrm{nbBin}}\mathrm{abs}\left({\mathrm{curHist}}_i-{\mathrm{prevHist}}_i\right)& \left(1\right)\end{array}$

[0054] where curHist_i is the number of pixels in current video frame with gray-level values defined for bin i, and prevHist_i is the number of pixels in previous video frame with gray-level values defined for bin i.

[0055] With the SADHist, the current histogram is further filtered with a SAD weighted temporal recursive filter. As the differences between the histograms increase i.e., higher SADHist, the weight of the current histogram is increased relative to the previous histogram. The SAD weight is then normalized with respect to the active window size and the width of histogram bins, and it is tunable by a gain factor, k₁. The filtered current histogram is provided by the formula (2),

[0056] $\begin{array}{cc}{\mathrm{curHist}}_i=\mathrm{wgtHist}\times {\mathrm{curHist}}_i+\left(1-\mathrm{wgtHist}\right)\times {\mathrm{prevHist}}_i\mathrm{where}& \left(2\right)\\ \mathrm{wgtHist}=\frac{\mathrm{SADHist}\times \frac{2^n-1}{2^n-\mathrm{binwdh}}\times k_1}{\mathrm{nbPix}},0\le \mathrm{wgtHist}\le 1& \left(3\right)\end{array}$

[0057] And where nbPix represents the number of pixels in the defined active window of the input video, n refers to bit rate, binwdh (binwdh=2ⁿ/nbBin) refers to the average gray level width of a histogram bin and the gain factor, k₁, is a user-defined constant controlling sensitivity to histogram change. The sensitivity of the filter is directly proportional to k₁ but inversely proportional to filtering levels. When more filtering is required, lower sensitivity (lower k₁) is required, and vice versa.

[0058] The processed SAD histogram is used for subsequent processing in the contrast enhancement process 100 as well as for computation of SADHist for a next video frame.

[0059] In accordance with the present embodiment, it is preferred that the histogram is in a full size pixel range. That is to say that for an n-bit image, it is required that the number of bins, nbBin, is equal to 2 to the power of n. Accordingly, the Histogram Pre-Process 102 provides conversions/expansions to the histogram when necessary. For the image where nbBin≠2ⁿ, the full size pixel range histogram bin, hist_i, is derived from the original histogram bin, Hist_j, by hist_i=Hist_j/binwdh, where j=i/binwdh. With the example where the 32 bins histogram for an 8-bit image is generated in the Histogram Calculation 101, the 32 bins histogram is subject to expansion to a 256 bins histogram by dividing each bin value by 8 and repeating the divided or average bin value 8 times along the defined bin region.

[0060] Still in the process of Histogram Pre-Process 102, the temporal filtered and possibly expanded histogram is further filtered by a smoothing filter to ensure better continuity between successive histogram bins. It is known that discontinuities between successive histogram bins generate a transfer curve that results in over-contrasted videos during histogram equalization process. Further, instead of a conventional histogram equalization process that flattens the distribution density equally among the pixel level range, which results in unnatural images, the Histogram Pre-Process 102 de-emphasizes the histogram data with a de-emphasis operator in the smoothing filter. In accordance with the present embodiment, it is preferred that the de-emphasis operator provides relatively higher smoothing factors in mid pixel levels than that of low and high pixel levels such that good contrasted images are not further contrasted. The smoothing filter is provided by formulas (4)-(6).
hist_i=hist_i−1+(hist_i+hist_i−1)×(ƒ(i)/S)  (4)
hist_i=hist_i+1+(hist_i+hist_i+1)×(ƒ(i)/S)  (5)
ƒ(i)=g×abs(i−2ⁿ⁻¹)+c  (6)

[0061] where, hist_i represents the number of pixel at gray level value i, and i is of increment from 0 to (2ⁿ−1), then of decrement from (2ⁿ−1) to 0; S is a smearing constant directly proportional to smoothing levels; ƒ(i) is a function that decreases the smoothing factor with the distance from mid gray level, 2ⁿ⁻¹, for an n-bit image and g and c are the gain and offset constants respectively describing function ƒ(i), where higher g provides higher differentiation of smoothing factor between different grayscale levels and higher c adjusts offset of the function ƒ(i) to give lower smoothing strength.

[0062] FIG. 2 shows graphs 201, 202 and 203, illustrating the Histogram Pre-Process 102. The graph 201 represents the histogram being filtered by the weight temporal filter. The graphs 202 and 203 show two smoothened line curves, which were obtained by smoothing the graph 201 via the smoothing filter. The graph 202 is formed without the de-emphasis operator, and the smearing parameters are S=8, g=0 and c=1. On the other hand, the graph 203 is formed with the de-emphasis operator and the smearing parameters are S=8, g= 1/128 and c=½.

[0063] Referring now to FIG. 3, the graph 203 of FIG. 2 is shown with some of its characteristics marked thereon. The characteristics of the graph 203 are obtained in the Peak Analysis 103, which include, a peak value 301, a peak position 302, a histogram average 303, a peak region 304, a peak width 305 and a peak density 306.

[0064] First, the Peak Analysis 103 identifies the peak value 301 of the graph 203. The peak value 301 is a maximum/highest point of the graph 203. Accordingly, the grayscale value of the peak value 301 is identified as the peak position 302, peak_pos. The peak position 302 is provided by the formula (7),
peak_pos=jε{hist_j=max{hist_i}₀²ⁿ⁻¹}  (7)

[0065] where j represents a position number in terms of grayscale value of a histogram bin, hist_j represents the histogram bin's value at position j, max{hist_i}₀²ⁿ⁻¹ is a function for seeking the maximum (highest) value amongst hist₀ to hist_2n−1, hist_i represents the histogram bin's value at position i and n represent the number of bits of the image.

[0066] The Peak Analysis 103 computes the histogram average 303, hist_avg, of the histogram. The histogram average 303 is an average pixel population that is independent to the peak value 301 provided by formula (8),

[0067] $\begin{array}{cc}\mathrm{hist\_avg}=\frac{\sum _{i=0}^{2^n-1}{\mathrm{hist}}_i}{2^n}\mathrm{or}\frac{\mathrm{active\_window}\mathrm{\_area}}{2^n}& \left(8\right)\end{array}$

[0068] where active_window_area means the total number of pixels found within the active window of the image.

[0069] Then, the histogram average 303 intersects with the graph 203 in at least two points (projected on x-axis as points m and n) and the area of interest according to the present embodiment is a boundary defined by two intersections that includes the peak value 301, herein defined as the peak region 304. The peak region 304 has a width identified as the peak_width 305, peak width, which is provided by the formula (9),
peak_width=m−n+1  (9)

[0070] where m and n denote an upper bound and a lower bound grayscale value of the peak region 304 respectively.

[0071] Next, the peak density 306 is defined as an average pixel population within the peak region 304 provided by the formula (10),

[0072] $\begin{array}{cc}\mathrm{peak\_density}=\frac{\sum _{i=n}^m{\mathrm{hist}}_i}{\mathrm{peak\_width}}& \left(10\right)\end{array}$

[0073] where

[0074] $\sum _{i=n}^m{\mathrm{hist}}_i$
represents the total number of pixels within the region bounded by the upper bound m and the lower bound n, i.e. the peak region.

[0075] The Peak Analysis 103 then identifies a peak distribution (not shown), peak_distribution, defined by the sum of histogram value in a section of grayscale level relative to a width of the section, which is provided by the formulas (11)-(14),

[0076] $\begin{array}{cc}\mathrm{peak\_distribution}=\frac{\sum _{i⋐{\sec }_X}{\mathrm{hist}}_i}{\mathrm{sec\_width}},x\in \left\{\mathrm{low},\mathrm{upp}\right\}\mathrm{where},& \left(11\right)\\ {\sec }_{\mathrm{upp}}=\left[\mathrm{peak\_pos}+1,2^n-1\right]& \left(12\right)\\ {\sec }_{\mathrm{low}}=\left[0,\mathrm{peak\_pos}\right]& \left(13\right)\\ \mathrm{sec\_width}=\{\begin{array}{cc}{2}^n-\mathrm{peak\_pos}& {\sec }_{\mathrm{upp}}\\ \mathrm{peak\_pos}& {\sec }_{\mathrm{low}}\end{array}& \left(14\right)\end{array}$

[0077] where sec_upp represents a position of the histogram bins from the histogram bin next to the peak to the last ((2ⁿ−1)^th) histogram bins at the right of the peak, sec_low represents a position of the histogram bins from the first (0^th) bins to the peak.

[0078] Once the required data are obtained in the Peak Analysis 103, the Contrast Enhancement Process 100 proceeds to the Histogram Redistribution 104. It is observed that when enhancing an image contrast by applying histogram equalization to the entire histogram region, risk of unwanted distortion exists. Further, having a flatter histogram shape over the entire histogram, the noise may be enhanced in un-uniformly colored areas of the image. Thus, in accordance with this embodiment of the present invention, it is preferable that the histogram modification for contrast enhancement is performed in a selective area of the histogram. In accordance with one embodiment, histogram modification is performed within the peak region 304.

[0079] As illustrated in FIG. 4, the Histogram Redistribution 104 is performed on the histogram which was filtered by the smoothing filter during the Histogram Pre-Process 102. Based on the peak region 304 defined in the Peak Analysis 103, the Histogram Redistribution 104 distributes pixels of the histogram bins within the peak region 304 symmetrically away from the peak position 301. To pre-empt over-stretching of pixel values around the histogram bin which corresponds to the peak position 302, the Histogram Redistribution 104 limits the distributions to only the pixels which fall above the peak density 306 and are within the peak region 304.

[0080] FIG. 4 shows histogram bins that are bound within the peak region 304. The histogram in FIG. 4 is a mock example, which is not accorded to any scale nor it is based on any real histogram illustrated above. It is only for elaborating the pixel distribution in accordance with the present embodiment. The histogram 400 has a histogram bin 410 that locates at the peak position 302, i.e. the highest histogram bin. The histogram bin 410 and the parts thereof are filled with cross-lines-pattern. A histogram bin 420, positioned immediately to the left of the histogram bin 410, and the parts thereof are filled with upward-diagonal-lines-pattern. The next left histogram bin is a histogram bin 425 that is filled with upward-diagonal-dash-lines-pattern. Then all the subsequent bins 429 at the left are filled with dots-pattern. A histogram bin 430, positioned immediately to the right of the histogram bin 410, and the parts thereof are filled with downward-diagonal-lines-pattern. The next right histogram bin is a histogram bin 435 that is filled with downward-diagonal-dash-lines-pattern. Then all the subsequent bins 439 at the right are filled with dots-pattern. In this mock example, only histogram bins 410, 420, 425, 430 and 435 have a pixel amount above the peak density 306. The histogram bins 429 and 439 have pixel amount lower than the peak density 306. The histogram 401 shows a histogram, which is after the Histogram Re-Distribution 104. In histogram 400, the bins with dashed-outline are subject to re-distribution. Further, the histogram 400 provides lines, each start with a dot and end with an arrow, are shown to illustrate the bin (pixel) distributions, wherein the dots indicate where in which the bin are clipped off, and the arrows indicate the directions in which the clipped off bins are moved.

[0081] Still in FIG. 4, starting from the histogram bin 410, the pixel value exceeds the peak density 306 is clipped and divided into half. Half of the clipped value is added to the histogram bin 420. As the histogram bin 420 exceeds the peak density 306 as well, the excess portion of the bin 420 is clipped off and added to the histogram bin 425 together with the add-on from bin 410. The clipping and moving processes are repeated to the histogram bins 429 until the distribution on the left reaches the left boundary, n, of the peak region 304. In this way, pixel values within the peak region 304 in excess of the peak density 306 recursively overflows in the left direction until it is truncated. Likewise, the other half of the clipped value of the histogram bin 410 is added to the histogram bin 430. As the histogram bin 430 exceeds the peak density 306 as well, the excess portion of the bin 430 is clipped off and added to the histogram bin 435 together with the add-on from bin 410. The clipping and moving processes are repeated to the histogram bins 439 until the distribution on the right reaches the right boundary, m, of the peak region 304 where excess overflows are truncated to the right of the peak position 302.

[0082] Still in the FIG. 4, as the Histogram Redistribution 104 occurs only within the peak region 304, unlike the common histogram distribution/modification known in the art, which flattens the mid level grayscale entirely, the contrast enhancement 100 according to the embodiment preserves a peak region to provide an overall natural output video.

[0083] The contrast enhancement process 100 provides gains in the adaptive gain computation 105 for controlling level/strength of the contrast enhancement. Instead of a constant contrast gain, the present embodiment provides an adaptive contrast gain, adaptive_gain, to prevent occurrence of over-contrasted video artifacts during histogram equalization, which may result in undesired contouring effects, and lacking of graduation. This is particularly useful for low contrast images/videos represented by fewer pixel levels to maintain a satisfactory naturalness of a processed image/video frame. The adaptive gain is computed based on a peak density factor (peak_density_factor), a peak position factor (peak_pos_factor) and a peak distribution factor (peak_dist_factor), which is provided in formula (15)
adaptive_gain=user_gain×peak_density_factor×peak_pos_factor×peak_dist_factor  (15)

[0084] where user_gain is a user defined gain.

[0085] FIG. 5 illustrates the characteristic of the peak density factor which is provided by the formula (16). The X-axis represents a division of peak density by histogram average, and the Y-axis represents an amplified peak density factor by a factor of 16. The constants k₅ and k₆ are both set at 2, for example.

[0086] $\begin{array}{cc}\mathrm{peak\_density}\mathrm{\_factor}=\{\begin{array}{cc}{k}_5\times \left(\left(\mathrm{peak\_density}/\mathrm{hist\_avg}\right)-1\right)& \mathrm{peak\_denisty}/\mathrm{hist\_avg}<1.5\\ 1& 1.5\le \mathrm{peak\_density}/\mathrm{hist\_avg}\le 2\\ k_6\times \mathrm{hist\_avg}/\mathrm{peak\_density}& \mathrm{peak\_density}/\mathrm{hist\_avg}>2\end{array}& \left(16\right)\end{array}$

[0087] where k₅ and k₆ are constants which may be defined by user.

[0088] As shown in FIG. 5, the peak density factor begins with graph section 503, which provides a linear raise graph that intersects with the X-axis at 1. Following that, a graph section 502 provides a unity peak density factor for a range of peak density between 1.5 and 2 times the histogram average. When the peak density grows more than two times the histogram average, the peak density factor have a inverse relationship with the peak density as shown in graph section 501. Back to the graph section 503, it provides control of a gradual transition of contrast strength between images that have peaks, which require contrast enhancement, and those already have good contrast, which require little contrast enhancement. The graph section 502 provides saturation, which is suitable for videos that require normal contrast enhancement. The graph section 501 provides a non-linear contrast gain adapted for peak density as the factor between the peak density and histogram average grows. As shown in FIG. 5, it is understood that range for graph section 501, 502 and 503 is for illustration only, not limitation. The aim is to provide a non-linear contrast enhancement based on the peak density and histogram average of the processing image. Depending on the quality desired, the peak density factor may provide a unity factor (normal contrast enhancement) at a range other than between 1.5 to 2 times the histogram average and less contrast enhancement for images having a ratio of peak density to histogram average falls outside the range designated for unity density factor.

[0089] Inclusion of the peak position factor for computing the adaptive gain is to prevent over-spreading of image histogram towards one end when the peak is detected at the other end. It is understood that human visual perception is a non-linear function of image brightness and the human eye is more sensitive to middle range of brightness. Thus, the present embodiment imitates the visual sensitivity by providing higher gains to peak intensities along the center of the pixel range (mid-tone), relatively lower gain for peaks towards the darker range such that sensitive details in relatively brighter regions are not over-stretched and very low gains for peaks towards the brighter ends so that more sensitive details in relatively darker regions are not stretched excessively.

[0090] To cater to the above need, it is provided that the peak position factor having characteristics as shown in FIG. 6. As shown, the graph section 602 provides saturation for peak position between quarter and mid brightness level (64<peak_pos<128), and both graph section 601 and graph section 603 provide a linear downward incline of the peak position factor towards the brighter end and the darker end respectively. The peak position factor is provided by formula (17),

[0091] $\begin{array}{cc}\mathrm{peak\_position}\mathrm{\_factor}=\{\begin{array}{cc}\left(2^n-\mathrm{peak\_pos}\right)/2^{n-1}& \mathrm{peak\_pos}>128\\ 1& 64\le \mathrm{peak\_pos}\le 128\\ \left(\mathrm{peak\_pos}\right)/2^{n-1}+0.5& \mathrm{peak\_pos}<64\end{array}& \left(17\right)\end{array}$

[0092] where n is the number of bit of the image. The aim for formula (17) is to provide non-unity peak position factor, depending on the pixel brightness (grayscale) within the same image. It is understood that, subject to personal preference, mid-tone range which is more eye pleasing may vary. The mid-tone rage (64<peak_pos<128) selected in FIG. 6 is for illustration only, not limitation and this range may be altered in accordance with another embodiment of the present invention.

[0093] Unlike adaptive gains in prior arts, the present embodiment does not limit the gain such that over-contrasted picture cannot be inverse contrasted. The peak position factor when applies in contrast mode, defined as the overall expansion of gray level i.e. a transfer curve is above y=x curve for lower section (left of peak) and transfer curve is below y=x for upper section (right of peak), and vice versa when the peak position factor is applied in inverse contrast mode. Accordingly, one way of finding out whether the peak position factor is applied in contrast mode or in inverse contrast mode is to calculate the difference, dev, of the areas under the transfer curve and the y=x curve, where dev is provided in formula (18).

[0094] $\begin{array}{cc}\mathrm{dev}=\sum _{i=p}^q\left(\sum _{j=p}^i{\mathrm{hist}}_j-\frac{i-p}{q-p}\times \sum _{j=p}^q{\mathrm{hist}}_j\right)& \left(18\right)\end{array}$

[0095] where,

[0096] $\sum _{j=p}^i{\mathrm{hist}}_j$
refers to the transfer curve value at any point i within [p,q];

[0097] $\frac{i-p}{q-p}\times \sum _{j=p}^q{\mathrm{hist}}_j$
refers to an identity transfer curve value, whereby the identified transfer curve is an y=x function; and

[0098] $\sum _{j=p}^q{\mathrm{hist}}_j$
refers to the total area under the transfer curve.

[0099] As a result, in contrast mode,

[0100] $\left[p,q\right]=\{\begin{array}{cc}\left[0,\mathrm{peak\_pos}\right]& \mathrm{dev}<0\\ \left[\mathrm{peak\_pos}+1,2^n-1\right]& \mathrm{dev}>0;\end{array}\mathrm{and}$

[0101] when in inverse contrast mode,

[0102] $\left[p,q\right]=\{\begin{array}{cc}\left[0,\mathrm{peak\_pos}\right]& \mathrm{dev}\ge 0\\ \left[\mathrm{peak\_pos}+1,2^n-1\right]& \mathrm{dev}\le 0\end{array}$

[0103] During pixel distributions, the pixels are randomly distributed on an image. However, such random distribution may result in unsightly contrasted patches on the image, especially when the pixels are distributed in the image as texture details. Thus, the peak distribution factor is provided to prevent excessive spreading among the histogram when the number of pixels in the distribution section is insignificant compare to the histogram average. The peak distribution factor is provided by formula (19),

[0104] $\begin{array}{cc}\mathrm{peak\_dist}\mathrm{\_factor}=\{\begin{array}{cc}1& \mathrm{peak\_dist}\ge \mathrm{hist\_avg}\\ 1-0.5\times \mathrm{hist\_avg}/\mathrm{peak\_dist}& 3\times \mathrm{hist\_avg}<3\times \mathrm{peak\_dist}<\mathrm{hist\_avg}\\ 0& 3\times \mathrm{peak\_dist}\le \mathrm{hist\_avg}\end{array}& \left(19\right)\end{array}$

[0105] FIG. 7 illustrates the relationship between the peak distribution factor and the peak distribution. The X-axis represents the division of histogram average by peak distribution or how large histogram average is a compared to peak distribution. The Y-axis represents the amplified peak distribution factor with a factor of 16. Graph section 701 shows unity saturation of peak distribution factor, preferably, when histogram average is equal or smaller than peak distribution. Graph section 702 shows that peak distribution factor decreases linearly as peak distribution decreases relative to histogram average. Graph section 703 shows zero saturation of peak distribution factor, preferably, when the histogram is at least 3 times larger than the peak distribution. At zero saturation, no contrast enhancement is performed, for example, for images that do not require any contrast enhancement.

[0106] When the adaptive gain is generated, the Transfer Curve Generation 106 generates transfer curves as shown in FIG. 8. The X-axis represents the grayscale of an input image and the Y-axis represents the grayscale of an output image, generally known as tone map. The tone map is commonly used in graphic editing tools for adjusting brightness and contrast. For this document, the explanation for brightness adjustment is disregarded.

[0107] Referring to FIG. 8, for a pre-process image where there is no change in the grayscale, the input image is equal to the output image, as represented by the graph Y=X 801. To adjust the contrast of the input image, the Y=X graph 801 needs to be transformed into an “S” curve or an inverse “S” curve. To achieve that, a pivot point 810 is defined along Y=X graph 801. The upper portion and lower portion of the Y=X graph 801, also known as cumulative density functions, are bent in two opposing directions about the pivot point 810, while the two ends of the Y=X graph 801 remains at Y=X. Effectively, the mid-tone range of the output image is changed which altered the contrast of the image. As the present embodiment targets to increase the contrast, an “S” curve is formed as transfer curve 802 and 803.

[0108] In FIG. 8, according to the present embodiment, the pivot point 810, x_p, is set as the peak position and the transfer curve 803 is provided by the formula (20),

[0109] $\begin{array}{cc}{y}^{\prime }=\{\begin{array}{cc}\frac{c\left(x\right)}{c\left(x_p\right)}{x}_p& x\le x_p\\ x_p+\left(2^n-1-x_p\right)\frac{c\left(x\right)-c\left(x_p\right)}{1-c\left(x_p\right)}& x>x_p\end{array}\mathrm{curTC}=x+\mathrm{adaptive\_gain}\times \left(y^{\prime }-x\right)& \left(20\right)\end{array}$

[0110] where curTC and x are the input and output gray-level value of pixel, x_p is the peak_pos and c(x) and c(x_p) are the cumulative function of x.

[0111] Still in FIG. 8, the transfer curve 803 is a transfer curve with two independent cumulative density functions in the lower and upper sections pivoted about the pivot point 810 to form an “S” curve. Each cumulative density function is then normalized by the section width and the peak position is mapped onto itself. The output transfer curve, curTC is then scaled linearly by the adaptive gain computed earlier.

[0112] Humans generally perceive total black in very dark areas and total white in very bright areas as visually more pleasing. In an alternative embodiment, black and white boost are added to the transfer curve 803 to form the transfer curve 802 such that dark areas are brought closer to black and bright areas are brought closer to white. The operation of the black boost 805 to the transfer curve 803 is such that the transfer curve in the lower section, fixed to the pivot point 810, is stretched vertically and clipped at the black level, resulting in a plateau at the lowest end of the pixel range. Similarly, the operation of the white boost is such that the transfer curve in the upper section, fixed to the peak position, is stretched vertically and clipped at the white level, resulting in a plateau at the highest end of the pixel range. The transfer curve 802, y″, is provided by the formula (22)-(24),

[0113] $\begin{array}{cc}\mathrm{curTC}=x+\mathrm{adaptive\_gain}\times \left(y^{″}-x\right)& \left(21\right)\\ y^{″}=\{\begin{array}{cc}\frac{c\left(x\right)}{c\left(x_p\right)}\left(x_p+\mathrm{bbst}\right)-\mathrm{bbst}& x\le x_p\\ x_p+\left(x_{2^n-1}-x_p+\mathrm{wbst}\right)\frac{c\left(x\right)-c\left(x_p\right)}{1-c\left(x_p\right)}& x>x_p\end{array}& \left(22\right)\\ \mathrm{bbst}=\frac{\mathrm{low\_sec}{\mathrm{\_width}}^2}{k_b}& \left(23\right)\\ \mathrm{wbst}=\frac{\mathrm{upp\_sec}{\mathrm{\_width}}^2}{k_w}& \left(24\right)\end{array}$

[0114] where, k_b and k_w are user defined constants for black and white boost respectively, upp_sec_width is a count of the grayscale level from the peak to (2ⁿ−1)^th grayscale level, lower_sec_width is a count of the grayscale level from the 0^th grayscale to the peak.

[0115] In the above formulas, the strengths of the black boost, bbst, and white boost, wbst, are adapted to the width of the lower and upper section with user adjustable factors k_b and k_w. An example parameter can be set as k_b=512 and k_w=2048. Typically, k_w is larger or equal to k_b such that the effect of white boost is less than that of black boost as darker saturated areas are more visually pleasing. The black and white boosts are integrated in the transfer curve generation and the output transfer curve is clipped within the defined pixel range.

[0116] The Transfer Curve Post-Process 108 processes the transfer curve with a weighted temporal filter that provides temporal stability in the contrast enhancement of video. The current transfer curve is compared with the previous transfer curve to compute SADTC for a defined pixel range, where the SADTC is provided in formula (25),

[0117] $\begin{array}{cc}\mathrm{SADTC}=\sum _{i=0}^{2^n-1}\mathrm{abs}\left({\mathrm{curTC}}_i-{\mathrm{prevTC}}_i\right)& \left(25\right)\end{array}$

[0118] where curTC_i is the output value of the current transfer curve with input i; and prevTC_i is the output value of the previous transfer curve with input i.

[0119] With the SADTC, the current transfer curve is subsequently filtered with a SAD weighted temporal recursive filter. As the differences between the transfer curves increase i.e. higher SADTC, the weight of the previous transfer curve is increased relative to the current transfer curve such that the transient change in transfer curve is smooth and impulse changes like flicker are avoided.

[0120] A transfer curve weight, wgtTC, is directly proportional to the SADTC and is adjustable by a gain factor, k₂, for a fixed transfer curve range. k₂ may be decreased for increased filter sensitivity to change and example value is k₂= 1/16. In addition, the possibility of scene change is considered in filter weights to reduce sharply the weight or remove the influence of the previous transfer curve when a scene change occurs. Scene change can be characterized by a stark difference in picture histogram. The calculated weight for the histogram filter wgtHist before clipping to range [0,1] in the Histogram Pre-Process 102 can be used as indication for scene change. The scene weight is a scaled and offset version of wgtHist'. Example values are k₃=256 and k₄=⅛. The final transfer curve filter weight is a clipped difference of transfer curve weight, wgtTC, and scene weight, wgtScene, as provided in the formulas (26)-(29)
wgtTC=SADTC/k₂, 0≦wgtTC≦1  (26)
wgtScene=wgtHist′/k₃−k₄  (27)
wgtTC=wgtTC−wgtScene, 0≦wgtTC≦1  (28)
curTC_i=wgtTC×prevTC_i+(1−wgtTC)×curTC_i  (29)

[0121] where the prevTC refers to the transfer curve right before the change of scene and curTC refers to the transfer curve right after the change of scene, and k₂-k₄ are constants.

[0122] In accordance with an alternative embodiment, further temporal stability and robustness can be achieved with gain control between consecutive fields. An exemplary embodiment is the limitation of the change in final adaptive gains for each section to be within ± 1/64 of the maximum gain of the contrast enhancement system such that the adaptive gains do not create sudden change in the enhanced video as a result of the analysis of its content.

[0123] In accordance with an alternative embodiment, the transfer curve may preferably be implemented in the form of a look-up table. For memory-saving implementations of transfer curve that has range or resolution less than the pixel range, the pixel mapping function may preferably extract the neighboring table entries and perform a linear interpolation to compute the output pixel value.

[0124] In the above description, the term video, video frame, and video image are used. It is understood to those skilled persons that embodiments of the present invention may be applied to different images sources, including broadcasted videos, video CDs, DVDs, or still images, to improve the contrast quality with user-defined contrast strength before displaying or encoding.

[0125] Referring now to FIG. 9, there is provided a schematic block diagram of a TV display processor 900 having the contrast enhancement process 100 embedded therein. The TV display processor 900 as described herein is adapted to receive analog signals, i.e. broadcasted signals from antenna, and digital signals, i.e. video signals from digital video players.

[0126] Also as in FIG. 9, when the TV display processor 900 received broadcasted analog signals, it converts the analog signals to digital composite signals via Analog to Digital Converter 901. The digital composite signals are then decoded into component YUV (Luminance-Chrominance) signals (one luminance and two chrominance components) in the Composite Decoder 902. These components may be processed by Analog Noise Reduction 903 to remove undesired Gaussian and impulse noise, which are introduced during transmission and conversion processes. Temporal and spatial noise filters may be used for Analog Noise Reduction 903. Likewise for digital input signals, often in the form of compressed bit streams for example MPEG, are decoded into component YUV signals through Video Decoder 904, for example MPEG decoder. The uncompressed video signals are further processed by Coding Noise Reduction 905 to eliminate artifacts resulting from coding processes using de-blocking and de-ringing filters. The noise reduced video signals are then converted to progressive signals in an Interlace to Progressive Conversion 906. The progressive signals are then scaled, cropped and zoomed to an appropriate size and format via the Image Size Control 907. The processed video signals are enhanced for better subjective effects with an Image Enhancement Circuit 912 prior to output.

[0127] In FIG. 9, the Image Enhancement Circuit 912 includes Contrast Enhancement 908, Transient Improvement 909 and Contrast, Saturation and Brightness Adjustment 910. Briefly, the Contrast Enhancement 908 performs the contrast enhancement processing 100 of FIG. 1, for increasing the video contrast. The Transient Improvement 909 improves the video edge details. The Contrast, Saturation & Brightness Adjustment 910 adjusts a dynamic range and saturation of the video according to user preferences. Subsequently, the enhanced video signals are converted to an appropriate color format, using Color Space Conversion 911 for outputting. The color format is dependent on the type of output device used for displaying, for example, RGB signal format is used for cathode ray tube (CRT) display.

[0128] According to an embodiment of the present invention, the contrast enhancement process 100 may be applied in any formats of videos or still images. FIG. 10 illustrates a schematic block diagram of contrast enhancement of the component YUV signals in accordance with one embodiment of the present invention, which occurs in the Contrast Enhancement 908 of the Image Contrast Enhancement 912 of FIG. 9. The luminance signal Y_in of the component YUV signals is processed by a Luminance Contrast Enhancement 1001 to provide an enhanced output signal Y_out. The Luminance Contrast Enhancement 1001 performs contrast enhancement based on the operations of the contrast enhancement process 100. The chrominance signals Cr_in and Cb_in of the component YUV signals are processed by a Chrominance Saturation Compensation 1002 in dependence of the change in luminance signals Y_in and Y_out to give compensated chrominance signals Cr_in and Cb_out. A known method of Chrominance Saturation Compensation may be found in US patent publication 2005/007497.

[0129] Still in FIG. 10, in conjunction with FIG. 1, the Histogram Calculation 101 generates a histogram for the luminance signal Y_in of the component YUV signals. As mentioned earlier, it is preferred that the histogram is in a discrete number of bins less than the full pixel level range. For example, the Histogram Calculator 101 provides a histogram with 32 bins for an 8-bits luminance signal, each bin includes a fixed bin width of 8 consecutive grayscales. It is also understood that, in accordance with a further alternative embodiment, the bin width of each bin may include varying number of consecutive grayscales, preferably, less bin width for mid-tone bins and gradually wider bin width for brighter and darker tone bins. As mentioned, the histogram of the 8-bits luminance signal will be expanded to full size pixel range (i.e. one grayscale per bin) during the Histogram Pre-processing 102, as preferred by the Contrast Enhancement Process 100.

[0130] In accordance with an embodiment of the present invention, it should preferably, but not limited, be applied on a relatively noise-free video or a video that has subjected to noise reduction to prevent undesirable effects of noise amplification. Contrast Enhancement Process 100 may be applied in conjunction with detail and color enhancement techniques to provide a full image enhancement solution.

[0131] The industrial fields of application include video-related consumer products like television, display sets, video CD player, DVD players and set-top-boxes and software products with image processing, video editing functions.

[0132] While this detailed description has set forth some embodiments of the present invention, the appended claims are sufficiently supported to cover and will cover other embodiments of the present invention which differ from the described embodiments according to various modifications and improvements apparent to those skilled in the art. For example, the contrast enhancement process may identify a plurality of peaks from the histogram for re-distributing the histogram. In addition, the contrast enhancement process may provide a fully automated adaptive gain computation either by excluding the user gain as one of the factor, or by pre-setting the user gain as 1.