Field of the Invention
This invention relates generally to an improved digital electronic image processing circuitry for sharpening an electronically recorded image and, in particular, to an improved digital image processing circuitry for sharpening edges and details of a digitally recorded image without amplifying noise in the input recorded image.
Background of the Invention
The human visual system perceives the high spatial frequencies of an image as sharpness and the low spatial frequencies as tonescale. That is, sharpens of an image is generally a function of how well the edges of objects in an image are perceived whereas the tonescale is a function of how the image's color and its gradual variation over the extent of the image are perceived. Since the human visual system perceives sharpened images much more favorably than unsharpened images, most electronic imaging systems which process recorded images attempt to provide sharpened images for human viewing.
In electronic imaging cameras, charge coupled devices (hereafter referred to as "CCDs") capture the image and introduce noise (hereafter referred to as "CCD noise") into the system. Typically, color processing circuitry then receives the recorded image with the inherent CCD noise and enhances and/or corrects the colors found in the recorded image. The color processing circuitry may transform the recorded image into a new tonescale and color coordinate system which is more suitable for processing by downstream circuitry and for eventual viewing. Unfortunately, the color processing circuitry also greatly amplifies the CCD noise.
A serially connected sharpening circuit then typically receives the color corrected recorded image in order to sharpen it. The sharpening circuit may receive the luminance components of the recorded image and sharpening algorithm. An example of such a mathematical sharpening algorithm is disclosed and explained in U.S. Pat. No. 4,941,190, entitled "Method and System for Enhancement of a Digitized Image", by Terrence H. Joyce which is incorporated herein by reference. Although different types of sharpening circuits attempt to differentiate between those high spatial frequency components attributable to noise and those attributable image information, such differentiation cannot be perfect and some portion of the high spatial frequency components of the noise is inevitably amplified. Thus, the sharpening as discussed above is always achieved at the expense of some increase in noise visibility. Therefore, the usefulness of sharpening circuitry is restricted by the amount of noise which becomes noticeably amplified after the recorded image is propagated therethrough.
As a result, a need exists in the art for an improved digital image processing circuit which does both color processing and image sharpening in a manner whereby the deleterious effects associated with the amplification of noise is minimized.
Summary of the Invention
Embodiments of the present invention solve the above-identified problem by enhancing the edges and the details of selected elements of a recorded image comprising of a plurality of pixel values received directly from a two-dimensional photosensitive array while color processing in parallel other selected elements of the recorded image. In particular, one embodiment of the present invention operates to selectively combine color separation elements of the recorded image which have not been processed by the color processing circuitry and adaptively sharpen these elements to provide high spatial frequency combined electronic image data signals. These high spatial frequency combined electronic image data signals are added together with low spatial frequency luminance signals which are parallel processed by the color processing circuitry to provide combined luminance signals of the recorded image. As is obvious, the high spatial frequency combined electronic image data signals avoid being processed by the color processing circuitry and thus also avoid having the CCD noise associated with these signals amplified. Accordingly, this approach provides color processed and sharpened recorded images with little inherent CCD noise and a high subjective quality.
Specifically, sharpening circuitry and color processing circuitry both receive input electronic image data signals from the CCDs. These input electronic image data signals represent, on a pixel by pixel basis, the image sensed in a red-green-blue color coordinate system. The color processing circuitry, which can be either one 3-D look up table (hereafter referred to as a "LUT") or a LUT-matrix-LUT-matrix combination, color corrects these input electronic image data signals. In the preferred embodiment which employs a LUT-matrix-LUT-matrix combination, a first LUT transforms the input electronic image data signals into pseudo linear space, expands the tone rendition scale of these signals, and performs tonescale transformations on these signals. A first matrix transforms these electronic image data signals into the industry standard SMPTE C, NTSC, or any other color coordinate system. A second LUT compresses the tone rendition scale of these signals back to their original size and then performs another tonescale transformation on these signals. Finally, a second matrix transforms these electronic image data signals to a luminance-chrominance-chrominance color coordinate system and low pass filtering circuitry then passes the luminance components of the electronic image data signals below a first spatial frequency and the chrominance components of the electronic image data signals below a second spatial frequency.
While the color processing circuitry is processing these input electronic image data signals, the sharpness circuitry is also processing in parallel the same input electronic image data signals. Specifically, the sharpness circuitry combines linearly, non linearly, or by weighted average, the input electronic image data signals. The combined electronic image data signals are then provided to both a variance circuit and a high pass filtering circuit. The variance circuit provides variance signals to a third LUT which, in turn, provides an adaptive amplification factor to a first combining means which combines the adaptive amplification factor with the high pass filtered electronic image data signals. The combination of the variance circuit, the third LUT, and the summation means, thus, adaptively sharpen, on a pixel by pixel basis, the high pass filtered electronic image data signals. The output from the first combining means discussed above are adaptively amplified high spatial frequency composite electronic image data signals. These signals are adjusted by a fourth LUT and then combined via a third combining means with the low spatial frequency luminance signals provided by the low pass filtering circuit discussed above in conjunction with the color processing circuitry to provide much sharper and crisper combined luminance signals.
Description of the Drawings
The novel features that are considered characteristic of the invention are set forth with particularity in the appended claims. The invention itself, however, both as to its organization and its method of operation together with other objects and advantages thereof will be best understood from the following description of the illustrated embodiment when read in connection with the accompanying drawings wherein:
FIG. 1 is block diagram representation of an improved digital image processing circuit according to this invention.
Description of the Preferred Embodiments
Electronic imaging cameras for recording either motion or still images are in common usage today. Such cameras generally include, as is shown in the sole figure, a two-dimensional photosensitive array 2 which may comprise a high resolution charge coupled device (CCD), charge injection device (CID), or other sensors which receives the image scene light in a well-known manner by way of an objective lens and shutter (not shown). The image sensing array typically comprises a plurality of image sensing elements or pixels arranged in a two-dimensional array with each image sensing pixel converting the image defining light reflectants from the scene into a corresponding analog voltage value. The sampling is done sequentially for the three primary colors red, green, and blue (hereafter referred to as "RGB colors") and the image sensing elements are preferably arranged in a plurality of columns and rows. For imaging applications, the resolution of the electro-optically sampled image usually comprises about 1656 image points or pixels per line (horizontal) and 600 lines (vertical). Accordingly, each image has in the aggregate 1656.times.600 pixels wherein each pixel is assigned one of the RGB colors.
When an image is captured by an electronic imaging camera, a steady stream of analog voltage values associated with the pixel values for a given row of the image sensing elements are presented to a video processing circuit 4. The video processing circuit 4 compresses the analog voltage values that are associated with the highlights and shadows of the recorded image while emphasizing the analog voltage values that are associated with the midtones of the recorded image. In essence, the video processing circuit 4 compresses the tone of the analog voltages by using a tone compression curve having a gamma of about 0.4.
An analog-to-digital converter 6 then transforms, row by row, the analog voltages into a plurality of digital electronic image data signals representing the recorded image in a RGB color coordinate system. A median filter interpolation circuit 8 then receives the digital electronic image data signals and interpolates for non sampled RGB colors in a manner that substantially reduces color fringing as is fully described in U.S. Pat. No. 4,724,395 entitled "Median Filter Reconstruction Missing Color Samples" by William T. Freeman and which is now fully incorporated by reference herein. The median filter interpolation circuit 8 then presents the digital electronic image data signals to color processing circuitry 10.
The color processing circuitry 10 can be either one large 3-D LUT (not shown) or a LUT-matrix-LUT-matrix combination and both achieve the same object, namely, to transform the electronic image data signals into a color coordinate system which is suitable for viewing. Referring to the sole figure, a LUT-matrix-LUT-matrix is shown as the color processing circuitry 10. A first LUT 12 expands the tone rendition scale of the electronic image data signals into a pseudo linear space by expanding the output from 8 bits to 12 bits. This linearization of the electronic image data signals is accomplished through the use of an inverse gamma curve that has a gamma of approximately 2.2. The LUT 12 also performs tonescale transformation on these signals. A first matrix 14 then transforms the electronic image data signals into either the industry standard SMPTE C, NTSC, or any other color coordinate system. A second LUT 16 compresses the tone rendition scale of the electronic image input signals back to 8 bits by employing a gamma correction curve having a gamma of approximately 2.2. This second LUT 16 also performs tonescale transformation on these signals.
A second matrix 18 transforms these electronic image data signals in an RGB color coordinate system to a luminance-chrominance-chrominance color coordinate scale (hereafter referred to as "YUV color coordinate system"). It should be noted that several different luminance-chrominance-chrominance color coordinate systems exists and may be employed by the second matrix 18, for example, YUV or YIQ. However, for brevity, only the YUV color coordinate system transformation will be discussed. The transformation into the YUV color coordinate system is accomplished very simply according to the following equations:
Y=0.299R+0.587G+0.114B
U=B-Y
V=R-Y
The electronic image data signals in the YUV color coordinate system are next low pass filtered by way of low pass filter 20. Specifically, the luminance components, i.e., the Y signals, are low pass filtered at a first spatial frequency while the pair of chrominance components, i.e., the U and V signals, are low pass filtered at a second spatial frequency. It should also be noted that each individual chrominance component of the electronic image data signals can also be low pass filtered at different spatial frequencies. The output low spatial frequency luminance signal is combined via first combining means 22 with a high spatial frequency combined electronic image data signal from sharpness circuitry 24 discussed below.
While the color processing circuitry 10 processes the electronic image data signals, sharpness circuitry 24 also processes in parallel these same electronic image data signals from the median filter interpolation circuit 8. Specifically, a summation circuit 26 combines either linearly or non linearly or by a weighted average, these electronic image data signals. These combined electronic image data signals are provided to both low pass filtering circuit 28 and variance circuit 30. The combination of the low pass filtering circuit 30 and a second combining means 32 effectively high pass filter the combined electronic image data signal which is then provided to a third combining means 34.
The variance circuit 30 provides variance signals in a well known manner to a third LUT 36. The variance signals are indicative of the statistical variance of the composite electronic image data signals for a select group of pixels surrounding each pixel to be enhanced by the sharpness circuitry 24. The third LUT 36 receives the variance signals and thereafter generates adaptive amplification factors which are provided to the second summation means 34. The adaptive amplification factors vary as a function of different values of variance signals so as to distinguish between signal noise and signal information in the composite electronic image data signals. A detailed description of the variance circuit 30 and the relationship of the adaptive amplification factors to the variance signals is disclosed and explained in U.S. patent application Ser. No. 351,930, entitled "Method for Adaptively Sharpening Electronic Images", by Woo-Jin Song, filed on May 15, 1989, which is hereby incorporated by reference into this application. The combination of the variance circuit 30, the third LUT 36, and the third combining means 34 adaptively sharpen on a pixel by pixel basis the high pass filtered composite electronic image data signals provided by the second combining means 32.
A fourth look up table 38 adjusts by a precompensation factor the adaptively sharpened high spatial frequency composite electronic image data signals. Finally, the high spatial frequency composite electronic image data signals are provided to the first combining means 22 and combined with the low spatial frequency luminance signals from the color processing circuitry 10. This provides much sharper and crisper combined luminance signals and hence electronic image data signals in which the amplification of high spatial frequency noise is minimized.
Other embodiments of the invention including additions, subtractions, deletions, and other modifications of the preferred disclosed embodiments of the invention will be obvious to those skilled in the art and are within the scope of the following claims.