BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to electronic image processing and, more particularly, to a system for improving the quality of the output of full-color images reproduced on monitors that display a comparatively small number of colors, or printers that print a comparatively small numbers of colors, using halftoning and error diffusion techniques wherein the quantization error of one color of a given pixel is used to modify the quantization of the other colors of that pixel.
2. Prior Art
In full-color images, each pixel is typically represented as one of 16 million colors. However, in conventional electronic displays or computer monitors, such as those used for the IBM PS/2 with VGA, only 256 colors can be displayed. It is therefore a problem to obtain a top-quality display of full-color images on a monitor that can display only a small number of colors. Similarly, conventional printers are limited in the number of colors of a full-color image that can be reproduced. The present invention is directed to overcoming this color reproduction problem utilizing halftoning and error diffusion techniques.
Halftoning
Halftoning, as the term is used herein, refers to electronic processing that accepts an input image for which each pixel can take on any one of a large number of shades of gray, or shades of colors, and produces an output image for which each pixel can take on any one of a smaller number of shades of gray, or shades of colors. With the proper choice of the output-pixel values, the output image, when viewed at a distance, may be made to appear to have a full complement of shades. For a good overview of halftoning techniques, reference may be had to the text by R. Ulichney, Digital Halftoning, Cambridge, Mass., MIT Press, 1987.
Error Diffusion
Error diffusion generally is a well known technique for halftoning images. Its first use is usually credited to Robert Floyd and Louis Steinberg, as set forth in "An Adaptive Algorithm for Spatial Gray Scale," 1975 SID International Symposium, Digest of Technical Papers, pp. 36-37. Many variations of error diffusion are also discussed in R. Ulichney's above-cited text, Digital Halftoning.
Error Diffusion Processing Steps
Error diffusion begins with the diffused errors, e.sub.i,j, at each pixel position equal to zero. Mathematically, this is described as: e.sub.i,j =0 i,j. Then the input pixels are processed in order, following the processing of a selected pixel at an initial or first pixel position, against which the diffused errors are determined, with the processing steps for each remaining pixel being:
1. The modified-pixel value, mp.sub.i,j, is computed as the sum of the input-pixel value, ip.sub.i,j, and the value of the diffused error at that pixel position. Mathematically, this is described as:
2. The output-pixel value, op.sub.i,j, is selected as one of the possible output values, q.sub.t, that is near the modified pixel value. Mathematically, this is described as:
where Q(x) selects one of the available output values, q.sub.t, near x.
3. The quantization-error, .delta..sub.i,j, is computed as the difference between the modified-pixel value and the output-pixel value. Mathematically, this is described as:
4. At pixel positions not yet processed, the diffused errors are incremented by amounts that are proportional to the quantization-error at the pixel position. Mathematically, this is described as:
subject to .SIGMA.c.sub.r,s =.gamma..
Many of the differences in the various error diffusion techniques are variations in the choices made for the diffusion coefficients, c.sub.r,s, used in step 4. For instance, in the above-cited Floyd and Steinberg article the coefficients are constants, while an example of the use of random variable coefficients is found in U.S. Pat. No. 4,654,721, issued Mar. 31, 1987, to Gerald Goertzel and Gerhard R. Thompson and entitled, "System for Reproducing Multi-Level Digital Images on a Bi-Level Printer of Fixed Dot Size".
Error Diffusion Applied to Three Color Planes Independently
Error diffusion is particularly suitable for application to the situation where the display palette is orthogonal in red, green, and blue. By an orthogonal palette in this situation is meant that:
if the red value .gamma. is in the display palette,
if the green value g is in the display palette, and
if the blue value b is in the display palette,
then the color triplet r,g,b is also in the display palette.
A color input image may be treated as three input images;
where
ip.sub.i,j.sup.c1 is the input image for color 1,
ip.sub.i,j.sup.c2 is the input image for color 2, and
ip.sub.i,j.sup.c3 is the input image for color 3.
Similarly, a color output image may be considered as three output images; where
op.sub.i,j.sup.c1 is the output image for color 1,
op.sub.i,j.sup.c2 is the output image for color 2, and
op.sub.i,j.sup.c3 is the output image for color 3.
With an orthogonal palette, the three input images that constitute a color input image may be processed independently, using error diffusion, to produce three output images which are then combined to constitute a color output image. By this is meant that:
ip.sub.i,j.sup.c1 is processed by the steps described above in Error Diffusion Processing Steps to produce op.sub.i,j.sup.c1.
ip.sub.i,j.sup.c2 is processed by the steps described above in Error Diffusion Processing Steps to produce op.sub.i,j.sup.c2.
ip.sub.i,j.sup.c3 is processed by the steps described above in Error Diffusion Processing Steps to produce op.sub.i,j.sup.c3.
This processing can produce good quality color images with accurate color and has been demonstrated and used. However, artifacts occur in the output image, the main one being the amount of visually-apparent texture in the image reproduction. The present invention is directed to improving on this processing by minimizing this main artifact.
SUMMARY OF THE INVENTION
The invention utilizes what may be called coupled-color error diffusion processing on input signals representing a full-color image to produce an output image with visually improved color reproduction and is particularly suitable for use with an orthogonal color palette. More specifically, signals representative of the colors of the pixels of a full-color image are input to a processing device, such as a personal computer, to ultimately produce a representation of the image on an output device, such as a display on a monitor or a printout from a printer, that is only able to produce a small number of colors in comparison to the number of colors represented by the input signals. To process the input signals in such a manner as to obtain an output in a form that best reproduces or represents the full-color image to a viewer of the output product, i.e., the image on the monitor or the image printed out by the printer, error diffusion is applied to the input signals using an improved technique as follows. An initial pixel position is selected which acts as the basis against which the diffused errors are measured. Then, beginning with the diffused errors for each color (e.g., three colors) and at each pixel position equal to zero, that is, e.sub.i,j.sup.ck =0 i,j for ck=c1,c2,c3, the input pixel signals of a color image are processed in order by the following steps:
a modified-pixel value for color 1 is computed for the pixel at a first or given position (immediately following the initial pixel position) as the color-1-input-pixel value+the value of the color-1-diffused-error, at that pixel position,
the color-1-output-pixel value at the given position is selected from among those possible near the modified-pixel value for color 1, and the color 1 quantization error is computed as the difference between the color-1-modified-pixel value and the color-1-output pixel value;
a modified-pixel value for color 2 is computed similarly for the pixel at the given position, as the color-2-input-pixel value+the value of the color-2-diffused-error, at that pixel position;
the color-2-output-pixel value at the given position is selected from among those possible near the sum of the color-2-modified-pixel value and the product of a constant and the color-1-quantization error;
the color-2-quantization error is computed similarly to the color-1-quantization error, i.e., the error is computed as the difference between the color-2-modified-pixel value and the color-2-output pixel value;
a modified-pixel value for color 3 is computed similarly for the pixel at the given position, as the color-3-input-pixel value+the value of the color-3-diffused-error, at that pixel position;
the color-3-output-pixel value at the given position is selected from among those possible near the sum of the color-3-modified-pixel value and the product of a constant and the color-1-quantization error and the product of another constant and the color-2-quantization error;
the color-3-quantization error is computed similarly to the color-1-quantization error; and
at pixel positions not yet processed, the diffused errors for color 1 are incremented by amounts computed from the color-1-quantization error at the given pixel position, and the diffused errors for color 2 and color 3 are similarly incremented at each position, using the amounts computed from the respective color-2-quantization error and color-3-quantization error at the given pixel position.
It will be seen that the ordering of these steps is not critical as many other orderings of the steps would still produce the same outputs. The result of combining these outputs is an output image representative of a full-color input image using a small number of colors in comparison to the number of colors in the input image but minimizing artifacts such as the amount of visually-apparent texture in the output image.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1(a) is a diagrammatic representation of a system for the quantization of the first color plane in accordance with the invention.
FIG. 1(b) is a diagrammatic representation of a system for the quantization of the second color plane in accordance with the invention.
FIG. 1(c) is a diagrammatic representation of a system for the quantization of the third color plane in accordance with the invention.
FIG. 2(a) is a diagrammatic representation of a system for incrementing the diffused errors for the first color plane in accordance with the invention.
FIG. 2(b) is a diagrammatic representation of a system for incrementing the diffused errors for the second color plane in accordance with the invention.
FIG. 2(c) is a diagrammatic representation of a system for incrementing the diffused errors for the third color plane in accordance with the invention.
FIG. 3 is a diagrammatic representation of a system for generating coefficients for random error diffusion.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
The present invention uses a technique which may be called coupled-color error diffusion and in the preferred embodiment to be described the invention is applied to a three color system, i.e., with three color planes, wherein green is used as color 1, red as color 2, and blue as color 3. The technique is directed to improving the quality of the output of full-color images on monitors that display a comparatively small number of colors, or printers that print a comparatively small number of colors, using the quantization error of one color of a given pixel to modify the other colors of that pixel. More particularly, in image processing using error diffusion, as noted above, a modified input signal is compared to a set of possible output signals and one of the output signals is selected. Typically, the "nearest" output signal is chosen. This process of selecting an output that is near the input is called "quantization". The difference between the input and the selected signal is called the "quantization error". For a monochrome (one color with many shades) image the determination of the "nearest" color is fairly straightforward; but, for a multi-color (e.g., with multiple shades of red, green, and blue) image, the choice of nearest color (and how to select it) is not apparent. Presently the quantization of color images is done by independently processing each of the color components (usually the three: red, green, and blue). In contrast, the present invention uses the quantization error generated by one color component for a given pixel to influence the quantization of subsequent color components for the same pixel, and in such a way that the color image, so rendered, appears less grainy. The invention essentially involves, specifying a new quantization method for multi-color images, rendered with orthogonal or separable color image palettes, that takes advantage of the calculations done by error diffusion.
To begin with, using a system and method in accordance with the present invention, signals representative of the colors of the pixels of a full-color image are input to a processing device, such as a personal computer, to ultimately produce an output, such as a display on a monitor or printout from a printer, that only consists of a small number of colors in comparison to the number of colors represented by the input signals. It is accordingly desired to process the input signals in such a manner as to obtain an output in a form that best reproduces or represents the full-color image to a viewer of the output product, i.e., the image on the monitor or the image printed out by the printer. To this end, using error diffusion on the input signals and beginning with the diffused errors for each color and at each pixel position equal to zero, that is, e.sub.i,j.sup.ck =0 i,j for ck=c1,c2,c3, the input pixel signals of a color image are then processed, in order, by the following steps:
1. The modified-pixel value for color 1 is computed as the sum of the color-1-input-pixel value and the value of the color-1-diffused-error at a first or given pixel position in the image:
The first or given position may be at any location within the image based on the selection of a convenient starting point. The starting point is an initial pixel position, at which the pixel input signals are used as the basis for determining the diffused errors of the remaining pixel positions. The pixel in the first pixel position in the upper right hand corner of the image is usually a convenient choice for a starting point so that the first or given position typically would be the second, i.e., horizontally-adjacent, position in the upper right hand corner.
2. The color-1-output-pixel value is selected as one of the possible color-1-output values, q.sub.t.sup.c1, near the modified pixel value:
3. The color-1-quantization-error is computed as the difference between the color-1-modified-pixel value and the color-1-output-pixel value:
An implementation of these steps 1 through 3 is illustrated in FIG. 1(a). In the figure the component .SIGMA. represents any device that can compute sums or differences of signals; and whether a signal is to be added or subtracted is indicated by a+or-, repectively. The block labelled quantization Q.sup.c1 represents the component performing the quantization operation. This operation may be carried out by computation but normally a look-up-table is used to implement it.
4. The modified-pixel value for color 2 is similarly computed as:
5. The color-2-output-pixel value is selected as one of the possible color-2-output values, q.sub.t.sup.c2, near the sum of the color-2-modified-pixel value and the product of a constant and the color-1-quantization-error:
6. The color-2-quantization-error is computed similarly to the color-1-quantization-error as:
An implementation of these steps 4 through 6 is illustrated in FIG. 1(b).
7. The modified-pixel value for color 3 is similarly computed as:
8. The color-3-output-pixel value is selected as one of the possible color-3-output values, q.sub.t.sup.c3, near the sum of the color-3-modified-pixel value, the product of a constant and the color-1-quantization-error, and the product of another constant and the color-2-quantization-error:
9. The color-3-quantization-error is computed similarly to the color-1-quantization-error as:
The implementation of these steps 7 through 9 is illustrated in FIG. 1(c).
10. At pixel positions not yet processed, the diffused errors for color 1 are incremented by amounts computed from the color-1-quantization-error at this first pixel position as:
subject to .SIGMA.c.sub.r,s.sup.c1 =.gamma..sup.c1.
An implementation of this step is illustrated in FIG. 2(a). As shown in the figure, counters are used to generate an index r,s, which when applied to a coefficient store, leads to the production of the coefficient c.sub.r,s.sup.c1 from the coefficient store. The index r,s is also added to the address i,j to form the indexed address, i+r,j+s, which when applied to an error store, leads to the production of error e.sub.i+r,j+s.sup.c1 from the error store. The coefficient produced is multiplied by the quantization error, .delta..sub.i,j.sup.c1, and added to the error from the error store to produce the updated error e.sub.i+r,j+s.sup.c1. After a delay, the updated error e.sub.i+r,j+s.sup.c1, is inserted in the error store, replacing the earlier version. This operation is repeated for each index generated by the counters.
11. The diffused errors for color 2 are similarly incremented as:
subject to .SIGMA.c.sub.r,s.sup.c2 =.gamma..sup.c2.
An implementation of this step is illustrated in FIG. 2(b). This implementation is similar to the implementation of FIG. 2(a), except that the quantization error for color 2, the coefficient store for color 2, and the error store for color 2 are substituted for the quantization error for color 1, the coefficient store for color 1, and the error store for color 1, respectively.
12. The diffused errors for color 3 are similarly incremented as:
subject to .SIGMA.c.sub.r,s.sup.c3 =.gamma..sup.c3.
An implementation of this step is illustrated in FIG. 2(c). This implementation is similar to the implementation of FIG. 2(a), except that the quantization error for color 3, the coefficient store for color 3, and the error store for color 3 are substituted for the quantization error for color 1, the coefficient store for color 1, and the error store for color 1, respectively.
As mentioned above, in the preferred embodiment color-1 is green, color-2 is red, color-3 is blue. The constants are made equal to 1.0, i.e., .alpha..sup.12 =1.0, .alpha..sup.13 =1.0, .alpha..sup.23 =1.0, and the sums of the coefficients are made equal to 1.0, .gamma..sup.c1 =1.0, .gamma..sup.c2 =1.0, .gamma..sup.c3 =1.0. Note that with these choices:
in step 2 the green output value is chosen to minimize the difference between the green input value and the green output value;
in step 5, the red output value is chosen to minimize the difference between the sum of the green and red input values and the sum of the green and red output values; and
in step 8, the blue output value is chosen to minimize the difference between the sum of the green, red and blue input values and the sum of the green, red and blue output values.
The coefficients in the error diffusion processing may be sets of constants or, if desired, may be random variable coefficients. An implementation of the latter processing is illustrated in FIG. 3. As shown in the figure, a random number generator is used to provide random number signals which are multiplied by respective constants from a constant store. The constant store is connected to the index-generating counters of FIGS. 2 along with the coefficient store therein, and the products of the random numbers and the constants are input to the coefficient store, the output of which acts to provide respective multipliers for the diffused errors.
It has been found that the invention gives significantly improved performance when compared with that of independent color error diffusion, as the output images produced feature much less apparent graininess, while little additional computation is required. Additionally, the use of the quantization-error from color planes already processed to modify other color planes could also be used to improve dithered color images. Further the invention will be found useful in color imaging applications particularly for displays that are severely quantized (e.g., LCDs) or with shallow frame buffers (e.g., IBM Model 8514A).