Background noise removal for a low-cost digital color copier

[0002] The present invention relates to document reproduction and more particularly to the removal of undesirable and unintended dots of toner or ink produced in a background area of a document. As will be detailed below, the primary use of the present invention is in connection with marking engines used in digital black/white and color copiers and printers. However, it is to be appreciated that the present invention can have uses in other fields where noise removed is beneficial.

[0003] FIG. 1 illustrates the image path of a color digital copier A, including scanner 10, image processing electronics 12 and printer 14. An original document is scanned by scanner 10, and converted into a stream of pixels. Image processing electronics 12 manipulates the pixel stream into a format suitable for use by printer 14, which selectively transfers toner or ink to a blank piece of copy paper, commonly a white piece of paper, thereby reproducing the images of the original document onto the copy paper.

[0004] Each scanned pixel is represented by three 8-bit values 20 that measure the intensity of the three primary colors (i.e. red, green and blue), such a representation being in what is known in the art as a 24-bit RGB color space. Black is represented by all three values of RGB being equal to 0, white is represented by all three values being equal to the maximum 8-bit value 255, and gray is represented by all three values being an equal value. In existing color digital copiers, the printer functions in a non-RGB color space which more closely mimics the human eye. Examples of such color spaces include CMYK, YCC, and LAB color spaces, the concepts of which are well known in the art. The whitest pixel produced by a CMYK printer consists of no toner or ink located at that pixel position. Therefore, the whitest pixel is limited by the whiteness of the copy paper.

[0005] FIG. 2 graphically represents the RGB color space where each pixel is represented by three 8-bit values (0-255). If a user wishes to increase or decrease the luminance of an output document, it is necessary to adjust each of the colors of the RGB color space by an equal amount such that the color of the pixels do not change, but rather only the luminance is altered.

[0006] On the other hand as can be seen in FIG. 3, which is a graphical representation of a color space such as YCC (where C0=the intensity value and C1 and C2 represent the hue of a color), intensity can be altered by changing the value of C0 without requiring a corresponding change to the C1 or C2 values. With attention to the C1 and C2 values of FIG. 3, the further away from the origin, the more saturated the color value (i.e. the deeper the hue). Also, if the C0 value is set to 0, then the values of. C1, C2 are irrelevant since the color will be black. Similarly, if the C0 value is made equal to its maximum, i.e. 255, then it does not matter what the values of C1, C2 are since the color will be pure-white.

[0007] Thus, by moving from the RGB color space to a color space more closely mimicking the human eye, such as the YCC or LAB color spaces, it is easier to control the luminance and hues of the documents being reproduced.

[0008] In order for scanners which scan images into the RGB color space and printers which operate in color spaces, such as CMYK, to function together in the same copier, a color space conversion takes place through the use of color space converter 16, whereby over 16 million colors in the RGB color space (2²⁴) are converted to the 16 million colors of the YCC or LAB color space (2²⁴). Thereafter, the pixels are provided to render 18 which converts the 16 million colors in the YCC color space to the 16 colors of the CMYK color space (2⁴) and then passes the pixels to printer 14. FIG. 1 also shows microcontroller 19, which provides intelligence to control operation of color digital copier A.

[0009] Another consideration regarding digital color copiers is that existing color marking engines (e.g. laser and ink jet) cannot satisfactorily reproduce extremely unsaturated colors (known as near-white). The resulting reproduction consists of a few widely scattered small dots of color (i.e. background noise), in the background of the copy paper. The term background refers to the region of the source document that has no image imposed on it so that the underlying paper is viewed. Common background types are white copy paper, colored copy paper, newspaper, magazine paper and photographs.

[0010] When the background of the source document is scanned, even if the paper appears white to a person, it appears off-white i.e. near-white, to a scanner. This off-white color is an extremely unsaturated color, hence the perceptually white background of the original is reproduced with colored dots scattered around the page. Viewed at a far distance, the region looks like the near-white original color but at a typical viewing distance the dots are quite noticeable and objectionable.

[0011] These near-white pixels are created by the combination of scanner inaccuracies, source document paper inconsistencies, and actual near-white pixels in the source document.

[0012] To overcome the above, the present invention provides a system that removes the undesirable background noise, in the form of unwanted dots of color, located in a background area of a document.

[0013] In accordance with one aspect of the present invention, provided is a system which eliminates background noise from a document to be reproduced. The system includes a device for acquiring statistics for at least some pixels of a pixel stream generated by a document scanner. Statistics and characteristics of the document are extracted from the pixels to determine a background color of the document to be reproduced. The value of the determined background color is then compared with incoming pixels of the scanned document to determine color dispersion in the background. Pixels found to be “near” background color values are manipulated to become pure-white by changing the luminance to maximum, pixels found to be “far” from the background color values are left unchanged, and pixels between “near” and “far” are altered in a linear manner. The manipulated pixels are then forwarded to a printer whereby a document is printed which has eliminated undesirable background dots, i.e. noise, within a document.

[0014] The present invention may be used in a digital color copier as well as black and white copiers wherein the system includes a statistics gathering (STS) module which acquires statistics on the pixels scanned by a scanner. The pixels are then provided to a controller MIC module which manipulates the statistics to determine background color values. These background color values along with additional statistics define areas where a pixel will be changed to white, where a pixel will not be changed, and where a pixel will be changed linearly.

[0015] In a more limited aspect of the present invention, a reference background value and a value of an incoming pixel are compared to determine the color distance the incoming pixel is located from the background reference value.

[0016] The RTE module accepts a pixel stream from the scanner and performs a transform. This transform is used to remove background noise. The RTE determines the three dimensional color distance squared between each incoming pixel and a reference pixel (supplied by the MIC module). The RTE module applies a linear stretch to a C0 value of each pixel. Pixels with a color “near” to the reference pixel are stretched so that the C0 component becomes pure-white. Pixels with a color “far” from the reference pixel are not changed. In a transition region between the “near” and “far”, pixels are stretched in a linear fashion.

[0017] In accordance with a more limited aspect of the present invention, the regions of transition are defined by an inner three dimensional ellipse and an outer three dimensional ellipse formed around a center reference point.

[0018] In accordance with another feature of the invention, a color digital copier implements the present invention wherein pixels scanned into an RGB color space are converted into a YCC or LAB color space where after undesirable background noise is eliminated.

[0019] In accordance with another aspect of the present invention, the elimination of undesirable background noise is performed in real time, whereby the pixels are manipulated during a single pass of the document such that the ability to eliminate undesirable background noise does not lower the throughput of the digital copier.

[0020] In accordance with still another aspect of the invention, when a background color value is not capable of being determined, the present invention includes a default background reference color to which the incoming pixels are compared.

[0021] With attention to still yet another aspect of the present invention, a saturated background color removal option is available, wherein when a saturated background color of an original document is altered to white. This is typically used to copy an image originally printed on colored paper.

[0022] A principal advantage of the present invention is the elimination of undesirable noise, i.e. dots in a background of a reproduced document.

[0023] Another advantage of the present invention is that the elimination of undesirable background color provides a linear manipulation of pixels in an intermediate area.

[0024] The invention may take form in various components and arrangements of components, and in various steps and arrangements of steps. The drawings are only for purposes of illustrating a preferred embodiment and are not to be construed as limiting the invention.

[0025] FIG. 1 is a block diagram of an image path of a color digital copier;

[0026] FIG. 2 is a graphical representation of the three dimensional RGB color space;

[0027] FIG. 3 is a three dimensional graphical representation of a 4-bit color space such as a YCC or LAB color space;

[0028] FIG. 4 is the image path of a color digital copier including the components of the present invention;

[0029] FIG. 5 is a flow chart providing an overview of the operation of the present invention;

[0030] FIG. 6 illustrates a relationship between C1 and C2 values of a pixel whereby the concept of two dimensional color distance is defined;

[0031] FIG. 7 is a graphical correlation between the two dimensional color distance and luminance upon which is defined a color space;

[0032] FIG. 8 defines a point of the C1 mean and C2 mean around which is formed a three dimensional ellipse;

[0033] FIG. 9 correlates Cmean and C0 values to define a Cref value around which is formed a three dimensional ellipse;

[0034] FIG. 10 illustrates outer and inner three dimensional ellipses formed around a Cref value;

[0035] FIG. 11 illustrates the histogram of the present invention;

[0036] FIG. 12 defines the STS data path of the STS module;

[0037] FIG. 13 is an example of a plot the contents of the histogram as counts versus bins;

[0038] FIG. 14 is a data flow diagram for the RTE module;

[0039] FIG. 15 provides a graphical illustration of the stretch transform;

[0040] FIG. 16 depicts how the luminance stretch slope (lss) is varied as a function of color space distance;

[0041] FIG. 17 is a top-level block diagram of the RTE module; and

[0042] FIG. 18 is the RTE data path of the RTE module.

[0043] Referring now to the drawings, wherein the showings are for illustrating a preferred embodiment of the invention only and not for purposes of limiting same. While FIG. 1 is a block diagram of an image path for an existing color digital copier, FIG. 4 provides such a block diagram including components directed to the present invention. Within imaging electronics 12 of digital color copier A, the present invention further includes STS module 20 which accumulates document statistics, MIC module 22 which extracts document background characteristics, and RTE module 24 used to manipulate a pixel stream generated by scanner 10 to remove background noise. STS module 20, MIC module 22 and RTE module 24 work together to perform a pixel transformation that maps selected pixels to pure-white in order to remove background noise. As previously noted, color space converter 16 converts pixels from scanner 10 which are in an RGB color space to another color space such as YCC or LAB pixels. Pixels in these color spaces are defined by components (also known as channels) C0, C1, and C2. Wherein the C0 component represents the luminance of the pixel, and C1 and C2 represent the hue. As similarly shown in FIG. 1, microcontroller 19 is used to control operation of the digital color copier A.

[0044] An overview of the operation of the present invention is described with attention to FIG. 5.

[0045] Initially, a user places an original document to be copied on a scanner, 30, and the scanner converts the document into a pixel stream in the RGB color space, 32. A color space conversion is undertaken to convert the pixel values from the RGB color space to a color space more closely mimicking the human eye such as a YCC or LAB color space, 34. The STS module observes the pixel stream from the scanner and collects statistical data 36. This data is used to develop statistics including:

[0046] a histogram of the C0 values;

[0047] a sum of all C1 values indexed by the C0 value;

[0048] a sum of all C2 values indexed by the C0 value;

[0049] a sum of all C1 values squared indexed by the C0 value;

[0050] a sum of all C2 values squared indexed by the C0 value.

[0051] In step 38, the MIC module uses the statistics gathered by the STS module to determine the background color. The background color is in part determined by looking for the brightest region of the histogram that is the same color. In the present embodiment, the background is identified as one of the four following choices:

[0052] near-white contone (e.g. white copy paper, newspaper)

[0053] near-white, halftone (e.g. magazine paper)

[0054] far-white, contone (e.g. photo or colored paper) and

[0055] far-white, halftone (e.g. magazine paper).

[0056] Next, in step 40, the range or dispersion of C0 values that comprise the background color and the corresponding C1/C2 means and standard deviations are obtained from the statistics. If the statistics appear to represent a near-white, contone background and enough of the background is visible, 42, the MIC module uses the statistics to measure the background of the paper and generate reference pixel values 44. Otherwise, default background reference pixel values are used, that are determined to work well for high quality paper (the most common type of document being copied), 46. The MIC module uses the background information to create a set of control parameters to program the RTE module for conversion of near-white background pixels to pure-white in order to eliminate background noise 48.

[0057] The RTE module receives the pixel stream originally generated by the scanner 50, and determines the three dimensional color distance squared between the values of each incoming pixel and the background reference pixel values (supplied by the MIC module based on the determined background values or the default values) 52. The RTE module applies a stretch algorithm to the C0 component of incoming pixels of the pixel stream 54. This stretching of the C0 component is accomplished in accordance with the determined distance of each incoming pixel from the background reference pixel. When a received pixel has a color value “near” to the reference pixel value it is stretched sufficiently so that the C0 component becomes pure-white (i.e. a value of 255). Pixels with a color value “far” from the reference pixel are not changed, and pixels in a transition region between “near” and “far” are stretched in a linear fashion. Thereafter, an improved copy is printed having background noise removed 56.

[0058] Turning attention to a more detailed review of the present invention, it will be useful to discuss the concept of two dimensional color distance. FIGS. 2 and 3 which depict the RGB and YCC color spaces, illustrate that these color spaces are represented in three-dimensions. A relationship between C1 and C2 is shown in FIG. 6, where C_{2d color distance}represents a two dimensional color distance and is found by the equation: $C_{2d\mathrm{color}\mathrm{distance}}=\sqrt{{\mathrm{C1}}^2+{\mathrm{C2}}^2}$

[0059] The two dimensional color distance, C_{2d color distance}, represents the amount of color existing in a pixel (i.e. how saturated is the pixel), wherein the farther from origin, O_rigin, the more saturated the pixel. Once C_{2d color distance}is obtained the horizontal plane of FIG. 6 may be collapsed and the value of C_{2d color distance}correlated to illuminance, C0, as shown in FIG. 7. The farther C0 is from O_origin, the greater the illuminance of the pixel. Similarly, the farther C_{2d color distance}is from the O_origin, the more saturated the pixel.

[0060] In connection with the C0, C_{2d color distance}relationship shown in FIG. 7, a desirably saturated color, which is not too bright as to be washed out or too gray, is located within a region, R_{color space}, which represents an entire color space such as the YCC color space. Since it is possible to increase luminance, C0, by multiplying the C0 value by some value, a pixel at location 60 within R_{color space}can be stretched by multiplying C0 by a number such that the pixel is moved to location 60′. By this operation, a portion of the color space has been increased such that it becomes washed out, and to the human eye appears pure-white.

[0061] The stretching of pixels out so as to extend R_{color space}so that pixels in the extended area appear pure-white to the human eye is applied in the present invention to achieve a reduction in undesirable color background noise.

[0062] As previously noted, the YCC, and other such color spaces, are represented in three dimensions. An aspect of the present invention is to determine a measure of the background color. The data to make this determination is obtained from STS module 10. Using information from STS module 10, the mean C0 value, the mean C1 value, and the mean C2 value are obtained. As shown in FIG. 8, the C1 mean value and the C2 mean value are combined to obtain a C_mean62, which is the mean hue value of the background. Thereafter, a standard deviation around C_mean, is determined (in this embodiment a +−3.0 standard deviation, around the C_mean), 62 and a two dimensional ellipse is generated 64.

[0063] Assuming the above ellipse is defined as being a slice of the color space (e.g., horizontal), it is also possible to take a further slice in another direction, (e.g. such as a vertical direction) by which a second two dimensional ellipse is formed 66. Combination of these slices define three dimensional ellipse 68.

[0064] Three dimensional ellipse 68 can then be graphed versus the C0 mean, as shown in FIG. 9. Thus, point Cref of three dimensional ellipse 68 is at the C0 mean, C1 mean and C2 mean. The above operations define at least a part of the three dimensional color space.

[0065] As depicted in FIG. 10, in addition to three dimensional ellipse 68 (which will be referred to as the inner ellipse), an outer three dimensional ellipse 70 corresponding to inner ellipse 68, but of a larger size, is also generated.

[0066] With continuing attention to FIG. 10, Cref represents the pixel value for background color and intensity. The value of Cref is compared to values of pixels which have been scanned by scanner 10 and transformed into the YCC or LAB color space. Dependent upon the values of the pixel being compared to the Cref value, certain operations will take place. For example, for a pixel 72 which has values placing it within inner ellipse 68, the C0 value of pixel 72 is multiplied by a scale factor which causes pixel 72 to become pure-white (a value of 255), i.e. the C0 value is moved out of R_{color space}Pixel 74, which is outside of outer ellipse 70, is multiplied by a unity factor, i.e. 1, such that no change is made to the pixel value C0. Pixel 76 which is outside of inner ellipse 68 but within outer ellipse 70 has its C0 value linearly increased, allowing for a smooth transition between a no change situation to forcing a pixel to pure-white.

[0067] Through the above operations, those pixels within inner ellipse 68 are forced to pure-white and those outside of outer ellipse 70 are not changed. Further, those pixels within outer ellipse 70, but outside inner ellipse 68, are provided with a smooth transition. It is noted that reference Cref is obtained from statistics collected by STS module 20. The use of the linear stretching is especially useful when part of a document being scanned includes a color bar which fades from a very saturated end to a highly unsaturated end.

[0068] In some cases, a clear indication of the background color will not be obtained. In these situations the present invention provides default values and these values are used for generation of the required information including that to generate three dimensional inner ellipse 68 and three dimensional outer ellipse 70.

[0069] The above-discussed comparison between incoming pixels and Cref includes obtaining a three dimensional color distance of the incoming pixel as to Cref. One manner of obtaining the three dimensional color distance is as follows (it is noted for use in the present embodiment scaling factors will also need to be included in this calculation): $3D\mathrm{color}\mathrm{distance}=\sqrt{{\left(\mathrm{C0}-\mathrm{C0}\mathrm{ref}\right)}^2+{\left(\mathrm{C1}-\mathrm{C1}\mathrm{ref}\right)}^2+{\left(\mathrm{C2}-\mathrm{C2}\mathrm{ref}\right)}^2}$

[0070] An alternative to obtaining the three dimensional distance using square root calculations, is to use a distance squared calculation. The above square root calculations would, in the present invention, require scaling factors for appropriate application. This is beneficial in the present invention as square roots are expensive in hardware terms. Whatever operation is used, it is important to note that the object is to use some metric that provides information as to a distance an input pixel value is from a reference pixel value. It is important to find this value, since then a proper scaling factor, i.e. stretching value, may be applied to incoming pixels.

[0071] The information used to determine the background color, is obtained during an initial scan of the original document. It is common in color scanners, that while all lines of a document are scanned, the first fifty or so lines are normally not printed on, this portion therefore is a useful source to gather data concerning the background color. This information is collected by STS module 20 and used as the reference color value in three dimensions.

[0072] Further, existing copiers perform their document generation in real time. Therefore, it is necessary for the operation of the present invention to occur in real time. Particularly, two passes cannot be made, where on a first pass the STS module collects statistics of an entire page, and then on a second pass the actual copying operation takes place. Rather, it is necessary that STS module 20 be able to collect its information in a single pass. It is to be appreciated that STS module could be programmed to use more or less than the fifty described lines, and that the present invention could be used with a multipass copier.

[0073] With more particular attention to the elements of the present invention, STS module 20 collects a histogram for a document which is to be copied, into an internal RAM. A microcontroller such as microcontroller 19 of the copier reads the histogram from the RAM and computes RTE parameters to enhance the document. It is to be appreciated that while FIG. 4 shows microcontroller 19 as part of the image processing electronics 12, the present invention can function in various configurations, including those where control of the copier is centralized or distributed within individual components.

[0074] The histogram, as shown in FIG. 11, is an array of 64 bins (0-63) each containing five tables (C0, C1, C1², C2, C2²) STS module 20 takes the last four C0 values and averages them, and the most significant 6 bits of the average are used to select one of the 64 bins. The averaging is used to help deal with halftones, without it, the average value of halftones could not be determined. The average is computed as (suffix_dn represents a delayed version of the value by n cycles):

[0075] C0 avg=(((C0+C0_d1)>>1)+((C0_d2+C0_d3)>>1)>>1)

[0076] This approach introduces a maximum error of −1 as compared to the more accurate approach of adding all four values and then right shifting by two. For example, assume the four values 0, 1, 0 and 3 are averages. The first approach produces an average of 0 whereas the latter approach produces an average of 1. While either approach may be used, it is noted the latter requires 10-bit sums to be stored whereas the first only requires 8-bit sums.

[0077] The average calculation produces the wrong result for the first three pixels on a line. This error is negligible for lines of any reasonable width. For the first line on the page, the C0_d1, C1_d2, and C0_d3 values will be 0. For other lines, the C0_d1, C0_d2, and C0_d3 values will be carried over from the previous line.

[0078] As a 6-bit value is being used in a 64 bin arrangement, any C value in the range of 0-3 is allotted to the first bino, any C value from 4-7 is allotted to bini, any C value from 8-11 is allotted to bin2, . . . , and any C value from 252-255 is allotted to bin63. Each bin has five pieces of information gathered, i.e. tables C0, C1, C1², C2, C2². When a pixel value is entered, and a bin is picked, information is input into the C0 value table and then is incremented, i.e. +1, to the C1 table and so on. The selected bin in the histogram is updated as follows:

[0079] Following the initial scanning, wherein values are input into the bins based on the first 50 scanned lines, the contents of the tables are then reviewed to obtain a count of values in each of the bins. Each entry in a table indicates how many pixels having a certain value were encountered during the scan. In particular, reading binO, for the C0 table will indicate the amount of pixels having a C0 value in the range of 0-3.

[0080] If the copier is being operated in a black and white mode, or if the present invention were used in a black and white copier, the information regarding C0 is the only table that would be necessary, i.e. luminance. However, in a color copier data regarding tables C1, C1², C2 and C2²will also be used.

[0081] The C0 table is used to determine the distribution of luminance values. The C1 and C2 tables are used to determine the distribution of the color means associated with the luminance values. The C1²and C2²values are used to determine the distribution of color standard deviations associated with the luminance values. The microcontroller computes the mean and standard deviation of the hue components (C1 and C2) for a range of bins. Since the STS module uses the biased values of C1 and C2, the effect of the bias-must be accounted for.

[0082] Therefore, letting the biased value be C, the unbiased value be X, the bias B, and the number of samples N. By definition, X=C−B. Then the unbiased mean is:

[0083] mean=SUM(X)/N

[0084] It is computed by the microcontroller as: $\begin{array}{c}\mathrm{mean}=\mathrm{SUM}\left(X\right)/N\\ =\mathrm{SUM}\left(C-B\right)/N\\ =\mathrm{SUM}\left(C\right)/N=N*B/N\\ =\mathrm{SUM}\left(C\right)/N-B,\mathrm{and}\end{array}$

[0085] The unbiased standard deviation is:

[0086] sdev²=SUM(X²)/N−mean²

[0087] It can be shown that biased and unbiased standard deviations are identical (since the bias is just a linear shift of the distribution). Therefore, the unbiased standard deviation is:

[0088] sdev²=SUM(C²)/N−mean²

[0089] The microcontroller, for example microcontroller 19, of FIG. 4, reads the contents of the STS histogram tables, and provides this information to MIC module 22 in order to make a determination regarding the background color. The above information is then provided to RTE module 24, and includes background color values, the size of the ellipses, and where the ellipses are located within the color space.

[0090] These operations occur at a rate early enough such that this information processing will be completed before pixels are actually printed out. It is to be appreciated that while ellipse type configurations are used in this explanation, other shapes may also be used.

[0091] The microcontroller collects statistics from the STS module according to the following sequence:

[0092] clears out the contents of the histogram;

[0093] programs STSLC (STS line count) location to the desired number of lines (e.g. first 50) to be examined for collecting statistics;

[0094] enables the STS module to collect statistics;

[0095] starts the flow of pixels;

[0096] the STS asserts an interrupt to the microcontroller when the programmed number of lines has been processed;

[0097] clears interrupt conditions in the STS;

[0098] the microcontroller reads the histogram from the STS module (typically it stops the flow of pixels but this is not a requirement);

[0099] the microcontroller provides the information from the histogram to the MIC module to compute parameters for the RTE module to enhance the remainder of the document;

[0100] the microcontroller loads the parameters into the RTE module; and

[0101] the microcontroller restarts the flow of pixels (assuming that it stopped the flow earlier).

[0102] Turning to a more specific description of STS module 20, the STS datapath, shown in FIG. 12, contains an adder, registers to hold temporary values, and a RAM. The RAM holds the histogram. All wires not marked are 8 bits wide. Also, unless otherwise specified, when a wire is reduced in bit width, it discards the most-significant bits.

[0103] The heart of the datapath is a fast 8-bit adder 80. The output of the adder 80 is fed to several registers and the din (data in) input of the RAM 82. The registers acc1, 84, and acc2, 86, operate as accumulators and hold results of temporary calculations. The carry output (cout) of the adder 80 is stored in register cout_d1, 88, and then presented to the adder's carry in (cin) input. This allows the adder 80 to perform additions greater than 8 bits.

[0104] When STS module 20 is collecting the histogram, it receives pixels (one component at a time) from color space converter 14. It stores component values (C0, C1, C1²low, C1²high, C2, C2²low or C2²high) in the tmp register 90. STS module 20 maintains a 4-pixel C0 context in the tmp 90, c0_d192, c0_d294, and cO_d396, registers. It adds these four values together using the adder 80 and the two accumulator registers, acc184 and acc286. The result of each addition is divided by 2 using a wired right shift. After the three additions, the average of the 4 values is stored in the acc1 register 84.

[0105] The adr_mux 102 selects the source of the RAM address as either the STSADR 104 register or the tb register 106 and bin register which is part of the RAM. The STSADR 104 register is selected when the microcontroller, such as microcontroller 19, requests access to the RAM 82. The STSADR register 104 which includes STSADRLO and STSADHI components (not shown), contains the address within the STS histogram that the microcontroller wants to access. The tb (table and byte) register 106 and bin registers are selected when the STS module 20 is processing pixels. The tb register 106 is incremented by the adder 80 so that each byte of a histogram bin can be accessed. The bin is specified by the most-significant 6 bits of the C0 average stored in acc184.

[0106] The tb_mux 108 and bin_mux 110 are used to select between the stored values of tb and bin and newly computed values of tb or bin. The adr mux 102 is used to select between the STSADRHI, STSADRLO, and tb registers. The selected value is incremented by the adder 80 and stored.

[0107] The adder's input operands are selected by the srcl mux 112 and src2 mux 114. The srcl mux 112 selects the RAM data output, part of the RAM address to increment, c0_d192, or acc2/c0_d386, 96. The src2 mux 114 selects the constant zero 116, tmp 90, co_d294, or acc198.

[0108] Programmable registers are written via the cpu_data bus, i.e. for communication with microcontroller 19, by storing the cpu_data into the tmp registers and then passing the data through the adder (by forcing acc2=0). The STSADR 104 register is incremented by adding 1 to STSADRLO and then propagating the carry to STSADRHI. The RAM is read via a RDPRE register (not shown), which is loaded with the RAM data whenever the STSADR register is written by the microcontroller.

[0109] Turning attention to MIC module 22, as previously discussed, the information from the histogram of STS module 20 is used by MIC module 22 to generate parameters, including, the background reference color values and luminance values, to be used by RTE module 24.

[0110] FIG. 13 plots the contents of the histogram as counts versus bins. The count value indicates the number of pixels collected within a particular bin during scanning of the predetermined number of lines, e.g. the first 50 lines. As it may be recalled, each bin of the histogram corresponds to three values, e.g. binO counts pixels having C0 values 0-3, whereas bin63 counts pixels having C0 values 252-255. The first 50 scanned lines will tend to represent the background color. Therefore, there should be more background color than any other color, i.e. the C0 bins with the highest count of pixels will likely represent the background color. In FIG. 13 the bins with the highest counts are shown with the highest peaks.

[0111] If operating in black and white mode, all that is required is to select a peak with the highest C0 value as a background color. However, in a color copier it is necessary to know more than just how light or dark an image is.

[0112] Nevertheless, MIC module 22 can use the C0 values to arrow the range of bins which appear to be the most appropriate selections as representative of the background. Particularly, those counts in the higher bins are more likely to represent a white background color. In the color mode, the bins are used as an index of an array for information to be provided by MIC module 22 to RTE module 24. Particularly, as recalled from FIG. 11, associated with each C0 value are C1, C1², C2, C2²values. These values are used to obtain reference values which are provided to RTE module 24, i.e. the calculation of the mean values and the standard deviation.

[0113] The invention can be configured to select a single bin as representing a peak value component or a plurality of bins within a range. If a range of background colors are selected then there is a summing of C1, C2, C1², C2²(e.g. if bins 53-58 are considered representative of the background color). The reference background value is then calculated and forwarded to RTE module 24.

[0114] Turning attention to a more specific discussion of RTE module 24, this module applies image enhancements to pixels using a one pixel context. It accepts pixels from scanner 10 and sends enhanced pixels to render 18.

[0115] Enhancements are grouped into two categories: luminance enhancements and hue enhancements. Luminance enhancements consist of a color-sensitive contrast enhancement, user control brightness, and colored-background removal. Additionally, in gray scale mode, an arbitrary curve is applied to the luminance component, i.e. C0, to account for non-linear TRC (Tonal Reproduction Curve) of the printer. Hue enhancements consist of user controlled color saturation (pale/vivid). All these enhancements are a function of the image statistics (collected by STS module 20) and user interface controls.

[0116] RTE module 24 has several parameters that control its image enhancement algorithms and a 256-bit luminance look-up table. All these values are stored in a RAM inside RTE module 24. It is the responsibility of the microcontroller to load the contents of the RAM appropriately. The luminance look-up table and some selected enhancement parameters can be loaded before processing of the document begins. The remaining parameters are loaded after appropriate statistics have been collected on the document.

[0117] RTE module 24 nominally applies its luminance enhancements to C0 and its hue enhancements to C1 and C2. When the pixels entering RTE module 24 are in the YCC or LAB color space the intended enhancements are implemented. However, when pixels in other color spaces (e.g. RGB) enter RTE module 24, the enhancements are programmed to just pass C0, C1 and C2 of the pixels through RTE module 24 unaltered.

[0118] RTE module 24 uses global color space signals to determine whether to operate in color or in black and white mode and to determine the hue coding biases.

[0119] On reset (hardware or software), RTE module 24 forces state machines to the idle state as long as a reset persists. An exception is made to this for RAM access. In this situation only a hard reset forces the RAM access state machine to idle. Therefore, the RTE RAM can be accessed independently of the state of soft reset.

[0120] With attention to FIG. 14, depicted is a data flow diagram for the RTE module enhancements. The following discussion corresponds to the previously set forth ellipse concepts.

[0121] A weighted three dimensional distance between the input pixel (represented by C0, C1 and C2) and a reference pixel is computed by 3D color space distance computation block 120. This distance is used to compute a luminance stretch slope by slope adjuster block 122. The input C0 value is multiplied by the slope and limited to the range of 0 to 255 in the luminance stretch block 124. This stretch C0 value is passed through a 256 bit look-up table 126 to produce the output C0 value. The table contains a printer TRC (black and white mode only) and a gamma brightness curve (all modes). The C1 and C2 input values are multiplied by a scale factor and limited to the range −128 to +127 to produce the C1 and C2 output values by C1 scaling block 128 and C2 scaling block 130.

[0122] Color-sensitive contrast enhancement and colored background removal are implemented by applying a stretch transform to the luminance (C0) value. FIG. 15 provides a more detailed look at the stretch transform. Particularly, any input C0 value less than or equal to the black point (black_pt) is transformed to 0 (pure black). Any input C0 value greater than or equal to the white point (white_pt) is transformed to 255 (pure white). Intermediate values are stretched by the slope of the line between the black and white points. This slope is known as the luminance stretch slope (lss), and is greater than or equal to 1. The white and black points are computed by the microcontroller as a function of the image statistics (from the STS module) and user interface controls, i.e. color and luminance selection controls.

[0123] The contrast of a document is not reduced by the stretch transform. This effect is obtained by the user adjusting the lightness/darkness control instead.

[0124] The equation used to implement the stretch transform shown in FIG. 15 is:

[0125] c0_stretched=(c0₋in—RTE—BLACK—PT)*lss (where: c0_stretched is limited to range [0, 255])

[0126] In black and white mode, the lss is a constant for the document and is provided by the microcontroller. In color mode, the lss is computed for each pixel.

[0127] A color-sensitive contrast enhancement varies the white point as a function of the distance between two points in the color space; the pixel being processed and a reference pixel. The distance only is in three dimensions. Pixels closer to the reference are stretched harder (i.e. smaller white point and larger slope) than pixels further from the reference. The reference is computed by the microcontroller as a function of the image statistics and the user interface controls. The intention is that the reference is the background color of the paper and is unsaturated. Therefore, unsaturated pixels are stretched harder than pixels that are more saturated. This helps prevent bright saturated colors from being washed out by the stretch but still allows unsaturated pixels to have a large contrast and remove background noise. This scheme is especially effective when processing documents that were printed on low-quality paper (such as newspaper) since the paper is not as uniformly white as higher-quality paper.

[0128] The colored-background removal enhancement uses the color-sensitive contrast enhancement and the color space distance is computed in three dimensions (C0, C1 and C2). Also, the reference pixel is set by the microcontroller to the background color of the paper even if the color is saturated. Now, only pixels with their color close to the background color and luminance close to the background luminance will be stretched the hardest (ideally to a C0 of 255 since this is pure white) and other pixels will be stretched lightest (ideally by unity).

[0129] The color space distance may be computed as:

[0130] dist=

[0131] ((c0_in−RTE_DIST_REF_C0)*RTE_DIST_SCALE_C0/4){circumflex over ( )}2+

[0132] ((c1_in−RTE_DIST_REF_C1)*RTE_DIST_SCALE_C1/4){circumflex over ( )}2+

[0133] ((c2_in−RTE_DIST_REF_C2)*RTE_DIST_SCALE_C2/4){circumflex over ( )}2

[0134] As previously noted, a true three dimensional distance would require taking the square root of dist. In the present embodiment, the square root calculations are not used in order to simplify the hardware with little loss in image quality.

[0135] The scale factors allow the three channels (C0, C1 and C2) to have different contributions towards the color space distance. These scale factors are based on the distribution of the background color in each of the three dimensions of the color space. The microcontroller uses the image statistics (including C1/C2 means and standard deviations) to determine the distributions and compute the scale factors appropriately. The color-background removal enhancement is disabled by setting its scale factor to zero.

[0136] For both the color-sensitive contrast enhancement and the colored-background removal enhancement, the stretch is a function of color space distance. This is accomplished by making the luminance stretch slope a function of color space distance since it controls the white point via the equation:

[0137] white_pt=(255/lss)+black_pt

[0138] FIG. 16 shows-how the luminance stretch slope (lss) is varied as a function of color space distance. Pixels close to a reference point have the largest stretch slope (lowest white point) and pixels far from the reference point have the smallest stretch slope (highest white point).

[0139] The luminance stretch slope (lss) is computed according to the following equation:

[0140] lss=RTE_LSS_SCALE*dist+RTE_LSS_INTER (lss limited to range [RTE_LS_MIN, RTE_LSS_MAX])

[0141] The color-sensitive contrast and colored-background removal enhancements may be disabled by setting RTE_LSS_SCALE to zero and the RTE_LSS_INTER to the desired stretch_slope. If this is done, all pixels will be stretched using the same implied white point.

[0142] The user-controlled brightness and non-linear printer TRC compensation are implemented by applying a lookup table to the C0 component (after being stretched). This 256-byte lookup table allows an arbitrary transformation to be applied to the C0 component. This is required since the TRC is a non-linear function and the lightness/darkness is an exponential curve. This is implemented by the equation:

[0143] c0_out=RTE_TAB_C0[C0_stretched]

[0144] The user-controlled color saturation (pale/vivid) is implemented by scaling the C1 and C2 channels by a constant scale factor found by:

[0145] c1_out=c1_in*RTE_OUT_SCALE_C1

[0146] c2_out=c2_in*RTE_OUT_SCALE_C2

[0147] C1 and C2 are converted to an unbiased form (two's complement signed) before being multiplied by the scale factors and then converted back to their biased form. The bias is implied by the color space. The hue enhancements are not performed in black and white mode.

[0148] FIG. 17 shows a top-level block diagram of RTE module 24, wherein C0 is stretched by an appropriate value obtained through use of a C0 table lookup, and C1/C2 are scaled prior to being output.

[0149] RTE module 24 uses a computational block to implement the luminance and hue enhancements except for the C0 table lookup. FIG. 18, which is the RTE data path of RTE module 24, shows the computational block consists of an ALU, muxes, and scratch registers. The enhancement algorithm parameters are stored in the RAM. All wires not marked are 16-bits wide. Also, unless otherwise specified, when a wire is reduced in bit width, it discards the most-significant bits.

[0150] ALU 132 performs a multiply or an add for each cycle. The result of the ALU 132 is stored in either one or two accumulators (ACC1 and ACC2) 134, 136 or in the DATA OUT register 138. The ALU 132 obtains its operands through the srcl mux 140 and src2 mux 142.

[0151] The srcl mux 140 in conjunction with the aux mux 144 and channel mux 146 select biased versions of C0/C1/C2, unbiased versions of C1/C2 from unbias blocks 147a-d, the constant zero 148, or ACC1134. The zero input 148 is used in conjunction with the adder to pass values from the src2 ALU input to the ALU output.

[0152] The src2 mux 142 selects COEFHI/COEFLO 150, 152 (containing an algorithm coefficient), ACC1134, or ACC2136. The coefficients are zero-extended to 16 bits. The coefficient is composed of the 8-bit COEFLO register 150 and 2-bit COEFHI register 152. If an 8-bit register parameter is read from the RAM 154, the COEFHI 150 register is cleared. Otherwise, the COEFLO 152 register contains the least-significant 8 bits of a parameter and COEFHI 150 contains the most-significant 2 bits. The COEFHI receiving its bits from passthru or C1/C2 bias block 155.

[0153] The multiplier 156 performs a signed 11-bit by 11-bit multiplication producing a signed 22-bit product. The conversion of the product to a 16-bit value depends on whether the multiplier is in integer or fractional mode. In integer mode, the multiplier output is the least-significant 16 bits of the product. In fractional mode, the multiplier output is either the most-significant 14 bits (bits 6 to 19) sign-extended to 16 bits or the 16 bits from bits 4 to 19. The first choice produces an implicit right shift of 6 and the second choice produces an implicit right shift of 4. These right shifts of register 157 are used to discard lower-precision bits of a fractional result.

[0154] The adder 158 performs a 16-bit addition producing a 16-bit sum. The carry-in and carry-out signals are unused. Control logic examines the most-significant 8 bits of the sum and controls the mux 160 on the output of the adder to select 0, 255, 1023, or the sum. This is used to perform saturating additions (when required). Negative values are limited to 0 and positive values are limited to 255 or 1023.

[0155] The src1 or src2 operand can be bypassed to the ALU output by replacing the adder output with srcl or src2. This is only used for a min/max operation that subtracts srcl from src2 and then based on bit 15 of the difference (the sign bit), it selects srcl or src2.

[0156] In color mode, pixels are received from color space converter 16 in C0/C1/C2 order and sent to render 18 in the same order. The C0 component comes from the COEFLO register 152 and the C1/C2 components come from the ACC2 register 136. In black and white mode, only one component is active so there is no component order issue.

[0157] RTE module 24 contains one lookup table loaded into 256 bytes of the RAM 154. The stretched C0 value from the FINAL_SUM 162 output of the adder/subtractor is used to address this 256 bytes of the RAM. The RAM 154 can also be read, or written indirectly through read/write operations, using a RAM_ADR input.

[0158] The operation described in the foregoing to remove background noise can include various safety features. Including that if the background color is determined to have a certain saturation value, the system will select a default operation and will not stretch pixels to the background color. This feature is available as it is not as necessary to place additional toner on a dark background as it is to remove undesirable spots from a white background.

[0159] Another alternative feature is that when a dark background color is encountered, it is possible to have an option to alter the background color to a white color. This is a valuable feature in situations where an original document has a dark background color and copies are to be made. By causing the background to be changed to white, less toner will be used.

[0160] As previously noted, the present embodiment will seek a maximum peak as being representative of the background color. Of course in some situations there may be several peaks of equal size. Therefore, to increase accuracy, the system may be configured to select the brightest/largest peaks. Similar to the foregoing, the system can be configured with minimum and maximum peak values which is not encountered results in default operations.

[0161] Once the background color is predicted, the present invention generates a confidence factor, wherein a high degree of confidence indicates that it is the actual background color represented by (1) or a low degree of confidence (0). When the confidence factor is high, the areas of change, represented by the ellipses, are sized to increase the background area to be reviewed. As the confidence factor declines, the size or dispersion of the ellipses are made smaller. The low confidence factor may be based on encountering numerous peaks of the same ize or that the detailed background color is a saturated color. When the confidence factor is below a certain predetermined value, a default option is implemented where minimal sized areas, i.e. ellipses are generated.

[0162] These default ellipses are sized conservatively and are generated from fixed values that tend to work well for a particular scanner. This default mode is beneficial for a situation where the fifty scanned lines are found to be very saturated, but as the document comes down the page, there are areas of near-white. In such a document if there was no default value, i.e. no ellipses at all, these near-white areas (which would contain undesirable marking) would not be improved. Thus, the default value ignores the material provided due to the statistics and uses the conservative default back-up.

[0163] While the foregoing modules have been discussed as individual components, it is appreciated that their functions can be combined or included with each other.

[0164] The invention has been described with reference to the preferred embodiment. Obviously, modifications and alterations will occur to others upon reading and understand the preceding detailed description. It is intended that the invention be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.