CIS (contact image sensor) RYB (red-luminance-blue) sampling

[0001] This patent application claims benefit of prior U.S. Provisional Patent Application Ser. No. 60/535,398, filed Jan. 9, 2004 by Derek T. Walton et al. for CIS RYB SAMPLING.

[0002] The above-identified patent application is hereby incorporated herein by reference.

[0003] This invention relates to office automation products in general, and more particularly to printers, fax machines, scanners, copiers and the like. Even more particularly, this invention relates to the integration of formerly separate functions into single devices such as those sometimes referred to as MFPs (Multi-Functional-Peripherals), and the integration of multiple functions into a single ASIC (Application Specific Integrated Circuit) such as those produced by Oasis Semiconductor, Inc. of Waltham, Mass.

[0004] This invention relates to a novel system to reduce the amount of digital computation required when performing color image processing on images acquired from scan systems based on color CIS (Contact Image Sensor) scanner devices.

[0005] To scan a document with a CIS device, the document is passed across the face of the CIS device (or, alternatively, the CIS device is passed across the document which is being scanned). The document is thus typically in contact with the CIS device. To scan an 8½″×11″ paper formats, for instance, a CIS device with an image sensor size of at least 8½″ is required.

[0006] The CIS device senses a document's scanlines, one scanline at a time, as the document passes across the face of the CIS device. The scanning process can thus be thought of (somewhat simplistically) as a sequence of repeated operations in which (i) the CIS device senses a scanline of the document and then (ii) the document is moved by a small amount across the face of the CIS.

[0007] The CIS device itself is an integrated unit consisting of a linear image sensor and a light source. For each scanline, the CIS device (i) illuminates a section of the document during a pulse period with a light source; (ii) collects the reflected light in its linear sensor array, and (iii) outputs a series of voltages representing the sensed light intensities. A computer chip collects this output and converts the data into a form suitable for performing image processing.

[0008] A color CIS device typically contains three different light sources which emit red, green and blue light, respectively. When performing color image sensing, the color CIS device is typically configured to illuminate and sense each document scanline position with three independent pulses of light—once with each of the three light sources. In this way, the red, green and blue tri-stimulus values can be obtained for each image pixel.

[0009] The CIS light sources are provided by arrays of red, green, and blue LEDs (Light Emitting Diodes). The LED arrays are typically configured in such a way that they have a common anode connection, but separate cathode connections, to the CIS device pins. This configuration allows the current which is passed through each of the LED arrays to be controlled independently, and thus, indirectly, allows independent control of the red, green, and blue light intensity.

[0010] Most scanner systems allow calibration of the red, green, and blue illumination used for each image scanline. For instance, calibration may be performed to obtain the optimal signal voltage from the CIS device. This is typically performed prior to carrying out a scan. Tuning of the illumination values is typically performed by adjusting the amount of time, during each image scanline time, for which current flows through the LED arrays. This is commonly referred to as tuning the LED on-times. The on-times of the red, green and blue arrays are typically calibrated independently, as the characteristics of the red, green, and blue LED arrays tend to differ.

[0011] During a first pulse for any particular scanline, the red LEDs are turned on for the optimal on-time (Tr) and the reflected light is collected by the sensor array for the entire pulse period (see FIG. 1). Then a second pulse for that scanline is begun with the green LED for a time (Tg) and the reflected light collected. Finally, a third pulse is carried out wherein the blue LEDs are turned on for a time (Tb), and the reflected light is collected by the sensor. The order in which the different colored LEDs are turned on is not significant.

[0012] When performing color image processing, it is advantageous to operate on color images represented in a luminance/chrominance color space rather than RGB (Red-Green-Blue) space for certain image processing outputs.

[0013] Examples of such luminance/chrominance color space are: YUV (Y—luminance, UV—color information), which is the basic color space of PAL, NTSC, and SECAM composite color video standards; YIQ (derived from YUV, where I is “in-phase” and Q is “quadrature”), which is also associated with the NTSC color video standard; YDbDr (also associated with SECAM); YCbCr (i.e., Luminance; Chroma:Blue; Chroma:Red), typically associated with digital video, from Recommendation ITU-R BT.601 (formerly CCIR 601)); Xerox/YES; PhotoYCC® (Eastman Kodak) and CIELAB.

[0014] Examples of image processing outputs for which it may be advantageous to work in the luminance/chrominance color space include, but are not limited to, standalone color copies and compression to the JPEG file format. Within MFPs, images in JPEG format are used, for instance, for scans to PC, for scans outputted directly to email, for color facsimile, for image archival onto a memory card, or for temporary image storage during collated copy operations.

[0015] Thus, before such image processing is carried out, it is generally necessary for the chip to convert the data obtained from the RGB space of the image sensor into the desired luminance/chrominance color space.

[0016] The traditional method of performing the conversion of RGB space to luminance/chrominance color space is to have the chip perform a set of computations that use digital representations of the RGB values as input.

[0017] To convert to YCbCr, for instance, a chip or firmware might perform the following set of computations:
Y=0.29900*R+0.58700*G+0.11400*B
Cb=−0.16874*R−0.33126*G+0.50000*B
Cr=0.50000*R−0.41869*G−0.08131*B

[0018] Alternatively, the chip may use a 3-D table lookup with interpolation to map from RGB space to the luminance/chrominance color space.

[0019] It would, therefore, be advantageous to reduce the computational load on a chip in converting from an RGB color space to a luminance/chrominance color space.

[0020] The present invention solves this problem by providing a method that reduces the amount of digital computation required when performing color image processing on images acquired from scan systems based on color CIS (contact image sensor) devices that requires the use of luminance/chrominance color space.

[0021] In one preferred form of the invention, there is provided a method of directly obtaining a Y value for a luminance/chrominance representation of a scanline of a color scanned image produced by a color contact image sensor device having a linear sensor array to collect reflected light, wherein said contact image sensor device comprises red, green and blue light sources and wherein an optimal on time for each of said light sources is calibrated, said method comprising the step of illuminating said scanline during a single pulse independently with said red, said green and said blue light sources and allowing the linear sensor array to accumulate the reflected light from all of said light sources, wherein:

[0022] illumination with said red light source is for a first fraction of the optimal on time determined for said red light source, wherein said first fraction is derived from a known relationship between Y and said red light source;

[0023] illumination with said green light source is for a second fraction of the optimal on time determined for said green light source, wherein said second fraction is derived from a known relationship between Y and said green light source; and

[0024] illumination with said blue light source is for a third fraction of the optimal on time determined for said blue light source, wherein said third fraction is derived from a known relationship between Y and said blue light source.

[0025] In another preferred form of the invention, there is provided a method of obtaining a chrominance value for a luminance/chrominance representation of a scanline of a scanned image produced by a color contact image sensor device having a linear sensor array to collect reflected light, wherein said contact image sensor device comprises red, green and blue light sources and wherein an optimal on time for each of said light sources is calibrated, comprising the steps of:

[0026] directly obtaining a luminance value (Y) of said luminance/chrominance representation using the method described above.

[0027] obtaining a R value by illuminating said scanline with said red light source for the optimal on time and collecting said reflected light;

[0028] obtaining a B value by illuminating said scanline with said blue light source for the optimal on time and collecting said reflected light; and

[0029] deriving said chrominance value from said luminance value and one of said R or B values based upon known relationships between all of said values.

[0030] In another preferred form of the invention, there is provided a method of producing an output of a scanned color image of a document in a luminance/chrominance representation using a color contact image sensor device having a linear sensor array to collect reflected light, wherein said contact image sensor device comprises red, green and blue light sources and wherein the optimal on time for each of said light sources is calibrated, said method comprising the steps of:

[0031] obtaining a Y value for each scanline of said document using the method described above;

[0032] obtaining a R value for each scanline by illuminating each scanline of said document with said red light source for the optimal on time determined for said red light source and detecting the reflected light in said linear sensor array;

[0033] obtaining a B value for each scanline of said document by illuminating each scanline with said blue light source for the optimal on time determined for said blue light source and detecting the reflected light in said linear sensor array;

[0034] deriving a first chrominance value of said luminance/chrominance representation for each scanline from said Y value and said R value for said scanline; and

[0035] deriving a second chrominance value of said luminance/chrominance representation for each scanline from said Y value and B value for said scanline.

[0036] In another preferred form of the invention, there is provided a computer chip programmed to control the production of a chrominance/luminance representation of a scanned color image of a document through the use of a color contact image sensor device, wherein said color contact image sensor device comprises (i) a linear sensor array to collect reflected light, and (ii) red, green and blue light sources; wherein said chip controls:

[0037] the calibration of an optimal on time for each of said red, green and blue light sources;

[0038] determining a Y value for each scanline of said document using the method described above;

[0039] determining a R value for each scanline by illuminating each scanline of said document with said red light source for the optimal on time determined for said red light source and detecting the reflected light in said linear sensor array;

[0040] determining a B value for each scanline of said document by illuminating each scanline with said blue light source for the optimal on time determined for said blue light source and detecting the reflected light in said linear sensor array;

[0041] determining a first chrominance value of said luminance/chrominance representation for each scanline from said Y value and said R value for said scanline;

[0042] determining a second chrominance value of said luminance/chrominance representation for each scanline from said Y value and B value for said scanline; and

[0043] inputting said Y, first chrominance and second chrominance values for each scanline into an image processing path to produce said scanned color image.

[0044] In another preferred form of the invention, there is provided a method of producing an output of a scanned color image of a document in a luminance/chrominance representation using a color contact image sensor device having a linear sensor array to collect reflected light, wherein said contact image sensor device comprises red, green and blue light sources and wherein the optimal on time for each of said light sources is calibrated, said method comprising the steps of:

[0045] obtaining a Y value for each scanline of said document using the method described above;

[0046] obtaining a D value for each scanline by illuminating each scanline of said document with one or more of said red, green and blue light sources for the optimal on time determined for one or more of said light sources and detecting the reflected light in said linear sensor array;

[0047] obtaining a W value for each scanline of said document by illuminating each scanline with one or more of said red, green and blue light sources for the optimal on time determined for one or more of said light sources and detecting the reflected light in said linear sensor array;

[0048] deriving a first chrominance value of said luminance/chrominance representation for each scanline from two or more of said Y, D and W values for said scanline; and

[0049] deriving a second chrominance value of said luminance/chrominance representation for each scanline from two or more of said Y, D and W values for said scanline.

[0056] The present invention provides a system for reducing the computational complexity of converting to luminance/chrominance color space. Fundamental to this invention is taking advantage of the ability of the CIS device to accumulate, in one sensing operation, intensities resulting from periods of red, green and blue illumination. This allows for a direct determination of the luminance (Y) value in any luminance/chrominance color space where the luminance and chrominance components are simple linear functions of the RGB tri-stimulus values, and the luminance scaling coefficients are positive. These include, but are not limited to YUV, YIQ, YDbDr, YCbCr, and/or Xerox/YES.

[0057] By way of example, a particular image processing system might require a YCbCr representation of a scanned image. Looking at the above equations relating YCbCr to RGB, it is apparent that the Y component is formed as the sum of weighted contributions from red, green and blue illumination. Thus, it is possible to obtain a representation of Y directly from the CIS device by arranging the red, green and blue LED on-times during a single pulse to be representative of the red, green and blue contributions to Y. During each of the on-times, the sensor accumulates the reflected light (as voltages) from the different color LEDs. The sum of those reflected light voltages is the value for Y.

[0058] More particularly, assume that the calibrated red, green and blue LED array on-times are Tr, Tg and Tb, respectively. The on-time required to create Y from the CIS device directly is thus as follows:
Ty,r=0.29900*Tr
Ty,g=0.58700*Tg
Ty,b=0.11400*Tb

[0059] The on-times required to create Y from the CIS device in other luminance/chrominance color spaces can similarly be calculated based upon the well-known relationship between Y and R, G and B in those other spaces.

[0060] Unfortunately, Cr (Chroma:Red) and Cb (Chroma:Blue) cannot be synthesized in a similar manner as Y because, in some cases, the R (Red), G (Green) and B (Blue) contributions are subtractive. Thus, instead of obtaining Cr and Cb, the standard R and B components from the CIS device are used. The sequence of LED on-times to achieve this selection of components RYB (Red-Yellow-Blue) is illustrated in FIG. 2.

[0061] After obtaining the standard R and B components from the CIS device, the pixels are now in a hybrid RYB color space. Significantly, the hybrid RYB color space can easily be converted to a YCbCr color space. With minor modifications, the equations mapping RGB to YCbCr can be rearranged to yield Cr and Cb as a function of RYB:
Cr=0.71327*(R−Y)
Cb=0.56433*(B−Y)

[0062] Thus, by changing the usage of the CIS device to obtain RYB rather than RGB, the computations required to obtain luminance/chrominance representation of pixels (i.e., the YCbCr color space in our example) are dramatically reduced.

[0063] For many image processing tasks, using a strictly defined luminance/chrominance color space such as the YCbCr color space in the foregoing example is not essential. For instance, an alternate luminance/chrominance color space can be defined which uses the same equation for luminance as the YCbCr color space but for which only subtract and shift operations are required in calculating the chrominance components, as follows:
Cr′=(R−Y)>>1
Cb′=(B−Y)>>1

[0064] This alternate luminance/chrominance color space preserves the characteristic of the YCbCr color space that neutral (i.e., gray only) tones occur when Cr=Cb=0. Significantly, these alternate Cr′ and Cb′ values are much simpler to compute because multiplication is no longer needed.

[0065] In many cases the image processing path requires the use of a 3-D color lookup table to convert from the image processing input color space to the output color space. This method is generally used to convert from the image processing input color space to the CMYK (Cyan Magenta Yellow blacK) color space which is typical of many printing devices. In one preferred embodiment of the present invention, the image processing path uses a 3-D color lookup table. In this embodiment, the chip calculates the simpler Cr′ and Cb′ values and performs image processing based on YCb′Cr′ as input. To correctly map to CMYK, the 3-D color lookup table used during the color space conversion step is simply configured to map from YCb′Cr′ to CMYK rather than from YCbCr to CMYK.

[0066] It should be appreciated that other luminance/chrominance systems can be utilized by modifying the calculations presented above. Such modifications will be readily apparent to one of ordinary skill in the art based upon known relationships between the chrominance and luminance value in any given system and between the chrominance and luminance values in such a system and R, G and B values.

[0067] It should further be appreciated that it may be advantageous to configure the CIS device to acquire samples which represent different linear combinations of R, G, and B tri-stimulus values than the RYB (Red-Luminance-Blue) components indicated in the above example. For instance, depending on the target luminance/chrominance system, it may be computationally more efficient to obtain GYB (Green-Luminance-Blue) components, RYG (Red-Luminance-Green) components, or even components of the form XYZ, where X and Z (like Y) represent linear combinations of the R, G, and B tri-stimulus values.