Color Signature Sensor

This invention is directed to a field of color signature sensor used in process automation which performs color recognition of objects for the purposes of identification, sorting or matching.

In this invention reflected (or transmitted) light from an illuminated object (or process) is transmitted optically to a spectral dispersive element in the form of a concave diffraction grating. Spectrally dispersed light from the grating is focussed on a photodetector array which generates a spatially discretized analog signal representing the color signature of the object. The signal is then converted to digital form and sent to a microprocessor for processing. The sampled and digitized signal is a unique and complete description of the colored object being observed. Cross-correlation and mean-value statistics are used by the microprocessor for color recognition.

FIG. 1 is a partial block diagram of an embodiment of the invention.

FIG. 2 is a typical spectral signature of standard blue and standard red color chips, and FIGS. 3 and 4 show sampled values comprising the analog signal for the blue and red colors, respectively.

FIG. 5 is a flow chart of the operation of the system shown in FIG. 1.

Referring now to the apparatus shown in FIG. 1 there is shown an illumination source (I) or lamp 10 for properly illuminating an object 11 to be observed. Object 11 may be, for example, individual piece parts moving along a conveyor belt of a production line in which it is needful to compare the color of each part against the color of a reference part. Color perception depends on the spectral energy density of the illumination source 10, the spectral reflectance (or transmittance) distribution of the object 11 and the spectral responsivity characteristics of the signal acquisition (the optics portion) system herein described.

Diffusely reflected (or transmitted) light from an illuminated object, such as 11, is collected by a fiber optic probe 12 that routes the light signal to the input of a monochromator shown in the form of a spectral dispersive element 13. The fiber optic path is exemplary only and the light from the object may be otherwise optically directed to the spectral dispersive element. Additional optics may be located at the input end 14 of the fiber to affect signal collection efficiency, angle-of-view, and the like. Similarly, additional optics may be located at the output end 15 of the fiber to affect better spot-size matching or far-field divergence angle matching to the spectral dispersive element 13.

The spectral dispersive element 13 may consist of a blazed concave diffraction grating as shown, or may consist of other spectral dispersive configurations (generally called monochromator) that are known to those skilled in optics. The essential function of concave diffraction grating 13 is to image the fiber output face 15 onto the input face of a detector array 16 and to disperse it spectrally (that is, to angularly decompose the input light distribution into the continuous spectrum of pure colors, each described by a specific wavelength) over the spatial extent of the acceptance aperture of array 16. The spatial characteristics of the array and of the individual photodiodes of the array, as well as its physical alignment with respect to 15 and 13, are chosen such that the spectrally dispersed signal of interest falls within the acceptance aperture of the array. The output of each photodiode in the array corresponds to the time- and space-integrated intensity of a specific band of pure color wavelengths in the light from object 11. This spectrally dispersed light distribution is the color signature L(x)=L'(.lambda.) of the illuminated object where .lambda. represents wavelength of light and x represents the spatial extension of the array. For colorimetry applications L'(.lambda.) extends from .lambda..congruent.400 nm to .lambda..congruent.700 nm; for other applications, the range of .lambda. is not restricted to the visible band thus permitting the inclusion of invisible wavelength bands for various additional applications such as counterfeit detection. The photodetector array may be similar, for example, to the EG&G RETICON solid state line scanners RL128G, the Honeywell TCL chip, or the like. This detector array 16 consists of a linear sequence of photodetectors that detects the light distribution L(x)=L'(.lambda.) and converts it to a time-dependent electrical current I(t) on wiring 17 that is clocked out in response to timing signals from the microcontroller/microprocessor 18. The array is scanned by the processor once every five to twenty milliseconds depending on the application with 16.67 milliseconds being one of the preferred intervals. A typical spectral distribution from a standard red and a standard blue color chip is seen in FIG. 2. In general, the signal I(t) on wiring 17 is an aperiodic, analog current that represents the signal data from the object 11 acquired by the optics hardware described above. Under static object illuminations, I(t) becomes periodic with successive scans and represents the acquired color signature. Multiple samples of the color signature of the same object under the same illumination can be used to reduce the noise present in each signature alone.

The signal I(t) first flows into a sample-and-hold analog-to-digital converter 19 that forms a digital word description sequence of the colored object 11. The digitized output from 19 is connected by wiring 20 to microprocessor 18. The microprocessor also interfaces with a system memory 21 by wiring 22 and also with an input-output (I/O) buffer device 23 by wiring 24. The processor can remember a number (such as eight) of color signatures each of which uniquely represents (eight) sample/reference colors. Sample/reference colors can be trained by putting a sample/reference object in the sensor's field-of-view, selecting a channel with a switch and pressing a "train" button. The color signature of the sample/reference object is then stored in memory. Data processing can be free-running at an exemplary speed of approximately sixty-eight spectrum-comparing operations being performed per second. The match/no-match outputs between incoming signature data and memory signature data can be updated after each compare operation or they can be latched by a higher level controller or external trigger when an object is properly in the sensor's field-of-view and thus when the outputs are valid. Under higher level control the color signatures can be trained, dumped or loaded through a serial port to facilitate day-to-day or product-to-product reprogramming and off-line training. For convenience the digitized signature output at 20 can be described as an m.times.n matrix I.sub.mn, where m is the number of photodetectors (or pure color sample bands) in the array and n is the bit precision of the digitization process, that is, n may be thought of as the number of bits resolved in the analog to digital converter. As an example m may be about 24-32 and n may be about 8-12. Generally m and n can be traded off with one another to meet various performance and cost constraints. The uniqueness and completeness of I.sub.mn as a signature description of a colored object is preserved in this digitization process provided m and n are chosen appropriately. All systematic aberrations occurring in the optical fiber 12 from the fiber input 14 to the fiber output 15, the diffraction grating 13, and the detector array 16 are incorporated implicitly in I.sub.mn. It is necessary to account for these aberrations only if the sensor output is an NBS-traceable color determination. Color recognition and discrimination outputs do not require aberration correction or compensation because all the acquired color signatures contain the same systematic aberrations. Any time-dependent or random system variations can be accounted for by acquiring cross-correlation and mean-value statistics for sequential acquisitions of the color signature of a static object. These statistics give what is equivalent to an autocorrelation function for this system (that is, system reproducibility and system error). This autocorrelation function essentially determines the color discrimination threshold for any two or more samples as described below. The simplest and probably the most useful data analysis scheme is to indicate whether the current color signature matches one or more reference signatures. Good success has been had using cross-correlation and mean difference formulae. The cross-correlation gives a statistical indication of "how good" the match (primarily in hue and chroma) is between two signatures, while the difference between the mean-values of each signature gives an indication of the difference primarily in brightness (or value).

The color sensor's basic programming is designed for process control monitoring for factory automation and on-line quality assurance and not for absolute color analysis. In many respects this mimics a human inspector in that the sensor can not readily describe the color that it is currently seeing, but by. comparing stored spectra to the current scene it can recognize very subtle color differences. The sensor's memory for colors and its speed, however, are far superior to the human eye and brain. Color recognition and discrimination are obtained according to the following description: Suppose the sampled and digitized signature from object 11 (sample A) is given as the sequence described collectively as ##EQU1## That is, ##EQU2## is a shorthand notation for the sequence of ordered numbers (a.sub.0, a.sub.1, a.sub.2 . . . a.sub.m-1). The a.sub.i 's are the weighted pure color values within the analog-to-digital converter 19 after sample-and-hold and analog-to-digital conversion. The a.sub.i 's represent the time- and space-integrated light values from each of the m photodetectors that constitute and collectively describe the object's color signature. That is, the color signature consists of an ordered sequence of weighted pure color bands with a one-to-one correlation of the color bands to the a.sub.i 's. This provides the common basis for measuring the similarity of one sample with respect to the next.

FIG. 3 shows an example of the ordered sequence of numbers ##EQU3## that represents the sample-and-hold values of the continuous color signature of the standard blue in FIG. 2. The choice of m=19 in FIG. 3 is exemplary only.

Similarly suppose the sampled and digitized signature from object 11 (sample B) is given as ##EQU4## The notation and meaning of I.sub.mn (B) is similar to that described for I.sub.mn (A), except that I.sub.mn (B) refers to sample B and I.sub.mn (A) refers to sample A. FIG. 4 shows the ordered sequence of numbers ##EQU5## that represents the sample-and-hold values of the continuous color signature of the standard red in FIG. 2. Again, the choice of m=19 in FIG. 3 is exemplary only.

The signature mean (or average) values are defined by ##EQU6## where .SIGMA. is the summation of m weighted values. Thus .eta..sub.a and .eta..sub.b may also be written

and

FIGS. 3 and 4 show the approximate values .eta..sub.a and .eta..sub.b corresponding to ##EQU7## for blue and ##EQU8## for red, respectively, in FIG. 2. The cross-correlation .psi. between I.sub.mn (A) and I.sub.mn (B) is given as ##EQU9## The numerator can be expanded explicitly as ##EQU10## and the denominator factors can be expanded explicitly as ##EQU11## The mean-value difference .phi. between I.sub.mn (A) and I.sub.mn (B) is given as

By definition, .psi.(A,A)=1; however, since nonzero noise sources are always present, .psi.(A,A')<1 where I.sub.mn (A') represents any sequential signature sample of A. Similarly, it is generally true that .PHI.(A,A').noteq.0. For convenience, we can take .psi.(A,A') and .PHI.(A,A') jointly to represent the total system error indicative of the system autocorrelation. (Similarly, .psi.(B,B') and .phi.(B,B') can be used as well in lieu of .psi.(A,A') and .phi.(A,A').) Thus, two or more colors can be discriminated provided

where .sigma..sub.cc and .sigma..sub.mvd are the discrimination thresholds for cross-correlation and mean-value difference, respectively. .sigma..sub.cc and .sigma..sub.mvd are small nonzero numbers. If

the sample colors of A and B are said to match.

It is noted that .psi.(A,A') and .PHI.(A,A') (or .psi.(B,B') and .PHI.(B,B')) may be insufficiently complete descriptions of the total system error, since many more samples of the object A (or B) are necessary in general. Similar arguments apply to .psi.(A,B) and .PHI.(A,B). Furthermore, it is well known in signal processing that averaging multiple signal acquisitions reduces the level of additive (random) noise and improves the signal-to-noise ratio of the final acquired signal. The signal-to-noise ratio improves in proportion to the square root of the number of waveforms averaged. Thus, for example, ##EQU12## where a.sub.i =(a.sub.i.sup.1 +a.sub.i.sup.2 + . . . +a.sub.i.sup.k)/k is the mean-value of a.sub.i over k samples, is a better indicator of I.sub.mn (A) in the determination of both .psi. and .PHI.. FIG. 5 shows in flow chart form the method steps performed by the aparatus which are described above.

If the optical hardware is designed and configured properly, color determination can be obtained straightforwardly from I.sub.mn of any sample. However, optical system imperfections (due to diffraction limitations, nonideal assembly or component performance, or performance/cost tradeoffs) give rise to distorted signals for L'(.lambda.) and I.sub.mn that cause errors in the color specification. In the terms of network theory, the system transfer characteristic is nonideal and "blurs" the system output. However, since the system impulse response for any spectrally dispersive system, as exemplified in FIG. 1, can be measured using a tunable, monochromatic illumination source over the wavelength band of interest, the optical system transfer characteristic can be determined. Within the spatial quantization error of the linear array (due to the limited number of photodiodes per unit distance x and the active/passive photodiode area ratio), it is possible to deconvolve the system transfer characteristic from the acquired signal to improve effectively the fidelity of the acquisition system. The spatial quantization error can be minimized by maximizing the effective lineal density of photodiodes in the array. This deconvolution concept can be implemented in a signal "deblurring" algorithm that permits use of relatively imprecise/low-cost components to achieve the performance obtainable in precise, high-cost spectrophotometric systems. In other words, this deconvolution concept enables the color signature sensor to emulate expensive, high-performance systems, in the function of color determination without many of the disadvantages (e.g. high cost, low speed, precise alignment and calibration) associated with these conventional systems.

In this invention it is thus recognized that detected color signature L'(.lambda.) is a characteristic and unique descriptor of the color of an illuminated object without need for additional processing or analysis such as is required, for example, for the L.sup.* a.sup.* b.sup.* specification. There is herein used a color discrimination algorithm based on signature cross-correlation and mean-value difference, without need for color determination as is conventional. There is also described the system deconvolution (or signature deblurring) to filter out or compensate for optical system aberrations. This allows the use of inexpensive system optics. Also there is shown use of system autocorrelation output to set recognition threshold for inter-signature cross-correlation and mean-value difference.