Photoelectric Color Sensor for Article Sorting

The present invention relates to an apparatus suitable for machine vision applications. Specifically, the present invention relates to sensing systems which can receive and process color information for inspection purposes. More particularly still, the present invention relates to an apparatus for determining the correctness of a target object's color.

The ability to quantify and reliably identify color has proven to be illusive. The perception of color has historically proved to be highly subjective. The matter is further complicated by the existence of several ways to specify color. For example, manufacturers of color monitors, video cameras and computer graphics often define color as a combination of the primary colors--red, green and blue (RGB). These additive primaries can be mixed in any combination to produce millions of colors in the visible spectrum. Another traditional color specification method, popular in the publishing industry for mixing inks, is based on combining the subtractive primaries--cyan, yellow, magenta and black (CYMK). Television broadcasting represents color in yet another method; RGB signals are encoded in luminescence (Y) and chrominance (I&Q) signals to facilitate broadcasting.

Recent advancements in optics and microelectronics is tending to expand color imaging into many fields including manufacturing process control, robotics and the like. Today, color is a very important factor in maintaining the overall quality of pharmaceuticals, food products and the like. Because the physical appearance of a product is frequently equated to a perceived if not actual standard of quality, it is incumbent upon a manufacturer to maintain consistency from unit to unit as well as to maintain adherence to specified product standards. By incorporating a color discerning sensor on a line as part of the manufacturing/production process, much more product can be monitored than heretofore was available with recognized off-line measurement techniques, ensuring that only acceptable product is produced. Subjective judgments as to appearance are no longer required whereby continuous product consistency can be achieved.

Although the foregoing concepts are widely recognized and accepted in principle, currently available color sensor technology has a number of deficiencies, limiting the applications thereof in many areas. Present sensors are typically, extremely expensive, bulky and are sensitive to ambient light changes and heat. As a result, sensors are typically employed off-line. Sensors which are used on-line frequently have their sensing heads and processing logic separated by bundles of fiber optic cables wherein the head is positioned to contact or be closely spaced from the target object and the processing portion of the device is physically remote to protect it from the manufacturing process. In the food industry, the problem is reversed in that the goods being produced are extremely susceptible to outside sources of heat and real estate on the production line tends to be extremely limited.

A further limitation of many prior art color sensing devices resides in their extremely short lifetime, requiring frequent shutting down of the production process to replace the sensor's light source or other failed components. Furthermore, because widely accepted prior art approaches at processing color information tend to be laborious, system response time suffers whereby such devices are not suitable for many applications requiring high speed processing.

Although color sensors are frequently employed with powerful data processors, they frequently are incapable of learning; that is, to compare color component information with prior process history to reprogram the system to adjust tolerances, anticipate changing ambient conditions and the like. To the contrary, prior art devices typically compare a simple measured signal with a derived fixed standard. Lastly, such devices also typically have a very limited sensitivity and, although able to differentiate one pure color from another, are incapable of discriminating between slightly varying hues of the same color.

The present invention provides a new and improved apparatus for rapidly determining the color of a target object as well as differentiating changes in hue of the target color extremely rapidly and cost effectively. The apparatus is packaged for convenient on-line application adjacent the product flow. Furthermore, the present inventive apparatus is adaptive, able to learn and remember the color of prior sensed target objects and to adapt to changing conditions.

The apparatus of the present invention comprises at least two sources of light disposed in a fixed array and focused upon a sensing region. Each source is operable to emit light having a limited characteristic range of chromaticity which differs from that of the other range or ranges. Photosensitive means are positionally fixed with respect to the array of light sources to receive light from each source which has been reflected off of a target object within the sensing region and to output a signal in response thereto. Logic means are connected in circuit with the light sources and photosensitive means which is operative to sequentially energize each light source and to receive resultant sample signals from the photosensitive means in response to the light source energization. The logic means is further operative to generate a resultant signal as a function of the sample signals, to compare the resultant signal with a predetermined reference standard and to generate a condition output signal when the difference between the resultant signal and reference standard exceeds a tolerance limit. This arrangement has the advantage of providing a compact assemblage of inexpensive components with simple yet adaptive local processing in a configuration well suited for on-line manufacturing process control. Furthermore, this arrangement provides for serial processing of multiple signals of differing frequency ranges which are subsequently combined for enhanced logical processing speed and reliability.

According to another aspect of the invention the light sources comprise red, green and blue LEDs disposed in a common plane with their mutual focal point within the target sensing region and the photosensitive means comprise at least one phototransistor disposed in a plane common with the central blue LED and the focal point normal to the LED plane of arrangement. This arrangement ensures that the line of travel from each LED to the target and back to the phototransistor is the same.

According to another aspect of the invention, the photosensitive means includes two phototransistors which straddle the center (blue) LED. This arrangement permits differential processing to offset errors induced by variances in target position between successive targets.

According to still another aspect of the invention, the logic means includes first and second amplifiers each of which are operative to generate a respective element signal in response to the light source energization. A third amplifier is provided to output the aforesaid sample signals as a function of both the first and second element signal. This arrangement provides the differential processing discussed hereinabove. Further, the logic means comprises a circuit operative to drain electrical current from the base junction of the phototransistors whenever all of the light emitting diodes are deenergized to offset the effect of ambient induced photocurrent in the phototransistors. This arrangement has the advantage of providing for ambient light compensation without affecting the speed or responsiveness of the color sensor.

These and other features and advantages of the present invention will become apparent upon reading the following specification, which, along with the patent drawings, describes and discloses a preferred embodiment of the invention in detail.

A detailed description of the embodiment of the invention makes reference to the accompanying drawings.

FIG. 1, is a top plan view of a sensor head employed in the preferred embodiment of the present invention;

FIG. 2, is a cross sectional side view of the sensor head taken on lines 2--2 of FIG. 1;

FIG. 3, is a cross sectional bottom view of the sensor head taken on lines 3--3 of FIG. 1;

FIG. 4, is a signal timing chart correlating various relationships within the present invention;

FIG. 5, is a schematic diagram of the preferred embodiment of the present inventive photoelectric color sensor;

FIG. 6, is a schematic diagram of the LED driver portion of the schematic of FIG. 5;

FIG. 7, is a simplified flow chart of software implemented in the microprocessor of the present invention;

FIG. 8, is a graphical correlation of the wave length/relative output of the various light sources employed in the present invention; and

FIG. 9, is a graphical presentation of experimentally derived color sensor output versus distance (from target) data.

Referring the drawing figures, a photoelectric color sensor 10 is illustrated which, in its intended application, is to compare the color of both ends of a pharmaceutical capsule 12 to the desired color for a particular product. Capsules 12 move past the sensor 10 on a conveyor line indicated generally by arrow 14 as they are inspected. As will be described in detail hereinbelow, sensor 10 performs a go, no-go comparison of sensed color to stored color data. Two sets of color data corresponding to the opposite ends of the capsule 12 are stored so that the color of both capsule ends can be inspected. The sensor 10 provides a reject signal if either end of the capsule 12 does not match one of the stored colors. Logic is provided to accept the capsule with either color first in the sequence. When the first color is found to be one of the acceptable colors, the second color must correspond to the remaining reference color. Sensor 10 is provided with a learn mode activated manually by a learn/run switch wherein color data from a reference capsule presented to the sensor in the learn mode is stored. In application, a succession of capsules 12 is presented to sensor 10 for color identification at a fixed vertical distance from the sensor head 16. In practice, sensor 10 can accommodate a lateral displacement of capsule 12 by an amount equal to plus or minus 10% of the capsule width as indicated in phantom in FIG. 2.

When implemented with componentry of the values specified hereinbelow, sensor 10 can sample and identify the color of both ends of a capsule within 3 ms. For allocating the time during the identification cycle, it is assumed that the 3 ms interval between capsules 12 occurs over a spatial distance of two capsule lengths, i.e. only half of the space is actually occupied by capsules. Sensor head 16 includes a shield (not illustrated) to minimize the intrusion of ambient light. However, as shall be described in detail hereinbelow, sensor 10 can accommodate residual ambient light from normal room lighting that may enter the target area or sensing region, shown generally at 18, beneath the edges of the light shield. It is contemplated that sensor 10 shall receive a marker signal from a host system to indicate the presence of a capsule 12. Sensor 10 makes provision for a second marker signal to provide a reference mark for the middle of capsule 12. Sensor 10 also makes provision for generating an internal time reference for the middle of the capsule based upon uniform line speed and timed from the first marker. Sensor 10 provides a solid state N-type reject output usable with a 10-30 VDC N-type input. The output of sensor 10 may be short-circuit protected. Sensor 10 makes quantitative measurements of red, blue, and green light reflection from the target capsule 12 when illuminated with suitable light sources. The color identification algorithm shall compare this color data using suitable tolerances, to stored color data obtained during the learn mode. Sensor 10 includes a logic circuit 20 employing an eight-bit single chip microprocessor 22 with internal A-D converters 24, and accepts the aforesaid marker signals, generates the timing cycle for internal sensor functions, provides set-up modes, captures and processes color data and provides a reject output signal. It is contemplated that suitable function and output indicators will be provided as well as an appropriate source of power. For the sake of simplicity, a number of above described incidental or peripheral circuit elements are not described in detail herein, it being understood that such functions are well understood by those of ordinary skill in the art and that suitable componentry is commercially available.

Referring to FIGS. 1-3, sensor head 16 is generally dish-shaped and formed of an opaque material. Red, blue and green light emitting diodes (hereafter LEDs) are received within through bores 32, 34 and 36, respectively, within sensor head 16. When energized, LEDs 26, 28 and 30 will emit light along a principle axis indicated by arrows 38, 40 and 42, mutually converging upon a focal point upon the surface of capsule 12 within sensing region 18. Likewise, two phototransistors 44 and 46 are received within through bores 48 and 50, respectively. Phototransistors 44 and 46 are each positioned whereby its light receiving axis 52 and 54, respectively, are inwardly converging upon the focal point of LEDs 26, 28 and 30. The axes of symmetry of LEDs 26, 28 and 30, axes 38, 40 and 42 as well as their focal point fall within a fixed plane generally corresponding with the section cut line 3--3 of FIG. 1. Likewise, LED 28, phototransistors 44 and 46 as well as their respective axes 40, 52 and 50 fall on a common plane which is normal to the plane of LEDs 26, 28 and 30. Accordingly, phototransistors 44 and 46 are positioned to be mutually equidistant from each of the LEDs 26, 28 and 30 along the line of light incidence and reflection off of capsule 12. The blue LED 28 is located in a favored position in the center of the head to compensate for lower light output capability than the red and green LEDs 26 and 30.

Referring to FIG. 4, LEDs 26, 28 and 30 are driven sequentially by logic circuit 20 to illuminate capsule 12 with blue, green and red light as illustrated. The reflected light from each color is detected by phototransistors 44 and 46 (see FIG. 5), amplified by operational amplifiers 56 and 58, respectively, and summed in operational amplifier 60. A potentiometer 62 has its fixed resistance portion across the outputs of op amps 56 and 58 and the wiper connected to the input of op amp 60 to balance sensitivity variations between phototransistors 44 and 46 Op amp 60 outputs resultant sample signals to A-D converter 24 within microprocessor 22. Microprocessor 22 generates timing pulses for the red, blue and green LEDs 26, 28 and 30, respectively, and in synchronism with each pulse samples the output of amplifier 60 and converts it to a digital value. These digital values for each color are stored in memory and may be processed with any of several algorithms to determine the color of the target being viewed. One such known algorithm or conversion relationships are known as Munsell equations which describes an exact color by way of a vector on a circle with its origin at the center of the circle. Alternatively, the red, green and blue values may be compared to a stored set of colors obtained by viewing a reference color during a "learn" mode at set-up. This is functionally described in FIG. 7. For this mode, a second potentiometer 64 is connected to a second A-D converter 24 channel of microprocessor 22 which is proportional to the tolerance to be used when comparing the red, green and blue signals to a set of stored reference values. As discussed hereinabove, more than one set of color reference values can be stored for use in, for example, sequentially checking the color of both ends of a pharmaceutical capsule 12.

Red, green and blue pulses from microprocessor 22 drive LED drivers 66 as shown in FIG. 6 which can be adjusted to individually control the red, green and blue LED currents through use of a digitally controlled gain circuit established by the microprocessor 22 and programmable feedback resistance in the final gain stage amplifier circuit 92 (FIG. 5). These are adjusted to provide equal red, green and blue pulse amplitudes out of amplifier 60 when a white target is viewed. This adjustment compensates both for variation in light output versus current among the LEDs and for variations in phototransistor 44 and 46 over the spectral range. An additional drive stage may be added to the blue LED 28 to provide .+-.12 volt bias for enhanced output.

Sensor 10 is made substantially insensitive to steady state ambient light through the provision of ambient compensating loops consisting of operational transconductance amplifiers 68 and 70, and capacitors 72 and 74. Operational transconductance amplifiers 68 and 70 function as amplifiers only when enabled by the ambient compensation enable (EAC) signals shown in FIG. 4, which are true whenever none of the red, blue and green LEDs 26, 28 and 30, respectively, are energized. Otherwise, the output of amplifiers 68 and 70 are an open circuit which does not provide a discharge path for the voltage stored in capacitors 72 and 74. When amplifiers 68 and 70 are energized, a voltage is stored on capacitors 72 and 74 which withdraws current from the base of the respective phototransistor 44 and 46 to compensate for ambient induced photocurrent in phototransistors 44 and 46. Capacitors 72 and 74 are large enough to retain this voltage during the signal measurement intervals when amplifiers 68 and 70 are not energized. The lowermost signal trace in FIG. 4 represent the resultant sample (RS) signals received by microprocessor 22 from amplifier 60.

The balance of the componentry illustrated in FIGS. 5 and 6 are listed hereinbelow, it being understood that they represent only one of any number of variants upon the represent invention inventive concept.

Referring to FIG. 8, the relative light output amplitude versus frequency characteristic derived for two blue, two green and one red commercially available LEDs is presented to illustrate that each diode in the red, green and blue categories have their own characteristic limited range of chromaticity which substantially differs from the others and that collectively they substantially encompass the entire visible light spectrum.

Referring to FIG. 9, the experimentally derived relationship of output versus distance of the target capsule 12 from the focal point is illustrated to highlight the color to color registration of sensitivities within the present invention.

Referring to FIG. 7, the basic logic program within microprocessor 22 is illustrated. The single-chip microprocessor 22 adds intelligence and versatility to the color sensing system 10 in a way that would be difficult to achieve by discrete means such as through programmable logic devices. The microprocessor chip 22 performs three general tasks in the operation of the color sensor 10. (1) it maintains control of the emitting and receiving photodiodes 26, 28, 30, 44 and 46, (2) it provides input analysis and response to the collected sensor data, (3) it manages local and remote communication interfaces. These functions are primarily originated by software driven inputs and outputs from the microprocessor chip 22. Each of these functions are expounded upon below.

The first sensor control task (1) is provided by software algorithms which switch the red, green and blue LEDs 26, 30 and 28 on and off in a proper order in conjunction with the inversion of the autozero signal. Routines are also available for a programmable gain control on the receiving input amplifier, thus allowing for self calibration, in order to adjust to target applications of varying reflective gains. The routine attempts to achieve a linear but maximal gain for all three hues without driving the receiving amplifier into saturation for any individual hue. Circuitry and software is in place to provide an interrupt mechanism which permits triggering of the sensor readings based upon target presence or relative position in the sensor field of view. Initiation of this triggering mechanism can be accomplished through remote programming or by on-board dip switches. There are several pushbuttons on the sensor that are interpreted by the microprocessor. The mode button is a multipurpose control switch for progressing the sensor operation from a learn-color mode to the normal run mode of operation. In particular, the first push of the mode button out of reset will learn the color in the sensor view, another push will cause it to learn the second color if the device has been previously configured for more than one color. After learning colors the next push puts it in the tolerance setting mode, and the next would put it in the normal run mode. If the mode button is pushed in the run mode, the device again returns to the tolerance set mode.

The second task (2) of the microcomputer chip 22 is for data analysis. After measuring the analog input and converting it to digital information, one of several paths can be taken to analyze the data, based on user preconfigured settings. The input readings may or may not be averaged, the choice being determined by how much response speed versus noisy signal suppression that is desired. Averaging slows the response time of the device but increases repeatability in the color determination. The readings may or may not be normalized, that is where each individual hue or LED light reflection reading is expressed as a ratio of the total of all three hue readings. The chief consequence of dealing with absolute rather than normalized voltage values for hue is that it renders the analysis more susceptible to variations in intensity of the reflected light due to sensor-to-target distance variation; this is not a problem when normalized. However, if the intensity of the reflected light can be held relatively constant, absolute readings will distinguish different tones of the same color hue, while normalizing masks the distinction. One-color or multi-color sensing can be selected by dip switch or remote terminal programming as well. In the current design, the output of the color analysis is made up of two go/no-go control lines which provide a go/no-go signal. One line fires high for a passing color, the other for a failure of the color match to the stored standard. Each line has a corresponding LED indicator for demonstration purposes. When there is a match to any stored color, the go-signal pin fires. Plug-in external sensors may serve as marker inputs to distinguish which color is being tested at a given time (mentioned above as part of the triggering mechanism.

The third task (3) incorporates the elements of the user interface. Simple interface elements are provided such as the reset button, which restarts the device and sets it in the learn color mode. The second button is the mode button, which as described above, progresses the device operation sequentially through the available modes. LED indicators under the control of the microprocessor, distinguish the state of the device operation. These indicators are the learn versus run mode, color good versus bad, learn color versus tolerance set mode, first color or second color learn, and power-on LEDs. User defined set-up configurations can be entered in two ways. The operating configuration can be set by either switching a bank of dip switches located on the sensor (local) or by serially communicating to the sensor from a (remote) personal computer or dumb ASCII terminal. Serial communication software is available in the microcomputer chip coding. Serial communication can be initiated during the run mode which will return the operating state to the learn mode.

It is to be understood that the invention has been described with reference to a specific embodiment to provide the features and advantages previously described and that such specific embodiments are susceptible of modification, such as will be apparent to those skilled in the art. For example, the preferred embodiment of the invention represents an application specific design for a particular shape target object as well as a specific set of colors and hues. Furthermore feedback resistor 92 can be replaced by a potentiometer or a programmable resistor to selectively adjust the gain of amplifier 60. Also any number of color standards can be learned and stored and any number of readings can be averaged. It is contemplated that the above recited component values and arrangements could be varied without departing from the spirit of this invention. Accordingly, the foregoing is not to be construed in a limiting sense.