Image subtraction of illumination artifacts

[0001] This application claims the benefit of the priority date of co-pending U.S. Provisional Patent Application Ser. No. 60/538,914, which was filed on Jan. 23, 2004.

[0002] In image processing, the goal is to examine an image of an object in order to extract some useful information about that object. This is most often done by analyzing variations in the brightness value of spatially separated elements of the image and comparing them to one another or comparing them to a priori determined values.

[0003] It is not uncommon that there are brightness value variations within an image of an object that do not convey information about the object, but rather convey spurious information about the illumination system used to light the object or the imaging system used to optically form the image on the image sensor or about other aspects of the object's environment. When it is not possible to control the environment or the lighting or the optical system in such a way as to eliminate these artifacts, the system that analyzes the image to extract useful information must cope with these artifacts; that is the system must extract the salient features of the object from the image while ignoring spurious information from the artifacts.

[0004] In imaging systems with an in-line coaxial illumination device are well known in the art and were first widely used in biological microscopes. In general, in-line coaxial illumination devices introduce a light source either between the image plane and the lens, or within the lens itself between the optical elements. The elements of the lens between the light source and the object may be used to focus or collimate the light and cause the light to impinge the object along a complimentary path to the image-forming rays. This method can result in increased contrast of the image of the object and also in more compact assemblies as additional space between the lens and the object is not required to introduce the illumination.

[0005] The problem with imaging systems such as these is that a portion of the illumination may be reflected by the optical elements between the illumination source and the object in a direction back toward the image plane where the illumination may register as an image artifact. Traditionally, the response to this has been to use careful optical design practices, including baffling and antireflection coatings, to minimize the artifacts from this type of lighting.

[0006] Artifacts due to in-line coaxial illumination are completely independent of the object being imaged, and the artifact is substantially additive to the image of the object. The artifact can be completely characterized a priori, such that the artifact may be automatically subtracted from an image of an object. This invention provides a method for removing the artifact from an image of an object by imaging an object and digitally processing the image to subtract a previously-characterized artifact. Specifically, the method includes the steps of obtaining a first image of an artifact resulting from an in-line illumination device of an imaging system using a first artifact illumination level, imaging an object to be analyzed using the imaging system wherein the in-line illumination device is using an object illumination level, and subtracting respective pixel values of an artifact image from respective values of the object image on a pixel address-by-pixel address basis. The artifact image could be the first image of the artifact or could be a scaled image of the first image. Of course, multiple images of the artifact could be taken at differing illumination levels prior to analyzing the object. Then, the image of the artifact having an illumination closest to the illumination level of the object could be used for the subtraction, or that image could be scaled to the illumination level of the object prior to the subtraction.

[0007] Preferably, this digital removal of the illumination artifacts would occur automatically, in-line, before the image was reported or presented for viewing, or before any subsequent image analysis was performed on the image to determine specific characteristics of the object being imaged.

[0008] By digitally subtracting out the in-line on-axis illumination artifacts, a system so configured may be more sensitive to features of the object, as they will now be rendered in the image with less interference from the illumination artifacts. Additionally, a system so equipped may be operated at overall higher illumination levels without undue interference from illumination artifacts, increasing the domain of object types that may be inspected and the type of information about the object that may be extracted from the image.

[0009] Other applications of the present invention will become apparent to those skilled in the art when the following description of the best mode contemplated for practicing the invention is read in conjunction with the accompanying drawings.

[0010] The description herein makes reference to the accompanying drawings wherein like reference numerals refer to like parts throughout the several views, and wherein:

[0011] FIG. 1 is an image of a machinist's rule with an illumination artifact;

[0012] FIG. 2 is an image of the machinist's rule of FIG. 1 where the artifact illustrated in FIG. 3 has been subtracted;

[0013] FIG. 3 is an image of black velvet including the illumination artifact of image 1;

[0014] FIG. 4 is a process flow chart illustrating a preferred embodiment of the present invention; and

[0015] FIG. 5 is one embodiment of an in-line illumination system that can be used with the present invention.

[0016] The invention involves digitally recording an illumination artifact associated with an imaging system. This artifact may then be later subtracted from subsequent images to remove the artifact from such images.

[0017] This invention is preferably utilized in combination with in-line illumination systems that can result in artifacts from internal reflections. One such in-line system is described in co-pending U.S. patent application Ser. No. 10/373,934, filed Feb. 26, 2003, which is incorporated herein in its entirety by reference. As shown in FIG. 5, such a system can include an imager 100 generally defined by an illumination source 118, an imaging lens 108, an electronic image detector such as a video camera 200, and a digital computing device (not shown) capable of accepting images electronically or optically from the electronic image detector and a set of instructions (a computer program) for digitally processing said image, typically for extracting information about the object or objects being imaged 102. The video camera 200 typically can include a CMOS or CCD sensor. In the preferred embodiment of the invention, at least one mode or aspect of the illumination system 118 is in-line with the imaging lens 108 and is projected through at least one lens element 110 before illuminating the object. The illumination is preferably controlled by the digital computing device.

[0018] The artifacts resulting from the system described in U.S. patent application Ser. No. 10/373,934 have some identifiable properties although other in-line illumination devices exhibit similar properties. First, the artifacts are substantially additive to the image of the object in intensity space (a map of the intensity of the returned illumination, pixel-by-pixel, commonly just referred to as the image). Second, they are substantially independent of the object. This means that the artifacts can be fully documented a priori and the information stored for use during operation, where the image subtraction may be done inline. Finally, the artifacts do not have to be characterized at every illumination level. Instead, they can be characterized at a few well-chosen illumination levels, and the characteristics required for subtraction can be interpolated or extrapolated from these few samples due to the direct relationship between the in-line illumination intensity and the characteristics of the artifact. This technique, of course, presumes that the illumination levels are known to the vision system.

[0019] It should be noted that the above described properties will not hold if the image goes into saturation. That is, if the light intensity exceeds the sensitivity of the camera. If the image does saturate, then information is lost. (Saturated values are indeterminate values, e.g., those where the dynamic range of the image sensor has been exceeded.) Saturation can be mitigated if the illumination levels are under control of the vision system. That is, the system can be programmed so that, as a first precaution, the intensity of the artifact is never saturated (i.e., a limit can be set). In a more sophisticated system, an automatic illumination-level control scheme can be programmed so that no part of the image is ever saturated.

[0020] At setup time, a black object (or no object) is placed before the imaging system. This setup may be performed at the time the imaging system is manufactured, or it may be performed when the system is deployed. With the imaging system configured so that there is no return image from an object, the image of the back-reflection of the in-line illumination source from the lens element or elements onto the electronic image detector is the only item imaged. This artifact is stored within the digital computing device. Images of the back reflection or artifact may be made for every illumination level of the in-line illumination source, or for only a few illumination levels.

[0021] With reference to FIG. 3 there is shown an image in which an artifact 10 is the sole content of the image. The image of FIG. 3 is shown as taken against a black backdrop. In a preferred embodiment, an illumination level that is one level below that required to cause image saturation is determined. That illumination level is stored by the digital computing device and used to set the maximum illumination level that can be set by the system for in-line illuminations of an object. Preferably, at least one image of an artifact at this maximum illumination level is stored on the digital computing device, and at least one additional image of an artifact at a lower illumination level is stored. Preferably, this lower illumination level is substantially lower that the maximum illumination level, i.e., a level such as one quarter or one half of the maximum illumination level, but above the noise level of the sensor/imager being used. The noise level of a sensor/imager varies with the configuration used and is a quantity that can be determined by those skilled in the art. Using a minimum illumination level above the noise level assures that there is sufficient intensity to register a usable image.

[0022] With reference to FIG. 1, during operation of the system, for each object 12 imaged, an image of the artifact 10 is also present. The artifact 10 may then be subtracted to yield only an image of the object 12 which is shown in FIG. 2. As shown in FIG. 1, initially, the image of the object 12 will include the artifact 10 from the in-line illumination, and after the subtraction, as shown in FIG. 2, the artifact 10 will be substantially removed from the image of said object 12. This results in a generally improved image with less spurious information that could interfere with the stated purpose of extracting information about the object from the image of said object 12.

[0023] If an image of the in-line illumination artifact corresponding to the same illumination intensity used to obtain the initial object image is not available, then scaling can be used to obtain substantially the same result. Recall the direct relationship between the in-line illumination intensity and the characteristics of the artifact. Thus, the intensities (gray-scale levels) of an available artifact image can be scaled using a ratio of the intensities used to make the artifact image and the object image. This suitably-scaled artifact image is used for the subtraction.

[0024] With reference to FIG. 4 there is shown a flow diagram illustrating an embodiment of the present invention. At 20, an in-line illumination device, preferably of the type described above, is provided. At 24 the artifact saturation point is determined and stored. At 22, images of an artifact associated with the imaging system are captured. In the preferred embodiment, multiple artifacts are captured at different illumination levels. As described above, the artifacts can be captured by imaging black velvet or any other blank surface that does not charge the video camera. In one embodiment two artifacts are captured and stored, one artifact associated with an illumination level near the saturation and one of roughly half that illumination level.

[0025] At 26 the images of the artifacts are stored with each artifact image having an associated intensity level. It is understood by those of ordinary skill in the art that each artifact image is stored by pixel value and pixel location. The pixel value and the pixel location of the artifact as described above, for any particular lighting condition, will reproduce itself in subsequent images regardless of the object being imaged so long as the imaging device has not reached saturation.

[0026] At 28 an object of an image is obtained using the imaging system including the in-line illumination device. At 30 the intensity or illumination level of the image captured at 30 is determined. The object will have been imaged at a known intensity level. At 32 it is determined whether one of the stored artifacts has the same illumination level as that of the image captured at 28. If the answer to the query at 32 is yes, that corresponding artifact is selected at 34 and the process moves to block 40.

[0027] If the answer to the query at 32 is no, an artifact is calculated at 36 for the illumination level of the image captured at 30 by interpolating or extrapolating an artifact with the known intensity from the stored artifact values. For example, if the known illumination intensity of a stored artifact is half of the level of the illumination intensity of the image captured at 30, the values of the stored artifact are doubled at 36 to create a new artifact image.

[0028] As described above the image of the object captured at 28 will include the artifact superimposed thereon. The artifact 10 of FIG. 1 is in the same location and is the same artifact as the artifact 10 in FIG. 3. At 40, the artifact supplied at block 34 or 36 is subtracted from the image captured at 28. This yields a subtracted image at 50 such as that illustrated in FIG. 2 in which the artifact is no longer present.

[0029] It is understood that in this description blocks 28, 30 and 32 are not necessary to practice the present invention and that those blocks merely improve the accuracy of the subtracted image.

[0030] Moreover, it should also be understood that the order represented is merely one order in which the steps can be performed. That order is useful for one time recording of the artifact image to be used with a variety of objects to be imaged and analyzed. Another useful embodiment results whereby the operator of the image system sets the illumination levels according to the particular object being imaged. Specifically, an object is imaged first. Then, a null or dark object as described previously is substituted to capture the artifact image at the illumination level used for the object. The subtraction of the artifact from the object image can then be performed. Any similar objects would then have the same artifact subtracted without adjustment due to changes in the illumination level. Although this embodiment would not be as convenient as that described previously when dissimilar objects are being analyzed it would be useful in a repetitive task, such as analyzing a large number of similar objects in an industrial conveyor-line setting.