Sensor-Illumination System for Use in Three-Dimensional Measurement of Objects and Assemblies of Objects

The present invention relates to sensor-illumination systems and in particular to an intelligent machine vision sensor-illumination system for use in three-dimensional measurement of objects and assemblies of objects.

In the manufacture of parts and assemblies by processes such as stamping, extrusion, drawing and casting from materials such as sheet or bulk metal, plastic, film rubber and the like, it is often required that accurate dimensional measurements be performed either on-line as the manufacturing process operates upon the part or assembly, or off-line upon samples temporarily removed from the manaufacturing production line. In order to obtain economies of manufacturing and produce high quality products, it is necessary that measurements be made to ensure that dimensional tolerances are maintained. Such measurements must be made rapidly at many points on each produced part or assembly and in such a manner that the part or assembly itself is not touched or otherwise affected substantially by the measurement process itself. Measurements are often required to be performed which determine the position in three-dimensional space of certain identified points or features which exist on the manufactured part or assembly. Such points or features typically include edges, corners, holes, designated portions of a surface, studs and intersections of surfaces. Special and general purpose computers exist which are capable of determining such measurements from image data produced by sufficiently intelligent sensor-illuminator systems.

Presently available sensor-illuminator systems are typically subject to several shortcomings. For example, many systems are highly specialized and provide limited three-dimensional data, thus limiting their application. Other more sophisticated systems tend to be overly complex and large in physical size making practical use of such systems virtually impossible. In addition, these systems also suffer from the problem of producing such complex image data that the required computer processing time to assimilate the data becomes excessive, thus further restricting practical application.

It is therefore an object of the present invention to provide an intelligent sensor-illuminator system that is capable of exercising significant self-control and yet is extremely compact in physical size.

In addition, it is an object of the present invention to provide a sensor-illuminator system that produces a unique structural illumination pattern which results in the production of image signals which permit an attached machine vision computer to dimensionally measure and locate a wide variety of surfaces and shapes, including planar surfaces at any orientation, curvilinear surfaces, holes and protrusions.

It is a further object of the present invention to provide a completely self-contained sensor-illuminator system that has permanently stored therein calibration and identification data which permits ready replacement of a faulty unit without the need for generating new calibration data.

It is also an object of the present invention to provide an intelligent sensor-illuminator system for making three-dimensional measurements which provides sufficient image data to an attached machine vision computer for such purposes without imposing an unduly complex computational burden on the computer.

In general, the present invention comprises a completely self-contained intelligent machine vision sensor-illumination system which includes a laser source preferably positioned normal to the surface of the part or assemblage of parts to be inspected, for projecting onto the surface a structured illumination which in the preferred embodiment is in the form of a cross hair pattern. A solid-state sensor device precisely located within the unit at a perspective angle relative to the angle of illumination and associated electronics produce image signals representing the projection of the structured light pattern on the surface of the part or assembly as viewed from such perspective angle. Control circuitry within the unit transmits to an attached machine vision computer specific calibration data prestored in the unit which is usable by the computer to accurately determine the three-dimensional position and orientation of the illuminated part or assembly from the image signals. The present invention also preferably includes control and communication elements permitting economical and rapid operation of a large number of sensor units in conjunction with a single attached machine vision computer and control cable, and under highly variable conditions of object light reflectance, sensor to computer cable connection distance and specific measurement point specification.

Additional objects and advantages of the present invention will become apparent from a reading of the detailed description of the preferred embodiment which makes reference to the following set of drawings in which :

FIG. 1 is a block diagram of a sensor-illumination system according to the present invention;

FIG. 1a is a more detailed block diagram of the sensor-illuminator system controller 26 shown in FIG. 1;

FIG. 2 is an exemplary diagrammatical view illustrating the preferred orientation of the present sensor-illuminator system relative to the part to be inspected;

FIG. 3 is an exemplary diagrammatical view illustrating the resulting images of the projected cross-hair light pattern, as perceived by the sensor, due to surface variations in the inspected part; and

FIG. 4 is an exploded perspective view of the preferred manner in which the sensor and illuminator are mounted within the housing.

Referring to FIG. 1, a block diagram of the sensor-illuminator system 10 according to the present invention is shown. The present invention comprises a triangulation-type sensor-illumination system wherein the sensor and illumination source are positioned at an angle relative to one another so that the sensor obtains a perspective view of the light projected on the part or assembly of parts to be inspected. However, for reasons to be subsequently explained in detail, the preferred application of the present invention departs from conventional practice and positions the light source normal to the surface of the part to be inspected and the sensor at a perspective angle relative thereto. As will be seen, significant advantages are derived from this orientation. In addition, the light source and sensor in the present invention are mounted at a predetermined angle within the same housing which, as will also be seen, permits the present sensor-illumination system 10 to be internally calibrated, thereby permitting convenient replacement of defective units on a test fixture.

The light source 12 mounted within the unit generates a narrow beam of light which may be either coherent light such as from a laser or laser diode, or non-coherent light, such as from a flourescent or incandescent source or light emitting diode. In the preferred embodiment a laser light source is used. One of the principal features of the present invention is the structured light pattern utilized to illuminate the object to be inspected. In the preferred embodiment, this structured light pattern consists of a cross hair configuration. This light pattern is created by structured light forming optics 14 which operate upon the beam of light emitted by the laser light source 12 to transform the beam of light into the desired projected light pattern. The resulting structured light pattern is preferably projected onto the object or assembly to be inspected at an angle normal to the surface of the part.

The sensor device 18 in the unit receives the reflected light image through a set of sensor optics 16 which may comprise conventional camera optics in combination with spectral, temporal, or polarizing filters. The sensor optics 16 serve to focus the reflected light image upon the surface of the sensor device and filter a portion of the light reflected from the object so that the image formed on the surface of the sensor device is restricted to only that resulting from the reflected light originating from the laser source. The sensor device 18 transduces the light image into electrical signals whose signal values are approximately proportional to the intensity of the light at each resolvable point of the image. The sensor device 18 may comprise a rectangular or other suitable two-dimensional array of electro-optical semiconductor devices including photo-diodes, charge-coupled devices, charge injection devices, or metal-oxide-silicon devices.

The electrical output signals from the sensor device 18 are provided to a sensor signal conditioning circuit 20 which filters and amplifies the image signals from the sensor. In addition, the sensor signal conditioning circuit 20 produces timing signals to permit the synchronous transmission of the image signals to a remotely attached machine vision computer. The sensor signal conditioning circuit 20 comprises conventional digital and analog communications circuitry well known to those skilled in the art. Similarly, the sensor device controller 22 comprises conventional electronic video frequency analog and digital circuits which provide timing and required address and control signals to the sensor device 18 for controlling the sequencing and operation of the sensor device to permit the device to develop a usable image under conditions of varying illumination levels, part reflectance characteristics, temperature and electrical power.

The sensor data storage circuit 24 comprises non-volatile, programmable or non-programmable read-only memory (e.g., EPROM or ROM), and is used to store data which uniquely identifies each individual sensor-illuminator system to any machine vision computer which might interrogate the sensor system. In addition, the data store circuit 24 is used to store the calibration data of the sensor-illuminator 10 for subsequent transfer, upon request, to the machine vision computer.

The sensor-illuminator system controller 26 is a microprocessor-based circuit which provides overall sequencing and signal flow control between all of the above noted sensor-illuminator system circuits and the machine vision computer. The interconnection between the sensor-illuminator system 10 and the machine vision computer includes an image signal channel via which the conditioned sensor signal from the sensor signal conditioning circuit 20 is transferred to the computer, and a control channel via which the machine vision computer interrogates or commands the sensor-illuminator system and receives acknowledgements of interrogation and commands and sensor data upon request.

With additional reference to FIG. 1a, a more detailed block diagram of the system controller 26 is shown. The system controller 26 operates, under control of its microcomputer logic, by interpreting each data transmission that occurs on the control channel to which it is aattached. When a data transmission is prefaced by the unique identification number of the sensor-illuminator unit 10, the microcomputer 40 accepts and stores the remainder of the data transmission and acknowledges the transmission by transmitting a response including its unique identity code. The system controller 26 then proceeds to execute the control actions indicated by the data sequence which it received and stored. These actions may include opening or closing the light source shutter, transmitting the signals for an image, or transmitting stored calibration data.

Turning to FIG. 2, a diagrammatical view of the preferred orientation of the sensor-illuminator system 10 of the present invention relative to a part to be inspected is shown. Contrary to prior art systems, the present sensor-illuminator system 10 is intended to be secured to a fixture so that the illumination source 12 is positioned substantially normal to the surface of the part 28 to be inspected. In this manner, the center of the cross hair light pattern 30 is projected directly onto the specific point of interest 29 on the surface of the part 28. The sensor device 18 within the unit 10 is thereby positioned at a specified angle relative thereto so that the sensor obtains a perspective image of the light pattern 30 on the part. Ideally, this angle of displacement is substantially equal relative to each light plane in the projected cross hair light pattern 30.

With additional reference to FIG. 3, the advantages of the cross hair light pattern will now be explained. In illustration (a) in FIG. 3, the perspective view of the sensor of the projected cross hair light pattern 30 on an exemplary square part in a preselected reference position is shown. Note that due to the perspective view of the sensor, the lines of the cross hair light pattern 30 are broken by the edges of the part 28. As will be appreciated by those skilled in the art, the position, size and orientation of the reference part are defined by the included angles 32 and 34 between the crossed lines of the projected light image and the locations of the end points 36 and 38, which define the lengths of the lines and the translation of the line segments projected on the surface of the part relative to the line segments projected off the part.

Thus, for example, a variation in the z-axis, as illustrated in (b) (ignoring for the moment the phantom lines), is readily detected by the present sensor-illuminator system due to the resulting translation in the line segments perpendicular to their direction, as identified by the change in the locations of the end points 36 and 38. Similarly, a change in the orientation of the part in the x-y plane, as shown in illustration (c), is readily identifiable by the resulting change in the lengths of the line segments projected onto the surface of the part, again as defined by the location of the end points 36 and 38. In illustration (d) the slope of the part surface has been changed relative to the reference position, which change is detectible by the change in the included angles 32 and 34 between the crossed lines. Thus, it will be appreciated that the use of the cross hair light pattern provides an effective means of readily detecting variations from a specified reference in all three dimensions. It will further be appreciated that similar three-dimensional observations can be made for other types of common part configurations and characteristics such as holes, edges, corners, studs, designated portions of a surface, and intersections of surfaces. Moreover, if a more complex configuration is to be examined, such as an irregularly shaped hole, the present invention is readily adaptable to such an application by modifying the structured light pattern to add additional radiating lines as needed to uniquely define the features of interest in the configuration of the hole.

Returning briefly to illustration (b) in FIG. 3, the advantage of orienting the light source 12 and sensor 18 as shown in FIG. 2 will now be explained. In conventional triangulation-type sensor-illumination type systems, it has uniformily been the practice to position the sensor device normal to the surface of the part or assembly to be inspected and to locate one or more light sources at perspective angles relative thereto so that the projected image or images appear directly below the sensor. The disadvantage with this approach is that while a variation in height (z-axis) can be detected, the actual amount of the variation at the point of interest on the surface of the part cannot be accurately measured. Specifically, if the positions of the light source 12 and sensor 18 are reversed relative to the positions as shown in FIG. 2, the resulting image of the projected cross hair light pattern, as viewed by the sensor, will appear as shown in phantom in illustration (b) in FIG. 3. While a variation in height can be detected by the translation of the line segments of the cross hair image projected on the surface of the part, the actual height of the part at the desired point of interest 29 can only be inferred from the height at point P. In other words, if the surface of the part is not flat, the height at point 29 cannot be accurately measured by this conventional arrangement. Thus, it will be appreciated that in order to accurately measure height variation with a triangulation-type sensor-illumination system, the precise location of the projected light image must be known. Hence, the present invention preferably locates the light source normal to the point of interest on the surface of the part.

Referring now to FIG. 4, an exploded perspective view of the construction of the sensor-illuminator unit 10 according to the present invention is shown. The various printed circuit boards containing the aforementioned circuits identified in connection with the description of FIG. 1 have been deleted for clarity. As noted previously, an important feature of the present invention derives from the fact that the illumination source 12 and the sensor device 18 are both precisely located within the same housing 11. In this manner, the precise geometric relationship between the two components is predetermined and fixed and therefore readily repeatable from unit to unit. In addition, the unit housing 11 is also provided with five precisely located mounting points--one on each exterior surface of the housing except the front--which, together with the self-calibration feature of the present invention, permits defective units to be replaced in a fixture without having to recalibrate the system.

Mounted to the output end of the laser light source 12 are the structured light forming optics 14 which transform the laser beam into the desired cross hair light pattern. Initially, the beam is applied to an optical assembly 14a containing diverging and focusing lens which focus the beam to the desired standoff distance. The focused beam is then applied to a beam splitter (i.e., a partially silvered mirror) and mirror located within assembly 14a which splits the beam into two parallel beams. Lastly, the beams are passed through an optic unit 14b containing a pair of cylindrical lenses oriented 90.degree. relative to each other, to form the desired line images. The cylindrical lenses are preferably mounted in eccentric holders to permit the crossing of the lines to be controlled by rotation of the lenses. In addition, it is preferable that the displaced beam be applied to the horizontally disposed cylindrical lens so that it produces the vertical light plane in the cross hair pattern. In this manner, the vertical plane can be steered upward to intersect with the horizontal light plane so that the center of the cross hair is aligned with the axis of the laser beam.

Once the unit 10 is fully assembled, calibration data for storage in the sensor data storage circuit 24 of the system is generated. This is performed in the following manner. Each sensor-illuminator unit 10 has associated therewith a specified point on the unit with respect to which all calibration data is referenced. The calibration data or "rectification table" generated for each sensor comprises a list by row and column number of each resolvable point or "pixel" in the rectangular array of the sensor, followed by the location of a point in three-dimensional space as measured from the specified reference point of the unit. Two such tables are generated for each sensor; one for each line in the cross hair light pattern.

In that the position of the light source 12 relative to the sensor 18 is fixed upon assembly of the unit 10, the illumination of each resolvable point or "pixel" within the sensor's array by reflectance from one of the two line images in the light pattern, uniquely defines in three dimensional space the position of an object. In other words, if a particular point in the sensor's array is being illuminated by reflectance from one of the lines in the light pattern, then the surface of an object must be located at a precise point in three-dimensional space relative to the reference point of the sensor-illuminator unit 10. It is the x, y and z measurements of this point relative to the specified reference point on the unit 10 which are stored adjacent the row and column number defining the pixel in question. The rectification table thus comprises, as noted, x, y and z measurement information for each pixel in the sensor's array.

Once generated, the rectification table, along with a unique identification code, are permanently stored in the sensor data store circuit 24 of the unit 10. When subsequently installed in a test fixture and connected to a machine vision computer, the sensor-illuminator unit 10 will upon request transfer the contents of its rectification table and its identification code over the control channel to the computer. in use, when a particular pixel in the sensor's array is illuminated, the computer can determine by reference to the appropriate rectification table, the location of the object in three-dimensional space. Note that from the lines defined by the end points 36 and 38 (FIG. 3), the computer can readily determine which line in the light pattern is illuminating the pixel in question.

Thus, it will be appreciated that with the sensor-illuminator system of the present invention, if a particular unit on a test fixture should fail, it can readily be replaced by another unit without having to recalibrate the new unit in the fixture. Moreover, because all image data transmissions from a sensor-illuminator unit 10 to a machine vision computer are accompanied by the unique identification code for the unit, the computer will readily be able to determine that a new unit has been installed and thereby avoid the possible error of applying the image data received from the new unit to the rectification table received from the old unit.

With conventional sensor-illumination systems the sensors and light sources are mounted separately. Therefore, the calibration data for each sensor cannot be generated until after the sensor and light sources are installed in the fixture at the test site. Accordingly, when a sensor failure occurs, the calibration process must be repeated for the new sensor before operation can continue. Because the calibration process is quite tedious and time consuming, this can present a severe disadvantage in production line operations which of necessity would therefore have to be shut down until the recalibration process was completed.

The following is a listing of the software for the microcomputer 40 in the sensor-illuminator system controller 26. The software is written in Intel's PL/M-51 version of PL1 language. ##SPC1##

While the above description constitutes the preferred embodiment of the present invention, it will be appreciated that the invention is susceptible to modification, variation and change without departing from the proper scope or fair meaning of the accompanying claims.