Stereo Optical Guidance System for Control of Industrial Robots

The invention relates to a stereo optical system for locating a visible indicia on a workpiece and for producing output signals which may be processed for the actuation of an industrial robotic device. The robotic device may be utilized to operate many types of industrial equipment, such as welders, x-ray machines, adhesive applicators, riveting machines, and other similar devices.

The invention will be described for illustration in the context of a robotic arc-welding, seam tracking system. However, the system is equally effective for guidance and control of any type of industrial robot which is required to locate and/or track indicia on a workpiece. The indicia may be in the form of a dot, a hole, a line, a groove, a seam or a similarly indicated path or point.

The use of optically controlled seam trackers in conjunction with robotic welders has been extensive in recent years. Examples of such systems are found in the following U.S. Patents: Taft et al U.S. Pat. No. 4,833,381 issued May 23, 1989; Gordon U.S. Pat. No. 4,831,233 issued May 16, 1989; Richardson U.S. Pat. No. 4,737,614 issued Apr. 12, 1988; Shibata et al U.S. Pat. No. 4,667,082 issued May 19, 1987; Polick et al U.S. Pat. No. 4,590,356 issued May 20, 1986; Smith et al U.S. Pat. No. 4,567,348 issued Jan. 28, 1986; and Richardson U.S. Pat. No. 4,501,950 issued Feb. 26, 1985.

Review of previous systems reveals critical shortcomings in the systems capabilities to provide an effective and inexpensive means of providing three dimensional guidance for robotic welding apparatus. It will be noted that these devices are not capable of providing a usable 3-D optical image of the seam unless the seam is grooved or spaced.

By use of the stereo vision concept with a single camera the present invention provides features not possible with the above listed or any known prior art. In addition to the ability of the present invention to track accurately, the stereo vision concept permits precise quantitative three dimensional control of an industrial robot. Use of a single camera, rather than two cameras, not only halves the camera costs but greatly simplifies the system electronics and reduces the size of the program software.

The present invention provides an improved optical system for the generation of basic electrical signals which are supplied to a computerized processing complex for the operation of industrial robots. The system includes a stereo mirror arrangement for the projection of views from opposite sides of a visible indicia formed on a workpiece. The views are projected onto halves of the retina of a single camera. The camera retina is of the CCD (charge-coupled-device) type and is therefore capable of providing signals in response to the image projected thereupon. These signals may then be processed for control of industrial robots or similar devices.

FIG. 1 is a diagrammatic illustration of the invention showing in particular the arrangement of optical components and the resultant light paths.

FIG. 2 is a frontal view illustrating the arrangement of the mirrors and the laser illuminators of the invention.

FIG. 3 is a diagrammatic view of the invention illustrating its use with a robotic welding machine.

As illustrated in FIG. 1, this stereo guidance system includes a housing 14, which encloses mirrors 9L, 9R, 10L and 10R, a camera 11, a light filter 11a, a lens 11b and a retina 12. The retina 12 of the camera 11 is of the CCD (charge-coupled-device) type which converts the image formed thereon into electrical signals which are transmitted by a cable 11d to a computer, processor complex 20, for operation of industrial type robots 22.

The mirrors 9L, 9R, 10L and 10R include the alphabetic left and right hand designations to simplify the description of the optical paths and the configuration of the system. Further, it will be noted that the mirrors 9L and 9R are located near the outer portion of the system and will be referred to as peripheral mirrors while the inwardly located mirrors 10R and 10L will be referred to as center mirrors.

As illustrated, the peripheral mirrors 9L and 9R direct light to the central mirrors 10L and 10R which in turn direct the light into the camera lens 11b. Because of the spaced apart, or stereoscopic locations of the left and right mirrors, two separate images will be impinged upon the retina 12 of the camera. One image is impinged upon each of the left and right portions 12L and 12R of the retina 12.

A workpiece 5 includes two metal portions 5L and 5R which are intended to be welded at a seam 6. It will be noted that seam 6 forms an indicia 7 which defines a line which is the only visible portion of the seam as seen by the system optics.

The indicia or visible line 7 is located within a simultaneous viewing polygon 8 which is defined by extreme undeviated rays 1, 2, 3, and 4. Light reflected to the left by the apparent line 7 will travel along the imaginary corridor bounded by rays 2 and 4, striking in turn mirrors 9L and 10L, filter 11a, lens 11b and will finally illuminate a line of pixels (image 12L) on the right half of the CCD array retina 12. Similarly light from the indicia 7 which is scattered in a rightward direction will travel along the imaginary corridor bounded by rays 1 and 3 and strike in turn mirrors 9R and 10R, pass through the filter 11a and the lens 11b, and will form the image 12R on the left half of the CCD array retina. It is understood that indicia of any type whether it be a hole, a dot, a line, a seam, or any visible marking which is located within the simultaneous viewing polygon 8 will be quantitatively locatable in three dimensions by the CCD retina.

Initial set-up adjustments of the system include a precise predetermined orientation of the workpiece to the system. This orientation is based upon a mathematical model of the relative positions of the optical components of the system. Thus, after such orientation it will be clear that indicia on the workpiece may be precisely located and that relative movement between indicia on the workpiece and the stereo system will be reproduce in the images which illuminate lines of pixels on the retina 12. The retina, being of the CCD array configuration, produces basic signals which, when received by a properly programmed computer, processor complex 20, may be processed to control the movement and operation of various types of industrial robots 22. The CCD array provides signals which vary in accordance with the intensity of the light which strikes it in any given area. This permits the system to react precisely to the varying shades of light of an image projected upon the array. This feature is referred to as "gray-scale" processing. Such computer and processing equipment is well known in the art and is therefore not illustrated in detail in the drawings. Use of cameras having a CCD or similar capability in conjunction with a computer, processor complex for operation of industrial robots is illustrated in Smith et al U.S. Pat. No. 4,567,348 issued Jan. 28, 1986 which is cited as related art earlier in the application.

FIG. 2 is a frontal view of the stereo vision system which illustrates the location of laser diode illuminators 16 through 19 relative to the peripheral mirrors 9L and 9R. The numerals 16 and 17 indicate linear laser diode illuminators mounted above the peripheral mirrors while global illuminators 18 and 19 are located outwardly from the mirrors. The global laser diode illuminators 18 and 19 provide general illumination of the workpiece while the linear laser diode illuminators 16 and 17 provide a structured stripe of intense light which is precisely directed to the indicia to be located or tracked.

FIG. 3 is a diagrammatic view of the arrangement of elements of the stereo vision system illustrated as applied to a robotic welding system. As an additional important feature of the invention, it will be noted that a welding wire 13, also illustrated in FIG. 1, is positioned between the peripheral mirrors of the system whereby the images received on the peripheral mirrors are not impeded by the wire. A wire feed positioner 22 is attached to a welding torch assembly 24. The torch 24 ejects a plasma stream 26 to effect the welding process.

In operation, the stereo vision system collects visual information necessary to locate and track indicia on the workpiece 5. This indicia may be in the form of a visible line, a marked point, a hole, or as illustrated, the line or indicia 7 as formed by seam 6 between the pieces of metal 5L and 5R to be welded together. This collection of visible information is accomplished by forming two images 12L and 12R of the visible indicia 7 on the CCD array or retina 12. This information (image 12L and image 12R) is combined with a precise mathematical model including the internal dimensions of the optical elements of the stereo vision sensor. The computer processor complex 20, appropriately programmed, will then deduce the precise location in space of indicia 7 with respect to the optics of the system. Knowing the location of the indicia 7 allows the computer to generate appropriate commands to a torch positioning device 22, keeping the torch aligned with, and properly spaced from, the indicia 7 during the welding process.

Initially the pieces to be welded, 5L and 5R, are positioned appropriately with respect to the optics of the system, that is, their visible common boundary, which the system views as indicia 7, lies within the simultaneous viewing polygon 8, defined approximately by the extreme undeviated rays 1, 2, 3, and 4. Thus it will be apparent that light reflected to the left by the apparent line or indicia 7 will travel along the imaginary corridor bounded by rays 2 and 4, striking in turn mirrors 9L and 10L, pass through filter 11a, and lens 11b, and will finally illuminate a line of pixels (image 12L of indicia 7) on the right half of the CCD array 12. Similarly, light from the visible part of the seam 7 which is scattered in a rightward direction will travel along the imaginary corridor bounded by rays 1 and 3 and strike in turn mirror 9R and mirror 10R, pass through filter 11a and lens 11b, and will form an image 12R of indicia 7 upon the left half of retina 12. Impingement of the images of the indicia upon the half portions 12L and 12R of the retina 12 will illuminate a line of pixels in the central area of each half of the retina. It will be seen that any change in the distance between the indicia 7 and the optics systems will be instantly sensed as a change in distance between the illuminated lines on the retina 12. Thus it will be appreciated that the stereo feature of this system provides the ability of the system to instantly react not only to length and width movements, but also to the very important third dimension of depth or distance. This feature is particularly useful in voltage controlled welding systems wherein the distance between the welding head and the workpiece is constantly changing to effect the best possible welding process. As previously mentioned the CCD type array included in retina 12 will react to images and image changes by production of output signals which are processed to effect guidance and control of various types of industrial robots 22.

While certain preferred embodiments of the present invention have been disclosed in detail, it is to be understood that various modifications may be adapted without departing from the spirit of the invention or scope of the following claims.