Video Measuring System

The present invention relates to a video measuring system and, more particularly, to a fast, efficient, user-friendly video measuring system.

It is known to employ a solid state TV camera for industrial process control. For example, U.S. Pat. No. 4,135,204 to Ray E. Davis, Jr. et al, which is entitled "Automatic Glass Blowing Apparatus And Method" and is assigned to the assignee of the present application, discloses the use of an analog video signal to control the growth of a thermometer end opening blister in a heated hollow glass rod by monitoring and iteratively controlling the growth of the edges of the blister using analog edge detection techniques. It is also known to employ a solid state TV camera in a video inspection system. For example, U.S. Pat. No. 4,344,146 to Ray E. Davis, Jr. et al, which is entitled "Video Inspection System" and is assigned to the assignee of the present application, discloses the use of such a TV camera in a high speed, real time video inspection system wherein the TV camera has at least sixteen levels of grey scale resolution.

The present invention represents an improvement over both of these prior art systems and complements the video inspection system of U.S. Pat. No. 4,344,146. In addition to being user-friendly, the present invention is highly efficient because it can effectively perform measurements using only a small part of the information obtained by the system. It is extremely fast while, at the same time, being relatively inexpensive and very reliable.

In a preferred embodiment, the present invention employs a pair of solid state TV cameras, a pair of interface/memory circuits (also known as "frame grabbers"), a pair of TV monitors, a computer, a keyboard, a joystick and strobe lights. In the system are stored a series of "menus" which guide the operator in defining those features of the object which are to be measured. These menus and the manner in which they are presented render the system very user-friendly.

Initially, the operator takes a picture of an object such as a package using the TV camera. The picture is stored in memory and displayed on the monitor. The operator then uses the joystick to manipulate a cursor on the monitor and specifies those features of the object to be measured. The operator designates points where the system is to start searching for the features and also specifies intensity gradient threshholds for the features. The intensity gradient is the rate of change of light intensity at a particular point on the monitor and has both a magnitude and a direction. It may be defined as the difference in intensity between neighboring picture elements.

If the object is a package having a closure and a label, the operator defines the package, defines the closure and defines the label. In addition, the operator specifies tolerances for these measurements. All of this is done with the assistance of various menus which are presented to the operator and provide step-by-step guidance for the operation of the system.

After this information has been entered and stored, the system is ready to operate. A picture is now taken of each package as it moves past the TV camera, for example along a high speed fill line. The picture is stored and the system measures the package, the closure and the label for each package. The system will indicate when these features are out of tolerance or missing altogether so that corrective action can be taken.

An important advantage of the present invention is that it permits accurate measurements but does not require large amounts of data to effect the measurements. Thus, to measure an object the system starts at specific points and searches along lines of picture elements or "pixels," looking for gradients which exceed the selected threshholds. It is not necessary for the system to examine more than a small percentage of the pixels in order to measure an object or a particular feature of the object. For example, if the TV camera comprises a two dimensional array containing over 50,000 photodetectors, it is possible to measure an object by examining fewer than 400 pixels, or less than one percent of the information captured and presented on the TV monitor. Similarly, it is possible to measure a series of features using less than five percent of the pixels.

Because the video measuring system is user-friendly, and because it is highly efficient in its use of information, it is an extremely valuable industrial tool. Thus, it can be used for process control in manufacturing operations, for the quality control of both raw materials and finished goods, and to provide sensory signals for robotics.

The present invention is described with reference to the following drawings which form a part of the specification and wherein:

FIG. 1 is a functional block diagram of a preferred embodiment of the video measuring system of the present invention;

FIGS. 2, 3 and 4 are line drawings illustrating ways in which the system of FIG. 1 can be used to define various features of the package shown in FIG. 1; and

FIGS. 5, 6, 7 and 8 are line drawings illustrating ways in which the system of FIG. 1 can be used to measure and analyze various features of the package shown in FIG. 1.

The basic system architecture of a preferred embodiment is shown in FIG. 1. The system employs two TV cameras 10 and 12, designated "A" and "B." Connected to TV cameras 10 and 12 are two interface/memory units 16 and 18, also designated "A" and "B." Associated with TV cameras 10 and 12 is a TV monitor 14 which is connected to either interface/memory 16 or interface/memory 18, depending on the position of switch 15. TV camera 10 and interface/memory 16 form channel "A," while TV camera 12 and interface/memory 18 form channel "B." Two channels are employed because when the system is used, for example, to inspect packages on a high speed fill line, these packages frequently have both front and rear labels and it is desirable to inspect both labels.

Interface/memory units 16 and 18 are connected to computer 22 via a conventional multibus arrangement. Also connected to computer 22 are joystick 26, strobe lights 28, keyboard 23 and monitor 24. The operator uses keyboard 23 to communicate with computer 22 and uses joystick 26 to manipulate the cursor on monitor 24. Strobe lights 28 illuminate package 30, which comprises a top closure 32 and a label 34 containing the letter "V." The strobe lights are synchronized with the TV camera and the movement of package 30.

Monitor 14 and monitor 24 may, for example, be a Panasonic TR-932 dual monitor made by Matsushita Electric, Osaka, Japan. Joystick 26 may be a 91 MOB-6 joystick made by Machine Components Corp., 70 New Tower Road, Plainview, NY 11803. Strobe lights 28 may be a Model 834 dual stroboscope control unit made by Power Instruments, Inc., 7352 North Lawndale, Skokie, IL 60076. Keyboard 23 may be a VP-3301 keyboard data terminal made by RCA Microcomputer Marketing, New Holland Avenue, Lancaster, PA 17604. Computer 22 may be an Am 97/8605-1 8086 16 bit MonoBoard Computer made by Advanced Micro Devices, 901 Thompson Place, P.O. Box 453, Sunnyvale, CA 94086. This computer is software transparent to code written for the SBC-86/05 and SBC-86/12A computers. A suitable program is included at the end of the specification. Inferface/memory units 16 and 18 may be "frame grabber" boards Model VG-120B made by Datacube, Inc., 4 Dearborn Road, Peabody, MA 01960. These units acquire a full screen of video information from any EIA-standard video source. The information is stored in an on-board memory for access by any MULTIBUS-based computer. The Model VG-120B frame grabber also generates EIA-standard video from the on-board memory for a TV monitor. Finally, TV cameras 10 and 12 may be Model KP-120 solid state TV cameras made by Hitachi Denshi America, Ltd., 175 Crossways Park West, Woodbury, NY 11797. This is a solid state black and white TV camera employing solid state imaging. It has a two-dimensional photosensor array with 320 horizontal and 244 vertical picture elements or 78,080 pixels. The frame grabbers capture information from an array of 320 by 240 photosensors or 76,800 pixels.

The system operation will now be explained with reference to a preferred embodiment of the invention using an illustrative object, in this case package 30 shown in FIG. 1. In the preferred embodiment, the invention employs a "Master Menu" from which the operator makes selections. The Master Menu includes the following operating routines.

1. Select Product

2. Teach Product

3. Measure

4. Run

5. Stop Run

6. Tally

Assuming the operator wishes to select a product and then teach that product to the system, the operator turns the power on, initiates the "Select Product" routine and enters the product number. Next the operator initiates the "Teach Product" routine, which has its own menu, and includes the following sub-routines.

1. Get Image

2. Teach Product Name

3. Define Package

4. Define Closure

5. Define Label

6. Define Feature 1

7. Define Feature 2

8. Teach Tolerances

The operator initiates the "Get Image" routine and then decides whether a continuous image or a single image is desired. A continuous image is used, for example, when the system is being set up, to adjust lighting levels. A single image is employed, for example, to capture the image of the package as it moves along a high speed fill line. Taking the image is synchronized with the physical location of the package on the fill line and the TV camera and involves the use of strobe lights 28 shown in FIG. 1. Once a satisfactory image is obtained, the operator so indicates and the image is stored in memory. The system then returns to the Teach Menu.

The operator now initiates the "Teach Product Name" routine and teaches the product name, either by selecting an existing name or by entering a new name. In the preferred embodiment up to ten product names may be stored in memory. The operator now decides whether to enable label A and/or label B. Label A may be the front label while label B may be the rear label. Enabling label A involves enabling TV camera A, interface/memory A and the associated strobe light and tells the system that label A should be taught. Enabling label B involves enabling TV camera B, interface/memory B and the associated strobe light and tells the system that label B should be taught. Once images of one or both labels are taken and stored, the system returns to the Teach Menu.

The operator now initiates the "Define Package" routine. This can more easily be understood by referring to FIG. 2, which shows package 30 drawn in outline on TV monitor 24. The first step is to designate the starting point 2A for locating the left edge of package 30. This is accomplished by using joystick 26 to move a cursor until the cursor has reached point 2A, which is then stored. It is necessary to designate a starting point to the left of the actual left package edge because, when the image of the package is obtained as the package is moving, the image will not always appear in the center of TV monitor 24. The cursor is now moved to point 2B, which is the left edge of package 30, which is temporarily held. Next the cursor is moved to point 2C, which is the starting point for locating the right edge of package 30, which is also stored. Thereafter, the cursor is moved to point 2D, which is the right edge of package 30, which is also temporarily held. The system then stores the difference between points 2B and 2D, which is the measure of the package width. Points 2B and 2D need not be stored. In a similar manner, joystick 26 is used to locate starting points 2E and 2G for determining the left and right top package edge points 2F and 2H. Note that points 2E and 2F are spaced to the right of the left package edge, while points 2G and 2H are spaced to the left of the right package edge. This ensures that the top edge of the package can be detected even if the image of package 30 is not centered on TV monitor 24 because of less than perfect synchronization. Only points 2E and 2G need be stored.

At points 2B, 2D, 2F and 2H there exist gradients in light intensity corresponding to the transitions at the edges of the package. In addition to locating the points 2B, 2D, 2F and 2H, the operator also selects gradient threshholds for those points, e.g., by selecting a value between minus 63 and plus 63 for each point. To assist the operator in choosing an appropriate gradient threshhold, the system will, on request, visually display the gradient which exists at any given point on the TV monitor. By selecting appropriate gradient threshholds for points 2B, 2D, 2F and 2H and storing them in memory, the operator ensures that the edges of the package can be accurately located.

Points 2A, 2C, 2E and 2G, together with gradient threshholds for points 2B, 2D, 2F and 2H, are stored in a package offsets table. See step number 246 of the computer program. Also stored in that package offsets table are the package width and the package elevation, which is the average of points 2F and 2H. The package elevation, which forms a horizontal reference, is also stored in a work table for later use. See step 247 of the program. Also stored in the work table is the package center, which is the average of points 2B and 2D, and forms a vertical reference. After these various values have been stored, the system returns to the Teach Menu.

Having completed the "Define Package" routine, the operator now initiates the "Define Closure" routine, since package 30 has a closure 32. If there were no closure, this routine would be bypassed. Referring to FIG. 3, the operator uses joystick 26 to position the cursor at point 3A, which is then stored. This is the starting point for locating the top closure. Next the operator moves the cursor to point 3B, selects an appropriate gradient threshhold (magnitude and sign), which is then stored. This process is repeated for the remaining points 3C through 3H, which together define top closure 32. Points 3A, 3C, 3E and 3G are stored. The difference between points 3B and 3F and the difference between points 3D and 3H are also stored, together with the gradient threshholds for points 3B, 3D, 3F and 3H. If, as package 30 travels down a high speed fill line, top closure 32 is either misaligned or absent altogether, this defect can be readily detected by the system and appropriate corrective action taken.

In the preferred embodiment, the absolute locations of points 3A, 3C, 3E and 3G are not stored. Rather, these points are stored relative to the horizontal and vertical package references previously computed and stored in the work table. This permits the closure to be located and measured irrespective of where the image of the package appears in the picture. The relative locations of points 3A, 3C, 3E and 3G, as well as gradient threshholds for points 3B, 3D, 3F and 3H, are stored in a closure offsets table. See step number 249 of the program. It should be noted that points 3A, 3C and points 3E, 3G need not be located on opposite sides of the closure. All may be located below the closure. All may be located above the closure. All may be located within the closure. The system will operate properly in each case.

Now the operator initiates the "Define Label" routine. Referring to FIG. 4, package 30 and label 34 are shown on TV monitor 24. Using joystick 26, the operator positions the cursor at point 4A, which is then stored. Next the cursor is moved to point 4B, which defines one edge of the label. An appropriate gradient threshhold is now stored for point 4B. This procedure is repeated for points 4C through 4L, all of which define the label and permit the label to be located when an image of the label is obtained as the package moves along a high speed fill line. As a result of the foregoing there are now stored in the system: (1) points 4A, 4C, 4E, 4G, 4I and 4K; (2) gradient threshholds for points 4B, 4D, 4F, 4H, 4K and 4L; (3) the difference between points 4B and 4F and/or the difference between points 4D and 4H; and (4) the difference between points 4J and 4L.

The various points and gradient threshholds for the "Define Label" routine are stored in a label offsets table. See step number 250 of the program. As with the "Define Closure" routine, the start search points for the "Define Label" routine are stored relative to the horizontal and vertical package references. Again, this permits locating the label irrespective of the location of the package in the picture. Note also that the label need not be defined using the edges of the label. It may be defined using information appearing on the label itself. Referring to FIG. 5, the operator uses joystick 26 to position the cursor at point 5A, which is then stored. Next the operator selects the horizontal and vertical distances from point 5A, which are also stored. These distances are 5B and 5C and define an area which will be searched. The operator now determines (1) whether the search will be from right to left or from left to right and (2) whether the search will be from top to bottom or from bottom to top. This information is also stored. In FIG. 5, for point 5A, the search pattern is from left to right and from top to bottom. Finally, the operator selects and stores a gradient threshhold. A similar procedure is employed for point 5D. The search area is defined by points 5E and 5F and the search pattern is from right to left and from top to bottom. This information is stored in the feature offsets table. See step number 251 of the program.

In addition to defining the label, the operator may define various features of the label and, in this way, determine not only that the label has been correctly applied to the package, but that the correct label has been applied. In the present illustrative embodiment, the label contains the letter "V." Features of this letter may be defined by the operator by initiating the "Define Feature 1" and "Define Feature 2" routines of the Teach Menu.

FIG. 6 illustrates how the present invention can accurately measure distances. Joystick 26 is used to position the cursor at point 6A, which is the starting point for locating the first edge of the feature to be measured. After point 6A is stored, the cursor is moved to point 6B, at which time the operator selects and stores a gradient threshhold. Point 6B is temporarily held. A similar procedure is followed for points 6C and 6D. The difference between points 6B and 6D is also stored. The system can now measure the distance between points 6B and 6D of the letter "V" of label 34 on package 30 as it speeds down a fill line. The unit of measure in the system is a "pixel," i.e., a picture element. The system measures distance by counting the number of pixels between, e.g., points 6B and 6D in FIG. 6.

It will be appreciated that, while the measurement of distances was illustrated in a rudimentary fashion using the letter "V," the ability to accurately measure objects or features of objects "on-the-fly" is extremely valuable and has numerous and wide-ranging applications. For example, one can use the present system to perform a 100% quality control check on the dimensions of parts, either as they are received from suppliers or as they are being used in an automated assembly operation. Also, one can use the present invention to do a 100% quality control check on the dimensions of goods as they are being manufactured and thus correct defects before the goods are shipped to customers. In addition to quality control applications, the present invention is also useful in the on-line control of manufacturing operations, for example, to measure increases or decreases in the size of features as well as increases or decreases in the distance between features.

In addition to accurately measuring distances, the present invention can also examine for line signatures. Referring to FIG. 7, the joystick is used to locate points 7A and 7B, which are the beginning and end of the line signature, and are stored. Next a gradient threshhold is selected and stored. The line signature routine may be used to examines a label for positive and negative transitions which exceed the gradient threshholds. For example, positive (dark-to-light) transitions which exceed the gradient threshhold may be assigned a binary one while negative (light-to-dark) transitions which exceed the gradient threshhold may be assigned a binary zero. The result of the line signature operation is then a series of ones and zeroes, which may be accumulated in a shift register. This binary signature may be used, for example, to differentiate between a front label having a line signature of "1010" and a rear label having a line signature of "0101."

The present invention can also be employed to measure area gradients. Referring to FIG. 8, the center of the search area is designated by moving the cursor to point 8A, which is then stored. Next the horizontal and vertical distances from point 8A are selected and stored. These are Points 8B and 8C and define the search area. Finally, a gradient threshhold is selected and stored. In determining the area gradient, the system sums and stores the number of transitions (light/dark and/or dark/light) which occur within the area to be searched and which exceed the gradient threshhold. If, for example, the area to be searched is a solid color, then essentially no transitions should be observed. If a number of transitions are observed, this indicates that the area being searched is not a solid color and may signify that an incorrect label has been applied or that the correct label has been applied upside down.

Having completed the foregoing, the system again returns to the Teach Menu where the operator initiates the "Teach Tolerances" routine. At this point the operator selects the tolerances for labels A and/or B. To set the tolerances the operator employs the "Measure" routine in the Master Menu. Using the joystick, the operator manipulates the cursor and designates two points, for example the points 2B and 2D in FIG. 2. The system counts the number of pixels between the two points, each pixel corresponding to, for example, 1/32 of an inch. The tolerance selected for the width of package 30 may, for example, be plus or minus two pixels. After the appropriate tolerances have been entered in the tolerance table (see step number 248 of the program), the system returns to the Master Menu and is now ready to run.

It should be noted that once the various values have been determined and stored in the package offsets table, the closure offsets table, the label offsets table, the feature offsets table and the tolerance table, this data may be used so long as the package does not change. Also, in the preferred embodiment, the system has the capability of storing such data for ten differenct packages. Thus, so long as these packages do not change, they need be taught to the system only once, even if the packages are used only infrequently.

In operation, the system captures and stores an image of the package as it speeds along the fill line. The system then searches along lines 2A-2B, 2C-2D, 2E-2F and 2G-2H until the appropriate gradient threshholds are detected so as to locate the package and measure its width (See FIG. 2). The system also determines the horizontal and vertical package references and stores them in the work table. Next the system verifies that the top closure is present and properly positioned. This is done by searching along lines 3A-3B, 3C-3D, 3E-3F and 3G-3H until the appropriate gradient threshholds are detected (See FIG. 3). Next the system locates the label by searching along lines 4A-4B, 4C-4D, 4E-4F, 4G-4H, 4I-4J and 4K-4L until the appropriate gradient thresholds are detected (see FIG. 4). The horizontal and vertical package references are taken from the work table and combined with the data from the label offsets table and used to analyze the image of the label. The skew, label references and label width are now stored in the work table. Finally, the label is analyzed in a similar manner to see if the label contains the proper information (See FIGS. 5-8). Note that in all of this searching, relatively few pixels are examined. Thus, in searching along lines 2A-2B through 4K-4L, less than about five percent and preferably less than about one percent of the pixels are actually utilized.

When it is desired for any reason to stop, the operator enters the stop run code via keyboard 23. In the interim, the system has kept a count of, e.g., the number of defective labels. These totals can be requested by the operator. If an unusually large number of defective labels has been detected, it may indicate the existence of a bad batch of labels, or it may indicate that the tolerances have been set too tight. Finally, upon request the system will display the error codes for the defects detected so that the operator knows precisely what is causing the defects.

The invention disclosed and claimed herein is not limited to the preferred embodiment shown or to the exemplary application of that embodiment to the inspection of packages on high speed fill lines since modifications will undoubtedly occur to persons skilled in the art to whom this description is addressed. Therefore, departures may be made from the form of the present invention without departing from the principles thereof. For example, the sequence in which various steps are performed is ultimately a matter of choice. Thus, while the preferred sequence is define package, define closure, define label, define feature and define tolerances, these steps may be performed in a wide variety of sequences. Also, while it is preferred to define, for example, points 2A and 2C before choosing gradient threshholds for points 2B and 2D, that sequence may be reversed if desired without affecting system operation. ##SPC1##