Field of the Invention
This invention relates to machine vision systems, and more particularly, to such systems in which binary values are assigned to pixels.
Background of the Invention
It is common in machine vision systems to perform a binary classification of pixels in an image to be analyzed; in other words, one of the values "0" and "1" is assigned to each pixel. Analysis of the image then proceeds on the basis of the binary classification of the pixels.
Since the information making up the image to be analyzed is often presented to the system in non-binary form, such as gray-scale levels, selection of a threshold for binary classification (a "pixel-discrimination threshold") is a fundamental task that must be performed before the analysis of the image. Moreover, it has been found in some machine vision systems, particularly those in which character recognition is to be performed, that the accuracy of the analysis and/or the amount of time required is dependent upon the threshold selected.
An application of machine vision in which setting of a pixel-discrimination threshold has been found to be critical is the automatic recognition of identification codes on semiconductor wafers. The recognition may be performed, for example, for the purpose of sorting or production control, record keeping and the like. FIG. 1 is an illustration of a typical semiconductor wafer, generally indicated by reference numeral 10. The semiconductor wafer 10 is generally planar, and circular in outline except for an indentation in the shape of a segment defined by a straight line edge 12. A multi-character identification code 14 is formed in a surface 16 of semiconductor wafer 10. The characters making up the i.d. code 14 may be formed by laser-etching or a similar process and are arranged in a character line that runs parallel to, and a short distance from, edge 12. The size of the characters as shown in FIG. 1 is not necessarily to scale; in a typical semiconductor wafer 10, the characters may be only a few millimeters high. Further, as will be recognized by those skilled in the art, the surface 16 of semiconductor wafer 10 may have been coated with a metal film or other coating that renders the surface 16 highly reflective. Moreover, although in the illustration the characters are shown as black-on-white, it should be understood that in practice the characters may be evidenced only by impressions in the surface 16, so that there is not a great contrast in tone between the characters of i.d. code 14 and the balance of surface 16.
Adding to the difficulties presented in accomplishing accurate and rapid machine recognition of the i.d. code 14 is the fact that a sequence of wafers bearing i.d. codes to be recognized may include wafers that vary significantly in reflectivity and color.
In setting up a machine vision system for recognition of i.d. code characters, it is often found to be useful to program or otherwise prepare the system to recognize characters having character lines of a predetermined thickness. However, it has been found that the particular pixel-discrimination threshold level used by a machine vision system can have a significant effect on the thickness of the character lines resulting from the binary classification of the pixels. The variations in thickness, in turn, can have a significant adverse effect on one or both of the accuracy of the character recognition and the time required therefor.
Objects and Summary of the Invention
Accordingly, it is an object of the present invention to provide a method of operating a machine vision system in which an appropriate pixel-discrimination threshold level is automatically established for each item presented for recognition to the machine vision system.
In accordance with the invention, a method of setting a pixel-discrimination threshold level for a machine vision system adapted to recognize indicia on a work-piece includes setting a plurality of test threshold levels for each work-piece of a test group of work-pieces and using the plural test threshold levels to recognize indicia on a work-piece; determining if the plural test threshold levels result in acceptable indicia recognition; calculating at least one threshold-setting parameter on the basis of the results of the determinations; and establishing a pixel-discrimination threshold level for each work-piece not included in the test group on the basis of a characteristic of an image of the work-piece not included in the test group and the threshold setting parameter or parameters.
According to a further aspect of the invention, a threshold-setting parameter is calculated on the basis of an estimated average number of pixels which represent the indicia to be recognized.
According to still another aspect of the invention, a threshold-setting parameter is calculated on the basis of an estimated average gray-scale level of the estimated average number of pixels.
According to another aspect of the invention, a method of setting a pixel-discrimination threshold level for a machine vision system includes the steps of selecting a test group of work-pieces, each of which has an indicia to be recognized by the machine vision system; establishing a range of gray-scale levels as test threshold levels; and, for each work-piece of the test group, forming an image of at least a portion of the work-piece so that the image includes a representation of the indicia and has a plurality of gray-scale levels, attempting to recognize the indicia by means of the vision system using each of the test threshold levels as a pixel-discrimination threshold for the image, entering into the system data representing the actual indicia of the work-piece, determining for each test threshold level whether the system correctly recognized the indicia on the work-piece, establishing a subrange of test threshold levels that includes only those test threshold levels at which the system correctly recognized the indicia, selecting a central threshold level of the subrange as an ideal threshold level for the work-piece, and calculating a piece parameter on the basis of the ideal threshold level. The method further includes calculating a threshold-setting parameter on the basis of the piece parameters.
According to a further aspect of the invention, the method also includes the steps of selecting a work-piece not included in the test group and which has an indicia to be recognized by the vision system; forming an image of a portion of the selected work-piece including a representation of the indicia and having a plurality of gray-scale levels; taking a histogram of the image, where the histogram includes a representation of a respective number of pixels for each gray-scale level of the image; and setting a pixel-discrimination threshold for the work-piece not included in the test group on the basis of the histogram and the threshold-setting parameter.
By using the methods described above, a satisfactory pixel-discrimination threshold can be automatically established for each item to be recognized on the basis of a relatively small quantity of data gathered during a brief initialization process.
The above, and other objects, features and advantages of the present invention will be apparent from the following detailed description thereof which is to be read in connection with the accompanying drawings.
Brief Description of the Drawings
FIG. 1 is a plan view of a semiconductor wafer which includes an identification code that is to be machine-recognized;
FIG. 2 is a partial schematic, partial block diagram of a machine vision system in accordance with the present invention;
FIG. 3 is an image formed by the machine vision system of FIG. 2 and including a representation of the i.d. code of the semiconductor wafer of FIG. 1;
FIGS. 4A-4C are histograms taken of images of the sort shown in FIG. 3;
FIGS. 5A and 5B are diagrams that illustrate results of pixel-discrimination threshold testing procedures in accordance with the present invention;
FIGS. 6A and 6B illustrate analyses of histograms performed in accordance with the present invention;
FIG. 7 is a flow chart of an initialization procedure for a machine vision system in accordance with the present invention; and
FIG. 8 is a flow chart of a pixel-discrimination threshold setting procedure in accordance with the present invention.
Detailed Description of the Preferred Embodiments
A machine vision system, and a method of setting pixel-discrimination threshold levels for the same, in accordance with the invention, will now be described with reference first to FIG. 2.
In FIG. 2, reference numeral 20 generally indicates a machine vision system which includes a camera 22, an analog-to-digital converter 24, an analysis module 26 for carrying out character recognition, and an operator interface 28. A lighting device 30 is associated with machine vision system 20 and provides light L for illuminating at least the portion of semiconductor wafer 10 which includes the identification code 14. (Although in FIG. 2, for reasons of scale, identification code 14 is represented only by a rectangular box, it should be understood that the identification code is actually in the form of a character string etched in the surface of semiconductor wafer 10 as shown and described in connection with FIG. 1.)
Light incident upon the identification code 14 is received by camera 22 and used by camera 22 to form a gray-scale image of a portion of the surface 16 of wafer 10 which includes the identification code 14. Camera 22 is preferably a conventional video camera or other well-known image-forming device and may be, for example, the model XC77 camera produced by Sony Corporation.
FIG. 3 is a representation of an image 32 formed by camera 22. The image 32 includes a substantially gray background area 34 and a representation 14' of the identification code 14. Although the representation 14' is shown in FIG. 3 in the form of an outline of the represented characters of the i.d. code 14, it should be noted that in the image itself the characters appear as white, or relatively light in tone, against the gray background 34. Image 32 also includes a rectangular frame 36 in the form of black lines defining a relatively small portion of the gray background 34 that includes all of the represented characters of representation 14'.
Camera 22 provides to A/D converter 24 analog signals representative of respective gray-scale levels of individual pixels of image 32. These signals are converted into digital data on a pixel-by-pixel basis by A/D converter 24, and the resulting data is provided for analysis and character recognition to analysis module 26. In a preferred embodiment of the invention, only the portion of image 32 within frame 36 is analyzed by analysis module 26 for the purpose of character recognition, the area within frame 36 contains an array of approximately 100.times.400 pixels so that about 40,000 pixels are in the part of the image 32 that is to be analyzed, and 128 grayscale levels are possible (i.e. each pixel is represented by a seven-bit data word).
Analysis module 26 may be a known type of electronic character recognition device, including a microprocessor and associated memory and data communication devices, and operating under control of a known type of character recognition software. It is also within the contemplation of this invention that analysis module 26 use neural network recognition technology of the type disclosed in co-pending U.S. patent applications Ser. Nos. 07/955,522; 07/955,526; 07/955,534; 07/955,555; and 07/955,563; all of which have a common assignee and a common inventor with the present application.
Operator interface 28 is connected for data communication with analysis module 26 and preferably includes conventional input/output devices such as a computer display monitor and keyboard. For example, operator interface 28 may include a conventional personal computer. Operator interface 28 may also be directly connected to camera 22 for receiving and displaying images formed by camera 22 and/or for making adjustments to settings of camera 22.
As is conventional with known types of character recognition equipment, analysis module 26 converts the pixel-by-pixel gray-scale data received from an A/D convertor 24 into a binary representation of the pixels within frame 36. In other words, each pixel is assigned either "0" or "1" representing respectively "black" or "white" (or vice versa). The analysis module 26 then attempts to recognize the characters represented by the representation 14' on the basis of the binarized pixel data. In a preferred embodiment of the present invention, the 128 possible values for the gray scale that may be output by A/D converter 24 have identifying data values 0-127. One of the values in this range is used by analysis module 26 as a threshold for the binary classification of the pixels. This value, as indicated above, will be referred to as the "pixel-discrimination threshold".
Lighting device 30 includes a box 38 enclosing a lamp 40. Light emitted by the lamp 40 is carried by an optical fiber tube 42 to a head 44 which includes a slit 46. The optical fiber tube 42 is flexible, allowing head 44 to be positioned so that light L emanating from slit 46 provides illumination of the i.d. code 14 on the wafer 10. Box 38 preferably includes a dimmer switch 48 which allows the level of light L to be conveniently adjusted.
An initialization procedure for selection of the pixel-discrimination threshold will now be described with reference to FIGS. 4-7. Referring initially to FIG. 7, the initialization procedure preferably begins with a determination whether an appropriate level of light L is being provided by the lighting device 30. At step 100, one or two work-pieces (i.e., semiconductor wafers) are selected. It is advisable that these wafers be among those which appear to have the greatest reflectivity. For each of the wafers an image 32 (like that shown in FIG. 3) is formed and the portion of the image within the frame 36 is analyzed to produce histograms of the types shown in FIGS. 4A-4C. Each histogram represents, for each of the 128 gray-scale levels, the number of pixels within frame 36 having that gray-scale level. Step 102 represents a decision by the operator, based on the histogram, as to whether the lighting level is satisfactory. If the histograms have the form shown in FIG. 4A, with a relatively sharp "peak" 50 toward the lower (darker) values and a rather long narrow "tail" 52 that stops short of the highest (lightest) value 127, then the lighting level is considered to be satisfactory. However, if a histogram like that shown in FIG. 4B is produced, lacking a "tail" then the lighting level is too dark, and meaningful threshold information will be difficult to generate. Even greater difficulties are presented by a histogram of the type shown in FIG. 4C in which a substantial number of pixels are at the highest (lightest) gray-scale level. This condition is known as "over-saturation" and will result in unreliable pixel-discrimination threshold data. If histograms of the sort shown in FIGS. 4B or 4C are found to be present at step 102, then the lighting level should be adjusted (step 104) and steps 100 and 102 repeated. It should be understood that if over-saturation (FIG. 4C) is found, than the lighting level should be adjusted downward while if an excessively dark image (FIG. 4B) is found, then the lighting level should be adjusted upward.
If at step 102 the lighting level was found to be satisfactory, step 106 follows, at which a test set of wafers is selected. A test set of 10 to 15 wafers has been found to be satisfactory, but it is advisable to have the wafers in the test set as representative as possible of the variations in reflectivity and color of the wafers that will be processed for recognition after initialization. After selection of the test set, operation of the machine vision system in an initialization mode begins with entry (preferably in response to a system prompt) of the number of items in the selected test step (step 108). Next, for a first wafer in the test set, an image 32 (like that shown in FIG. 3) is formed (step 110), and the operator then enters into the system 20, via operator interface 28, data which represents the identification code 14 as observed by the operator (step 112). That data may be, simply, the identification code itself as displayed in the image 32. There then follows a loop in which the system 20 attempts to recognize the identification code 14 while using, in turn, each of the 128 gray-scale levels 0-127 as a pixel-discrimination level. More specifically, at step 114 a test threshold level (initially, for example, level "0") is set as the test threshold level. Step 116 follows, at which the system attempts to recognize the i.d. code 14 using the test threshold level as the pixel-discrimination threshold. The loop proceeds next to step 118 at which the system 20 compares the i.d. code as recognized at step 116 with the i.d. code as entered by the operator at step 112. If the two i.d. codes match then the recognition is considered to be "successful"; if not, the recognition is considered "not successful". In either case, the result ("S" or "N" as the case may be) is stored with respect to the present test threshold level (step 120). Next, it is determined whether there remain additional test threshold levels to be tested (step 122). If so, the test threshold level is incremented (step 114) and the loop repeats until all of the 128 gray-scale levels have been used as a pixel-discrimination threshold level for the wafer currently being tested.
The routine then proceeds to step 124, at which, on the basis of the results of the 128 attempted recognitions and comparisons, the system determines the gray-scale levels making up the longest range of levels for which "successful" recognition is found. Further details of this step will be explained with reference to FIGS. 5A and 5B. Typically it has been found that when the "darkest" gray-scale levels are used as a pixel-discrimination level the recognition is not successful. Then, in an intermediate range of gray-scale levels, the recognition is successful, and then using the lightest gray-scale levels as the discrimination threshold, the recognition is again not successful. Accordingly, the entire range of levels 0-127 may be viewed as seen in FIG. 5A in which the level "L" is the lowest gray-scale level at which recognition was successfully performed, and the level "H" is the highest gray-scale level for which recognition was successful. Thus there is a range R, defined by the two arrows in FIG. 5A and inclusive of the gray-scale levels "L" and "H" in which the recognition was successful, the recognition not being successful at any of the other levels.
However, another possibility is that some "successful" test threshold levels may be interspersed among "not successful" levels. Such a case is illustrated in FIG. 5B. In this case, the range R, as defined by the two arrows, includes only "S" gray-scale levels, but there are some "S" levels outside of the range R. To deal with this possibility, range R should be defined as the longest run of consecutive "successful" gray-scale test threshold levels, with the lowest and highest of the levels in this range respectively selected to be "L" and "H". Of course, application of these rules to the situation shown in FIG. 5A would not change the gray-scale levels to be identified as "L" and "H" as shown in FIG. 5A.
The routine then proceeds to step 126, at which an "ideal" threshold level I is calculated for the particular wafer being tested on the basis of levels L and H. Level I preferably is the central level of the "successful" range R and may be obtained simply by averaging L and H. Thus, the ideal threshold level can be defined as:
If necessary, the resulting value is to be rounded up or down so that I is an integer.
The procedure then continues with step 128, at which the vision system 20 forms a histogram of the image formed at step 110. (Alternatively, it is possible to take the histogram immediately after the image was formed at step 110 or at another point between steps 110 and 128). As before, the histogram represents the number of pixels in the image frame 36 for each of the 128 gray-scale levels.
Step 130 then follows, at which system 120 adds up the total of corresponding numbers of pixels for each gray-scale level at or above the ideal threshold level I. The resulting sum n can be expressed mathematically as ##EQU1## where I is the ideal threshold level, U is the uppermost gray-scale level of the histogram (i.e., the highest gray-scale level having at least one pixel), and Pix(k) is the respective number of pixels for the kth gray-scale level of the histogram.
This sum may be considered to be an estimate of the total number of "white" pixels, which are the pixels making up the characters of the representation 14' of the identification code 14. This number n of "white" pixels can also be thought of as the area of the histogram within a range P.sub.1 (FIG. 6A) starting with the ideal threshold I and ending with the uppermost threshold level U. (Note that in FIGS. 6A and 6B the vertical scale in the "tail" portion of the histograms has been exaggerated to allow clearer illustration.)
The routine proceeds next to step 132, at which it is determined whether there are wafers in the test set which have yet to be processed. If so, the process described above with respect to steps 110-130 is performed for each of the remaining wafers. When this is complete, the routine proceeds to step 134, at which a parameter N is calculated as the arithmetic mean of the parameters n obtained for each of the wafers in the test set. The parameter N can be considered to be an estimated average number of the pixels which represent the i.d. code 14 (that is, the "white" pixels).
Step 136 follows in which a "centroid" (or average "white") gray-scale level CNT is calculated for each of the wafers in the test set on the basis of the parameter N. Step 136 will be described with reference to FIG. 6B. Before the centroid CNT is calculated, an upper range P2 of the histogram is established as the part of the histogram consisting of the uppermost gray-scale levels having a sum of the numbers of respective pixels that is greater than or equal to the parameter N. The lowest gray-scale level of this range will be referred to 23 as level ST, and can be expressed as: ##EQU2## where U is the uppermost gray-scale level of the histogram and Pix(k) is the respective number of pixels for the kth gray-scale level of the histogram. The centroid gray-scale level CNT is calculated as a weighted average of the gray-scale levels in this range P.sub.2 using the following formula: ##EQU3## The value of the centroid CNT for each wafer can be thought of as the average gray-scale level of the N "whitest" pixels in the image of the wafer.
After each centroid CNT is calculated (or alternatively after all of the centroids are calculated) the routine proceeds to step 138, at which a parameter k is calculated for each of the wafers according to the following formula: k=CNT/I. That is, k is the ratio of the centroid gray-scale level for the wafer to the ideal threshold level for the wafer. After all of the k values are calculated for the wafers, the system calculates at step 140 a parameter K as the arithmetic mean of the piece parameters k. Then the parameters N and K are established as pixel-discrimination threshold setting parameters to be used for the wafers that were not included in the test set.
Referring now to FIG. 8, the use of these threshold-setting parameters N and K for setting pixel-discrimination thresholds for wafers not in the test set will be described. The procedure of FIG. 8 begins with step 200, at which an image 32 is formed with respect to a wafer that was not part of the test set. Next, at step 202, a histogram is taken of the image within the frame 36. Based on this histogram and the threshold setting parameter N, a centroid or average gray-scale level of what are assumed to be "white" pixels is calculated (step 204) in the same manner as was discussed with respect to step 136. The first or starting level of the range over which the centroid is calculated will be referred to, for the purposes of step 204, as gray-scale level "ST1" which is calculated according to the following formula: ##EQU4## is greater than or equal to N, where U1 is the uppermost gray-scale level of the histogram and Pix(k) and N are as previously defined.
The centroid CNT is calculated using the same weighted average formula given above, and then, in step 206, a pixel-discrimination threshold for the wafer is calculated simply by dividing the centroid gray-scale level CNT by the threshold-setting parameter K, with rounding to provide an integral gray-scale threshold level. The routine then proceeds to perform recognition (step 208) using the pixel-discrimination threshold level calculated at step 206.
The following Tables 1-3 show results obtained by carrying out the procedures of FIG. 7 for a number of test sets.
In each of these tables, "low" and "high" represent respectively the lowest and highest gray-scale levels of the "successful" range of test threshold levels (i.e., "L" and "H" of FIGS. 5A and 5B), while "ideal" is the "ideal threshold level" I obtained by averaging L and H. "cntr" is the centroid gray-scale scale level CNT obtained according to step 136, and "k" is the piece parameter calculated as the ratio of the centroid CNT to the ideal threshold level I. "thrsh" is the pixel-discrimination threshold level calculated according to steps 200-206 (i.e., "cntr" divided by K). "change" is "thrsh" minus "ideal" and "idpix" is the value of the parameter n for each of the wafers. Also shown are the values N and K (respectively the means of the piece parameters n and k) for the respective test sets. "sigmacnt" is the standard deviation of the values of "thrsh" from the ideal thresholds "ideal" (I). Further, the values "pixthres" are the values that would be obtained for the starting level ST1 of the range P.sub.2 by applying step 204 to each of the wafers in the test set. It should be noted, as will be discussed in more detail, that this value ST1 can also be taken as an approximation of an ideal threshold level and so can be used as an alternative pixel-discrimination level. "Pixchg" is "pixthres" minus "ideal" and "sigmapix" is the standard deviation of "pixthres" from the ideal threshold levels.
As indicated above, a simplified approach to the pixel-discrimination threshold setting procedure described above dispenses with the threshold-setting parameter K and the piece parameters k. According to this simplified approach, steps 136-140 (FIG. 7) are omitted and step 142 consists only of setting N as the threshold-setting parameter. Moreover, in FIG. 8, step 206 is omitted and step 204 ends when the gray-scale level ST1 has been found. This level ST1 is then used as the pixel-discrimination threshold level. In other words, the initialization procedure is used only to obtain an estimated average number of "white" pixels which make up the characters in the i.d. code.
According to an alternative simplified approach, the test set is used to obtain an average centroid-to-ideal-threshold parameter K on the basis of an a priori estimate of the number of white pixels N. In this case, steps 130 and 134 of FIG. 7 are omitted, with N having been established as a parameter input by the operator, preferably as a result of previous testing or experience. It has been found that values of N in the range of 2000 to 5000 provide satisfactory results. However, the best results have been obtained by establishing N and K using the initialization procedure of FIG. 7 and by using both of these threshold-setting parameters to establish the pixel-discrimination thresholds for each of the wafers not in the test set.
Although the embodiments herein described have referred to semiconductor wafers as the work-pieces to be recognized, it should be understood that the procedures described herein can be applied to many different types of work-pieces and are particularly useful in recognizing codes or other information on highly reflective work-pieces such as compact disks.
It should also be understood that the features on the work-piece which are to be recognized are not limited to identification codes in the form of alphanumeric characters, but may also include other alphanumeric information, bar codes, geometric shapes, or other features that can be machine-recognized. A particular advantage of this system and method is its ability to facilitate recognition of indicia or other features which are not in sharp contrast with a background.
Having described a specific preferred embodiment of the present invention with reference to the accompanying drawings, it is to be understood that the invention is not limited to that precise embodiment, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope or spirit of the invention as defined in the appended claims. As used in the appended claims, "piece parameter" should be understood to include a parameter n or a parameter k as previously described.