Character Recognition Method

Filed with this application is a microfiche appendix containing a suitable program for implementing the preferred embodiment of the invention. The microfiche appendix comprises a total of two microfiche and a total of sixty frames.

The present invention relates to a character recognition method and, more particularly, to a user-friendly alpha-numeric character recognition method.

It is known to employ a solid state TV camera in a video inspection system. For example, U.S. Pat. No. 4,344,146 to Ray E. Davis, Jr. et al, which is entitled "Video Inspection System" and is assigned to the assignee of the present application, discloses the use of such a TV camera in a high speed, real time video inspection system wherein the TV camera has at least sixteen levels of grey scale resolution. It is also known to employ a solid state TV camera in a video measuring system. For example, Ser. No. 596,842 filed Apr. 4, 1984 by Ray E. Davis, Jr. et al, now U.S. Pat. No. 4,628,553 which is entitled "Video Measuring System" and is assigned to the assignee of the present application, discloses the use of such a TV camera in a fast, efficient, user-friendly video measuring system useful for both guality control and process control. Further, it is known to employ a vidicon camera in a method and apparatus for identifying images found, for example, on a "cents off" coupon. Such a system is shown in GB No. 2,031,207A.

The present invention is a computer-based vision system having the capability of reading printed characters and enables the user to verify, sort and tabulate products quickly and accurately. The system can read individual alpha-numeric characters in almost any printed font and can form words from these characters. The words can then be used in data analysis by another computer system or by the instant character recognition system. Examples of such data analyses include sorting of products by product name, tabulating for automatic factory invoicing and verifying the print quality of date and lot codes.

In a preferred embodiment the present invention employs a pair of solid state TV cameras, a pair of interface/memory circuits (also known as "framegrabbers"), a pair of TV monitors, a computer, a keyboard, a joystick and strobe lights. In the system software are stored "menus" which guide the operator in the use of the system. These menus and the manner in which they are presented render the system very user-friendly.

The basic steps of the character recognition process of the present invention are set forth below. These steps are used both in "teaching" a known character to the system, in the course of building up a character library, and in "reading" an unknown character and attempting to match the unknown character with a known character stored in the library. The basic process steps are set forth below:

1. Acquire an image from the TV camera using a framegrabber.

2. Define a window search area using the joystick to move a cursor on the TV monitor.

3. Search for a character by scanning the search window to locate the left, right, top and bottom most points of the character.

4. Determine the character width and height in terms of pixels, each pixel having a grey scale value.

5. Calculate the number of pixels per dot required to fit the character into a dot matrix, distributing any extra pixels uniformly among the dots.

6. Calculate a single grey scale value for each of the dots in the dot matrix by combining the values for the pixels associated with each dot.

7. Generate a grey scale histogram using the newly calculated grey scale values for the dots in the dot matrix.

8. Determine a grey scale threshhold for the dots in the dot matrix.

9. Convert each dot data point from its grey scale value to a binary value.

The present invention is described with reference to the following drawings which form a part of the specification and wherein:

FIG. 1 is a functional block diagram of a preferred embodiment of the video measuring system of the present invention;

FIG. 2 is a diagram showing the search pattern within a search window;

FIG. 3 is a diagram showing a character broken down into an 8 by 10 dot matrix;

FIG. 4 is a diagram showing grey scale values for the pixels associated with certain of the dots in the dot matrix;

FIG. 5 is a grey scale histogram of the dot matrix;

FIG. 6 is a diagram of a portion of the dot matrix following threshholding; and

FIG. 7 is a diagram of a portion of the dot matrix following conversion to binary values.

The basic system architecture of a preferred embodiment is shown in FIG. 1. The system employs two TV cameras 10 and 12, designated "A" and "B." Connected to TV cameras 10 and 12 are two interface/memory units 16 and 18, also designated "A" and "B." Associated with TV cameras 10 and 12 is a TV monitor 14 which is connected to either interface/ memory 16 or interface/memory 18, depending on the position of switch 15. TV camera 10 and interface/memory 16 form channel "A," while TV camera 12 and interface/memory 18 form channel "B." Two channels are employed because when the system is used, for example, to inspect packages on a high speed fill line, these packages frequently have both front and rear labels and it is desirable to inspect both labels.

Interface/memory units 16 and 18 are connected to computer 22 via a conventional multibus arrangement. Also connected to computer 22 are joystick 26, strobe lights 28, keyboard 23 and monitor 24. The operator uses keyboard 23 to communicate with computer 22 and uses joystick 26 to manipulate the cursor on monitor 24. Strobe lights 28 illuminate package 30, which comprises a top closure 32 and a label 34 containing the alpha numeric character "A". The strobe lights are synchronized with the TV camera and the movement of package 30.

Monitor 14 and monitor 24 may, for example, be a Panaso nic TR-932 dual monitor made by Matsushita Electric, Osaka, Japan. Joystick 26 may be a 91 MOB-6 joystick made by Machine Components Corp., 70 New Tower Road, Plainview, N.Y. 11803. Strobe lights 28 may be a Model 834 dual stroboscope control unit made by Power Instruments, Inc., 7352 North Lawndale, Skokie, Il. 60076. Keyboard 23 may be a VP-3301 keyboard data terminal made by RCA Microcomputer Marketing, New Holland Avenue, Lancaster, Pa. 17604. Computer 22 may be an Am 97/8605-1 8086 16 bit MonoBoard Computer made by Advanced Micro Devices, 901 Thompson Place, P. 0. Box 453, Sunnyvale, Calif. 94086. This computer is software transparent to code written for the SBC-86/05 and SBC-86/12A computers. A suitable program for implementing the preferred embodiment of the invention disclosed herein has been filed as a microfiche appendix. Interface/memory units 16 and 18 may be "framegrabber" boards Model VG-120B made by Datacube, Inc., 4 Dearborn Road, Peabody, Mass. 01960. These units acquire a full screen of video information from any EIA-standard video source. The information is stored in an on-board memory for access by any MULTIBUS-based computer. The Model VG-120B framegrabber also generates EIA-standard video from the on-board memory for a TV monitor. Finally, TV cameras 10 and 12 may be Model KP-120 solid state TV cameras made by Hitachi Denshi America, Ltd., 175 Crossways Park West, Woodbury, N.Y. 11797. This is a solid state black and white TV camera employing solid state imaging. It has a two dimensional photosensor array with 320 horizontal and 240 vertical picture elements or 76,800 pixels. The framegrabbers captures this information.

The system operation will now be explained with reference to a preferred embodiment of the invention.

The first step in the character recognition process is to acquire an image from the camera using the framegrabber. When the package is properly positioned, an indication is supplied to the computer which causes the strobe lights to fire, thereby freezing the package motion. The framegrabber digitizes the camera RS-170 video signal and stores the single frame of camera data as a 320 by 240 array of pixels.

The computer now initiates a search for the first alpha numeric character within a predefined search window. For example, the computer starts the search at the upper left hand corner of the search window and searches down the window search area until the bottom of the window is reached or the first character is found. This search pattern is shown in FIG. 2, wherein 40 designates the search window and 41 the direction of search. If the bottom of the scan area is reached without finding the left most edge of the character, then the scan shifts to the right one pixel at the top of the search window and scans down again.

The left most edge 44 of the character is found when the calculated intensity gradient over a 3 pixel vertical line exceeds the preselected gradient threshhold level. The intensity gradient is the rate of change of light intensity at a particular point and has both a magnitude and a direction. This gradient edge detection is, in effect, a form of high pass filtering.

After the left most edge 44 of the character has been located, scanning is continued from the top of the search window to the next top edge 45 of the character. Each time a new character top edge is located, the information is stored and later used to determine the top most edge 42 of the character. The pattern of scanning from the top of the search window to the top edge of the character continues until the scan hits the bottom of the scan area. The preceding character top edge 43 then becomes the right most edge of the character.

The system next locates the bottom most edge of the character. This is accomplished by scanning up from the bottom of the search window, starting at left most edge 44. Scanning continues incrementally, left-to-right, until right most edge 43 is again located. Each time a new character bottom edge is located, the information is stored and later used to determine the bottom most edge of the character.

Having determined the left, right, top and bottom most edges of the character, the character pixel width and pixel height are now calculated by differencing the left and right most edges and the top and bottom most edges, respectively. The character pixel width and height are then used to determine the pixel grouping reguired to fit the entire character into an 8 by 10 dot matrix. Thus, the character pixel width is divided by 8 to determine the number of pixels per dot in the horizontal direction. Similarly, the character pixel height is divided by 10 to determine the number of pixels per dot in the vertical direction. This determination takes into consideration the fact that the number of pixels in the horizontal and vertical directions may not go evenly into 8 by 10 dots. The extra pixels in the horizontal and vertical directions are inserted evenly as extra pixels in the 8 by 10 dot matrix.

In the example shown in FIG. 3, the character is 20 pixels wide and 25 pixels high. Dividing the 20 pixels by the 8 horizontal dots yields 2 pixels per dot, with 4 pixels left over. The 4 extra pixels are assigned to every other dot. Thus, the number of pixels per dot horizontally are 2, 3, 2, 3, 2, 3, 2 and 3. In the vertical direction, dividing 25 pixels by 10 dots yields 2 pixels per dot, with 5 pixels left over. These extra pixels are assigned to every other dot so that the number of pixels per dot vertically is 2, 3, 2, 3, 2, 3, 2, 3, 2 and 3.

FIG. 4 shows a portion of the 8 by 10 dot matrix with actual pixel grey scale values (0 to 63) for the pixels assigned to certain of the dots. Because the character was 20 by 25 pixels, there are 500 pixels associated with the 8 by 10 dot matrix. These 500 pixel character data points are now transferred to the on board computer data random access memory "RAM") to speed up processing. A single value for each of the 80 dots in the 8 by 10 dot matrix is now computed by forming, for example, an average grey scale value from the pixels associated with each dot. Thus, in FIG. 4 the six grey scale values 43, 39, 44, 41, 44 and 42 average out to 42. This reduction of the number of data points from 500 (20 by 25 pixels) to 80 (8 by 10 dots) reduces the processing time and improves the effective signal-to-noise ratio of the character representation. This is, in effect, a form of low pass filtering.

The next step in the character recognition process is to generate a grey scale histogram for the 80 grey scale data points in the 8 by 10 dot matrix. The histogram plots grey scale value versus counts (the number of dots having that grey scale value). A typical grey scale histogram is shown in FIG. 5. The grey scale histogram is then used to threshhold each of the 80 dots in the 8 by 10 dot matrix. The threshhold value will then be used to convert each grey scale dot in the 8 by 10 dot matrix into a binary dot data point.

When the lighting and image contrast are properly adjusted, the grey scale histogram will have most of the data within a relatively narrow range of grey scale values. The left and right endpoints 50 and 52 of the grey scale histogram are shown in FIG. 5. The threshhold value is the midpoint 51 between the two endpoints of the grey scale histogram. If, for example, the threshhold is 35, then dots having a grey scale value of 35 or above are given a threshhold value of 63 (white). Similarly, those dots having a grey scale value below 35 are given a threshhold value of 0 (black). Each threshhold data point is then converted to a binary data point, with 63 becoming binary 1 (white) and 0 becoming binary 0 (black). This threshholding operation is, in effect, a form of high pass filtering.

The result of the threshholding and conversion operations is a character defined by 10 horizontal rows of 8 dot data points. Therefore, each row is an 8-bit byte. The character, which is now defined by ten 8-bit bytes of digital data, can be stored in a library for future reference. An entire library may be built up in this fashion.

In attempting to "read" an unknown character, the character is first reduced to ten 8 bit bytes in the manner just described. The digital representation of the unknown character is then compared, data point by data point, with characters in the library of known characters. The comparison is effected by EXCLUSIVE ORing each byte of the unknown character with each byte of the character library. The results of each EXCLUSIVE OR operation is the number of bits by which the unknown character differs from the library character with which it is being compared. The number of differences per character are saved until all characters in the library have been compared with the unknown character. The library character with the lowest number of different dots is most likely to be the correct character.

If the unknown character does not match any of the characters in the library, then the left most edge of the character is shifted to the right by one pixel and the character is reprocessed to generate a new digital representation of the unknown character. This single bit shift of the pixel data points corrects for a possible error in locating the left most edge by doing a simplified form of cross correlation. This reprocessing of the unknown character data will normally happen only once. If after two complete tries at processing the character error count (the number of different dots) is still too large (i.e., exceeds a predetermined threshhold), then the unknown character is deemed to be the library character with the smallest error count, but the character is flagged as being of guestionable validity.

Successive characters are processed in the manner just described. When all characters on the current image plane have been processed, then the system will compare this set of characters with known words in the library and, if the system is being used for guality control, either accept or reject the product based on the result of that comparison.

The invention disclosed and claimed herein is not limited to the preferred embodiment shown or to the exemplary application of that embodiment to the inspection of packages on high speed fill lines since modifications will undoubtedly occur to persons skilled in the art to whom this description is addressed. Therefore, departures may be made from the form of the present invention without departing from the principles thereof. For example, it is not necessary to use two TV cameras. Similarly, it is not necessary to use a joystick to define the window search area. The coordinates of the window search area may be entered through the keyboard. Further, the scan directions need not be vertical but may be horizontal or a combination of both. The intensity gradient need not be calculated over a three pixel interval. A different interval may be employed. Also, the pixels need not be fitted to an 8 by 10 dot matrix. Other size matricies may be employed. These and other variations are design choices within the level of skill of those skilled in the art.