Methodology for OCR Error Checking Through Text Image Regeneration

The invention is in the field of optical character recognition (OCR) and is directed to reducing errors in OCR methods and apparatuses.

The ability to accurately recognize characters by scanning hard copy images is extremely important for many forms of automated data processing and has wide applications ranging from automatic text recognition for word processing to banking applications wherein numerical data is scanned, processed and stored. Accuracy is essential in most applications.

A great deal of effort has been devoted to correcting errors which invariably result from commercially available OCR devices. In some commercially available OCR devices, a confidence level indication is provided with the recognized character in order to permit flagging certain characters which are recognized with a low degree of confidence. In such cases, an operator may subsequently check the flag character and correct it if necessary by reference to the original scanned hard copy image.

An additional method for lowering the error rate for OCR devices is that of employing multiple OCR units each with their associated confidence factor and the use of certain algorithms for combining these confidence factors to obtain the most probable character. See for example U.S. Pat. No. 5,257,323; U.S. Pat. No. 5,418,864; and U.S. Pat. No. 5,455,872 incorporated herein by reference.

While the above approaches to improving error rates in OCR devices have some advantages, the current OCR recognition techniques are still inadequate for a number of reasons. First, the various confidence factors provided by the manufacturer of OCR equipment may, in fact, be incorrect and there are numerous instances in which characters are identified incorrectly without an associated low confidence factor, as for example, in substitution errors. The character "c", for example, may be incorrectly recognized as a character "e" with a 100% degree of confidence so in this case the OCR device does not detect a clear substitution error. In the multiple OCR environment, the typical majority voting or weighing techniques utilized in the prior art may, and often do, yield incorrect results. Further, most systems have no mechanism for dealing with characters which are completely unrecognizable.

Of course, manual techniques have also been utilized for checking the accuracy of scanned data. Some approaches utilize both machine scanning/recognition (OCR) and manual keying so that the data has effectively has been input twice, once by machine and once by hand. Subsequent machine comparisons then reject characters on the basis of inconsistencies in the two sets of data. Such manual keying is tedious, time consuming and expensive.

In accordance with the principles of the invention, there is provided a method for reducing errors in OCR systems comprising the steps of:

(a) digitizing an image of an object wherein the image contains alpha-numeric characters;

(b) storing a bitmap of all or part of the digitized image as a scanned document file (SDF);

(c) performing a standard OCR procedure to obtain candidate characters;

(d) storing the candidate characters as a textural results file (TRF);

(e) determining the font and/or position of the characters in the digitized image;

(f) storing the determined font and/or position as a regeneration library file (RLF);

(g) generating a regeneration image file (RIF) using the TRF and RLF files;

(h) comparing the regenerated image file with the bitmap of the digitized image stored in the scanned document file;

(i) outputting candidate characters if the results of the comparison step (h) indicate a match of the regenerated image file with the bitmap of said scanned document file; and

(j) performing further processing to resolve the mismatch if a match is not found.

FIG. 1 illustrates a black diagram of the apparatus which may be utilized to implement the invention;

FIG. 2 shows a flow diagram illustrating the operation of the computer program in accordance with the principles of the invention;

FIG. 3 illustrates the further processing step in the event of a mismatch in accordance with an embodiment of the invention; and

FIG. 4 shows an alternative embodiment of the invention in block diagram form similar to that of FIG. 1.

As illustrated in FIG. 1, the error checking apparatus 10 in accordance with the invention, comprises a scanner 12, computer 14 and output 16. The computer 14 is preferably a program digital computer which has stored therein an optical character recognition program 18 which may consist of any standard, commercially available OCR program, e.g., Xerox's ScanworX or Textbridge. Many different OCR processing approaches have been utilized and are commercially available. Some techniques utilize neural networks, feature extractions such as line or circle extraction, edge examination and the like.

A confidence factor is usually provided by such conventional OCR program 18 as a measure of the degree of confidence associated with the candidate character. The output 16 may be routed to one or more of a display device, storage device, printer or data transmitter. In most cases, the output 16 would be routed to at least a CRT display so that the operator may key in flagged characters using keyboard 20 connected to the computer 14.

FIG. 2 illustrates a flow chart governing operation of the apparatus shown in FIG. 10 and including program aspects of the computer 14.

The scanner 12 is first utilized to scan a hard copy of a document containing alpha-numeric text. The digitized image is stored as a bitmap in a scanned document file SDF as indicated in step 30. The conventional OCR program 18 then performs the OCR algorithm or procedure specific to the manufacturer's design as indicated in step 32. In step 34, the results of the OCR processing are stored as a textural results file TRF within a memory of computer 14. In step 36, the program computer 14 analyzes and determines the font or fonts of characters in the SDF and may also determine a placement characteristic such as the position of the character being identified within the document or its relative position with respect to an initial character or the document edge or some other reference point. The determination of the specific font may be done with template matching techniques or other procedures per se known in the art. Likewise, determination of the position of the character being analyzed may be determined according to known techniques. The font information determined in step 36 may include the typeface information as well as the size or scale of the character being examined. Techniques for determining the font and position are included in some conventional OCR programs 18 but may alternatively be provided by separate programs in accordance with well known techniques, see for example U.S. Pat. No. 5,253,307.

The font and placement characteristic determined in step 36 are then stored as a regeneration library file (RLF) in step 38. Next, in step 40, a bitmap is generated utilizing the TRF and RLF previously stored. This bitmap is termed a regenerated image file and is based on stored font data and candidate character data from the textural document file. The regenerated image file may also utilize the positional data of the candidate character within the document. In step 42, at least a portion of the regenerated image file (bitmap image) is compared with the corresponding portion of the bitmap from the scanned document file to determine if there is a match. The placement characteristic may be used to determine the corresponding portion of the SDF that aligns with the portion of the regenerated image file. If a match occurs as determined in step 44, the program proceeds to step 46 wherein the matched portion of the textural results file (TRF) is utilized as the correct output text. If a match does not occur then further processing takes place as indicated in step 48.

The comparison step 42 may take place with respect any portion or the whole of the document being digitized. The portion may be a character, word, text string, sentence, paragraph, page, multiple pages or the whole document. For example, the portion may simply be that portion of the regenerated image file that corresponds to a particular character (e.g., the first character of the document) with the corresponding character in the textural results file. Error checking in this embodiment simply proceeds on a character-by-character basis. In another embodiment, the portion may be a page, and the comparison may take place on a page-by-page basis. In this case, the determination of a match in step 44 is relative to the entire page, and if there are no errors, the entire page is accepted as correct. If there is a "no match" condition, the entire page is further processed in step 48.

The match condition may be set to some predetermined criteria such that the bitmap overlap between the candidate bitmap generated in step 40 and the bitmap from the scanned character file stored in step 30 are congruent within a predetermined percent such as 99% or the like. Two pixels are congruent if they each have a "dark" or "1" bit at a pixel location and are not congruent if one pixel location indicates a dark spot (bit 1) and the other pattern indicates a light spot (bit 0).

The further processing step (48) may include machine resolution of mismatches by alternative or additional OCR algorithms, alternative regeneration engines utilized in step 40 or alternative preset criteria or confidence levels utilized in steps 42 and 44. In its simplest implementation, the further processing step 48 entails utilizing a display and providing a special character at a position corresponding to the character in question so that a human operator may examine the original hard copy text object or the displayed SDF and determine from the indicated position or the special character on the display the corresponding position on the hard copy to thus identify the correct character and key in same. The display may utilize a flashing and/or special character (e.g., a "tilde") as a predetermined output indication to assist the operator in locating the corresponding image on the hard copy. Alternately, a reverse video image may be utilized or any other method to flag the particular character as one that needs to be checked by the operator. All or portions of the SDF and TRF may be displayed side-by-side in a split-screen mode to facilitate operator arbitration of mismatches between corresponding characters.

The further processing of step 48 (FIG. 2) is illustrated in FIG. 3. In step 60 mismatches are identified with respect to any portion or whole of the document(s) scanned. In one embodiment, mismatches are routed to be further analyzed by means of machine resolution techniques as in step 62 with subsequent human arbitration in step 64 if the results of step 62 do not resolve the error. Alternately, the mismatch condition may be directly routed for human arbitration as in step 66. Machine resolution techniques include alternate or additional OCR algorithms (including voting algorithms); alternate regeneration engines; alternate comparison confidence levels or other techniques. Human arbitration generally comprises visual inspection and inputting in a correct character(s) or selecting a character among several displayed choices. The inputting/selecting may be done by keyboard, voice command, touch screen, mouse or any other technique.

The application of the principles of the invention to multiple OCR engines is straightforward and is illustrated in FIG. 4. Scanner 112 is utilized to scan the alpha-numeric text of the hard copy document and the digitized results are fed to a plurality of OCR engines such as indicated by 118-1, 118-2 . . . 118-N. These OCR engines are software programs stored within one or multiple computers 114. An output device at 116 is comparable to device 16 in FIG. 1. When multiple OCR engines are utilized, a voting technique or some other algorithmic technique is employed to appropriately combine and weigh the various results of the different OCR algorithms. A single output is provided representing the most probable character candidate. See, for example, U.S. Pat. Nos. 5,455,872 and 5,418,864 incorporated herein by reference. Once the most probable candidate is determined, the candidate character is processed according to step 34 in FIG. 2 and steps subsequent thereto. Thus, FIG. 2 is appropriate for the embodiment of FIG. 4 except that the OCR algorithm step 32 is now replaced by a multiple OCR algorithm step and a selection among the OCRs of the most probable character.

While the invention has been described with reference to preferred embodiments, it is understood that alternative embodiments will be apparent to those of skill in the art, and the invention is intended to cover all such alternative embodiments as are encompassed by the attached claims.