System to Achieve Automatic Recognition of Linguistic Strings

The present invention relates to systems to achieve automatic recognition of text.

Attention is called to a number of papers authored by the present inventors (Srihari and Hull) together with another: "Integrating Diverse Knowledge Sources in Text Recognition," Srihari et al. (June 21-23, 1982, Conference Abstract); "Integrating Diverse Knowledge Sources in Text Recognition," Srihari et al. (June 21-23, 1982, Proceedings Supplement); "Integration of Bottom-up and Top-down Contextual Knowledge in Text Error Correction," Srihari et al. (June 1982); "Knowledge Integration in Text Recognition," Srihari et al. (August 1982); and "Integrating Diverse Knowledge Sources in Text Recognition," Srihari et al. (1983). The papers are drawn upon heavily in the present specification.

The capabilities of present commercial machines for producing correct text by recognizing words in print, handwriting and speech are very limited. For example, most optical character recognition (OCR) machines are limited to a few fonts of machine print, or text that is handprinted under certain constraints; any deviation from these constraints will produce highly-garbled text. There is described herein an algorithm for text recognition/correction that effectively integrates a bottom-up refinement process that is based on the utilization of transitional probabilities and letter a posteriori probabilities, known as the Viterbi Algorithm (VA), together with a top-down process based on searching a trie structure representation of a lexicon. The algorithm is applicable to correcting text containing an arbitrary number of character substitution errors such as that produced by OCR machines. It is directly applicable to the classification of electrical signals from a scanner, that characterize the letters of words in the text. It is also applicable to the classification of set of shape features derived from visual images of letter input to an optical scanner.

The VA is a method of finding the word with the maximum a posteriori probability over all possible letter combinations and not necessarily those in a lexicon; it is based on a dynamic programming formulation which leads to a recursive algorithm. The algorithm can be viewed as a procedure to find the shortest path through a directed graph called a trellis. The negative of the log transitional probabilities are associated with the edges of the trellis and the negative log letter a posteriori probabilities are associated with the nodes. The cost of a path is then the sum of all the edge and node values in the path. Some generalizations of the VA have used either a fixed but small number of alternatives per letter of the input word, called the Modified Viterbi Algorithm (MVA) or a variable number of alternatives per letter; these alternatives can be determined by the letters that have the highest a posteriori probability.

The VA (and its variations) is a purely bottom-up approach whose performance may be unacceptable due to the fact that the resulting strings do not necessarily belong to a dictionary. One approach to this problem is to use top-down contextual information, in the form of a dictionary of allowable input words, to aid the bottom-up performance of the VA.

The trie data structure is suitable for determining whether a given string is an initial substring, or prefix of a word in the lexicon. Essentially, the trie represents words as ordered lists of characters, elements of which are represented as nodes in a binary tree. Since the VA proceeds by computing the most likely prefix of letters given an observed string of symbols (letters, feature sets, or electrical signals), the trie is an efficient data structure.

According to the present teaching, simultaneous search of the VA trellis using a variable number of character alternatives per input letter and the trie structure is achieved using a binary array A. Element A(j i) is set to 1 if the jth letter of the alphabet is a possible correction for the ith symbol of the input word, and O otherwise. Thus the paths of the trellis that need to be evaluated are only those data that begin at the 1's of the first column of A and proceed through the 1's of the subsequent columns. Before evaluating the likelihood of the path that proceeds from one column of A to the next column, that path is determined to be legal with respect to the trie. The computational complexity of the resulting algorithm is of the same order as the VA.

Accordingly, it is an object of the present invention to provide a new, one-pass, low-cost method for recognition of handwritten, typewritten or typeset text.

Another objective is to provide a method that allows the maximum a posteriori estimate to be computed for every input string of symbols, where output words are constrained to be found in a dictionary of acceptable or allowable words.

Still another objective is to provide a text-recognition scheme that greatly increases the recognition rate of a linguistic string without significantly increasing the search time or performing an exhaustive search through the lexicon.

A further objective is to provide a method of correcting words that may contain an arbitrary number of substitutions of incorrect letters for correct ones.

These and still further objectives are addressed hereafter.

The foregoing objectives are achieved, generally, in a onepass system to achieve automatic language recognition wherein language in the form of input strings or structures is analyzed on the basis of: channel characteristics in the form of probabilities that a letter in the input is a corruption of another letter or, more generally, class-conditional probabilities (i.e., probability of the observed symbol for each possible state of nature); the probabilities of the letter occurring serially with other recognized letters that precede the letter being analyzed or particular strings of letters occurring serially; and lexical information in the form of acceptable words represented as a graph structure. The input language is typically text received from an optical character reader and the output typically is a printed output; however, the input can be speech and the output can be speech generated, for example, from the output of an optical character reader.

The invention is hereinafter described with reference to the accompanying drawing, the single FIGURE of which is a block diagram of a system to practice the invention.

Turning now to the drawing, the system shown at 101 serves to achieve automatic recognition of text 2. In the disclosed embodiment the text at 2 is scanned by an optical scanner 1 which provides electrical signals derived from the text scanned, which electrical signals are the electrical analog of words in the text. The electrical signals are transmitted along a conductor 5 to an analyzer 3 which performs analysis to achieve automatic recognition of the individual letters in a word on the basis of: channel characteristics in the form of probabilities that a particular letter is a corruption of another letter or class-conditional probabilities associated with the observed electrical signal or feature sets derived therefrom; the probabilities of the letter occurring serially with other recognized letters; and lexical information in the form of acceptable words (or prefixes) represented as a graph structure. The output of the analyzer 3 is fed to an output device 4 (e.g., a printer). Typically the optical scanner 1 and the analyzer 3 are one piece of apparatus as indicated by the broken line 102 which may be an optical character reader (OCR) whose output 6 is connected to a word processor at 4, the signals at 6 being in the form of binary digital bits. On the other hand, the output 6 can be fed to a device at 4 which is capable of generating speech (e.g., for the blind).

The present system effectively merges a bottom-up refinement process based on the use of transition probabilities with a top-down process based on searching a trie structure representation of a dictionary. As above indicated, two types of bottom-up information are employed: probability of letter shape (represented as a feature set) conditioned on true letters and the probability of a letter occurring when the previous letters are known (transitional probability). Results of experimentation employing the present teachings show a significant increase in correction rate over predecessor systems that do not use lexical information and show no increase in the order of complexity. Owing to its superior performance, the concept herein is suggested as a word hypothesization component in a system focusing global contextual knowledge on the text-recognition or speech-recognition problem. A formal statement of what the inventors call procedure dictionary-viterbi, is given below. The procedure is stated as an error corrector for text that may contain substitution errors. It can be simply modified for classifying electrical signals or feature sets by allowing X to consist of a set of m observed electrical signals or feature sets.

The above statement is explained in some detail in what now follows, wherein there is described, in order, the procedures initialize and dictionary-trace, and the procedure select. Procedure initialize selects the d most likely alternatives for each letter of the input word. This is done by choosing those d letters for which the sum of the log-confusion and log-unigram probabilities is greatest. This procedure refers to the following primitive function: max(G:array [l . . . r] or real; var u:integer), which returns the maximum of the elements of an array G, with the side effect that u contains the index of the maximum element.

An alternative way of implementing initialize is to select a variable number of alternatives for each letter based on a threshold t. In this case the k loop above is replaced by

Procedure dictionary-trace, described next, returns a set of letter strings in vector S whose costs are defined by vector C. In addition to calling function max, as defined for procedure initialize, this procedure refers to the following primitive procedure: concat(s1, s2:string; L.sub.j :char), which concatenates character L.sub.j at the end of string s2 and returns the result in s1.

Procedure select returns the most likely word by considering the cost of the transition from the final symbol to the trailing delimiter B using the cost vector C and the survivor vector S. If all the values in C are equal to minus infinity, X is rejected and a null value is returned.

The integration of the dictionary into the VA is done in procedure dictionary-trace by maintaining a vector of pointers into the trie. Each element corresponds to a survivor string. At each iteration of the k loop the kth element of this vector is passed to access-trie. If the corresponding initial substring concatenated with the letter indicated by the j index is a valid dictionary string (access-trie is true), the probability calculation is carried out. A weight of minus infinity is given to this alternative when a false value is returned to preclude the possibility of nondictionary words being considered. At some iteration in the j loop, if all attempted concatenations fail to produce a valid dictionary string, the survivor for the corresponding node and its pointer are assigned null values. For some value of i, if all survivors are null, the input word is rejected as uncorrectable. This may occur when fewer than r possibilities are considered for each letter but will never happen when all candidates are allowed. This phenomenon is discussed further in the papers cited above.

A variation of procedure dictionary-viterbi is not explicitly to maintain survivor strings but to derive them from a vector of pointers Q. However, this would necessitate son-to-father pointers to allow a path to be traced from each node back to the first level of the trie to retrieve the string. This would yield a storage economy when the number of nodes was less than the number of locations required for the survivor strings due to the additional backward pointers.

Procedure dictionary-viterbi can be modified to handle second-order transitional probabilities by making two simple changes. In step a of procedure dictionary-trace, replace P(L.sub.j .vertline.L.sub.k) by P(L.sub.j .vertline.(S[k]).sup.i-1,L.sub.k), where (S[k]).sup.i-1 denotes the (i-1)th letter of the kth string in vector S. In step b of procedure select, replace P(B.vertline.L.sub.k) by P(B.vertline.S[k]).sup.m-1, L.sub.k).

The description herein emphasizes the text-recognition aspects of the invention as may be employed in the context of an OCR, for example, and it is with respect to such apparatus that the system described has been employed; however, the present inventors believe that the teachings can be used in the context of a speech input. In the latter context, the device 1 would be a mechanism that converts a vocal input of words to an electrical analog which would be analyzed and coverted to an output at 6 that could be printed. For example, the mechanism at 1 could be a process that segments a speech signal and thereafter extracts the phonological features contained therein.

Further modifications of the invention herein disclosed will occur to persons skilled in the art and all such modifications are deemed to be within the scope of the invention as defined by the appended claims.