Character recognition system

[0002] The present invention relates to a character recognition system.

[0003] In a conventional character recognition system to read out the image of characters written on paper and produce the read-out characters as character codes capable of being processed on a computer, a character area is extracted from the document image, each of the characters is extracted from the extracted character area, and character recognition is executed for each division area.

[0004] In such a conventional character recognition system, since the character area is extracted before the character recognition process there may be errors in the extraction of the character due to causes such as blots and blurs of the image. If such errors occur, it is impossible to restore or compensate for the error, reducing the character recognition rate.

[0005] It is therefore an object of the present invention to provide a character recognition system having an improved recognition rate.

[0006] According to the present invention, there is provided a character recognition system comprising: feature extraction parameter storage means for storing a transformation matrix for reducing a number of dimensions of feature parameters and a codebook for quantization; HMM storage means for storing a constitution and parameters of a hidden Markov Model (HMM) for character string expression; feature extraction means for scanning a word image given from an image storage means from left to right in a predetermined cycle with a slit having a sufficiently smaller width than the character width and thus outputting a feature symbol at each predetermined timing, and matching means for matching a feature symbol row and a probability maximization HMM state, thereby recognizing the character string.

[0007] According to another aspect of the present invention, there is provided a character recognition system comprising: an image scanner; an image information storage means for storing document image data read out from the image scanner and pertaining image area information; a feature extraction parameter storage means for storing a transformation matrix to reduce a number of dimensions of feature parameters and a codebook to obtain feature symbols, the transformation matrix expressing each feature parameter which comprises multivariates as a small number of variates to minimize information loss, and previously calculated from a training sample feature parameter through main component analysis, the codebook being a set of codevectors used for quantization to express the transformed feature parameter with low bits, and previously calculated from the training sample feature parameter; a character string HMM storage means for storing a constitution and parameters of a character string HMM expression, the character string HMM being obtained by preparing one HMM for each character and adding a state transition from the completion state of each character to an initial state of each character HMM; an image extraction means for extracting an area with characters written therein from the document image and extracting rows and words from the extracted character area and storing word image area information thus obtained in the image information storage means; a feature extraction means for scanning the image of each word area from left to right in a predetermined cycle with a slit having a sufficiently smaller width than the character width and producing a feature symbol at each predetermined timing, and a matching means for making correspondence of the feature symbol sequence with a HMM state so as to maximize the probability of recognizing character string by utilizing the resultant optimum state transition sequence.

[0008] Other objects and features of the present invention will be clarified from the following description with reference to the attached drawings.

[0009] FIG. 1 shows an embodiment of the present invention;

[0010] FIG. 2 shows a block diagram of the feature extraction means 32 in FIG. 1;

[0011] FIG. 3 shows an example of the transformation matrix in the embodiment;

[0012] FIG. 4 shows an example of the codevector on the embodiment;

[0013] FIG. 5 shows a constitution and parameters of HMM in the embodiment;

[0014] FIG. 6 shows an example of the input image in the embodiment; and

[0015] FIG. 7 shows a diagram for explaining the feature extraction method.

[0016] The present invention will now be described with reference to the drawings. FIG. 1 shows an embodiment of the present invention. The embodiment comprises an input unit 1 such as an image scanner, a storage unit 2 for storing information, a data processor unit 3 operable under a program control, and an output unit 4 such as a display. The storage unit 2 includes an image information storage section 21, a feature extraction parameter storage section 22, and a character string HMM storage section 23. The term “HMM” is an abbreviation of “Hidden Markov Model”. The data processor unit 3 includes word image division means 31, feature extraction means 32 and matching means 33.

[0017] The image information storage section 21 stores document image data read out from the image scanner and pertaining image area information.

[0018] In the feature extraction parameter storage section 22, a transformation matrix for reducing the dimension number of feature parameters and a codebook for obtaining feature symbols are prior to scanning the image. The transformation matrix is employed to express each feature parameter which comprises multivariates as a small number of variates to minimize the information loss, and it is previously calculated from a training sample feature parameter through main component analysis or the like. The codebook is a set of codevectors used for the quantization to express the transformed feature parameter with low bits, and it is previously calculated from the training sample feature parameter. The codebook may be obtained by, for instance, a method using a LBG algorithm or the like as disclosed in IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. COM-28, NO.1, JANUARY 1980.

[0019] The character string HMM storage section 23 stores the constitution and parameters of HMM of the expression of the character string. Character string HMM is obtained by preparing one HMM for each character and adding a state transition from completion state of each character to initial state of each character HMM. Character HMM is constituted as a left-to-right HMM which has 2 to 8 states depending on the kinds of characters, and only self-transition and transition to the next state. Such parameters as the output symbol probability and state transition probability are previously determined for each HMM by using a large number of training samples. Parameters of the HMM may be determined by, for instance, a method using Baum-Wealch algorithm, as disclosed in “Speech Recognition with Probability Models”, Seiichi Nakagawa, edited by the Society of Electronics Information Communication Engineers of Japan, 1988, pp. 55-60.

[0020] Document image data supplied from the input unit 1 is stored in the image data storage section 21. The image division means 31 extracts an area with characters written therein from the document image and divides rows and words from the extracted character area. Word image area information thus obtained is stored in the image information storage section 21. The extraction method of rows from the document image may be made by utilizing, for instance, a method disclosed in 1987, 35-th Information Processing Conference, in Japan, 5k-8. The extraction of words may be executed by a method such that when a blank in the image which is obtained by a longitudinal image projection has a length in excess of a predetermined threshold value, the detected blank is regarded to be an end of a word. This method is not directly relevant to the present invention, so it will not be described in detail.

[0021] The feature extraction means 32 scans the image of each word area from left to right in a predetermined cycle with a slit having a sufficiently smaller width than the character width and produces a feature symbol at each predetermined timing.

[0022] The feature extraction means 32 will now be described in detail with. reference to FIG. 2. As shown in FIG. 2, the feature extraction means 32 includes feature parameter extraction means 321, feature parameter transformation means 322 and quantization means 323.

[0023] The feature parameter extraction means 321 extracts from the slit image feature parameters such as the number of strokes at the center of the slit, center position, run length, etc. The feature parameter transformation means 322 transforms the parameters using the transformation matrix stored in the feature extraction parameter storage section 22 to reduce the number of feature parameter dimensions. The quantization means 323 performs vector quantization of the transformed feature parameter by determining a codevector with the minimum distance from the transformed feature parameter among from the codevectors in the codebook stored in the feature extraction parameter storage section 22, and produces the obtained codevector's number (feature symbol).

[0024] The matching means 33 makes correspondence of the feature symbol sequence with a HMM state so as to maximize the probability of recognizing the character string by utilizing the resultant optimum state transition sequence. The optimum state transition sequence may be calculated by a method using a Viterbi algorithm disclosed in the foregoing “Speech Recognition with Probability Models”, Seiich Nakagawa, edited by the Society of Electronics Information Communication Engineers in Japan, 1988, pp. 44-46. From the optimum state transition sequence thus obtained, a transition from the final state of each character HMM to a different state is extracted. Next, characters corresponding to the transition known are connected together to obtain the character string. Then, a character in the state corresponding to the final symbol in the feature symbol row is coupled to the end of the character string. The result is supplied as a result of recognition to the output unit 4.

[0025] Now, the embodiment will be described by taking a specific example. For the sake of brevity, it is assumed that the feature parameters are of four dimensions, i.e., number of center strokes, center position and first and second run lengths; the number of transformation dimensions is 2. The transformation matrix is as shown in FIG. 3. The number of symbols is 4. The codebook is as shown in FIG. 4. The character set to be recognized consists of three characters of “a”, “b” and “c”. The character HMM state number is 2, and the constitution and parameters of HMM are as shown in FIG. 5. In FIG. 5, the vertices and the directed arcs correspond to the states and the state transitions of the HMM. Output probability and transition probability of each state are shown in the table.

[0026] It is further assumed that one of the word image areas obtained as a result of division of an image given from the input unit 1 in the image division means 31, has an image as shown in FIG. 6.

[0027] Referring to FIG. 7, first, the feature extraction means 32 scans the word image from left to right in a predetermined cycle with a slit having a sufficiently smaller width than the character width and supplies a feature symbol at each predetermined timing. For example, with slit No. 10 the feature parameter extraction means 321 extracts, as parameters, 2 as the number of strokes, 20 as the center position, and 5 and 7 as the first and second run lengths, respectively. The feature parameter transformation means 322 transforms these parameters with the transformation matrix (FIG. 3) to obtain a vector (9.0, 34.4). The quantization means 323 selects a codevector closest to the vector (9.0, 34.4) from the codebook (FIG. 4) to obtain symbol No. 2.

[0028] Now, it is assumed that a feature symbol sequence of “t=1, 2, 2, 8, 3, 4, . . . , 1” is obtained in this way. This sequence and the HMM (FIG. 5) stored in the HMM storage section are matched using the Viterbi algorithm to obtain an optimum state transition sequence.

[0029] It is assumed that an optimum state transition sequence of “S1, S1, S1, S1, S2, S2, S2, S2, S5, S5, S5, S5, S5, S6, S6, S6, S6, S6, S3, S3, S3, S3, S3, S4, S4, S4, S4, S4” is obtained in this way. In this case, transitions from the final state of each character to a different state that are extracted are one from S2 to S5 and one from S6 to S3. Characters that correspond to the transition known state of these transitions are “a” and “c”. By adding these characters and then connecting, to the end of this character string, character (b) corresponding to the state S4 matched to the final symbol in the feature symbol sequence, a recognized character string of “acb” is obtained. This result is supplied to the output unit 4.

[0030] As has been described in the foregoing, in the character recognition system according to the present invention the division and recognition of word image are made simultaneously using HMM. It is thus possible to reduce recognition errors due to division errors.

[0031] Changes in construction will occur to those skilled in the art and various apparently different modifications and embodiments may be made without departing from the scope of the present invention. The matter set forth in the foregoing description and accompanying drawings is offered by way of illustration only. It is therefore intended that the foregoing description be regarded as illustrative rather than limiting.