Image Processing for Pattern Extraction

1. Field of the Invention

The invention relates to image processing method and apparatus which can correctly extract desired information from input image information and can recognize the extracted partial information.

2. Related Background Art

A conventional apparatus which can recognize characters from image information is constructed as shown in FIG. 12. Reference numeral 41 denotes a scanner for converting a character image on an object to be read into an analog electric signal; 42 a preparation signal processor to binarize the analog signal and to eliminate noise from the signal; 43 a character extractor to separate a character train into individual characters; 44 a feature extractor for extracting a feature which is peculiar to the particular character being examined and for producing a feature vector in accordance with a predetermined algorithm; 45 a recognition dictionary to store statistics (mean value, distribution, and the like) pertaining to each character type of the feature vector; 46 a comparator for comparing the feature vector obtained from the input character image and the recognition dictionary, thereby selecting the optimum candidate; 47 a word dictionary to store the results of translations of words; 48 a dictionary retrieval unit to extract the corresponding translated word from the recognized character train with reference to the word dictionary; and 49 a display to display the translated word.

The above conventional technique, however, has the following two drawbacks.

(1) What are called blank characters, which are written in white on a black background, cannot be recognized. FIGS. 7 and 10 show examples of such blank characters.

As a conventional technique, there is known a method whereby a ratio of the number of black pixels in the whole image buffer is counted, and when the ratio is equal to or larger than a predetermined value, the characters are determined to be blank characters, those blank characters are inverted from white to black, and, thereafter, a recognizing process is executed. Such a method, however, has a drawback in that it takes a long time to count the number of black pixels, and consequently the whole processing time increases. As shown in an example of FIG. 9, on the other hand, there are cases where, when character lines are very thick, those characters are erroneously identified as blank characters. Or, as shown in FIG. 10, there are cases where, in spite of the fact that the characters are really blank characters, they cannot be so identified because there are a large number of white pixels. There is, consequently, a drawback in that it is not always possible to judge correctly whether given characters are blank characters.

(2) As shown in FIG. 11, when a ruled line exists before a character train to be recognized, the characters of the train (here, the word "take") cannot be extracted.

Hitherto, when the operator tries to extract character information by tracing an outline of the character information by using a certain point of an input image as a start point, so long as a ruled line exists as shown in FIG. 11, the tracing starts from the start point and reaches the edge of the input image, and the tracing ends at such an edge. Therefore, the outline tracing operation doesn't reach the character train "take" and "take" cannot be extracted.

It is an aspect of the invention that when desired image information is extracted from input image information by tracing an outline, in the case where the outline tracing point reaches the edge of the input image, the start point of the outline tracing operation is moved and the outline tracing operation is again executed, thereby making it possible also to perform an extraction of desired image information from an image in which a delimiter image such as a ruled line or the like exists between the original start point of the outline tracing operation and the desired image.

Another aspect of the invention is that the shifting of the above start point and the outline tracing operation are repetitively executed, and when the number of shifting of the start point or the amount of a shift thereof exceeds an arbitrary value (some number of times or length), the input image is determined to be blank characters, and is inverted from white to black, and thereafter, desired image information is again extracted, thereby making it possible also to execute extraction of the blank characters from the image.

To solve the above problems, it is an aspect of the invention to provide an image processing method whereby an outline of character information is traced by using a certain point of input image information as a start point and, when the outline reaches an edge of the input image, the start point is moved, and the outline is again traced from the moved start point.

To solve the above problems, according to one preferred aspect of the invention both of a tracing of the outline and the movement of the start point are repeated until the character information can be extracted by the outline tracing operation.

To solve the above problems, according to another preferred aspect of the invention, when the outline tracing operation and the movement of the start point are repeated, the number of movements of the start point is counted, and when the count value exceeds a predetermined value, the image information is inverted.

To solve the above problems, according to still another preferred aspect of the invention, both the outline tracing operation and the movement of the start point are repeated until the character information can be extracted by the outline tracing operation.

FIG. 1 is a block diagram showing an embodiment of the invention;

FIGS. 2A, 2B and 2C are external views of an apparatus of the embodiment of FIG. 1;

FIG. 3 is a diagram showing a storing method used in an image buffer;

FIG. 4 is a flowchart showing the overall process in the embodiment of FIG. 1;

FIG. 5 is a flowchart showing an extracting process;

FIG. 6 is a diagram for explaining a character extraction by an outline tracing operation;

FIG. 7 shows an example of blank characters;

FIG. 8 shows an example of ordinary characters;

FIG. 9 shows an example of ordinary characters (bold characters);

FIG. 10 shows an example of blank characters;

FIG. 11 shows an example of ordinary characters in an image also including a ruled line; and

FIG. 12 shows an example of a construction of a conventional electronic dictionary of the recognizing type.

FIG. 1 is a block diagram showing an embodiment according to the invention and shows a construction of an electronic dictionary of the recognizing type in which a document on which an English word has been printed is read by a manual reading operation, characters are recognized and a Japanese translation of a word is displayed by searching a dictionary by using the result of the character recognition.

In the diagram, reference numeral 1 denotes a scanner to convert image information of an object to be read into an analog signal; 2 a binary converter to convert the analog signal generated from the scanner 1 into the binary digital signal; 3 an encoder which is synchronously rotated when the object is read by a manual scan and which generates a signal synchronized with the manual scan by detecting the rotation of the encoder by a photo interrupter; 4 a reed switch for monitoring a depressing state of a roller (i.e., whether the roller is depressed or not) to impart rotation to the encoder and for detecting the end of the reading operation; and 10 a central processing unit (CPU) comprising a large scale integrated circuit (LSI) including an interruption input port, an interruption control circuit, clock pulse generator, an instruction decoder, registers, an ALU, an input port, an output port, and an image input shift register. Reference numeral 11 denotes a random access memory (RAM) having a readable and writable memory section which is assigned to every address. As functions of the memory section, there can be mentioned a memory function to store data, a flag function to store the result of the discrimination process described below, a counting function to store a state by a count value, a register function for temporary storage, and the like. Reference numeral 12 denotes a read only memory (ROM) in which microprograms which are sequentially executed by the CPU 10, a recognition dictionary, an English/Japanese dictionary, and constants which are used in various kinds of discriminations or the like, have been encoded and stored. Reference numeral 13 denotes a liquid crystal display driver (LCD driver) for receiving output data and generating a signal to display on a liquid crystal display (LCD) 14; 15 a keyboard; and 16 an external bus line including an external address bus and an external data bus. The addressing of the ROM and RAM, the transmission and reception of data, and the like are executed through the external bus line 16.

FIGS. 2A, 2B and 2C are diagrams showing external views of an apparatus of the embodiment of FIG. 1. Reference numeral 21 denotes an upper casing; 22 a lower casing; 23 an array of input keys; 24 a liquid crystal display; 25 a roller to impart rotation to the encoder; 26 a reading unit; 27 a mark for reading; and 28 a cavity for a thumb to facilitate holding the apparatus.

First, image buffers which are provided in the RAM 11 will be described with reference to FIG. 3. A scanner used in the embodiment is a one-dimensional line sensor and reads image data line by line. The invention can be also applied to a line sensor which can read image data several lines at a time. Reference numeral 31 indicates image buffers which are provided in the RAM 11; the x direction is called a column, and the y direction is called a row. The image data of one line which has been read is sequentially stored from the 0th column in the direction from the left to the right. Since the image data has already been binarized by the binary converter 2, a "1" in the image buffer corresponds to a black pixel and "0" corresponds to a white pixel in the original image.

FIG. 4 is a flowchart showing the overall processing of this embodiment. The processing is executed under the control of the CPU 10 in accordance with programs stored in the ROM 12. A character image is read from the scanner 1 in step S401. In step S402, what is called a character extraction to determine a rectangular region of each character is executed by the CPU 10. In step S403, a checking process is executed for every rectangular region by the CPU 10. In step S404, the CPU 10 searches the dictionary in the ROM 12 by using the character train obtained. In step S405, the translation results are displayed by the LCD 14.

The extracting process in step S402 will now be described in detail in accordance with a flowchart of FIG. 5. Processes in the flowchart of FIG. 5 are executed under the control of the CPU 10 in accordance with the program stored in the ROM 12.

In the first step S501, initial values are set into the data in the RAM 11. That is, a value of a variable p to store the x coordinate of the extraction start position, a value of a variable "cline" indicative of the number of columns replaced to the white pixels, and a value of a flag "rflag" indicating whether the whole contents of the image buffers have been inverted, are set to "0". The first column of the image buffers is set to the white pixels in order to perform the extraction by outline tracing. FIG. 6 shows an example of character images stored in the image buffers in the RAM 11. In step S502, a character is extracted by outline tracing beginning from a point a.sub.1 at the left edge central position in the image buffer and tracing to the right. The pixels are examined from the point a.sub.1 to the right. The outline tracing is executed from a point at a position where the outline tracing first encounters a black pixel. A rectangular region which is given by the minimum value (x.sub.1, y.sub.1) and the maximum value (x.sub.2, y.sub.2) of the outline coordinates obtained by the outline tracing is set as a region containing one character. The algorithm of the outline tracing is a well-known technique. In step S502, each time the outline tracing is started, the x coordinate is substituted as a new value of p. After the region of one character is derived, the pixels are examined moving from a position a.sub.2 of the coordinates (x=x.sub.2, y=(y.sub.1 +y.sub.2)/2) to the right in order to extract the next character, and processes similar to those mentioned above are executed.

In the case of the blank characters as shown in FIG. 7 or 10 or in the case where a ruled line exists before the word to be read as shown in FIG. 11, the outline tracing position reaches the upper/lower or left/right edge of the image buffer during the outline tracing operation and, after that, outline tracing cannot be performed. Even in the case of ordinary characters as shown in FIG. 8, after all the characters are extracted, the outline tracing position finally reaches the right edge of the image buffer, so that a discrimination by the CPU 10 regarding whether the extraction has succeeded or not is executed in step S503. That is, when the outline tracing position reaches an edge after one or more characters are extracted, the outline tracing is returned. When cline reaches the edge without extracting any character, the processing routine advances to step S504. In step S504, when the value of the variable cline in the RAM 11 is equal to or less than a predetermined threshold value N, step S505 follows. If it is larger than N, step S509 follows. In step S505, the value p of the x coordinate at which the outline tracing has been started is increased by only a predetermined value d. In step S506, the values of the line x=p are stored in the line memory buffers set in other addresses different from those of the image buffers in the RAM 11. The line memory buffer can store the data of N lines. In step S507, all of the values of x=p in the image buffers are set to "0". In step S508, the value of cline in the RAM 11 is increased by "1" and after that, the processing routine is returned to step S502 and the extracting process is again executed.

In the case where a ruled line is included as shown in FIG. 7, the processes in steps S505 to S508 for setting all of the values in the image buffers of one line to "0" are executed once so long as the ruled line is a thin line. Those processes generally are repeated only few times even when such a ruled line is a thick line. Due to this, the outline tracing position can get over the ruled line, so that the word subsequent to the ruled line can be accurately extracted.

On the other hand, when cline in the RAM 11 is larger than N in step S504, step S509 follows. When rflag in the RAM 11 is equal to 1, step S510 follows the data of N lines stored in step S506 is returned to the image buffers and the original image data is reconstructed. In step S511, all of the values in the image buffers are inverted. That is, "0" is inverted into "1" and "1" is inverted into "0". By the above process, blank characters as shown in FIG. 7 or 10 are also inverted and can be recognized in a manner similar to ordinary characters.

To store an indication that the inverting process has been executed, "1" is set into rflag in the RAM 11 in step S512. After that, step S502 follows and the extracting process is again started. In step S512, both cline and p in the RAM 11 are reset to "0".

When rflag in the RAM 11 is equal to 1 in step S509, this means that the inverting process has already been performed once. There is no reason to invert again. Therefore, the CPU 10 determines that the input image is an image that cannot be recognized. The processing routine is now finished.

In this embodiment, after the image data is stored in the whole image buffers, the extracting process is started. The extracting process can also be executed in parallel with the transfer of the image data from the scanner. In such a case, when it is assumed that the image data has already been transferred until x=x.sub.1 at a certain time point t, it is necessary to wait for the execution of each process in steps S502, S505, and S511 in the flowchart of FIG. 5 so as not to exceed x=x.sub.p.

In this embodiment, although the data is transferred from the scanner into the image buffer line by line, the data transfer can instead be performed row by row.

Again, in this embodiment, an English word is read and the corresponding word in Japanese is displayed. However, the particular language to be read is not limited so long as it is one in which spaces exist between consecutive characters (characters are separately written).

The invention also doesn't depend on the particular recognition algorithm used and word extracting algorithm but is also effectively applied with other algorithms.