Image Corner Detector Using Haar Coefficients

1. Field of the Invention

This invention pertains to digital processing machines for processing two-dimensional images. More particularly, this machine pertains to the detection of corners in video images where the images are transformed in two dimensions by means of the Haar transform and are represented in the transform domain by two-dimensional arrays of Haar transform coefficients.

2. Description of the Prior Art

Various systems have been developed to recognize specific images, see e.g. "Pattern Identifying Systems", U.S. Pat. No. 3,869,697; "Character Recognition Apparatus", U.S. Pat. No. 3,846,752; and "Pattern Recognition Apparatus", U.S. Pat. No. 3,597,731. These systems operate on the two-dimensional video picture to recognize a specified image such as a letter in the alphabet.

Other machines have been developed for detecting lines and edges utilizing the Hadamard transform, see e.g. "Pattern Recognition Machine for Analyzing Line Orientation ", U.S. Pat. No. 4,005,385. A system has also been developed for recognizing corners when the outline of the object is defined as a series of points, see "Automatic Corner Recognition System", U.S. Pat. No. 3,576,980. The latter system operates in the image domain and requires as an input the location of a series of points which lie on the boundary of the image.

The invention described in this application utilizes the two-dimensional Haar transform to detect image corners without first requiring the outline of the image be determined. Because this invention utilizes Haar transform coefficients to detect corners, it is suitable for use in connection with other, more general, image processors which also utilize the Haar transform. Furthermore, because this invention recognizes a component (a corner) of more general images, it can be used in connection with other processors to detect and identify a large number of different images based on the number, location, and relative position of the corners identified by this invention.

This invention Haar transforms, in two dimensions, digital data representing two-dimensional images. The absolute magnitude of each Haar transform coefficient, within the subset of transform coefficients that are sensitive to corners, is compared to a threshold. Whenever a corner sensitive, transform coefficient exceeds the threshold, this invention generates an output indicating the existence of a corner and outputs the associated transform coefficient and its identifying address. The threshold may be fixed at a value preselected by the operator or the threshold may be automatically adjusted in an adaptive manner dependent upon the number of corners detected in previous sets of transform coefficients.

FIG. 1 is a block diagram of the invention; and

FIG. 2 is a block diagram of the portion of the invention which compares the transform coefficients with a threshold and adjusts the threshold in an adaptive manner.

Referring now to FIG. 1. Digital data representing two-dimensional images is input to two-dimensional Haar transformer 1. The digital data may be in series or parallel form, but in either case the data is input as successive sets of data representing two-dimensional arrays of picture elements. In most applications, so that memories within the invention can be modest in size, the two-dimensional array of picture elements represented by the digital data is a subset of the much larger two-dimensional image which is processed by the invention. Methods for obtaining successive two-dimensional subarrays from a much larger two-dimensional image are straightforward and well-known. For instance, the sequence of picture elements (pels) obtained from the raster scan of a two-dimensional video scene, together with identifying addresses can be processed by a logic circuit to select only those pels that are members of a specified subarray. In such an arrangement, the remaining pels, not belonging to the subarray, are ignored. During successive periods of time, the location of the subarray is shifted so as to cover, in steps, the entire video scene. As an alternative, the entire video scene could be scanned once and all the pels stored in the buffer memory from which the subarrays would then be selected, one subarray at a time, by logic circuitry, for processing by the image corner detector. See U.S. Pat. No. 3,980,809 which describes such a system as part of its "data compressor". If higher processing rates are desired, a plurality of image corner detectors could be operated in parallel to simultaneously process each of the subarrays within the video scene. In the embodiment described in this application, a single image corner detecting device is used to process successive subarrays obtained from the video scene by any convenient method including those described above.

Each successive n.times.n array of digital data is transformed by two-dimensional Haar transformer 1 into an n.times.n array of transform coefficients. A two-dimensional transformer suitable for use in this application has been described in U.S. Pat. No. 3,981,443 and U.S. Pat. No. 4,189,748. Two-dimensional Haar transformer 1 outputs successive two-dimensional arrays of transform coefficients, together with their addresses, to comparator and thresholder 3. The transfer of this data may be in either serial or parallel form. The addresses are also output by two-dimensional Haar transformer 1 to a read only memory 2 which contains a list of the subset of transform coefficients in the n.times.n array which are sensitive to corners. By the way of example, the coefficients in a 16.times.16 array which are sensitive to corners are indicated by "x's" in Table 1. The embodiment described here utilizes 16.times.16 arrays of picture elements and of transform coefficients. Larger or smaller arrays, however, could be used, although "n" is limited to be a power of 2.

The corner sensitive coefficients are those coefficients with row and column address of "i" and "j" for which 1+2.sup.k-1 <i,j.ltoreq.2.sup.k, where k is +1, +2, +3, . . . . The coefficients specified above are sensitive to corners in the sense that a corner detecting coefficient will be significant in size if a significant change in image brightness occurs in both the vertical and horizontal directions within the subset of picture elements within the n.times.n array of picture elements for which the particular transform coefficient is sensitive to such changes. The subset of picture elements for which each of these transform coefficients in sensitive may be obtained from an examination of FIG. 4 in U.S. Pat. No. 3,981,443 which contains an example of the one-dimensional Haar transform for a 16.times.16 array of elements. It must be remembered that the transform illustrated in FIG. 4 is in one dimension, so in order to determine the relationship between the subset of picture elements in the image domain and the corner sensitive transform coefficients, the transform in FIG. 4 must be applied twice, once in the vertical direction and once in the horizontal. Table 2 gives the approximate location of the areas within a 16.times.16 array of picture elements which are associated with the respective Haar coefficients that are sensitive to image corners.

The read only memory 2 provides an enabling signal to comparator and thresholder 3 to output a corner indication signal for each corner sensitive transform coefficient whose absolute magnitude exceeds a threshold that has been preset in the comparator by the operator. Whenever the corner sensitive coefficient exceeds the threshold, an indication of the corner is output together with the value of the coefficient and its address.

The information output by the comparator and thresholder 3 can be used by a more general image processing machine which may, for instance, utilize the corner information to identify a particular image from its library of the number and relative positions of the corners for different images. The corner indicator also can be used in conjunction with a device which would replace the particular transform coefficient for which a corner is indicated by the maximum value that is utilized by the machine for representing such transform coefficients. Upon retransformation of the array of coefficients into the image domain, the detected corner then would appear as a bright and/or dark area within the reconstructed image.

The threshold contained in comparator and thresholder 3 may be a fixed threshold preselected by the operator. However, in some applications, an adaptive threshold would be more useful. In such circumstances, the preferred embodiment would include provisions for an adaptive threshold. FIG. 2 is a more detailed block diagram of comparator and thresholder 3 which contains a comparator and provides for automatically adjusting the threshold. Referring now to FIG. 2, the transform coefficients, addresses, and an enabling signal from read only memory 2 enter comparator 4, together with a threshold signal. For each edge sensitive transform coefficient whose absolute magnitude exceeds the threshold, the comparator 5 outputs a corner indication, together with the transform coefficient and its address, in the same manner as that described for comparator and thresholder 3. However, the corner indication from comparator 4 also is output to counter 5 which counts the number of such indications that are output for each n.times.n array of transform coefficients. At the end of the processing of each n.times.n array of coefficients, the counter outputs the number of corners indicated in the array to threshold and threshold adjuster 6. Threshold and threshold adjuster 6 maintains a value for the threshold and also decreases the threshold by a preselected amount at the end of the processing of the n.times.n array of transform coefficients if the number output by counter 5 is less than a preselected number and increases the threshold by a preselected amount if the number output by counter 5 exceeds a predetermined number. The threshold that is output by threshold and threshold adjuster 6 is input to comparator 4.