Method and Apparatus for Augmenting Binary Patterns

This invention relates to systems for converting linear video data into binary representation and in particular to a technique and means for processing binary patterns so as to improve pattern measurements.

There exist many schemes for converting linear video type data into simpler binary patterns. These are easier to manipulate and require less data storage. For instance, consider a portion of a video raster, P, which is quantized spatially into m .times. m picture elements (PIXELS), P.sub.ij ; i, j.epsilon.{0,1,2 . . . m}. The video level, or intensity of each PIXEL is represented by a quantized digit or gray level where P.sub.ij .epsilon.{0, g max}. For the sake of illustration, it will be assumed here that m=32 and g max=255. For this particular instance, 8.times.32.times.32=8192 bits of memory would be required to store this original image. On the other hand, if an algorithm is used to generate a binary pattern, the gray level information is compressed into a single bit. This binary pattern can be stored in a smaller memory, is easier to manipulate and requires less bandwidth for transmission. Needless to say, other reasons exist for converting linear video into a binary representation, in particular, separation of targets from background. In such schemes PIXELS which are determined by some algorithm to be representative of targets are replaced with a "1," the others are set to zero. Another scheme is to subject the gray level values of the individual PIXELS to a threshold test. PIXELS with gray level values greater than the threshold, g.sub.th, are replaced by a "1" otherwise by a "0."

The binary pattern is useful for pattern measurements since it is often more representative of the true pattern geometry than the linear video which can be influenced by glints and temperature variations. Characteristics of the pattern are often expressed in terms of measured parameters. Measurements that are of special interest are the centroid (x,y) and the spatial variances (.sigma.x.sup.2, .sigma.y.sup.2, .sigma.xy): ##EQU1## By way of illustration, consider a binary pattern consisting of a single line of "1" PIXELS, P.sub.ki, for i=0 through n.

The x centroid coordinate is given by equations one and six as: ##EQU2## Note the strong dependence on the length of the line of ones. Now, if the original linear pattern is noisy, the resulting binary pattern may not be a good representation. This is particularly true if a thresholding technique is used to form the modified pattern. Cells whose gray level value is near the threshold will have their gray levels altered by the noise in a manner such that they can appear or disappear from frame to frame, even though the object does not change. PIXELS near the edge of the pattern are particularly sensitive. In such cases the deletion or addition of a simple binary cell will cause the centroid to shift by 1/2 element. For many applications, such errors are intolerable. The present invention comprehends a procedure and an apparatus for implementing this procedure that generates a multilevel or augmented pattern which is less sensitive to the erroneous deletion and/or addition of cells. The method identifies and classifies cells according to their location with respect to the edge of the pattern. All cells composing the pattern are then assigned a value between 0 and 31 depending upon their type and the cell type of the four closest adjoining cells. What results is a pattern of PIXELS with values assigned on the basis of relative location. These values may be interpreted, in a loose sense, as probabilities that a given PIXEL is part of a pattern. Since the edge cells which are more likely to be misassigned, are given less weight, the pattern measurements tend to remain more stable.

The invention comprehends a means of processing simple binary patterns so as to improve pattern measurements. The resulting augmented patterns have PIXEL values that are dependent solely upon relative geometrical location with respect to the initial binary pattern. The invention is realized by a technique that identifies and locates various types of PIXELS in the original binary pattern and weighs them numerically in accordance with their own PIXEL type and the PIXEL types of their neighbors. This is accomplished by: locating by row and column scans, and designating as "T" type PIXELS, the PIXELS at the edges of each contiguous group of "1" type PIXELS in the rows and columns of the original binary pattern; locating and designating as "P" type PIXELS the PIXELS created by the Exclusive OR of "1" type PIXELS in row and column scan intermediate patterns; developing an intermediate pattern of "0," "1," "T" and "P" type PIXELS; and, assigning a numerical value to each PIXEL in the intermediate pattern. The numerical values are derived by considering each PIXEL as a base PIXEL and inputting the logical value of its type into a PIXEL Weight table; sequentially assessing the logical values of each of the four near neighbor PIXELS of the base PIXEL and inputting those values into the weight table; deriving from the weight table a numerical value for each of the four combinations of base PIXEL and near neighbor PIXELS; and accumulating those minimum values to provide the augmented pattern PIXEL value. Mechanization of the invention is characterized by operations that are easily implemented using standard digital logic devices.

It is a principal object of the invention to provide a new and improved method for augmenting binary patterns.

It is another object of the invention to provide new and improved apparatus for augmenting binary patterns.

It is another object of the invention to provide augmented binary patterns that are less sensitive to edge cell deletion than simple "0" and "1" type binary patterns.

It is another object of the invention to generate from a simple binary pattern an augmented binary pattern that provides improved data upon which to base pattern measurements.

These together with other objects, features and advantages of the invention will become more readily apparent from the following detailed description when taken in conjunction with the illustrative embodiment in the accompanying drawings.

FIG. 1 is a block diagram illustrating the operating environment of the augmented pattern generator of the invention;

FIG. 2 is a block diagram illustrating the mechanization of the technique of the invention;

FIG. 3 is a block diagram illustrating details of the "T" cell designator of FIG. 2; and

FIG. 4 is a block diagram illustrating the neighbor search logic function of FIG. 2.

The operating environment of the augmented pattern generator of the invention is illustrated in FIG. 1 wherein essential elements and functions are shown in block diagram form and wherein the control paths and pattern data paths are designated respectively by single and double interconnecting lines.

The augmented pattern generator requires a binary pattern as input. This is provided by means of time modulator 6, A/D converter 7, digitized pattern memory 8, processor 9, and binary pattern memory 11. Video compatible data from a sensor is digitized into a gray level by the analog to digital convertor 7. Time modulator 6, driven by a sample clock and derived sync from the video data, controls the timing of the digitization which is equivalent to a spatial quantization of the raster. It also provides address data for the digitized pattern memory 8, which retains the converted video of the raster segment of interest, and timing signals (if required) for the initialization of the preprocessing algorithm. The preprocessor 9 executes an algorithm which converts the multi-gray level digitized representation into a binary pattern, 1 bit per PIXEL. The details of the operation of the preprocessor are unimportant. It may be a simple thresholding device or it may perform a more complicated target designation function. The resulting output of the preprocessor 9 is retained in the binary pattern memory 11. The augmented pattern is generated by pattern generator 14 from this data and then made available for further processing e.g. pattern moment measurements by means of pattern memory 13 and pattern measurement means 12.

Central to the generation of the augmented pattern is the location of cells at the edge of a contiguous group of "1" cells in any given row or column of the binary pattern. Should any row or column contain more than one separate group of "1" cells, each such group is considered separately. The cells at the ends of each such group are designated "T" cells and they are found by the following procedure. The rows are scanned first. (This is not a requirement; scanning columns first will yield identical results.) For the i'th row of the binary pattern, if PIXELS P.sub.ij, thru P.sub.i j+n are a continuous group of "1" cells, then the "T" cells are designated according to the number of cells in this group (n+1):

This is best illustrated by example. PATTERN 1 depicts a binary pattern and PATTERN 2 shows that the T cells produced by applying the rules of Table 1 to the rows of the pattern. The same operations are performed on each column to yield the "T" cells of PATTERN 3.

The "T" cells of the row and column scans are then used to derive a multilevel logical pattern, P, whose cells, P.sub.ij, have one of four logical values: "0," "1," "T," and "P" representing a fourth type of cell. The PIXELS of P are defined as follows:

where + and .cndot. indicate the Boolean operations of logical OR and AND respectively. Thus a "p" cell is created by the Exclusive OR of "1" cells in the row and column scan intermediate patterns. PATTERN 4 depicts this resulting pattern.

The next step is to use PATTERN 4 to derive the augmented pattern. This is accomplished as follows: each cell of the multilevel pattern, P.sub.ij, is considered, in turn, as base cell. The logical value ("0," "1," "P" or "T") of this base cell serves as an input to a table. Next the logical values of the four nearest neighbors, P.sub.i-1j, P.sub.i j+1, P.sub.i+1 j and P.sub.ij j-1, are sequentially accessed. These sequential values serve as additional inputs to the same table. The Table W provides a numerical value for each of the four combinations of base cell and neighbor. These values are accumulated to yield the value of the augmented pattern cell A.sub.ij corresponding to the initial base cell P.sub.ij. ##EQU3## Table I shows the values accumulated for each neighbor dependent upon the base cell value. Note that for base cells with a value of "0" or "1," the augmented value, A.sub.ij, is independent of the neighbor cells. This is to insure that cells which have the highest probability of being target or background are not changed. This also enables some simplification in mechanization of the algorithm. As one final concession to practicality, the maximum accumulated value will be limited to 31 (instead of 32) in order to limit the number representation to five binary digits.

Applying the final phase of the augmentation algorithm, given by equation 7 and Table I, to the intermediate pattern, P (PATTERN 4), yields the augmented pattern of PATTERN 5. Unlike the original binary pattern, all cells do not have the same value.

PIXELS Near the center of the pattern have larger weights than the edge cells. Being derived solely from the binary pattern and geometrical considerations, the cell values do not depend upon the intensity values of the original linear pattern. Because of this and the strong influence of the location of the binary pattern edges on the algorithm, the augmented pattern values, A.sub.ij can, in a loose sense, be interpreted as the probability (unnormalized) that the cell is a part of the target.

The advantages of the augmented pattern over the binary pattern are demonstrated by considering the effects of cell deletions on the pattern measurements. PATTERNS 6a and 6b depict the binary pattern of PATTERN 1 and the resultant augmented pattern when single edge cell is deleted.

Such deletions of cells cause changes in the values of the pattern moments. Table II tabulates the mean and spatial variances for the original binary and augmented patterns (PATTERNS 1 and 5) as well as the noisy patterns of PATTERNS 6a and 6b. Note that the shift in the x centroid coordinate is almost three times greater for the binary pattern than it is for the augmented pattern. Also there is a 7.1% and 8.4% change in the values of .sigma.x.sup.2 and .sigma..sub.x.sup.2 respectively between the original and noisy binary patterns.

The changes in the same measurements for the augmented patterns are 3.2% and 0.3% respectively. This example indicates the augmented patterns are less sensitive to spatial noise than their binary counterparts.

A few final comments are in order. The rules which define "T" cells are somewhat arbitrary. The thickness of these transition regions in patterns can be adjusted according to known point spread functions of various sensors. The values of the weight in Table I have been chosen with respect to ease of implementation. Spatial optimality constraints are not deemed to be important criteria. The entire augmentation scheme is to be used in conjunction with a preprocessor and a device to measure pattern moments. Therefore, great care has been taken to insure that all processing can be realized with simple logical operations. This will insure compatibility with the other processors and introduce minimal delay.

The mechanization of the augmentation algorithm is depicted in block diagram format in FIG. 2. The binary pattern is loaded by the preprocessor into the Binary PIXEL RAM 15. After this is accomplished T cells are determined by clocking PIXELS (BP) sequentially out of this memory first by rows and then by columns. The 1's and 0's pass through logic 21 which designates the presence of a T cell with a binary "1." This bit is stored in a RAM 22 corresponding to the scan format. An address generator 16 is required to generate address for the binary PIXEL RAM 15 for row and column scans. An address modifier 17 contains logic which adjusts the Binary PIXEL RAM 15 for the row and column scan RAMS 18, 19, since there is a four PIXEL delay in the "T" cell designator 20. Appropriate buffering and additional clock pulses are supplied by this device.

The T cell designator 20 consists of four cell storage elements, A1-A4 (e.g. D-Type Flip Flops 28-31) a logic table e.g. programmable read only memory 26 and a counter 27 and gating or reset circuit 32. This is depicted in the block diagram of FIG. 3. The counter tallies the number of consecutive 1's in any row or column segment of the binary pattern. Its maximum value is limited to a count of five since the rules for finding "T" cells remain unchanged above this value. The value of the counter, the value of the next binary cell and the present state of the storage elements are provided as inputs to a logic table, e.g. a programmable read only memory (PROM). The next PIXEL transition stores the present status of A4, which if set, indicates a T cell four positions earlier, in the appropriate scan RAM. The state of the storage elements are changed according to Table 3.

Note that the sequence of operations defined by the table is similar to, but not exactly like a shift register. Care must be taken to insure that all information is loaded into the correct scan RAM.

Once the two scans that define "T" cells have been completed, the data in the two scan RAMS 18, 19 as well as the Binary Pattern RAM 15 is supplied to logic 21 which detects "P" cells. The logic also codes the type of all cells according to Table 4 and loads this information into the composite pattern RAM 22. This is the operation equivalent to the generation of PATTERN 3. Once this RAM is loaded, the final phase of the processing begins as shown by FIG. 4. The functions and components utilized are shown in block diagram form and comprise sequence counter 33, control logic 34, multiplexers 35, 36, counter 37, latches 38, 39, PROM 40, decoder 41 and accumulator 42.

The composite pattern RAM 22 is accessed five cells at a time. A ten bit counter 37 is used as an address generator for a base cell (P.sub.ij) address which access the base cell and latches its value. This is changed at the end of a five count sequence. If base cell value is such that no neighbor cell searches are required, logic is provided to terminate sequence immediately and begin on a new base cell. If neighbor cell information is required, logic is provided to modify the base cell address and generate all neighbor coordinates. This is controlled by a sequence counter 33 and multiplexers 35, 36. Once the address for the composite pattern RAM 22 is formed and applied to the memory the accessed data is stored in the neighbor cell latch 38. (Logic is provided to enable the system to skip cells over the edge boundaries.) This value, along with the base cell value is led to a logic table 40, e.g. a PROM which is the binary realization of Table I. This determines the incremental weight to be stored in the accumulator 42. At the end of the five count cycle, this accumulator contains the augmented pattern value which is made available for further processing.

The mechanization described above seems more complicated than what is actually required. Integrated circuit memories are available in a 1024 words by 1 data bit organization. Thus all memory requirements could be met utilizing 5 integrated circuit packages. Using available MSI devices, the entire process would consist of approximately 25 standard packages.Using a basic clock rate of 10 MHz, the entire augmentation process could be completed in less than a millisecond. This is a small delay when viewed in light of the improvement in the accuracy of the pattern measurements.

While the invention has been described in terms of its preferred embodiment it is understood that the words which have been used are words of description rather than words of limitation and that changes within the purview of the appended claims may be made without departing from the scope and spirit of the invention in its broader aspects.