Image Enhancement Using on-Line Spatial Filtering

The desirability of enhancing optical scenes, aerial photographs, radar and infrared images so as to cue an observer to the presence of targets having specific characteristics has been well recognized. Many of the prior approaches to the spatial filtering of images, with filter bandwidths adjustable by the human observer so that continuous observation can be had of dynamic images with selected spatial bandwidths have appeared theoretically possible, but they have not heretofore been feasible due to their great expense, the voluminous equipment required, and the slowness of operation. The best known prior art is that contained in U.S. Pat. No. 3,283,070 to patentee M. D. Prince and U.S. Pat. No. 3,566,027 to patentee T. O. Paine.

The invention provides near-real-time filtering of a video image with spatial frequency bands of the viewer's choosing. The filtering provides an approximation to known filtering processes in the human visual system and permits the results of these filtering processes to be observed in near real time with dynamic imagery. No known prior art device performs visual response function filtering in substantially real time. Observation of such filtered images aides in target, shape, object, and feature recognition of objects of different size. When the band pass filtered scene is viewed on the monitor, edges and corners of objects in the scene can be selectively made more or less visible to the observer by adjusting the filter center frequencies with the filter frequency band equal to or not equal to the expected target size. A thresholding operation can be performed on the resultant image leaving only important edges or corners thereby removing much background clutter unimportant in target detection. In some situations filtering is also beneficial with adjustable high pass or low pass filtering only.

FIG. 1 is a simplified block diagram of an embodiment of the invention;

FIG. 2 is a more detailed block diagram of a typical embodiment of the invention;

FIG. 3 is a simplified schematic diagram of a typical filter circuit in the invention;

FIG. 4 is an illustrative plot showing the combining of the responses of two Gaussian filters; and

FIG. 5 illustrates a typical total filter impulse response characteristic.

The spatial filtering of imagery can assist in pattern, shape and object size recognition in the analysis of complex imagery. The mathematical computation and subsequent display of spatially filtered images is complex and could not previously be achieved on-line in near real time with reasonable computer capacity or special-purpose computers. By scanning the image, this invention transforms the two-dimensional spatial dependence of the image function into functions of time. The invention then carries out the normally complex spatial filtering by performing equivalent frequency filtering in the time domain. Retransformation into the space domain produces an image display approximately the mathematically rigorous spatially filtered image. Advantages of this invention are: (1) Near real-time display of the spatially filtered image. (2) Observation of imagery by an observer with continuously variable spatial filtering under his control. (3) The observation of spatially filtered dynamic imagery presently not accomplished in real time. (4) Use of readily available frequency filters in the time domain for special-purpose image processing equipment. The device disclosed is used to cue the observer of optical scenes, aerial photographs, radar and infrared imagery to the presence of targets of a specific size in the visual field by proper tuning of the x and y spatial frequency filters. The device is also used to remove the effects of spatial frequency masking by viewing the scene through a narrow band of spatial frequency components. This type of viewing will frequently reveal underlying structures that are masked when the unfiltered scene is viewed, and permit the viewer to see through certain types of camouflage.

A typical embodiment of the invention is shown in FIG. 1. The line-by-line scanning properties of a video camera 11 are utilized to transform the spatial distribution of intensities present in a scene 12 into a time-varying intensity function. The time-varying function is frequency filtered in real time by passing it through an electrical filter 13 with adjustable pass band and center frequency, and the filtered image is written on a storage tube 14 or any other fast-memory array. The stored image has now been spatially filtered in the x direction. The storage tube (or memory) is scanned electrically by scanner 15 and read line by line (in the y direction) with the new scan lines perpendicular to the direction in which the image was written on the storage tube (or memory array). This step transforms the spatial distribution of intensities on the storage tube (or memory array) into a time-varying function of intensities. This time-varying function is passed through a second filter 16 with adjustable pass-band and center frequency. The doubly filtered image is displayed on monitor 17 and represents the initial image spatially filtered in both x and y directions with filter pass bands and center frequencies chosen by the observer.

A more detailed block diagram of an embodiment of the system is shown in FIG. 2. A slow scan TV camera 11 scanning at a typical rate of 12 frames per second is suitable for this invention. The filters 13 and 16 are tapped adjustable delay lines, such as the type manufactured by RETICON for example. A typical filter has 64 taps and is typically constructed as shown in FIG. 3. The .tau. symbols in FIG. 3 represent signal delay times between sample points. The 64 R values are adjustable so that the voltage drop across a resistor R.sub.i (where i=1, 2, 3, . . . 64), for an impulse input is equal to the desired filter impulse response at time i.times..tau.. The desired value of .tau. for a slow scan TV is 0.3 msec. The resistance values R.sub.i may be set according to the equation R.sub.i=R e.sup.- (i-.mu./2 .sigma..sup.2) where i is the index of the resistor (1 to 64), .mu.=33 and .sigma. can range from 0.5 to 16. (Preferred values.) R is a constant value chosen so as to match the impedance of the associated equipment. The impulse response specified above is a Gaussian function and the filters F.sub.1 (t) and F.sub.2 (t) of FIG. 1 would be made up of two such filters the output of which are summed together. One filter produces a positive Gaussian function such as curve 41 of FIG. 4 while the other produces a negative Gaussian function, such as curve 42 of FIG. 4. The sum of the two is similar to receptive fields measured in the human visual system. A typical total impulse response of F.sub.1 (t) then is shown by curve 51 in FIG. 5 and is given by the Equation ##EQU1## The preferred value of T.sub.o is T.sub.o =.tau..times.33 and t is given by i.times..tau.. The typical response shown is a bandpass characteristic filter. If B=0 it is a low pass filter of adjustable bandwidth. If B=A and .sigma..sub.1 <.sigma..sub.2 a high pass filter would be approximated. A HUGHES 1139 scan converter is a suitable memory device and scan converter for use in the invention. A TEKTRONIC Model 632 video monitor is a possible display device 17 for use in the invention. The controller 21 is a conventional timer and synchronous pulse generator for coordinating the operation of the elecronic circuits. It uses synch signals from the video camera 11 to control the read and write cycles of the scan converters and the clock times of the filters. The electronic switch 22 controls which conventional scan converter 23 or 24 receives the next image frame. One scan converter maintains image on the monitor 11 while a new frame is written onto the other scan converter. This prevents flicker in the video monitor 17. It is to be remembered that input 25 is read onto the storage tube horizontally and the output 26 is read off the storage tube vertically.