Adaptive Thresholding Technique

1. Technical Field

This invention relates to image processing techniques and, more particularly, to a method and apparatus for thresholding analog video signals.

2. Discussion

One of the major problems in performing efficient digital processing of an image is to properly digitize the analog video signal from the detector or source. Traditionally, the amplitude or intensity of the analog video signal from the detector is compared against a fixed threshold. Analog levels above the threshold are converted to a binary or digital one whereas analog signal levels below the fixed threshold are converted to a digital zero level.

The chosen thresholding technique should allow detection of the target signal in the various kinds of noise that are possible. There are two primary kinds of noise. One is low-frequency shading or scan noise that can be likened to a parabola along the scan direction. The other kind of noise is high frequency noise generated by the detector itself. As shown in FIG. 1, a fixed thresholding technique creates false detections that are due to noise instead of actual scene content.

In selecting criteria on which to base a threshold level, a designer is usually trapped between conflicting limitations. If the threshold is weighted too heavily toward the background (a "low" threshold), excessive digitization or clutter may occur. On the other hand, if the threshold is set too "high", a loss of digitized target pixels may result. The use of a low-pass filter does not provide advantageous results because the low frequency noise component would cross the threshold, just as a high-pass filter would allow the high frequency noise component to cross the threshold. A band pass filter is not easily implementable because there is little reliable information within the target spatial frequency prior to target selection. This is due to the uncertainty in target size, range to the target or the unknown effects of image blur of the target signal.

In accordance with the teachings of the present invention, an adaptive threshold level is set for each pixel position against which the analog signal is compared. The adaptive threshold level is a function of a running average of the analog video signal to which an offset valve is added. The offset value is a function of a global measure of the noise as well as a function of the noise generated by the detector.

The various advantages of the present invention will become apparent to those skilled in the art after reading the following specification and by reference to the drawings in which:

FIG. 1(A-B) are graphs illustrating a fixed thresholding technique of the PRIOR ART;

FIG. 2 is a block diagram of system components for carrying out the teachings of the present invention;

FIG. 3 is a graphical illustration of how the analog video signal is binarized by the adaptive thresholding technique of the present invention; and

FIG. 4(A-B) are illustrations of two frames of video input signals showing the derivation of the running average for each column of pixels.

In accordance with the teachings of the present invention, a method and apparatus is provided for thresholding analog video signals in such a way that real objects or potential targets can be detected although various kinds of noise exist in the video signal. The analog signals can be derived from a variety of sources such as an infrared scanning focal plane array disclosed in various publications by the assignee of the present invention. One of the primary objectives of this invention is to provide a thresholding technique that distinguishes the target signal from low-frequency scan noise or shading, as well as high frequency detector noise. To this end, this invention provides an adaptive threshold for each pixel. As is known in the art, each frame of video data is divided into a matrix of pixels having a number of rows and columns. By way of a nonlimiting example, a bidirectional scanning focal plane array with two sets of 120 detectors each can be staggered and multiplexed together to generate a 240 pixel X 240 pixel array or field of view (FOV). The intensity of the analog signal at each pixel position is ideally a function of a characteristic (e.g., infrared radiation emanating therefrom) of objects within the scanned scene. Unfortunately, the intensity of the analog signal is detrimentally also a function of noise. As indicated in the discussion of FIG. 1, the two primary kinds of noise are low-frequency shading and high frequency noise from the detector. Because the magnitude of noise may vary from one detector to the next and also along each scan, an adaptive thresholding approach is provided for each pixel. Pixels with intensity values above their corresponding adaptive threshold limits are digitized or binarized for further processing.

In accordance with the present invention, the adaptive threshold comprises a running average of the analog signal to which an offset value is added. The offset value is a function of a global measure of noise (alpha), as well as a function of noise specifically associated with the detector which is referred to herein as the "Sum Absolute Difference" (SAD).

The thresholding technique can be used in this embodiment to convert live video into "relative video", as well as "binary video". Binary video comprises an array of digital ones or zeros depending on whether the intensity value of the analog signal is above or below the adaptive threshold for a given pixel. The relative video provides data, for each pixel position, that is a function of the relative difference between the intensity of the analog signal and the running average used to calculate the adaptive threshold for each pixel position. The adaptive threshold is calculated by adding an offset to a baseline average of the incoming signal. The baseline average is set by calculating a running average of the video signal from each detector to smooth high frequency noise components, and to counter low-frequency non-uniformities from the detector. The offset is calculated by multiplying the global measure of the noise (alpha) by a measure of the noise per detector. The size of the offset can affect the number of false alarms and the target size. The offset is adjusted as a balance between minimizing the noise crossings while ensuring a high probability of detection. The noise per detector is calculated by taking an average of the difference between the live video and the running average in a sample window.

The adaptive threshold level for each pixel is defined by the following equation (1): ##EQU1## The Sum Absolute Difference (SAD.sub.j) is defined in accordance with equation (2): ##EQU2##

The method of the present invention can be carried out in a variety of different hardware/software approaches. FIG. 2 illustrates a hybrid approach where hardware is used to process the live video to generate certain parameters while the software provides many of the calculations, as will appear.

In FIG. 2, the live video from the detector 10 is applied over line 12 to circuitry 14 for calculating the running average. With additional reference to FIG. 4, a 16 pixel running average has been chosen. Thus, N equals 16 for equation (1). The running average (RA) is calculated for each column as illustrated in FIG. 4 by adding the intensities of a group or window of pixels along a column and dividing the sum by the number of pixels (here, 16) in the window. The running average for each column thus provides a measure of the local noise mean. A local standard deviation is measured by calculating the "Sum Absolute Difference" (SAD) by circuitry 16. Circuitry 16 accepts the running average (RA) output on line 18 and the live video which has been delayed by suitable delay circuitry 20. Provision can be made for defining the SAD window on line 22. The SAD gate defines the SAD window length M in equation (2). For the estimate of the local standard deviation of the noise to be correct, only noise should be included in the sample window. However, the target could be located anywhere in the video frame prior to target selection. To counter this problem, the SAD window position alternates from top to bottom of the frame based upon the scan direction as shown in FIG. 4. This ensures that the target will not consistently be located in the SAD window. The SAD window is controllable by software and may range from 4-16 pixels. In this embodiment, M equals 16.

Circuitry 16 outputs on line 24 the Sum Absolute Difference of equation (2). The SAD values for each of the SAD windows within a frame is stored in memory 26 and provided to the software through a suitable input/output interface 28.

Prior to target selection, the early detect function block 30 is disabled and the software proceeds along the acquisition line to read the noise information from RAM 26 and reposition the SAD window, if desired. This function is represented by block 32 in FIG. 2. As represented by block 34, the software then proceeds to define alpha. The calculation of alpha is based on the current false alarm rate (P.sub.FA) and the predetermined system probability of detection (P.sub.Det). Alpha is a positive number and is initialized to a value corresponding the desired probability of detection where: ##EQU3## If the desired probability of detection is 0.99 and a constant false alarm rate of three false alarms or less per field in the field of view should maintain, this implies that the minimum TNR should be four. However, because the SAD measurement is approximately 0.8 of the actual noise standard deviation, alpha should be initialized to 5.0 (four divided by 0.8) for a 240X240 field of view. The value of alpha is software controllable to compensate for incorrect assumptions made about the noise or target intensity so that alpha can be adjusted to maintain the desired false alarm rate. For example, alpha can later be determined based upon cluster count. If there are too many clusters, alpha should be raised whereas if too few clusters are detected, alpha should be lowered. Consequently the threshold level can be optimized. Once alpha is chosen, the software calculates the offset by multiplying alpha by the previously generated SAD data. In the acquisition mode, this offset value passes through functional block 36 back to the alpha-SAD memory 38 in the hardware.

Summing node 40 serves to add the offset from memory 38 to the running average to generate the adaptive threshold on line 42 for a given pixel. Summing Node 41 subtracts the running average from the pixel intensity to generate the Relative Video value on line 43. The live video on line 12 has been suitably delayed by delay 44 to synchronize the presentation of the pixel intensity value on line 46 with its corresponding threshold on line 42. Comparator 48 provides the binary video output by comparing the analog video signal intensity value on line 46 with its corresponding threshold on line 42. If the intensity value is above the threshold, the comparator provides a digital one binary output. If the intensity is below the threshold, then a binary zero level is provided by comparator 48. This process is graphically illustrated in FIG. 3. The analog video input signal generated on line 12 (FIG. 2) is labeled "VIDEO" in FIG. 3. The running average generated on line 18 (FIG. 2) is labeled "RUNNING AVERAGE". When the offset (alpha X SAD) is added to the running average, the ultimate threshold is generated. The threshold bears the label "THRESHOLD" in FIG. 3 and is provided on line 42 in the FIG. 2 implementation. FIG. 3 illustrates the digitization of one scan line. This process continues for every scan line in the field of view or frame. The thresholding process continues frame by frame. The offset or (alpha X SAD) term remains constant for each column of video, and the alpha term remains constant for each frame of video. If the estimated SAD term is zero then a predetermined minimum constant is added to the running average to obtain a new threshold value. A study of FIG. 3 reveals that the use of the running average only as the adaptive threshold would compensate for low frequency noise or shading, however, it would not operate to filter out the high frequency noise. Consequently, the offset is added to the running average to accomplish this function.

The previous discussion was directed to the method of thresholding that is implemented prior to target selection. For target track mode, this method could be adjusted so that only one object is thresholded within the track or background gates. Once this point is reached, alpha could be kept constant to ensure the stability of thresholded target image from jitter. This is provided in FIG. 2 by enabling the Early Detector control of block 30. After target selection, the tracker can generate a reliable estimate of the target intensity. A more accurate threshold can then be set by using an estimate of the target's intensity, rather than basing the threshold on an estimate of noise. When this stage is reached, the software switches from the Acquisition mode to the Target Track mode. Block 50 is then used to calculate a different adaptive threshold.

In view of the foregoing, those skilled in the art should appreciate that the present invention provides an adaptive thresholding technique that can be used in a wide variety of applications. The various advantages should become apparent to those skilled in the art after having the benefit of studying the specification, drawings and following claims.