Method for Detecting Signals in Non-Gaussian Background Clutter

The present invention relates to the field of signal processing, and more particularly to a method for detecting weak signals in a non-Gaussian, non-stationary background.

Many signal detection schemes in use today, including intensity-only and Doppler processors, are based on optimal detection formulas derived from Gaussian noise models. For example, signal detection schemes employing constant false alarm rate (CFAR) algorithms use the power detector, which is obtained from optimal detection theory by assuming a random signal and Gaussian noise, with an adaptive noise power estimator.

FIG. 1 shows the components of a basic CFAR detector. The input data is a collection of successive intensities (norm squared) of baseband demodulated radar returns from a set of range bins organized into a range-by-pulse matrix X=(x.sub.ij), where R is the number of range bins, P is the number of pulses and 1.ltoreq.i.ltoreq.R, 1.ltoreq.j.ltoreq.P. The steps in the processing include estimating the power in the i.sup.th bin using the formula: ##EQU1## The noise power estimate for the i.sup.th bin, .eta..sub.i, is then the output of a filter, O, that may be defined using an order statistic that selects a predetermined percentile, as:

where g is the number of guard bins on either side of the test bin, and L is the number of range bins used to form the estimate of the noise power.

The detection statistic for the i.sup.th bin, t.sub.i, may be calculated for each range bin by normalizing and summing the input data according to the formula: ##EQU2##

The threshold may be set as a fixed number, T, calculated using a predetermined false alarm rate and the assumption that under the noise only hypothesis the statistics, t.sub.i, are unit chi-squared random variables on 2P degrees of freedom. Alternatively, the threshold for the i.sub.th bin may be set using a function of the values of t.sub.l for l near i or determined by an operator.

To avoid an excessive false alarm rate when applied to non-Gaussian noise, many CFAR processors use a high threshold which makes it unlikely that weak signals will be discerned from the noise. Therefore, a need exists for a detector which can detect weak signals from a non-Gaussian noise background, yet which has a low false alarm rate.

The invention provides a method for detecting signals in non-Gaussian background clutter, comprising the steps of: 1) filtering input data to segregate Gaussian input data from non-Gaussian input data; 2) estimating the intensity level of the Gaussian input data; 3) determining the probability distributions of the non-Gaussian input data; 4) determining the background noise level of the input data; 5) estimating noise parameters of the input data; 6) estimating the residual intensity of the input data using the noise parameters; 7) determining a detection statistic M from the input data, estimated noise parameters, and the residual intensity of the input data; 8) determining a standard deviation N.sub.i of the detection statistic, M.sub.i ; 9) determining a normalized detection statistic S.sub.i, where S.sub.i =M.sub.i /N.sub.i,; 10) determining a maximum value, S.sub.imax of S.sub.i ; 11) determining a mean value S.sub.Mean and a standard deviation S.sub.Std dev of all S.sub.i excluding S.sub.imax ; 12) determining a threshold .tau..sub..alpha. of a zero mean unit variance normal distribution corresponding to a predetermined false alarm probability, where .alpha. is a predetermined false alarm probability; and 13) generating an output signal if ›S.sub.i -S.sub.Mean !/S.sub.Std dev .gtoreq..tau..sub..alpha..

The invention may also be characterized as providing a method for detecting signals in non-Gaussian background clutter, comprising the steps of: 1) filtering input data to partition Rayleigh input data from non-Rayleigh input data; 2) estimating the intensity level of the Rayleigh input data; 3) determining the probability distributions of the non-Rayleigh input data. 4) determining the background noise level of the input data; 5) estimating noise parameters of the input data; 6) estimating the residual intensity of the input data using the noise parameters; 7) determining a detection statistic M.sub.i from the input data, estimated noise parameters, and the residual intensity of the input data; 8) determining the standard deviation N.sub.i of the detection statistic, M.sub.i ; 9) determining a normalized detection statistic S.sub.i, where S.sub.i =M.sub.i /N.sub.i, 10) determining a maximum value, S.sub.imax of S.sub.i ; 11) determining a mean value S.sub.Mean and a standard deviation S.sub.Std dev of all S.sub.i excluding S.sub.imax ; 12) determining a threshold .tau..sub..alpha. of a zero mean unit variance normal distribution corresponding to a predetermined false alarm probability, where .alpha. is a predetermined false alarm probability; and 13) generating an output signal if ›S.sub.i -S.sub.Mean !/S.sub.Std dev .gtoreq..tau..sub..alpha..

The invention may further be implemented as a method for detecting signals in non-Gaussian background clutter, comprising the steps of: 1) filtering input data to partition exponential input data from non-exponential input data; 2) estimating the intensity level of the exponential input data; 3) determining the probability distributions of the non-exponential input data; 4) determining the background noise level of the input data; 5) estimating noise parameters of the input data; 6) estimating the residual intensity of the input data using the noise parameters; 7) determining a detection statistic M.sub.i from the input data, estimated noise parameters, and the residual intensity of the input data; 8) determining the standard deviation N.sub.i of the detection statistic, M.sub.i ; 9) determining a normalized detection statistic S.sub.i, where S.sub.i =M.sub.i /N.sub.i, 10) determining a maximum value, S.sub.imax of S.sub.i ; 11) determining a mean value S.sub.Mean and a standard deviation S.sub.Std dev of all S.sub.i excluding S.sub.imax ; 12) determining a threshold .tau..sub..alpha. of a zero mean unit variance normal distribution corresponding to a predetermined false alarm probability, where .alpha. is a predetermined false alarm probability; and 13) generating an output signal if ›S.sub.i -S.sub.Mean !/S.sub.Std dev .gtoreq..tau..sub..alpha.. The invention may also be implemented as computer program instructions written on a computer readable storage device. When executed, the program instructions cause the computer to perform the above-referenced method steps.

An advantage of the invention is that it selects noise-only mixture parameters using order statistics.

Another advantage of the invention is that the detector may optionally estimate the highest state variance using global data, and the combination of the high state parameters with the parameters estimated for each range bin.

Still another advantage of the invention is that it uses an estimate of the signal strength to modify the locally optimal detector so that the resulting detector will be able to detect signals that are not vanishingly small.

A further advantage of the invention is that it sets thresholds using the moments calculated from the multi-state mixture parameters used to process the data.

These and other advantages of the invention will become more apparent upon review of the accompanying drawings and specification, including the claims.

FIG. 1 is a block diagram of the basic components of a constant false alarm rate (CFAR) detector.

FIG. 2 is a block diagram of the basic components of a system for detecting signals in non-gaussian background clutter which embodies various features of the present invention.

FIG. 3 is a block diagram showing the basic components of another embodiment of a system for detecting signals in non-gaussian background clutter embodying various features of the present invention.

FIG. 4 is a block diagram of the hardware elements which may be used to implement the invention.

Throughout the several figures, like references refer to like elements.

The following description is of the best mode presently comtemplated for carrying out the invention. This description is not to be taken in a limiting sense, but is made merely for the purpose of describing the general principles of the invention. The scope of the invention should be determined with reference to the claims.

The present invention provides a method for detecting weak signals in a non-Gaussian and non-stationary background, such as radar sea clutter, acoustic data containing moving interferers, and seismic data. The invention preferably may be implemented as computer readable program instructions which may be encoded onto a program storage device such as magentic tape, a floppy disk, magnetooptical disk, compact disk, or the like. The storage device embodies the program of computer executable instructions for implementing the function of detecting weak signals in non-Gaussian and non-stationary backgrounds.

Referring to FIG. 2, input data 20 is provided to a computer readable program instructions referenced as a filter 24 of a detector 10. The filter 24 divides the input data 20 into data 26 which may include Gaussian data, Rayleigh data, or exponential data, and into data 30 which may include non-Gaussian data, non-Rayleigh data, and non-exponential data 30. If data 26 includes Gaussian data, then data 30 includes non-Gaussian data. If data 26 includes Rayleigh data, then data 30 includes non-Rayleigh data. If data 26 includes exponential data, then data 30 includes non-exponential data. The input data may be partitioned into groups of data 26 and 30 using a suitable statistical goodness of fit test, such as the Kolmogorov-Smirnov (KS1) test or Chi-Square test to reduce computational requirement. Next, the data 26 is provided to computer readable program instructions referenced as an intensity estimator 28 which generates estimates 29 of the intensity level of the input data 26. The input data 30 is received by computer readable program instructions referenced as a density estimator 32 which fits the data 30 to a family of first functions by estimating first parameters of the functions to determine the probability distributions of the data which are provided as output 34. The functions may be exponential mixture probability density functions. An example of an m-state exponential mixture probability density function with parameters (p.sub.1, . . . , p.sub.m, v.sub.1, . . . , v.sub.m) such that ##EQU3## and v.sub.1 <. . .<v.sub.m is: ##EQU4## The expectation maximization (EM) density estimator 32 estimates for the i.sup.th range bin, p.sub.k.sup.i and w.sub.k.sup.i for prescribed values of m, wherein different values of m may be used to suit the requirements of a particular application. The values of p.sub.k.sup.i and v.sub.k.sup.i are provided as output 34 from the EM density estimator 32. The first functions may also be Gaussian mixture probability density functions. Alternatively, the EM density estimator 32 may apply a quasi-Newton optimization algorithm to the output of the expectation and maximization algorithm to possibly speed the parameter estimation procedure.

Computer readable program instructions referenced as a robust parameter selector 36 employs the outputs 29 and 34 to determine the parameters 37 of the background noise of the input data 20. The input data 20 contains noise components and possibly components representing a signal of interest. However, a detection statistic M, which is a function of the noise parameters 37 and the input data 20, relates the input data 20 to a threshold in order to discern a possible signal of interest. Therefore, noise parameters 37 of the input data 20 are estimated, and are determined in the robust parameter selector 36 which may be implemented as computer readable subroutine ROBPARSEL10.m (Appendix 1), written, by way of example, in MATLAB.TM.. The robust parameter selector 36 transforms the outputs 29 and 34 into adjusted estimate outputs 37 of a selected set of the first parameters p.sub.k.sup.i and v.sub.k.sup.i, by using order statistics applied to subsets of the parameters p.sub.k.sup.i and v.sub.k.sup.i, where if m=2, then k=1, and a "clutter plus noise parameter," cpn(.theta.), where ##EQU5## The robust parameter selector 36 mimimizes the generation of false alarms by detector 10.

ROBPARSEL10.m, presented by way of example, in APPENDIX 1, determines exponential mixture parameters 37 of the noise in the input data 20. For each range bin ROBPARSEL10.M determines robust estimates of the two-state exponential mixture density for the noise component of the input data. The input and output parameters of ROBPARSEL10.m are described below in Tables I and II, respectively. ROBPARSEL10.m, lines 52-58 determines a robust noise power estimate by using an order statistic as defined in equation (2). Alternatively, a smoothing filter, such as the mean or the sensored mean, may be used in place of the order statistics used here. ROBPARSEL10.m, lines 59-62 use an order statistic to estimate the value of v.sub.1 corresponding to the noise component of the input data 20. At lines 63-67, ROBPARSEL10.m modifies the value of p.sub.1 based upon the revised value of v.sub.1. Under appropriate condition, lines 72-87 of ROBPARSEL10.m collapse the current two-state parameters to one state. Lines 99-101 of ROBPARSEL10.m use an order statistic to re-estimate the noise power of the input data using a possibly different number of guard bands than was done above in lines 52-58. At lines 102-104, ROBPARSEL10.m uses an order statistic to estimate the value of p.sub.2 of the noise component of the input data 20. Lines 106-115 of ROBPARSEL10.m calculate the value of v.sub.2 of the noise component of the input data. 20. Depending upon the value of v2oflag, lines 131-185 of ROBPARSEL10.m scale v.sub.2. At lines 187-196, ROBPARSEL10.m scales p.sub.1, depending upon the value of the input parameter pLflag,. The variable v.sub.1 is scaled at lines 197-210 of ROBPARSEL10.m depending upon the value of vlflag.

Still referring to FIG. 2, for the i.sup.th range bin, a residual variance calculator 40 transforms the output 37 from the robust parameter selector and the input data 20 into an estimate output 42 of the signal strength of any signal of interest which may be in the input data 20. The residual variance calculator 40 may produce the signal strength estimate v.sub.s.sup.i (residual intensity) used by the detection statistic M according to the formula: ##EQU6##

Next, computer readable program instructions referenced as detection statistic calculator 46 transforms the input data 20 and the outputs 37 and 42 into an exponential mixture amplitude detection statistic, M, or simply, "detection statistic" which may be determined as follows: Let .theta..sup.i =(p.sub.1.sup.i, . . . ,p.sub.m+1.sup.i,v.sub.1.sup.i, . . . ,v.sub.m+1.sup.i, v.sub.s.sup.i) be the outputs 37 and 42, then the detection statistic, M, for the i.sup.th range bin is calculated according to the formulas: ##EQU7##

Computer readable program instructions referenced as parameter dependent normalization factor calculater 50 uses output 37 to determine a normalization factor, N, for each range bin. The normalization factor may be obtained by determining k moments of the distribution of the detection statistic M, where k is a positive integer and preferably, where k=2. The moments may be calculated or obtained from a look-up table.

The k moments of the detection statistic may be determined in accordance with the following procedures: M may be approximated by a piecewise polynomial function obtained from piecewise polynomial approximations of the state weighting functions w.sub.k.sup.i (x.sub.ij)=p(k.vertline.x.sub.ij, .theta..sup.i). For an m-state mixture lim.sub.u.fwdarw..infin. w.sub.m (u)=1. Select K so that if u>K then w.sub.m (u).apprxeq.1. Select the degree, d, and number, p, of the polynomials to be used in the approximation. Define a partition:

Then for each 1.ltoreq.j.ltoreq.m-1 and each 1.ltoreq.l.ltoreq.p let P.sub.lj be the Lagrange interpolating polynomial of degree d defined by ›u.sub.(l-1)d, . . . , u.sub.ld ! and ›w.sub.j (u.sub.l-1)d, . . . , w.sub.j (v.sub.ld)!. Other conditions, such as continuity of the derivatives up to certain orders at the points (u.sub.r) or vanishing of certain derivatives at 0 and K, may also be imposed. Also, define: ##EQU8## For a set A, let .chi..sub.A be the characteristic function of A, i.e. .chi..sub.A (u)=1 if .mu..epsilon.A and 0 otherwise. Define the piecewise polynomial approximation of the weighting functions by: ##EQU9## and the approximate detector by ##EQU10## Since P M is piecewise polynomial, its moments are easily calculated for Gaussian mixture, Rayleigh mixture, or exponential mixture noise and a Gaussian signal having variance v.sub.s. Other signal models require appropriate modifications. For 1.ltoreq.l.ltoreq.p+1 define: ##EQU11## and define u.sub.(p+1)d =.infin.. Then, ##EQU12## Since Q.sub.l is a truncated polynomial, the integrals in (11) may be evaluated using the incomplete gamma function.

The normalization factor, N, of the detection statistic M for each time series is calculated as follows: Under the noise only hypothesis, the mean of the detection statistic, E(PM.sup.1), is zero, and the second moment, E(PM.sup.2) is obtained from equation (11). Therefore, N=›E(PM.sup.2)!.sup.1/2.

Computer readable program instructions referenced as comparator 54 uses the detection statistic M and the normalization factor N to determine whether a signal is present and to generate an output signal A if a normalized detection statistic is greater than a threshold, as described below:

Given a false alarm probability, .alpha., and letting .PHI. be the unit normal distribution, the comparator 54, determines a threshold .tau..sub..alpha. for the zero-mean unit-variance normal density which is defined as

The probability of a false alarm is equal to a predetermined false alarm probability suitable for a particular application. A suitable false alarm probability is based on one's tolerance for investigating alarms, based on the cost of a misdetection. For example, if missing a signal of interest would be very costly in terms of man-hours, level of risk, severity of consequences, etc., then a higher false alarm rate would be tolerated. Conversely, if missing a signal of interest would not be very costly, one could tolerate a lower false alarm rate.

Then, the comparator 54 determines the first normalized detection statistic S.sub.i, where S.sub.i =M.sub.i /N.sub.i,, N.sub.i is the output of the parameter dependent normalization factor calculater 50, and M.sub.i is the detection statistic which is the output of detection statistic determinator 46, for each ith range bin, where i is an index. Then, the maximum value, S.sub.imax of S.sub.i is identified; the mean value S.sub.Mean of all S.sub.i except for S.sub.imax is determined; and the standard deviation S.sub.Std dev of all S.sub.i except for S.sub.imax is determined. Finally, a signal is declared present in range bin i if:

Referring to FIG. 4, if a signal is declared present, a computer 70 generates an output signal A The output signal A may be provided to an output device 72, such as a video monitor, printer, audio alarm or other output device capable of providing a human intelligible message indicating that a signal of interest has been detected.

The present invention may also be implemented to further include computer readable program instructions referenced as tail parameter estimator 60 and parameter combiner 64, shown in FIG. 3. The tail parameter estimator 60 receives the data 30 and provides parameter outputs (v.sub.m+1 and p.sub.m+1) 62 of an additional state of a Gaussian mixture model or exponential mixture model. The parameters v.sub.m+1 and p.sub.m+1 account for high intensity noise that might not otherwise be observed in the neighborhood of a particular range bin. The values v.sub.m+1 represents the average value of high intensity data 30, and p.sub.m+1 represents the probability of observing such high intensity input data 30. The values of v.sub.m+1 and p.sub.m+1 may be determined by fitting the K highest intensity values (the "tail") of the complementary cumulative distribution of the non-Guassian input data 30 to a decaying exponential function.

The tail parameters may be estimated in accordance with the following steps:

a. The cumulative distribution function, C(x), of the data 30 is calculated using well known techniques.

b. A limit, L, is defined so that the top p percent of the data 30 are greater than L.

c. Using a linear regression, a linear function is fitted to the function log (l-C(x)) for x>L, and the slope, d and y-intercept b of this line are returned.

d. The m+1-state parameters (v.sub.m+1, p.sub.m+1) are then defined as

The parameter combiner (as opposed to the parameter selector) 64 receives the robust estimated noise parameters 37 from the robust parameter selector 36 and the estimated tail parameters 62 from the tail parameter estimator 60, and adds the parameters p.sub.m+1, v.sub.m+1 of the data 30 in accordance with the rules set forth as follows to provide estimated noise parameters 68: If v.sub.m+1 >v.sub.m.sup.i, then the parameter combiner 64 combines the tail parameters 62 and the robust parameters 37, (p.sub.1.sup.i. . . , p.sub.m.sup.i, v.sub.1.sup.i, . . . , v.sub.m.sup.i) to produce estimated noise parameters 68 of an m+1 state exponential mixture density function for each ith range bin, (p.sub.1.sup.i, . . . ,p.sub.m+1.sup.i,v.sub.1.sup.i, . . . ,v.sub.m+1). The estimated noise parameters 68 generated by the parameter combiner 64 are defined and determined in accordance with the following formulas: ##EQU13##

Additionally, the parameter combiner 64 coalesces states of the exponential mixture density function or Gaussian mixture density function if its variance terms v.sub.k.sup.i are sufficiently close, i.e., within some predetermined limit, to be practically considered as one state.

Still referring to FIG. 3, for the i.sup.th range bin, the residual variance calculator 40 transforms the output 68 and the input data 20 (including both data 26 and data 30) into an estimate output 42 as described above with reference to FIG. 2. Next, the detection statistic calculator 46 transforms the input data 20 and the output 68 and 42 into an exponential mixture amplitude detection statistic, M. The parameter dependent normalization factor calculater 50 uses output 68 to determine a normalization factor, N, for each range bin. The comparator 54 uses the detection statistic M and the normalization factor N to determine whether a signal is present and to generate an output signal A if normalized detection statistic is greater than a threshold. In all other respects, the operations of the residual variance calculator 40, parameter normalization factor calculator 50, and comparator 54, are described above with reference to FIG. 2.

In one alternative embodiment of the invention, the robust parameter selector 36 may be replaced by a minimax parameter selector to provide the robust estimates of the noise parameters 37. The minimax parameters are calculated as follows.

a. Define uncertainty intervals for p.sub.j and v.sub.j denoted I.sub.pj and I.sub.vj respectively, for 1.ltoreq.j.ltoreq.m, and let .DELTA..theta.=I.sub.p1 .chi.. . . .chi.I.sub.pm .times.I.sub.v1 .chi.. . . .chi.I.sub.vm. For each range bin the endpoints of the intervals may be determined from the minimum and maximum values of the parameters of the input data on nearby range bins.

b. Select the minimax parameter vector, .theta..sub.mm, from .DELTA..theta. as follows. Let .mu.(.theta., .theta..sub.0, 0) and .sigma.(.theta., .theta..sub.0,0) be the mean and standard deviation in the noise-only case, as calculated from equation (11), of the processor that uses parameter vector .theta. when the true parameter vector is .theta..sub.0. The value of v.sub.s used in the weighting function (9) is

The threshold is set at the desired PFA, assuming that the processor parameters, .theta., are correct, i.e. if .lambda..sub..alpha. is the threshold of the standard unit normal distribution corresponding to PFA=.alpha., then since .mu.(.theta., .theta., 0)=0, the threshold is .tau.(.alpha., .theta.)=.lambda..sub..alpha. .tau.(.theta.,.theta., 0). The actual probability of false alarm is then ##EQU14## where .PHI. is the unit normal distribution. By definition, ##EQU15## Equation (14) may be substituted into equation (15) and the resulting equation may be solved numerically.

Obviously, many modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described. ##SPC1##