Image Processing Method Including Image Segmentation

The present invention relates to the field of digital image processing and more particularly to a method for segmenting an image signal line by line into segments of constant variance or contrast.

Image segmentation is an initial step in many image processing methods such as feature extraction, pattern recognition, image enhancement, and noise reduction. One type of image segmentation called amplitude segmentation separates an image into segments having similar amplitude (luminance) employing a luminance threshold operation. See "Digital Image Processing" by W. K. Pratt, pp. 534-538, John Wiley & Sons, 1978. Often luminance thresholding is not adequate to segment an image, and other information such as edge information has been used to improve the image segmentation. See for example "Image Segmentation by Pixel Classification in (Gray Level, Edge Value) Space" by D. P. Panda and A. Rosenfeld. Abdou and Pratt have outlined a statistical procedure for segmenting an image signal. See the artical "Qaulitative Design and Evaluation of Enhancement/Threshold Edge Detectors" Proceedings of the IEEE, Vol. 67, No. 5, May 19, 1979, page 753-763. At page 756 of the article the authors suggest employing a maximum likelihood ratio test on the pixel values of an image for the hypothesis that an edge exists at any given pixel, based upon a statistical model of the edge probabilities in the image. However, the authors do not derive actual operating equations for the purposed method due to two difficulties which they duly note. First, there does not exist a single edge model for any large class of images of interest, and therefore, one must have a prior knowledge of the edge probabilities for a limited class of images to which the image to be processed belongs. The second difficulty is that for many complex edge models, no analytical solution exists to derive the maximum likelihood estimators employed in the maximum likelihood ratio test. Because of these problems, after mentioning the possiblity of a statistical approach to image segmentation, the authors turn their attention to techniques employing matched filters to perform segmentation. A problem with these prior art image segmentation methods is that they require as inputs ad hoc threshold levels, and require that empirical assumptions be made about the nature of the images that are to be segmented. In addition, statistical techniques that are based on pixel values can achieve at best two-pixel resolution because such techniques require independent sample sets and since both the mean and variance can change within an image, each sample set requires two pixels to estimate the two unknowns. It is an object of the present invention to provide an image segmentation method that requires a minimum of ad hoc inputs, and produces a more accurate segmentation for all classes of images of interest to a human observer.

The objects of the present invention are achieved by using a generalized-likelihood-ratio test (GLR test) on the variance values of adjacent pixel difference signals. The GLR test examines the validity of the null hypothesis H.sub..phi. that two segments have the same variance, assuming a Laplacian probability density function (pdf) for the image difference signal. The image signal processing method employs the steps of: forming an image pixel difference signal by subtracting adjacent pixel values; forming a maximum likelihood (ML) estimator for the variance signal of a distribution having a Laplacian pdf; forming a generalized-likelihood-ratio test (GLR test) for the null hypothesis H.sub..phi. that a first set of values from a Laplacian distribution having a variance .sigma. are statistically similar to a second set of values from a Laplacian distribution having a variance .sigma..sub.0 ; estimating the variance .sigma..sub.0 of the values of the samples in a first segment of image difference signals using the ML estimator; estimating the variance .sigma. of the values of samples in a neighboring segment of image difference signals composed of the next successive image difference signal in a line of image signals using the ML estimator, applying the GLR test to the .sigma. and .sigma..sub.0 values; and if the null hypothesis is true adding the second segment to the first segment to form a single segment, and repeating the steps of estimating and testing until the entire line is processed, if the null hypothesis is false, identifying the first segment as a recognized segment, and beginning a new first segment with the next successive image difference sample, and repeating the steps of estimating and testing until the line of image signals is processed.

FIG. 1 is a block diagram illustrating apparatus for practicing the image signal processing method according to the present invention;

FIG. 2 is a graph showing the generalized-likelihood ratio t plotted vs. the sigma ratio .sigma./.sigma..sub.0 ;

FIG. 3 is a graph showing the critical sigma ratio vs. segment length for various levels of significance .alpha.;

FIG. 4 is a flow chart illustrating the image segmentation method according to the present invention; and

FIG. 5 is a graph illustrating a portion of an image signal useful in explaining the operation of the present invention.

Before describing the method of the present invention in detail, the derivation of the generalized-likelihood-ratio (GLR) test for the variance of an image difference signal will be described. The methods of maximum likelihood estimation and the generalized-likelihood-ratio test are tools of mathematical statistics and can be found in many textbooks on mathematical statistics such as "An Introduction to Mathematical Statistics" by R. J. Larsen and M. L. Marx, Prentice Hall, Inc., Englewood Cliffs, N.J., 1981.

To determine whether or not two regions of an image are statistically equivalent it is necessary to establish a criterion that distinquishes between the null hypothesis H.sub..phi. that they are the same and the alternative hypothesis H.sub.1 that they are not. If Y(1), Y(2), . . . , Y(n) are a random sample from a distribution Y with the unknown parameter p, the objective is to test

versus

at a specified level of significance .alpha.. Where p is the parameter value being tested, and p0 is the value of the parameter, that satisfies the null hypothesis.

First, it is necessary to define two parameter spaces, w and W, where w is the subset of W that includes all unknown parameter values admissible under H.sub..phi.. If p is restricted to be a positive real number, as it is for the sigma value of a Laplacian for which the hypotheses will be tested, the spaces in set notation are

Let L be the joint probability density function (pdf) of the Y(i)'s where p is considered as an independent variable and the y(i)'s are considered fixed, then the likelihood of observing the y(i)'s is given by ##EQU1##

Maximizing the likelihood function under w and separately under W and taking the ratio is then defined as the generalized likelihood ratio, t. Symbolically it is given by ##EQU2##

If t is close to 1.0, the spaces w and W are highly overlapped and the hypothesis H.sub..phi. is most likely true. If t is much less than 1.0, w is much smaller than W and the hypothesis is most likely false.

The numerator of t is the maximum likelihood (ML) estimate under w which is simply p0. The denominator of t is the ML estimate under W. Determining W requires a knowledge of the pdf of Y. The stationarity of the assumed pdf is essential to the success of the GLR test.

It is known that the statistics of the adjacent pixel difference data for images that are of interest to a human observer are more stationary than the actual pixel values and are well modelled by a Laplacian. Because of this, the GLR test will be based on the difference data rather than the actual data, and the assumed pdf for a Laplacian distribution is (3) ##EQU3## Defining p as .sigma. (the variance of a Laplacian pdf), and Y as .DELTA.S (the image difference signal), the likelihood function under W is given by ##EQU4## where .DELTA.S(n) are successive samples of the image difference signal.

Differentiating ln(L) with respect to .sigma. and setting the result to zero to obtain the ML estimate of the variance .sigma. under W results in ##EQU5## The generalized likelihood ratio (GLR) is then ##EQU6## where .sigma..sub.0 is an estimate of the variance .sigma. under w. This reduces to

The numerical behavior of t considered as a function of (.sigma./.sigma..sub.0) is such that t decreases to 0 as (.sigma./.sigma..sub.0) becomes either increasingly less than 1 or increasingly greater than 1. When (.sigma./.sigma..sub.0) equals 1, t equals 1. Table 1 illustrates this pattern for the particular case where n=10.

For a given level of significance .alpha., the likelihood ratio principle states that H.sub..phi. should be rejected for any t<t*, where Prob(t<t*.vertline.H.sub..phi.)=.alpha.. As Table 1 indicates, the shape of t(.sigma./.sigma..sub.0) requires that the decision rule be divided into two critical regions, one for values of (.sigma./.sigma..sub.0) close to 0 and the other for values of (.sigma./.sigma..sub.0) much larger than 1. This is illustrated in FIG. 2. As the figure shows an equivalent statement of the decision rule is to reject H.sub..phi. whenever ##EQU7##

Here it is necessary to make a slight approximation by setting the probability of each critical region to .alpha./2. The two tails of the critical regions indicated by cross hatching in FIG. 2 do not have exactly the same probability, nevertheless, the two are numerically close enough that the likelihood ratio criterion is not substantially compromised. Equation (8) then becomes

To apply this decision rule the pdf of (.sigma./.sigma..sub.0) given H.sub..phi. is required. Since .sigma..sub.0 is a fixed, known value, it is only necessary to determine the pdf of .sigma. given H.sub..phi.. This can derived as follows: Let S(1), S(2), . . . , S(n) be n independent Laplacian random variables defined by the parameter .sigma..sub.0 (i.e. H.sub..phi. is given). We wish to find the pdf of ##EQU8## First, consider the case where n=1. It is easy to show that the pdf of .vertline..DELTA.S(1).vertline. is an exponential, which is not surprising since a Laplacian is nothing more than a two-sided exponential. To derive the pdf of .sigma.n/.sqroot.2, we resort to moment generating functions. The moment generating function for an exponential pdf defined by the parameter .sigma..sub.0 /.sqroot.2 is given by ##EQU9## for all i, and using the fact that the moment generating function for a sum of independent random variables is the product of the moment generating functions for each variable, the moment generating function for .sigma.n/.sqroot.2 is ##EQU10## This is identical to the moment generating function for a gamma distribution with parameter values n, and .sqroot.2/.sigma..sub.0, and using the theorem that if two random variables have the same moment generating functions for some interval containing 0, they also have the same pdf's, the pdf for .sigma.n/.sqroot.2 is given by ##EQU11## where z=.sigma.n/.sqroot.2.gtoreq.0. The decision rule is now to reject H.sub..phi. whenever either of the following is true

where a and b are the unknowns.

In integral form the probabilities are given by ##EQU12## By making the substitution ##EQU13## The integrals are incomplete gamma functions which can be written as summations, and the equations for the unknowns a and b become ##EQU14## Given n and .alpha., these can be solved for the critical regions a and b, and the decision rule becomes reject H.sub..phi. whenever either of the following is true ##EQU15## or equivalently

It remains to specify how the regions for estimating .sigma..sub.0, and .sigma. are defined. After k stationary line segments have been isolated extending from the first pixel to the jth pixel, the value for .sigma..sub.0 is the ML estimate for the (k+1)th segment extending from the (j+1)th pixel value to the last value deemed consistent with H.sub..phi.. The value for .sigma. is then the ML estimate for the difference between the last value and the next pixel value. The search is reinitialized at the beginning of each line with the absolute difference between the first and second pixels defining .sigma..sub.0 (the ML estimate requires only one difference value), and the absolute difference between the second and third pixels to determine .sigma..

Stationary segments grow or close along line segments depending on whether the hypothesis H.sub..phi. is accepted or rejected for the next pixel, and .sigma..sub.0 .sigma. are updated appropriately. As defined, stationary regions can be isolated with single pixel resolution because only one parameter is required for each segment and it can be estimated with a single difference signal, and the estimates for .sigma..sub.0 and .sigma. can be calculated recursively making the technique relatively simple to implement. The equations for a and b are not recursively computable due to their non-linear dependence on n. For a given level of significance, values of a and b can be computed off-line and stored in a buffer for a reasonable range of values of n. FIG. 3 shows the dependence of a and b on n for three levels of significance (0.5, 0.1, 0.01). As expected, the criterion for rejection of H.sub..phi. becomes less stringent, i.e., the ratio of .sigma./.sigma..sub.0 can be closer to 1.0 and still result in a rejection, as n increases. This is intuitively reasonable because adding one difference value to the ML estimate will not change it significantly. The criterion also becomes less stringent as the level of significance increases. Recalling that the level of significance is related to the confidence in accepting H.sub..phi., as .alpha. increases acceptance of H.sub..phi. becomes more difficult and rejectance more common. As such, the GLR test is self-adaptive because it non-linearly accounts for variations in n, and .alpha. in a statistically optimal manner.

Referring now to FIG. 1, generic image processing apparatus employing the image segmentation method according to the present invention will be described. An image or scene is sampled by scanning apparatus 10, such as a video camera or graphic arts scanner. The sampled image is digitized by an analog to digital converter 12 and the digitized sampled image is segmented by GLRT image segmentation apparatus 14. The sampled image signal is delayed by a delay device 18 to allow time for the image segmentation hardware to identify the image segments before the image is processed further. The delayed image signal and the image segmentation information generated by the image segmentation apparatus are supplied to a digital image processor 20. The image segmentation apparatus and the digital image processor comprise a programmed general purpose digital computer or alternatively a custom designed digital signal processing circuitry. The digital image processor performs the image processing on the image, using the image segment information, and supplies the processed image signal to an output device 22, such as a CRT display or hard copy generator in the case that the image processing was performed to improve the appearance of the image. The image segmentation apparatus 14 includes a memory 16 that contains the tables of values of a and b as a function of the number n for different values of the confidence level .alpha.. The image segmentation apparatus receives an operator input specifying the desired level of confidence .alpha., and selects the appropriate table of values accordingly.

The image segmentation method will now be described with reference to the flow chart of FIG. 4. The following description assumes that the image signal is in a linear form resulting from a raster scan. At the start of each scan line, the internal counters n indicating the number of pixels in a segment, j indicating the location of the pixel in the line, and k indicating the number of segments and a point Seg (k) that points to the starting pixel for the kth segment are initialized. Counters n and j are incremented by 1 and then the image difference signal .DELTA.X.sub.1 between the first and second samples of the image signal is formed. A maximum likelihood estimate .sigma. of the variance of the difference signal .DELTA.X.sub.1 as given by equation (10) is then computed.

Next, a check is made to see if this is the first pixel in a segment (n=1). If so, .sigma..sub.0 (an estimate of the variance of the segment as generated thus far) is set to equal the square root of two times the absolute value of .DELTA.X.sub.1 and n is incremented by 1. The next difference signal .DELTA.X.sub.n is formed, and the maximum likelihood estimate .sigma. of the variance of the difference signal is formed. This time, n does not equal 1, and the null hypothesis (H.sub..phi.) that the sigma value of the present segment is equivalent to the sigma value of the present segment plus the next image difference signal sample is checked. If the null hypothesis is true and it is not the end of the scan line, .sigma..sub.0 is updated by factoring in .sigma., and another .DELTA.X is formed and added to the segment for testing. If the null hypothesis is false, the segment is ended, k is incremented by 1, and the value Seg (k), is stored in a register. If it is the end of the line, the location of the kth segment is stored and the process is repeated for the next line. If it is not the end of the line, .DELTA.X.sub.1 is set to .DELTA..sub.n, .sigma..sub.0 is set to the square root of two times the absolute value of .DELTA.X.sub.1, n is set to 1, and the next segment building process is started. When the last pixel of a scan line is reached k and n are incremented, Seg (k) is stored in the register for the last segment of the line.

The check on the truth of the null hypothesis H.sub..phi. involves testing whether inequalities (19) are satisfied by referring to the values of a and b for the given level of confidence .alpha. that were computed from equations (17) and stored in the memory.

Application of the method of image segmentation will now be described with reference to a numerical example shown in the graph of FIG. 5. FIG. 5 shows the first seven samples in a line of image information. The sample values X.sub.j are shown above the graph. The level of significance .alpha. was chosen to be 0.25, i.e. 75% confidence that the rejection of the Null Hypothesis H.sub..phi. is correct. A table of values for a and b for segments of given length n at a confidence level of 75% are shown in Table 2 below. These values were computed from equations (17) and stored in a table look-up memory.

The signal shown in FIG. 5 is segmented by the segmentation method as shown by the values in Table 3 where .DELTA..sub.ij =.vertline.X.sub.i -X.sub.j .vertline..

From the values in Table 3 it can be seen that the first segment identified by the method extends from the first sample X.sub.0 to the third sample X.sub.2. The next segment extends from sample X.sub.2 to X.sub.3, the next from X.sub.3 to X.sub.5, and the last segment from X.sub.5 to X.sub.7.

The image segmentation method can be employed in image processing methods such as image coding, restoration, enhancement, and pattern recognition where it is useful to separate the image into regions of constant variance or contrast. In particular, the method is advantageously employed in a statistical image restoration scheme using a maximum a posteriori (MAP) filter to remove noise from an image signal. The noise restoration method is the subject of copending Application No. 802,107, filed Nov. 25, 1985, by the present Inventor. A major advantage of the image segmentation method is that only one empirically selected parameter, the level of significance .alpha. of the GLR test, is required. Another advantage is that since the image difference signal is highly stationary, the statistical test is optimal in a mathematically tractable manner. Other advantages are that the method is self-adaptive, recursively computable, and simple to implement in image processing hardware. It also provides for single pixel resolution which is unachievable with statistical techniques that operate on the pixel values rather than the difference pixel values.