Better Contrast Noise by Residue Image

The invention relates to a method for contrast-enhancing in a multi-dimensional image. The method comprising the steps of:

sampling the image according to a regular array of instants

based on said instants locally detecting a contrast quantity

detecting said quantity as being either in a low range or in a high range

when in said high range locally relatively enhancing said contrast with an enhancement factor that is a monotonically non-increasing, non-uniform function of said local contrast quantity.

A method of this kind has been described in F. P. de Vries, Automatic, Adaptive, Brightness Independent Contrast Enhancement, Signal Processing Vol. 21 (1990) 169-182. There, the enhancement is applied to the logarithm of the image data. Uniform windows are used, and no sub-sampling is applied. The enhancement factor is (mu+kap) if the local residue energy c(p) is lower than the variance var, and (mu+kap*var/contrast) otherwise. The meanings of these parameters will be explained hereinafter. A problem, as shown in FIG. 5, infra, is that low-contrast features that are minimally above a threshold are enhanced, whereas slightly higher contrast values are relatively obscured. In a particular region, the output contrast even decreases as the input contrast increases. This clearly leads to unwanted suppression of useful information. The present invention is based on the insight that applying the method according to the reference looses information when contrast is just above the threshold, and therefore the reference method is quite inflexible.

Amongst other things, it is an object of the present invention to improve the known method for retaining an optimum amount of displayable information. Now, according to one of its aspects, the invention is characterized in that in said high range, the contrast after said enhancing is a monotonically increasing function of said contrast before said enhancing. As discussed more in detail hereinafter, the resulting enhancement has both objectively and subjectively been satisfying for a broad range of applications.

By itself, earlier work of the present inventor has been described in B. Escalante-Ramirez and J. B. O. S. Martens, Noise Reduction in Computerized Tomography Images by Means of Polynomial Transforms, J. of Visual Image Communication and Image Representation, Vol. 3, (272-285), Sept. 1992, and related Patent Application EP 513 906, corresponding U.S. Ser. No. 07/874,660 to the same assignee as the present application. This reference was directed to relatively attenuating all image variation if its amplitude was considered insignificant. The present invention, on the other hand, is also directed to enhancement, and in particular has realized that contrasts just above the threshold should be enhanced preferentially relative to higher contrasts. Under certain circumstances, the present invention gives results that are functionally equivalent to the Escalante et al reference, by leaving out the contrast enhancement just above the threshold. In that case, the implementation according to the present invention is more efficient than in the reference.

Advantageously, the non-uniform function is monotonically decreasing, in particular, has an asymptote with an asymptotic value for high contrast equal to one. This has been found very satisfying in practice.

Advantageously, the method comprises before said detecting measuring a contrast occurrence spectrum in a second region that is larger than a region of said locally detecting and from said spectrum deriving a boundary between said low range and said high range. Although this means that the method must first assess the larger region before being able to execute the enhancing proper, this provides an automatic setting to the best value.

Advantageously, the method comprises detecting an overall contrast weakness in a third region that is larger than a region of said locally detecting and thereupon at least notionally retrocoupling an averaged image signal of the second region for contrast enhancing at a larger window granularity. Regions of low overall contrast may thus be processed in an overall manner that saves processing effort and thus gives higher speed.

The invention also relates to a device for practising the method, and comprising a first branch for through a deriving means generating from an input signal a window-wise local average signal and a second separate branch having detection means for generating a windowed image variability signal. The said second branch has deriving means for from said variability signal deriving a first control signal having an inverse response to said variability signal and a second control signal having a non-inverse response to said variability signal, a first size-controller of said average signal being controlled by said first control signal. The first branch has first processing means for processing said size controlled average signal through an inverse device of said deriving means. The device has a third branch with a second size-controller wherein said input signal is size controlled through said second control signal and said third branch having second processing means corresponding to said first processing means. The device has adder means for adding output signals of said first and third processing means. This represents a particularly straightforward implementation of the method according to the invention. Various further advantageous aspects of the invention are recited in dependent claims.

These and other aspects and advantages of the invention will be discussed in detail with respect to preferred embodiments disclosed hereinafter, and in particular with reference to the appended Figures that show respectively:

FIG. 1 illustrates a structure of adaptive residue image processing;

FIG. 2 illustrates a structure for deriving the energy of the residue signal;

FIG. 3 illustrates a structure for deriving the energy of the signal derivative;

FIG. 4 illustrates generation of enhancing and attenuation factors;

FIG. 5 illustrates various examples of response curves;

FIG. 6 illustrates a histogram of the contrast measure.

In order to determine the residue amplification factor kap, a strategy is specified for enhancing and/or noise-reducing the image. If the contrast measure c is large, then the signal is clearly visible, so that we can reasonably assume that no enhancement is needed, i.e. kap(c)=1 for large c. If c is small, such as c<c.sub.t. then the signal can be assumed constant, and if required the residue signal can be deleted by setting kap(c)=0 for c<c.sub.t, This requirement is not satisfied in the Vries reference, wherein kap(p)=mu+kap for c(p).ltoreq.sig.sub.n.sup.2. This high amplifying factor in noisy regions limits the usefulness of the reference's method. For c>c.sub.t, the signals are judged significant, but must be amplified to become clearly visible. The output contrast measure after amplification is kap.sup.2 (c)*c. This output contrast must be an increasing function of the input contrast. This condition is not satisfied in the reference, that has an intermediate minimum at c=sig.sub.n.sup.2 *(kap/mu). This is an unwanted property, because image features with a contrast well above the threshold are depicted as less relevant than the (noisy) image features that are only just above the threshold. For large values of the input contrast c, the output contrast is ›c*mu.sup.2 +2mu*kap*sig.sub.n.sup.2 !. Hence, even for mu=1, the output range is slightly expanded. Herein, the following class of functions is suggested, together with some approximations thereof, that allow fast processing: ##EQU1## Another, more simple realization is as follows: ##EQU2## Herein parameters c.sub.t, c.sub.v nd s occur, with 0.ltoreq.s.ltoreq.1. The performance of this expression is comparable to the more complicated one given above, but the necessary calculations are relatively elementary.

FIG. 1 shows a structure of an adaptive residue image processing operation according to the invention. The block-wise structure as shown may be mapped on a so-called video signal processor that consists of multiple processing elements which each may execute one or more elementary processing operations, so that the total hardware may effectively execute the whole processing as shown. By itself, a video signal processor of this kind is known from U.S. Pat. No. 5,103,311. The signal of the image arrives at input 40, as sourced by an appropriate camera, large-scale storage memory, or otherwise 41. The input may be according to various sequential organizations, such as line-by-line. The device then has sufficient memory available to store pixel values or, as the case may be, intermediate processing results, for producing output results of the various stages that represent all input quantities necessary therefore. In certain cases, additional frame buffers could be required. Elements 20, 22 together form a filter 23 for calculating a local average value of the pixel intensities. The dimension of the local environment is according to the structure of the picture, but for a television-like size (order of one million pixels) a square of 25 pixels could be appropriate. This size is not a restriction. Non-square shapes of the environment could be useful. By itself, element 20 may be a device for windowing of the kind that is in wide use for picture processing. Element 22 is a downsampling device for thereby limiting the amount of data to be processed in the subsequent processing stages. Downsampling can be by an integer or non-integer factor. If sufficient processing power is available, downsampling could even be foregone, so that element 20 alone would generate the local average. If downsampling is not done, later interpolating is not necessary.

Thereby, the output signal of block 22 is an array of average values as indicated. Block 24 receives the input values as well and calculates two adaptivity values that are functions of position (p). The first one ›mu(p)-kap(p)! is the mean value weighting function, the second one ›kap(p)! is the residue weighting function. Element 26 is a multiplier. The set of multiplied values is sent to block 28 that executes upsampling. Finally in block 30 an interpolation is executed, according to expressions that are inverse to the ones effected in block 22, although they may be somewhat different. Likewise in block 32 an upsampling on the results from block 24 is executed, followed by an interpolation in block 34. In element 36 a multiplication between the residue weighting function and the original pixel values is executed. The output signal arrays from elements 30, 36 are added in adder element 38 to produce the upgraded output signal as indicated. Such elements as necessary for correcting timing discrepancies have not been shown. It should be remarked that block pair 28 and 32 can be mutually identical, which is also valid for the block pair 30 and 34.

Just as in the Escalante and Martens reference, supra, the present invention can be used in a pyramidal or hierarchical organization. To that effect, the output signal of block 22 is used for constituting a larger integration area if the output of this block 22 varies sufficiently little over such larger area. For convenience, the necessary detector has not been separately shown; it compares the local variability signal c(p) to a particular threshold. If c(p) is lower than the threshold, the output signal of block 22 is functionally or notionally retrocoupled to input 40 and processed again with a coarser granularity. If the local variability is above the threshold, the setup as shown is used only once, and no retrocoupling is effected. The process of thresholding can be executed several times in succession, so that a pyramidal organization results, with successively coarser granularities and increased window sizes.

The adaptive residue filtering of the foregoing assumes that we can derive the mean-value amplification factor and the residue amplification factor from the input signal. Such factors are functions:

and

of the mean signal value fm(p) and some contrast value c(p) that expresses how much the signal locally deviates from its mean value. The mean signal value is defined as:

All contrast measures that have been considered can be derived by some kind of quadratic filtering of the input signal. An advantageous choice is as follows:

which is equal to the weighted energy in the residue signal. A further refinement can be attained by letting the weight factors mu and kap depend on the average value of the function f. For brevity this is not considered further.

FIG. 2 shows a structure for deriving the total energy of the residue signal according to the above expression for use in the arrangement according to FIG. 1 (cf. also FIG. 4). The blocks in the Figure are again elementary elements of one or more video signal processors and in fact, execute part of the functionality necessary for block 24 in the arrangement of FIG. 1. The input signal f(x) has been shown in FIG. 1. Block 50 successively calculates the square of the pixel values. Blocks 52, 54 functionally correspond to blocks 20, 22, respectively. Likewise, blocks 56, 58 correspond to blocks 20, 22, respectively. Now, the final block 60 executes a squaring of its input values. Block 62 calculates the difference of the two input values received as pertaining to the same positions, which boils down to the difference between the average of the square and the square of the average. If this quantity is large, locally there is much contrast; if this quantity is small, there is little contrast; if zero, the pixels have uniform values. The output signal ›c(p)! is used for further processing. The set up requires only one additional filter operation since the mean value, fm(p), is already produced in FIG. 1 according to the above expression. The necessary hardware is in the lower branch of FIG. 2, to wit, only blocks 56 and 58.

All residue signals are not always equally significant. Therefore, a second quadratical energy contrast measure is considered has a directional character and in consequence tunes preferentially for instance to edges in the image. Such contrast measures have the general form

wherein d.sub.i (x) is a derivative filter along the i-th dimension of the D-dimensional signal space. By itself, this contrast measure was already used in the Escalante and Martens reference.

FIG. 3 shows a structure for according to the above deriving of the energy of the signal derivative for use in the arrangement according to FIG. 1. The arrangement consists of one or more parallel branches that each determine the signal derivative in a particular direction 1 . . . D (blocks 64, 70) followed by downsampling (blocks 66, 72) and squaring/energy determination (blocks 68, 74). Block 76 represents an overall adder to produce output signal ›c(p)!.

For determining the value of kap, we herein suggest in particular to use the following class of functions: ##EQU3## for the residue amplification factor if c>=c.sub.t and kap(c)=0 otherwise. The residue amplification factor is equal to kap(c.sub.0)=kap.sub.0, with kap.sub.0 >1, at the contrast c=c.sub.0. The output contrast, kap.sup.2 (c)*c is an increasing function of the contrast c, and has a derivative equal to s.sub.0 at c=c.sub.0, where 0<=s.sub.0 <=1. The power factor .beta.=0 controls the asymptotic behaviour of the residue amplification factor. It is advantageously set equal to .beta.=1 or .beta.=(1-s.sub.0)/(kap.sub.0.sup.2 -1). The input contrast and the output contrast are equal for large values of c, as can be seen from FIG. 5. The choice .beta.=(1-s.sub.0)/(kap.sub.0.sup.2 -1) results in the following simplified expression for the residue amplification factor ##EQU4## where .gamma.=.beta.+1=(kap.sub.0.sup.2 -s.sub.0)/(kap.sub.0.sup.2 -1). The output contrast is c+(kap.sub.0.sup.2 -1)*c.sub.0 *(c.sub.0 /c).sup..beta., which has also been shown in FIG. 5. Visual inspection of processed images learns that the influence of the parameter .beta. is little, so for convenience the value .beta.=(1-s.sub.0)/(kap.sub.0.sup.2 -1) is most often adopted. The simplified expression for kap(c) given earlier behaves in similar manner and will not be considered in detail hereinafter.

Instead of specifying absolute values for the contrast c.sub.0 and c.sub.t, we prefer to specify them as multiples of the expected contrast output for uniform regions, i.e. the expected output is only noise is present. This expected output is c.sub.n =(1-.alpha..sub.2)*sig.sub.n.sup.2 if the contrast measure is equal to the energy in the residue signal, and c.sub.n =lam*D*sig.sub.n.sup.2 if the contrast measure is equal to the energy in the signal derivative. Herein, the quantity lam=.intg..sub.F d.sub.1.sup.2 (x)dx= . . . lam=.intg..sub.F d.sub.D.sup.2 (x)dx, each function d being a directional derivative in direction 1 . . . D. For two-dimensional images D=2, and for a sequence of two-dimensional images, D=3. The quantity .alpha..sub.n is calculated according to the expression .alpha..sub.n =.intg..sub.F w.sup.2n (x)dx with usually taken n=2.

Furthermore, the quantity lam is determined along the following line of thought. Because the derivative filters d.sub.i (x) along the D different dimensions are similar, we have the quadratic form of equation (15) has one non-zero eigenvalue

of order D. In the case of white input noise with variance .sigma..sub.n.sup.2, the first characteristic function of the quadratic form is equal to ##EQU5## with M=D/2 and d.sub.n =2.lambda..sigma..sub.n.sup.2. The signal response is denoted by ##EQU6## Since the first characteristic function is of the same form as in the above case of the energy measure, the analytic expressions in equations (37), (40) and (41) for the PDF apply, with e.sub.b (p) and e.sub.n replaced by d.sub.b (p) and d.sub.n respectively. The expressions for the mean value

and variance

of the contrast measure have to be changed accordingly. The peak in the histogram of c occurs, for uniform regions, at c.sub.0 =.sigma..sub.n.sup.2..lambda..(D-2), and hence cannot be used to estimate the noise variance .sigma..sub.n if D=2. If D=2, we can use the peak in the histogram of .sqroot.c, which occurs at .sqroot.d.sub.n /2==.sigma..sub.n..sqroot..lambda..

For example, in the specific case of a binomial window w.sup.2 (x) or order B, the derivative filter d.sub.i (x) is a separable filter with impulse response ##EQU7## for j=0, . . . , B along the i-th dimension, and impulse response ##EQU8## for j=0, . . . , B, along the other dimensions ›16!. The non-zero eigenvalue ##EQU9##

The contrast c.sub.0 up to which the maximum residue amplification factor of kap.sub.0 is applied is set to c.sub.0 =T.sub.0 *c.sub.n, where T.sub.0 is usually close to one. Since the threshold contrast c.sub.t distinguishes noisy regions from significant image features, choosing c.sub.t =T.sub.c *c.sub.n proportional to the expected output in noisy regions is a reasonable strategy. Typically, T.sub.c has a value larger than 1.

FIG. 4 shows an arrangement for generating the enhancing and attenuating factors, thereby realizing block 24 in FIG. 1. Block 80 generates signal c(p) and may be formed as the arrangements shown in either of FIGS. 2, 3. Block 82 is a lookup-table device for generating ›kap(p)!, block 84 a second look-up table device for generating ›mu(p)-kap(p)! containing the appropriate translation functions. Herein, kap(p) is the residue weighting function, and ›mu(p)-kap(p)! the mean value weighting function. In FIG. 4, these weighting functions are assumed to be independent of the mean value fm(p).

FIG. 5 shows various examples of response curves. Along the horizontal axis figures the input contrast, along the vertical axis the output contrast. Straight lines through the origin like 100, 108 amplify each contrast with a uniform factor that by itself does not make the picture any more instructive. Now, curve 102 shows the result obtained by De Vries, supra, for mu=1, kap=2 and var=1. It is clear that the usage of the reference method actually has a negative slope in the output response just above the threshold. The present inventor has found that the slope above this threshold should be monotonic, and moreover, that the response curve should preferably have an asymptote that is the line for unchanged contrast. Curve 104 corresponds to the expression (59), with .beta.=1, c0=1, kap0=3, and s0=0. Curve 106 corresponds to expression (60), with c0=1, kap.sub.0 =3, and s0=0.

For the response below the threshold, various curve shapes are useful. In the Figure, below the threshold a uniform contrast enhancement has been assumed as represented by a straight line. Another approach is according to the Escalante et al reference, which in that region suppresses all contrast but transmits a uniform level. Generally, it is clear that the choice of the various parameters allows straightforward adaptation to incidental requirements of the actual image.

FIG. 6 shows a histogram of the contrast measure, that is the energy in the residue signal of a particular image that was used to design policy and optimum values of parameters. This histogram is used to estimate the noise variance over the image, under the assumption that the image comprises a sufficient amount of uniform regions, so that the mode of the histogram is not shifted by the image structure. The noisy appearance of the histogram necessitates some smoothing to obtain a reliable estimate of the mode. A preferred method is to use uniform windows for such smoothing. In this example, the standard deviation has been estimated as sig.sub.n =1.3. It has been found that assuming this value gives very reliable results over a broad range of noise variances.