Methods and Apparatus Particularly Useful for Blind Deconvolution

The present invention relates to methods and apparatus particularly useful for performing a blind deconvolution operation, e.g. for controlling an adjustable linear filter (sometimes called an equalizer) receiving the output of an unknown system in order to produce a desired response for recovering (deconvolving) the input to the system.

The problem of blind deconvolution, as illustrated in FIG. 1 of the drawings, arises when there is an unknown system having an output which is observed, but whose input is unobserved. In such a system it is desired to recover (deconvolve) the input by identifying the inverse of the unknown system. This is done by using an adjustable linear filter (sometimes called an equalizer) which is adjusted to recover, or deconvolve, the input up to a constant shift (delay), and possibly a constant phase or a sign ambiguity.

The problem of blind deconvolution is of considerable interest in many fields of engineering and applied science. For example, in data communication, the unobserved input is a transmitted sequence of data symbols (bits), and the unknown system represents the distortion in the transmission channel (e.g., a telephone line) which causes the undesirable phenomena of inter symbol interference (ISI). The equalizer (adjustable linear filter) in this case attempts to recover the transmitted sequence (i.e., the message) by canceling, or inverting, the distortion caused by the channel.

A similar problem occurs in image restoration or reconstruction, wherein the unobserved input is the original image (scene), and the unknown system represents the imperfections in the electronic or photographic medium which essentially causes blurring effects. The equalizer (adjustable linear filter) in this case attempts to reconstruct the original image by de-blurring, that is canceling the imperfections in the image formation system. This problem is particularly important in fields such as astronomy, remote sensing, and medical imaging, where the blurs cause a severe degradation in the quality and resolution of the observed image. Similar deconvolution problems can be found in the analysis of geophysical signals, in underwater acoustics, and the like.

According to the present invention there is provided a method for controlling an equalizer to be coupled to an unknown system having an input and an output which equalizer receives the output of the unknown system in order to produce a desired response for recovering the input to said system, characterized by iteratively adjusting the equalizer such that the unknown system combined with the equalizer behaves essentially as a linear system whose (t, n) taps, for some combinations of t and n, are iteratively adjusted according to the following rule: ##EQU2## where s.sub.t,n denotes the (t, n) tap before the iteration, s'.sub.t,n denotes the (t, n) tap after the iteration, I is a preselected integer greater than or equal to one, .alpha..sub.i i=1,2 . . . I are preselected scalars that may vary from iteration to iteration, and p.sub.i, q.sub.i i=1,2, . . . I are preselected non-negative integers such that p.sub.i +q.sub.i .gtoreq.2.

As will be described more particularly below, the present invention involves a new class of iterative deconvolution methods that converge very fast to the desired solution in which the inverse of the unknown system is identified, and the input is recovered. These methods are computationally efficient and statistically stable. They are capable of identifying both the magnitude and the phase of the unknown system transfer function, as distinguished from the known linear prediction methods that are incapable of identifying the phase. Another important feature of the proposed methods is that they do not suffer from spurious/ambiguous solutions, and they always converge to the desired response. Error analysis indicates that in many cases the method of the present invention is superior to linear prediction methods based on the minimum phase assumption in the sense of statistical stability.

A futher important advantage of the novel method is that it is universal in the sense that it does not impose any restrictions on the nature of the input, and therefore can be applied to a wide range of problems. Thus, the tap index n in the above relationships may be one-dimensional, as in the data communication problem where n represents normalized sample time. The index n may also be multi-dimensional, as in the image reconstruction problem.

The proposed method can be used for the purpose of system identification where the equalizer can be viewed as the model for the inverse of the unknown system. The proposed method can also be used for the purpose of speech/image enhancement and compression, where the equalizer parameters correspond to the speech/image parameters, e.g. the LPC parameters in speech.

Further features and advantages of the invention will be apparent from the description below.

The invention is herein described, by way of example only, with reference to the accompanying drawings, wherein:

FIGS. 1,2 and 6 are diagrams helpful in explaining the invention;

FIGS. 3,4,5 and 7 are flow charts illustrating four algorithms in accordance with the present invention for iteratively adjusting the equalizer in order to identify the inverse of the unknown system and to recover (deconvolve) the unknown system input from measurements of its input;

FIG. 8 is a block diagram illustrating a digital, quadrature amplitude modulation (QAM) demodulator constructed in accordance with the present invention, and

FIG. 9 is a flow chart illustrating the blind deconvolution method for controlling the demodulator of FIG. 8.

It is believed a discussion of mathematical preliminaries at this point will be helpful in understanding the description of the present invention following this discussion.

A. Notations:

We denote a tensor (a matrix) by an upper-case letter (e.g. X), a column tensor (a column matrix) by a lower case bold faced letter (e.g. x), and a scalar by a lower-case non-bold letter (e.g. x).

We denote by x.sub.n the n.sup.th element of a set of real/complex numbers. For example, x.sub.n may be the n.sup.th time sample or the n.sup.th component in a sequence, or it can represent the value of a sampled image at location n.

.SIGMA..sub.n --sum over all possible values of n

.pi..sub.l=n.sup.m (.).sub.l --the product (.).sub.n.(.).sub.n+1 . . . (.).sub.m

x.sup.T --transpose of a tensor x

x.sup.+ --conjugate-transpose of a tensor x.

x*--complex conjugate of a scalar x.

.vertline.x.vertline.--complex magnitude (absolute value) of a scalar x.

B. Linear Systems:

The relation between the input x.sub.t and the output y.sub.t of a linear system (filter) is given by (See FIG. 2): ##EQU3## where h.sub.t,n is the (t,n) tap of the linear system.

The linear system whose (t,n) tap is h.sub.t,n convolved (combined) with the linear system whose (t,n) tap is g.sub.t,n is the linear system whose (t,n) tap is .SIGMA..sub.k g.sub.t,k.sup.h.sub.t-k,n-k

If the linear system is shift invariant then h.sub.t,n is invariant of t, in which case we denote by h.sub.n =h.sub.t,n its n.sup.th tap.

The transfer function (frequency response) of a linear shift invariant system whose n.sup.th tap is h.sub.n is given by: ##EQU4##

C. Cumulants and High-Order Spectrum

The joint cumulant of the random variables x.sub.1,x.sub.2, . . . x.sub.m is defined by: ##EQU5## where ##EQU6## is the joint derivative of order m with respect to the variables .mu..sub.1, . . . .mu..sub.m ; ln(.) is the natural logarithm (to the base of e), j=.sqroot.-1 and E{.} is the statistical expectation of the bracketed quantity.

A general formula relating cumulants to moments of the random variables x.sub.1,x.sub.2, . . . x.sub.m is given in the book: D. R. Brillinger, "Time Series, Data Analysis and Theory" Holden-Day, California, 1981.

Thus, for example: ##EQU7##

For notational convenience, we denote by: ##EQU8##

Let x.sub.t be a set of random variables such that cum(x.sub.t ;x.sub.t-t.sbsb.1 ; . . . x.sub.t-t.sbsb.p-1 ; x.sub.t-t.sbsb.p *; . . . x.sub.t-t.sbsb.p+q *) is independent of t for all combinations of t.sub.1, . . . t.sub.p+q ; Then, its (p,q)-order spectrum is defined by: ##EQU9##

D. Tensors

A tensor A is an array of real/complex valued numbers whose (n,m) element is denoted by A.sub.nm.

The product of a tensor A by a tensor B is a tensor C=AB whose (n,m) element is: ##EQU10##

The unit tensor is a tensor whose (n,m) element is equal to 1 whenever n=m and equal to zero whenever n.noteq.m.

The inverse A.sup.-1 of a tensor A is a tensor whose product by A is equal to the unit tensor.

The transpose A.sup.T of a tensor A is a tensor whose (n,m) element is A.sub.mn. The conjugate-transpose A.sup.+ of a tensor A is a tensor whose (n,m) element is A.sub.mn *.

A column tensor b is a tensor whose (n,m) element is 0 whenever m.noteq.0. We denote the (n,0) element of b by b.sub.n.

Finally we note that if the subscripts n and m are one-dimensional then the tensor A and the column tensor b are a matrix and a column-matrix (vector), respectively.

As described earlier, the present invention involves systems for processing the output of an unknown system, shown as 10 in FIG. 1, and controlling an adjustable linear filter (equalizer), shown as 12 in FIG. 1, receiving the output in order to produce a desired response for recovering (deconvolving) the input to the system. The invention involves the development of iterative algorithms in which the unknown system whose (t,n) taps, for some combinations of t and n, are iteratively adjusted according to the following rule: ##EQU11## where s.sub.t,n denotes the (t,n) tap before the iteration, s'.sub.t,n denotes the (t,n) tap after the iteration, I is a preselected integer greater then or equal to one, .alpha..sub.i i=1,2 . . . I are preselected scalars that may vary from iteration to iteration, and p.sub.i, q.sub.i i=1,2, . . . I are preselected non-negative integers such that p.sub.i +q.sub.i .gtoreq.2.

According to preferred embodiments of the invention described below, the equalizer is adjusted such that the unknown system combined with the equalizer behaves essentially as a linear shift invariant system whose taps are iteratively adjusted according to the following rule: ##EQU12## where s.sub.n denotes the n.sup.th tap before the iteration and s'.sub.n denotes the n.sup.th tap after the iteration.

Preferably I=1, such that:

where .alpha. is a preselected scalar that may vary from iteration to iteration, and p and q are preselected non-negative integers such that p+q.gtoreq.2.

To simplify the exposition, only the procedure in (2) has been considered. To further simplify the exposition, only the case where the equalizer is a linear shift invariant system has been considered. However, the indicated generalizations may also be useful. Following are descriptions of several algorithm implementations of the methods in (2).

If the unknown system input can be modeled as a set of statistically independent random variables--which is typically the case in digital communications--and the equalizer is a linear shift invariant system, then the equalizer taps are iteratively adjusted such that:

where c' is the column tensor whose n.sup.th element, c'.sub.n, is the n.sup.th tap of the equalizer after the iteration, R is the tensor whose (n,m) element is:

where y.sub.t is the given data at the input to the equalizer, and d is the column tensor whose n.sup.th element is:

where z.sub.t is the output of the equalizer given by: ##EQU13## where c.sub.n is the n.sup.th tap of the equalizer before the iteration, and where the subscripts n and m belong to a pre-specified set.

We note that the multiplication by R.sup.-1 in (3) corresponds to spectral whitening of the observed data y.sub.t. A flow chart of the algorithm is illustrated in FIG. 3.

The elements of d can also be computed by: ##EQU14## Equation (5) suggests to compute joint cumulants of y.sub.t and z.sub.t at each iteration of the algorithm. Equation (7) suggests to first compute the cumulants of the given data set y.sub.t, then to perform the iterations with respect to the taps of the equalizer until convergence is accomplished, and only then to generate z.sub.t according to (6). A flow chart of this algorithm is illustrated in FIG. 4.

We note that the subscripts n, m, t, etc. in the above equations may be one-dimensional as in e.g. speech processing and time series analysis problems, or multi-dimensional as in e.g. image processing problems. If n is one-dimensional then R becomes a matrix and d becomes a column vector.

If the unknown system input can not be modeled as a set independent random variables, then it is convenient to work in the frequency domain. The resulting iterative algorithm consists of iteratively adjusting the transfer function (frequency response) of the equalizer such that: ##EQU15## where C'(w) is the transfer function of the equalizer after the iteration, C(w) is the transfer function of the equalizer before the iteration, S.sub.y.sup.1,0 (w) is the (1,0)-order spectrum (power spectrum) of y.sub.t, S.sub.y.sup.p,q (w.sub.1, w.sub.2, . . . ,w.sub.p+q) is the (p,q)-order spectrum of y.sub.t, and S.sub.a.sup.1,0 (w) and S.sub.a.sup.p,q (w.sub.1 w.sub.2, . . . ,w.sub.p+q) are preselected functions which are related to the (1,0)-order spectrum and the (p,q)-order spectrum of the unknown system input, and where w and w.sub.1, w.sub.2, . . . w.sub.p+q-1 belong to a prespecified set of frequencies. We note that w may be one-dimensional or multi-dimensional depending on the application. A flow chart of the algorithm is illustrated in FIG. 5.

In order to ensure the convergence of all the proposed algorithms, it is suggested to select .alpha. every few iterations such that

where z.sub.t is the output of the equalizer, and .eta..sub.t are preselected set of numbers, e.g. .eta..sub.t =cum(a.sub.t ;a.sub.t *) if it is known a-priori.

An important feature of the proposed methods is that they do not suffer from spurious/ambiguous solutions, and they converge to the desired solution regardless of initialization. Another important feature of the proposed methods is that they are capable of solving the blind deconvolution problem regardless of the phase characteristics of the unknown system, unlike the conventional linear prediction methods that can solve the problem only if the unknown system is minimum-phase. Moreover, in many cases of practical interest, the proposed methods are statistically more stable (i.e., have smaller variance and smaller ISI than linear prediction methods).

We may consider combining several of the proposed algorithms for different pairs of p,q in order to improve the robustness and statistical stability of the outcome results e.g. as suggested by (1). We may also consider combining linear prediction methods with the proposed methods in order to improve statistical stability and yet to obtain a method that is capable of identifying phase characteristics.

In many practically interesting situations the observed data y.sub.t is contaminated by additive noise which we denote by e.sub.t as illustrated in FIG. 6. If the noise cumulants/spectra are known a-priori or can be measured separately, it is recommended to eliminate its effect by subtraction, that is to compute the elements of R and d used in the iterative algorithm in (3) as follows: ##EQU16##

Similarly, to eliminate the effect of additive noise in the iterative algorithm suggested by (8), it is recommended to replace S.sub.y.sup.p,q (-w.sub.1,-w.sub.2, . . . -w.sub.p+q-1, w) and S.sub.y.sup.1,0 (w) in (8) by [S.sub.y.sup.p,q (-w.sub.1,-w.sub.2, . . . -w.sub.p+q-1,w)-S.sub.e.sup.p,q (-w.sub.1,-w.sub.2, . . . -w.sub.p+q-1,w)] and [S.sub.y.sup.1,0 (w)-S.sub.e.sup.1,0 (w)] respectively, where S.sub.e.sup.p,q (w.sub.1,w.sub.2, . . . w.sub.p+q) and S.sub.e.sup.1,0 (w) are, respectively, the (p,q)-order spectrum and the (1,0)-order spectrum of e.sub.t.

We note that if e.sub.t is Gaussian then its (p,q)-order cumulant/spectrum is identically zero whenever p+q.gtoreq.2, and therefore it affects only the computation of covariance matrix R or the (1,0)-order spectrum S.sub.y.sup.1,0 (w).

We also note that in the blind deconvolution problem, if we want to minimize the means square restoration error between the recovered data set at the output of the equalizer and the unobserved input of the unknown system, it is recommended to perform a final iteration of the algorithm (3) or (8) in which only the contribution of the noise to the high order cumulants/spectra is subtracted, but the contribution to the second order statistics is not subtracted. Thus, in the case where e.sub.t is Gaussian--which is a very common case--in order to minimize the mean square restoration error we should perform a final iteration of the algorithm without any subtraction of the noise contribution.

We note that in the implementation of the algorithm in (3) the cumulants are typically replaced by their sample estimates based on the given data at the input and output of the equalizer. Similarly, in the algorithm given by (8) the (1,0)-order spectrum and the (p,q)-order spectrum are replaced by their sample estimates.

The iterative algorithm given by (3) can be converted into a sequential algorithm by performing the iterations sequentially with the data such that the equalizer is adjusted sequentially.

Thus, assuming for simplicity that t is one dimensional, that the equalizer is a finite impulse response (FIR) filter whose vector taps is c=(c.sub.L.sbsb.1 c.sub.L.sbsb.1.sub.+1 . . . c.sub.L.sbsb.2).sup.T, and that y.sub.t is zero-mean, then in case p=1, q=1 we obtain the following sequential algorithm in which we perform one iteration per data sample:

where

is the vector equalizer taps after t iterations, ##EQU17## where Q.sub.0 is a preselected matrix; c.sup.(0) is a preselected vector; L.sub.1 and L.sub.2 are preselected numbers; and .beta..sub.t, .gamma..sub.t, .delta..sub.t are preselected sequences of numbers.

In case p=2, q=1 we obtain the following sequential algorithm:

where .rho..sub.t is preselected. A flow chart of the algorithm is illustrated in FIG. 7.

If we eliminate the multiplying matrix Q.sub.t in (15) (i.e., replacing Q.sub.t by the identity matrix) then under certain choice of .beta..sub.t and .rho..sub.t our algorithm reduces to Godard's algorithm for blind deconvolution as described in the paper: D. Godard, "Self recovering equalization and carrier tracking in two-dimensional data communication systems," IEEE Trans. Commun. Vol. COM-28, No. 11, pp. 1867-1875, Nov. 1980. The additional multiplication by Q.sub.t in our algorithm may significantly improve convergence rate, statistical stability, and sensitivity to additive noise.

The sequential algorithms in (12) and (15) may be particularly useful in time series analysis and modeling, in speech enhancement and compression and the like. Similar algorithms for other combinations of p and q can also be derived. We may also consider combining the two algorithms, and perhaps other algorithms in the class in order to improve statistical stability and robustness of the algorithm.

FIG. 8 is a block diagram illustrating a known all-digital quadrature amplitude modulation (QAM) demodulator (see e.g. the book: J. G. Proakis, "Digital Communications", McGrew-Hill, 2nd edition, 1989) constructed in accordance with the present invention. It includes an equalizer 80, in the form of a linear filter which is adjustable by a processor 81 in accordance with the method of the present invention. In the example illustrated in FIG. 8, and in its respective flow chart of FIG. 9, the integration algorithm is that of equation (3) with p=2, q=1. Thus, by controlling the equalizer 80 to recover the input signal sequence to the demodulator, the illustrated system enables operation of the demodulator without the need for training sequences, and even without the need to know a-priori the constellation of the transmitted data symbols.

The illustrated demodulator is a system which detects transmitted data symbols from a measured QAM analog communication signal. The demodulator consists of several sub-units whose function is described below.

The sampler unit 82 converts the QAM analog communication signal into a discrete sequence x.sub.t that is sampled at the BAUD rate. Note that in this specific application t is a one-dimensional variable corresponding to the index of the time sample.

The timing control unit 86 supplies sampling delays such that the demodulator is synchronized with the modulator.

The baseband conversion unit 83 generates a complex-valued sequence y.sub.t, whose real and imaginary parts are lowpass filtered versions of the sampled signal multiplied by a cosine wave and a sine wave, respectively, at the modulation frequency.

The equalizer unit 80 processes the output y.sub.t of the baseband conversion unit through an equalizer in order to recover the transmitted data symbols a.sub.t up to a delay and a possible phase shift.

The phase recovery unit 84 adjusts the phase of the recovered symbols at the output of the equalizer using a decision feedback loop.

The detector unit 85 processes the output of the phase recovery unit to produce the estimated (recovered) data symbols a.sub.t, obtained by minimizing the magnitude of the restoration error .epsilon..sub.t using a memoryless nearest neighbor (minimum distance) decision rule.

We shall now turn to the equalizer adjustment unit 81, which constitutes our novel contribution to the demodulator system. The proposed algorithm is a special case of (3) where p=2 and q=1. A flow chart of the algorithm is given in FIG. 9.

The relation between the input y.sub.t and the output z.sub.t of the equalizer is given by: ##EQU18## where the parameters of the equalizer L.sub.1, L.sub.2 and c.sub.L.sbsb.1, . . . c.sub.L.sbsb.2 are adjusted by the equalizer adjustment unit. Detailed description of the proposed algorithm is given below:

(A) Algorithm Initialization

Accumulate N samples y.sub.1, . . . y.sub.N at the input to the equalizer.

Set L.sub.1 =-M.sub.0, L.sub.2 =M.sub.0 where M.sub.0 is preselected positive integer (M.sub.0 is on the order of the length of the equalizer needed to recover the transmitted data symbols).

Set the L.times.1 vector of equalizer taps c=(c.sub.L.sbsb.1, . . . c.sub.L.sbsb.2).sup.T such that c.sub.0 =1 and c.sub.n =0 for all n.noteq.0, where L=L.sub.2 -L.sub.1 +1 is the number of equalizer taps.

(B) Cumulants estimation

Compute the L.times.L matrix R whose (n,m) element is: ##EQU19## Compute the L.times.1 vector d=(d.sub.L.sbsb.1, . . . d.sub.L.sbsb.2).sup.T whose n.sup.th element is ##EQU20## Comment: If the cumulants of the additive noise at the input to the equalizer are known a-priori can be measured separately, it is recommended to subtract them from R and d as suggested in (10) and (11) respectively.

(C) Equalizer Taps Adjustment

Compute the L.times.1 vector c'=(c'.sub.L.sbsb.1, . . . c'.sub.L.sbsb.2).sup.T : c'=R.sup.-1 d

Where we note that this computation can be carried out efficiently (order of L.sup.2 operations) using the Toepliz form of R (See the book: I. C. Gohberg and I. A. Feldman, "Convolution Equations and Projection Methods for their Solution", American Mathematical Society, 1974).

(D) Normalization

Compute the L.times.1 vector c"=(c".sub.L.sbsb.1, . . . c".sub.L.sbsb.2).sup.T : ##EQU21## where .eta.>0 is preselected, e.g. .eta.=E{.vertline.a.sub.t .vertline..sup.2 } is the average power of the input symbol if it is known a-priori.

(E) Equalizer's Domain Adjustment

Calculate ##EQU22## where .mu..gtoreq.0 is preselected, e.g. .mu.=0.01. Comment: D measures the distance between the equalizer taps before and after the iteration.

Set L'.sub.1 to be the maximal integer K such that ##EQU23## Set L'.sub.2 to be the minimal integer K such that ##EQU24## where 0<.gamma.<1 is preselected (e.g. .gamma.=0.1) and M.sub.1, M.sub.2 are preselected positive integers.

Set ##EQU25## Set

(F) Filtering

Compute ##EQU26##

(G) Stop?

If D>0 then

Goto step (B)

else

Stop equalizer adjustment (exit).

After the iterations of the proposed algorithm have stopped, we may switch to a decision directed equalization (DDE) provided that the constellation of the input data symbols is known a-priori or can be identified from the recovered symbols at the output of the equalizer. We then switch back from the DDE mode of operation to the proposed algorithm whenever is needed.

While the invention has been described with respect to several algorithms and one application, it will be appreciated that many other variations, modifications and applications of the invention may be made.