Combined Cross Polarization Interference Cancellation and Intersymbol Interference Equalization for Terrestrial Digital Radio Systems

This invention relates to terrestrial digital radio systems and, in particular, to cross polarization cancellation and intersymbol interference equalization in said systems.

Transmission of M-state quadrature amplitude modulated (QAM) signals via orthogonally polarized carriers results in the conveyance of increased information within a given bandwidth, resulting further in economic advantages. The main obstacle in the way of realizing these advantages is the unavoidable presence of cross polarization interference (CPI) between the dually polarized signals which arise due to factors such as multipath fading, antenna misalignments, and imperfect waveguide feeds.

The theory of linear and decision feedback equalization/cancellation, to mitigate the effects of intersymbol interference (ISI) and noise in the transmission of a single digital signal, is well established in an article entitled "Digital Communication over Fading Radio Channels," Bell System Technical Journal, Volume 62, Number 2, Part 1, pages 429 to 459, February 1983 by Messrs. G. J. Foschini and J. Salz. But a combination of data aided compensation of CPI cancellation and ISI equalization in the presence of noise in the aforesaid manner has not been solved heretofore.

In accordance with the illustrative embodiment of the present invention, the problem of cross polarization interference (CPI) cancellation and intersymbol interference (ISI) equalization is solved by using data aided decision feedback techniques. Most importantly, finite tap transversal structures which are implemented adaptively are used in a receiver. A window sample of the signal that is received is processed in two sections. The first section is equalized and the interference in the remaining section is canceled. Furthermore, the taps are not co-located. That is, the equalizer taps operate over a range where the canceler taps are not operative. Likewise, the canceler taps operate over the remaining range where the equalizer is not operative.

The receiver configuration is based on a matrix structure from the theory of optimal detection. The structure is optimal when the mean square error (MSE) of the received signal from the ideal signal is minimized. The optimal structure comprises a matrix matched filter in cascade with a transversal filter and an intersymbol interference and cross polarization interference canceler. The canceler uses detected data symbols to estimate the interference to be canceled. Data aided operations presume correct knowledge of detected data symbols.

A dually polarized channel is modeled by a particular four by four real matrix impulse response or its Fourier transform followed by additive noise. The two by two block diagonal elements of this matrix represent the co-polarized (in line) responses, while the off-diagonal two by two block entries represent cross coupled and cross polarized interfering responses. Each matrix channel characterizes a snapshot of a multipath fading event which in the present of noise limits the achievable error rate of the receiver for a given data rate.

For a reasonable co-polarized and cross polarized propagation model and a severe centered fade, 40 dB notch depth with a secondary ray delay of 6.3 ns, over an approximately 22 MHz channel bandwidth, the performance of transversal filters with a finite number of taps deployed in a decision feedback canceler structure is substantially (6 dB) better than linear equalization, and the difference can be up to 10 dB for offset fades. It can be shown that a 3 dB increase in MSE translates into about 1 bit/sec/Hz decrease in data rate efficiently at a fixed error rate or an order of magnitude increase in outage probability. Hence, linear equalization may not be adequate in deep fades.

Decision feedback/canceler structures achieve the ultimate matched filter bound with only nine matrix taps provided that error propagation is neglected.

Nine linear equalizer taps essentially achieve the performance of the infinite-tap linear equalizer. This method is, of course, free of error propagation.

For milder centered fades, 20 dB depth with a secondary ray delay of 6.3 ns, the linear equalizer configuration with nine taps is only 1 dB inferior to the decision feedback structure over a 22 MHz channel. If the channel bandwidth is increased to 40 MHz, however, the performance of the linear equalizer is worse than that of the decision feedback structure by 2.2 dB and the difference can be up to 3 dB for offset fades.

Decision feedback/canceler configurations are less sensitive to timing phase than linear structures.

FIG. 1 shows a block diagram of a dual polarized system;

FIGS. 2, 3 and 4 show details of a receiver embodying the present invention; and

FIGS. 5 through 11 show curves for evaluating the performance of the aforesaid receiver embodying the present invention.

Referring to FIG. 1, there is shown a prior art transmission system which utilizes a dually polarized digital radio communication channel supporting two independent quadrature amplitude modulated (QAM) data signals. The system shows an ideal QAM modulator 10, that is transmitter, and demodulator 50, that is receiver. Four independent synchronous data signals S.sub.1v (t),S.sub.1h (t), where 1=1, 2 and which are represented generically as ##EQU1## amplitude modulate two linearly polarized carrier waves in quadrature at multipliers 12, 14, 16, and 18.

The vertically modulated signal,

is transmitted over the vertically polarized channel 21, while

the horizontally modulated signal, is transmitted over the horizontally polarized channel 23. The carrier frequency is .omega..sub.0 and the real data symbols

are assumed to be independently drawn from a lattice of points with odd integer coordinates. The QAM constellations associated with equations (2) and (3) are, therefore, rectangular. The scalar pulse, g(t), is selected to satisfy limitations on transmitted power and bandwidth.

The individual transmission channels are characterized by bandpass impulse responses or by their Fourier transforms, ##EQU2##

To accommodate coupling between the polarized channels 21 and 23, two pairs of impulse responses, one pair 30, 32 associated with the co-channel and the other pair 34, 36 associated with the cross-channels, are used to characterize completely the medium.

Two independent noises,

and

are added to the cross polarized signals, respectively, at multipliers 62 and 64. These multipliers are located at receiver 50 of FIG. 1 where the signal plus noise is then coherently demodulated.

It is convenient to view the linear system as a four input port four output port network and characterize it as a four by four matrix impulse response or its Fourier transform which is the overall system frequency response.

The cross polarized signals on lead 63 having been mixed with the aforesaid noise signals are bifurcated into paths 51 and 53 and then delivered, respectively, to mixers 52 and 54 where the signals are mixed with the delayed carriers cos (.omega..sub.o t+.theta.) and sin (.omega..sub.o t+.theta.). Likewise, the cross polarized signals on lead 65 having been mixed with the aforesaid noise signals are bifurcated into paths 55 and 57 and then delivered, respectively, to mixers 56 and 58 where the signals are mixed with the carrier signals cos (.omega..sub.o t+.theta.) and sin (.omega..sub.o t+.theta.). These signals are then low pass filtered through elements 72, 74 and 76, 78 and then sampled by the sets of samplers 82 and 84; and, 86 and 88, respectively. Signals on output leads 81, 83 correspond with input leads 11, 13 and output leads 87, 89 correspond with input leads 17, 19.

Referring to FIG. 2, there is shown a block diagram useful in disclosing the present invention and comprising a transmitter and channel 10 and receiver 111. Receiver 111 comprises intersymbol interference equalizer 100, a linear matrix filter, the output from which is supplied to detector 140. Detector 140 compares the signal input thereto with an expected signal and produces an error from which a mean square error signal is generated and this error signal is transferred to cross polarization interference canceler 150 as well as to linear matrix filter 100. The characteristics of the aforesaid linear matrix filter 100 are selected so as to minimize the total mean square error. Canceler 150 produces adjusting signals proportional to the interferene caused by cross polarization and delivers these signals to subtracters 102 and 104. After the signals which adjust for cross polarization interference are subtracted from the filtered estimates, the resulting signals are substantially free from interference. Further detail of receiver 111 is shown in FIGS. 3 and 4.

Referring to FIG. 3, there is shown an input signal I.sub.1 on rail 71 and input signal Q.sub.1 on rail 73. These two signals correspond with the signal on leads 63 of FIG. 2 prior to bifurcation. In order to effect comparison with FIG. 1, identical numbers are used where convenient. Likewise, the input signals I.sub.2 on rail 75 and Q.sub.2 on rail 77 correspond with the signals on lead 65 of FIG. 2 prior to bifurcation.

Input signals are sampled by samplers 82, 84, 86 and 88 prior to delivery to the feedforward transversal filters 110, 112, 114 . . . 139 for a digital system. For an analog system, the samplers appear after the filters as shown in FIG. 1 hereinabove.

Input signal I.sub.1 is delivered to feedforward transveral filters 110, 114, 126 and to 128. Likewise, each of the other input signals is delivered to four feedforward transversal filters. The taps of each feedforward transversal filter are adjusted adaptively by error signals from a detector to be described hereinbelow. The output from feedforward transversal filter 110 is fed to mixer 105.

Because the other signals are coupled with this input signal I.sub.1 during transmission, the other input signals, Q.sub.1, I.sub.2, and, Q.sub.2, must be decoupled. Decoupling of these signals from input signal I.sub.1 is effected by feeding the output signals from feedforward transversal filters 112, 118 and 122 to mixer 105. Likewise, each of the signals is decoupled from the others. In the preferred embodiment sixteen transversal filters are used. This is apparent because there are four input signals. Details of design are not disclosed in this application because such design is trivial to one skilled in the art.

The decoupled signals must be adjusted further by the output signals from the feedback transversal filters to be disclosed with reference to FIG. 4 hereinbelow. This is effected in mixers 106, 108, 96 and 98. The output from mixer 106, for example, is then sent to detector 141. The input to detector 141 is compared with the output therefrom in comparator 142 and the output is sent as error signal ES.sub.1 to a plurality of filters for adaptive adjustment of the taps therein. Error signal ES.sub.1 is sent, for example, to feedforward transversal filters 110, 114, 126 and 128 and to feedback transversal filters 151, 152, 155 and 156. Likewise, the taps of the remaining filters are adjusted adaptively by error signals from comparators 144, 146, and 148. The output signals from the detectors are substantially free from CPI and ISI.

Referring to FIG. 4, there is shown canceler 150 of FIG. 2 comprising a plurality of feedback transversal filters 151, 152, 153 ... 166 each being adjusted by an error signal from detector 140 of FIG. 2. Each feedback transversal filter has input thereto an output signal from detector 140. Thus, filter 151, for example, has input thereto the signal I.sub.1 from detector 141 of FIG. 3. The output signal from feedback transversal filter 151 is delivered to mixer 167. Also delivered to mixer 167 are output signals from filters 153, 159 and 161 so that these signals may be decoupled. The output from mixer is then delivered to mixer 106 of FIG. 3. Likewise, the other feedback filters of FIG. 4 cause the remaining signals to be decoupled.

The theoretical basis for the aforesaid apparatus of FIGS. 2, 3 and 4 is developed hereinbelow. If matrix filter 100 has an impulse response of W(t) and its output is evaluated at t=0, a column vector for the overall system response will be ##EQU3## where H(t) is the transmitter and channel overall impulse response matrix.

A practical solution of the aforesaid equations comprises the construction of reasonable estimates of cross polarization interference and intersymbol interference, which are not necessarily optimum, and then subtracting these estimates from the signal incoming to the detector.

Assume that over a finite set of sampling instants, S, vector data symbols, A.sub.n, where n is an element of S, are available at the receiver. Before making a final decision on the current symbol, A.sub.o, a portion of the interference, ##EQU4## is subtracted from D.sub.o (0). This is feasible because prior to n=0, symbols have been decoded. It is presumed that use is made of previously decoded symbols to improve on the current estimate of A.sub.o. Because practical systems are not realizable relative to a large delay, there is a problem in using symbols which have not yet occurred. This problem can be overcome by introducing a delay, making tentative decisions, and then returning to modify the A.sub.o decision.

This answer depends on the system error rate prior to cancellation. For example, when the error rate is 10.sup.-4 and the cancellation window size is small relative to 10.sup.4, the probability that almost all of the symbols in this window have been correctly detected is fairly large. Thus, after cancellation, the error rate may be much improved. On the other hand, if the error rate prior to cancellation is high, no improvement after cancellation can be expected because the estimation of the interference is not reliable. Decision-directed cancellation as proposed here is successful over a certain range of error rates but fails when the error rate is high.

The performance criterion used is the least mean square error (MSE) normalized to the transmitted symbols variance, denoted .sigma..sub.d.sup.2. This is a mathematically tractable criterion to work with, and by minimizing MSE one also minimizes an exponentially tight upper bound on the error rate. Its use is also practically motivated because it lends itself to easy estimation, and it can be used to update transversal filter tap coefficients in practical adaptive systems.

The error vector is defined as the difference between D.sub.o (0), the output from equalizer 100, minus the output vector, from canceler 150, and the desired vector data symbol, A.sub.o, ##EQU5## where, n.noteq.0, and C.sub.n represents tap values in canceler 150. ##EQU6## where, "tr" stands for trace of a matrix, E{.} denotes the mathematical expectation with respect to all random variables, and .gradient. represents the complex conjugate transpose.

The computation of equation (10) is straight forward and yields ##EQU7## where,

M is the total number of QAM signal states,

and

The set of canceler matrices, C.sub.n, n S, can immediately be determined. If they are not identically set to U.sub.n, they can only increase the value of MSE. Consequently, if C.sub.n is set to be equal to U.sub.n, where n S, then the residual MSE results in a function of the matrix impulse response, W(t), and the size of the cancellation window.

The minimization of MSE with respect to the matrix W(t) is accomplished by the use of the calculus of variations. After substituting for U.sub.n, defined in equation (6), ##EQU8##

In order to determine the optimum W, replace the matrix W in equation (12) by

where, .eta..sub.ij is arbitrary, and then set

at .epsilon..sub.ij =0, i,j=1, 2, 3, 4.

It is easy to verify that ##EQU9## where, the matrices, .eta..sub.ij.sup.o, where (i, j)=1, 2, 3, 4 have the entry "1" in the (1.sub.ij).sup.th position and zero everywhere else. By computing the trace of equation (14), the following expression is obtained: ##EQU10## Because equation (15) must hold for all functons of .tau. and .eta..sub.ij.sup.o (.tau.), the matrix integral equation that must be satisfied by the optimum matrix W.sub.o (.tau.), is ##EQU11##

The structure of W.sub.o (.tau.) is interesting practically. It consists of a matched filter followed by a matrix tapped delay line where the matrix taps are zero for n S. In other words, the linear transversal filter or equalizer specified in equation (16) operates over a range of matrix tap coefficients where the canceler is not operative. This is to avoid interaction between co-located taps and possible unstability problems. The structure is shown schematically in FIG. 2, described hereinabove. In practice, this structure is approximated and implemented by a finite transversal filter whose taps are adaptively updated as shown in FIGS. 3 and 4 hereinabove.

After multiplying equation (16) by W.sup..gradient. (-.tau.), integrating, and then comparing the result with equation (12), an explicit formula for the optimum MSE is obtained:

where U.sub.o is obtained by solving a set of infinite linear equations obtained by multiplying equation (16) by H(.tau.-kT) and then integrating. Thus, ##EQU12##

In order to evaluate the merits of the present invention, a solution for U.sub.o must exist. The task for solving equation (18), however, is complicated. It is made difficult by the fact that the matrix equations are not specified over the finite set, S. While the number of unknowns is infinite, the values at the gap window are not specified. A way around this dilemma was found in the scalar case solved by M. S. Mueller et al and disclosed in an article entitled, "A Unified Theory of Data-Aided Equalization," B.S.T.J. Volume 60, Number 11, Pages 2023 to 2038, November 1981. The same techniques are applied here.

First, equation (18) is separated into two equations, one for k=0 and the other for k.noteq.0. Thus, ##EQU13## where the set J is defined as

and

where .delta..sub.ok is the Kronecker delta function. The solution of equation (21) is facilitated by introducing a set of matrix variables ##EQU14## and a set of unknown matrices ##EQU15## Using these matrices, equation (21) is written as: ##EQU16##

For these doubly infinite sets of matrix equations to coincide identically with equation (21), the following constraints must hold:

and

If these can be satisfied, the solution to equation (24) will be identical to the solution of equation (21) with Y.sub.n =U.sub.n, n J, and this is the sole purpose for introducing new variables. Evidently, equation (24) is easy to solve because it is in a form of a convolutional equation. To this end, define the inverse matrix sequences, ##EQU17##

Now, insert equation (26) into equation (24) to obtain explicity the desired solution: ##EQU18##

From equation (27), a finite set of equations in the unknown matrices .LAMBDA..sub.k can be obtained because V.sub.n =0 for n J, ##EQU19##

By substituting the definition of M.sub.k from equation (23) into the left hand side of equation (28) and making use of equation (26), the desired equations for the unknown constraint matrices .LAMBDA..sub.k, where k J, is obtained: ##EQU20##

Returning to equation (24), when k=0 ##EQU21## where the last equality derives from the fact that V.sub.n =0, n J, V.sub.n =U.sub.n, n J and R.sub.n =M.sub.n, n J. Finally, by substituting equation (20) into equation (30)

and solving for I-U.sub.o yields

Substituting this into equation (17) provides an explicit expression for MSE.sub.o in terms of .LAMBDA..sub.o only,

When the canceler window is doubly infinite in extent, one obtains the very best possible result. In other words, ISI and CPI have been eliminated substantially. In this special case, N.sub.1 =-.infin. and N.sub.2 =.infin., and equation (29) is now easy to solve ##EQU22##

By evaluating the Fourier series of both sides of equation (34)

where a generic Fourier series pair representation is ##EQU23## and ##EQU24##

Because M(.theta.)=.sigma..sup.2 I+R(.theta.) and M.sup.(-1) (.theta.) is in fact the inverse, M.sup.-1 (.theta.), it can be determined from equation (35) that .LAMBDA.(.theta.)=R(.theta.). Consequently, the zeroth coefficient of .LAMBDA.(.theta.) is ##EQU25## and when this is substituted into equation (33) the desired matched filter bound is obtained:

This will serve as a lower bound to attainable performance to which all other results are compared.

In this case, the canceler is absent and so N.sub.1 =N.sub.2 =0. Here, equation (29) reduces to

and solving for M.sub.o.sup.(-1) : ##EQU26## It can now readily be derived that ##EQU27##

In this application, it is assumed that all the casual terms, which depend only on past decisions, are canceled in addition to a finite number of noncasual terms. This implies that N.sub.2 =.infin. and N.sub.1 is finite. When N.sub.1 =0, the canceler becomes a decision feedback equalizer because causal interference can be canceled by a feedback circuit. Here, it is necessary to determine MSE for the more general case when N.sub.1 is not necessarily zero.

It is more convenient to solve for U.sub.o directly from equation (21) than through equation (29). Equation (21) is rewritten as: ##EQU28## which is recognized to be a matrix Weiner-Hopf equation, and its solution depends on being able to factor positive definite Hermitian matrices.

Proceeding with the solution of equation (40), the following sequence of matrices is introduced

such that ##EQU29##

Substituting equation (41) into equation (40) and rearranging yields two sets of equations ##EQU30## and ##EQU31##

The procedure for solving these is to solve first for Y(.theta.) from equation (42) in terms of M.sup.- (.theta.), an easy task in terms of the Fourier transforms of {M.sub.n.sup.- } and {Y.sub.n }. Having obtained Y(.theta.), one proceeds to solve equation (43) for U(.theta.) in terms of M.sup..gradient. (.theta.). Note that equation (41) implies

and because M(.theta.) is Hermitian, M(.theta.)=M.sup..gradient. (.theta.), implying (M.sup.+ (.theta.)).sup..gradient. =M.sup.- (.theta.), (M.sup.- (.theta.)).sup..gradient. =M.sup.+ (.theta.), and the factorization problem is reduced to finding a matrix M.sup.+ (.theta.) such that

where the entries in M.sup.+ (.theta.), (M.sup.+ (.theta.)).sub.ij are such that (M.sup.+ (.theta.)).sub.ij has a Fourier series with only positive frequency coefficients.

Multiply both sides of equation (43) by M.sup.+.sub.-m and sum m from -.infin. to -N.sub.1 as a first step to determine the sequence of y.sub.m. This gives ##EQU32## Recalling that M.sub.k =R.sub.k +.sigma..sup.2 .delta..sub.ko I, equation (42) can be placed in the form ##EQU33## When this is compared with equation (41),

and when substituted into equation (44), ##EQU34## From equation (30), ##EQU35## and so it can be concluded that ##EQU36## Substituting again for R.sub.o =M.sub.o -I.sigma..sup.2 in equation (49) and rearranging, ##EQU37## because ##EQU38## hence, ##EQU39## Substituting equation (50) into equation (33) gives the result ##EQU40## When N.sub.1 =0, that is, when there is not anticausal cancellation,

where M.sub.o =M.sub.o.sup.- =(M.sub.o.sup.+).sup..gradient.. This is the formula for decision feedback equalization derived by Foschini et al for QAM transmission over a single channel which they cast in a matrix formulation in a paper entitled, "Theory of Minimum Mean-Square-Error QAM Systems Employing Decision Feedback Equalization," B. S. T. J. Volume 52, Number 10, pages 1821 to 1849, December 1973. The form of the answer generalizes to arbitrary dimensions.

The theoretical results derived so far apply to an ideal canceler of any window size and an infinite tap linear equalizer whose matrix taps vanish inside the cancellation window. In order to assess the penalties incurred by a finite tap linear equalizer, outside the cancellation window, the least MSE formulae applicable to this case are derived. With these formulae the merits of equalization and cancellation using only a finite number of matrix taps can be evaluated. An insight into the best way of deploying the total number of available taps, furthermore, can be obtained. Also inherent in the theory derived so far is the independence of MSE on sampling phase. This is so because the transversal equalizer and canceler is preceded by a matched filter whose structure presumes knowledge of sampling phase. Here, this condition is relaxed and the MSE is derived for a front end filter matched to the transmitter filter only rather than to the overall channel response and, thereby, bring out the dependence of MSE on timing phase.

The finite tap delay line matrix filter is represented by ##EQU41## where the two sets F and S are disjoint and F now is a finite set,

In equation (53), g(t), as before, is a scaler pulse shape, while {Q.sub.n }.sub.n F is a 4.times.4 matrix sequence. The objective now is to select the Q.sub.n 's which minimize the total MSE, equation (12), ##EQU42## Substituting equation (53) into equation (54) yields ##EQU43## where the H.sub.n 's are defined in equation (12) and ##EQU44##

Setting the derivatives of equation (55) with respect to the elements of the matrices {Q.sub.n }.sub.n F to zero, a set of linear matrix equations is obtained for the unknowns, {Q.sub.n }.sub.n F, namely, ##EQU45## where

The solution of equation (57) is straightforward and will be discussed hereinbelow.

For now, label the solution to equation (57) by Q.sub.1.sup.o, the optimal Q.sub.1 's. Multiply the result by Q.sub.n.sup.1, then sum over n F and substitute the result into equation (57). This yields the desired formula for the least means square error (MSE) ##EQU46##

Referring to FIG. 2, when linear equalizer 100 has a finite tap window size, the structure of optimum receiver 111 comprises a receive filter (not shown) matching the transmit filter (not shown) followed by a matrix transversal filter 100 and a matrix canceler 150. The most general case under this assumption is when the matrix canceler has a finite member of causal and anticausal taps and the solution of equation (29) for .LAMBDA..sub.o provides a means of calculating minimum means square error by use of equation (33). In order to solve for .LAMBDA..sub.o, block matrices M.sub.k.sup.(-1) 's defined in equation (26) have to be determined first. One way to determine the M.sub.k.sup.(-1) 's is to solve equation (33) by a Levinson-type algorithm where the entries are block matrices. Thus, matrix convolution equation (26) is represented as: ##EQU47## where I is the identity matrix. As observed, the block Toeplitz matrix equation can be solved for M.sub.k.sup.(-1) 's, with the M.sub.K 's given in equation (23). Having the M.sub.k.sup.(-1) 's and expressing equation (29) in the form: ##EQU48## it is possible to evaluate .LAMBDA..sub.o.

When matrix canceler 150 has knowledge of an infinite number of past data symbols, it becomes a decision feedback equalizer. In addition, it may also employ a finite member of anticausal taps to operate on future symbols in which case it becomes a finite window canceler. This can be accomplished with a finite delay. As shown in equation (40) to determine MSE.sub.o, a matrix Weiner-Hopf equation has to be solved. This involves determination of anticausal factors of the M(.theta.) matrix as explained hereinabove.

There are at least two computational algorithms available for solving a matrix Weiner-Hopf equation. One method converts the matrix that has to be factored directly into a nonlinear difference equation of a Ricatti type, which converges to a stable solution. Another method, which is adopted in this invention comprises a Bauer-type factorization of positive definite polynomial matrices. This algorithm is suited to sampled data applications and takes advantage of the periodic and positive nature of the channel covariance matrix, M(.theta.), as in this embodiment. It performs the factorization in the following steps. Suppose one desires to factor the n.times.n matrix M(.theta.):

This matrix has a Fourier series expansion, ##EQU49## whose n.times.n coefficients, ##EQU50## approach O.sub.m as .vertline.m.vertline. becomes infinitely large. One now follows these steps:

Step 1:

Form the following variable size Toepliltz matrices ##EQU51## of respective sizes

that are Hermitian and real.

Step 2:

For every m.gtoreq.0, perform the Cholesky's factorization

where L.sub.m.sup.T is the transpose of L.sub.m, and L.sub.m is a square, real and lower triangular matrix with positive diagonal scalar entries, ##EQU52## All blocks in equation (66) are real n.times.n and all L.sub.rr.sup.(m), r=0, . . . , m are lower triangular.

Step 3:

It has been proved by D. C. Youla et al that for every fixed r and k, r.gtoreq.k.gtoreq.0,

m.fwdarw..infin.

The factorization of equation (65) and the evaluation of M.sub.r-k.sup.+ in equation (67) is performed numerically. To find an upper bound on m for stopping the calculations, a convergence point must be established. This can be done by checking the trace of M.sub.r-k.sup.+ in each iteration in order to determine whether it has reached a level of constancy, and, if so, for what value of m. This completes the factorization. Indeed, Youla et al showed that a constant trace as a function of m corresponds to a minimum of a quadratic functional: ##EQU53## where P(.theta.) is a polynomial matrix of the form ##EQU54## with the quadratic functional of equation (68) expressed as

where T.sub.m was defined in equation (64) and x.sub.r 's represent the elements of X. Hence, there is a theoretical base for establishing the convergence point.

Finally, consider the case where the matrix linear equalizer operates on a finite set of taps that do not overlap with those of the finite tap matrix canceler. This is a case of great practical interest. Here, the receiver filter is assumed to have a square root Nyquist transfer function matching the transmit filter. Becuase it no longer matches the overall channel and transmitter characteristics, MSE.sub.o is a function of timing phase. Therefore, an optimum timing reference has to be established before the optimum nonstationary covariance matrix can be determined. This is accomplished here by minimizing the mean square eye closure (MS-EC) which is a measure of the amount of received level perturbation caused by CPI and ISI. In this invention, it is assumed that the demodulator removes the channel phase at the optimum sampling time reference. Once an optimal set of samples is found, the covariance matrix, G.sub.nm, of equation (56) is formed as: ##EQU55## In terms of the H.sub.n 's defined in ##EQU56## the covariance matrix can be expressed as: ##EQU57## Hence, by adding .sigma..sup.2 to the diagonal elements of G.sub.nm, the matrix R.sub.nm is formed, as expressed in equation (58). The Q.sub.n 's, that is the coefficients of the finite window equalizer, can be computed as follows:

These coefficients are used in equation (59) to determine the optimum MSE.

The cross polarization fading propagation model employed has the frequency characteristics represented by the complex matrix: ##EQU58## where the functional form of C.sub.11 (.omega.) and C.sub.22 (.omega.) is that of a single in line fading channel model, with the generic representation,

where a and .rho. are real variables representing flat and dispersive fading levels, .phi. is related to the fade notch offset, and .tau. is the delay between direct and reflected paths assumed to be 6.3 nano seconds. Also in the model,

which is in the same form as C.sub.11 (.omega.), except for an additional variable .DELTA..omega. that allows non co-located fade notches to occur on the two polarization signal transfer characteristics. Cross polarization paths are assumed to behave as

and

where K.sub.1, K.sub.2, K.sub.4, and K.sub.5 are constants which incorporate the nonideal properties of antennas and waveguide feeds at both ends of the channel, typically taking on values varying from one hop to another in the -35 to -20 dB range. The last term in equations (77) and (78) represents a nondispersive cross polarization response contributed by an independent ray. In the present invention, R.sub.3, R.sub.6, and .DELTA..omega. are assumed to be zero and the K.sub.i 's are assumed to be -20 dB

Computation of the channel covariance matrix is the initial necessary step behind all th MSE.sub.o calculations.

In the case of the infinite window size equalizer discussed hereinabove, the receiver filter is assumed to be a matched filter; hence, no reference timing establishment is necessary. The peak of the correlation function serves as a timing reference. By computing the sampled correlation matrix of equation (19) the normalized MSE.sub.o can be calculated.

In application of the finite window equalizer which is shown in detail in FIG. 3, a set of optimum samples of overall impulse response is found by establishing a timing reference, t.sub.o, for which the mean square eye closure of the received in line signal is a minimum, and at this reference the channel phase is removed. This has to be done for the two polarized signals independently.

The overall transfer function matrix is given by

where C(.omega.) is the propagation transfer matrix and P(.omega.) is the Nyquist shaping filter transfer matrix. For instance, if the impulse response of the vertical in line signal is

where p(t) is a Nyquist shaped pulse, the channel phase becomes ##EQU59## and the upper row block matrices of the overall impulse response matrix have to be multiplied by e.sup.-j.theta.(t.sbsp.o.sup.). I, I being the unity matrix, in order to remove the channel phase at t.sub.o.

In order to provide a single set of curves for MSE.sub.o, independent of the number of transmit states in M-QAM signal space, the MSE.sub.o is normalized as defined in equations (33), (36), (39), (51), (52), and (59) by dividing the formulas by .theta..sub.d.sup.2, that is, the transmitted symbols variance. In addition, the normalized MSE.sub.o is computed for only one of the M-QAM signals that comprise the dually polarized signal, S.sub.v (t).

If the unfaded signal to noise ratio, SNR, is defined by .GAMMA. it can be verified that in the case of a matched filter receiver, the normalized MSE.sub.o in the absence of any cross polarization interference, that is, K.sub.1, K.sub.2, K.sub.4, K.sub.5, R.sub.3, R.sub.6 =0, is simply ##EQU60## and, consequently, for a large unfaded SNR, it becomes .GAMMA..sup.-1. Hence, equation (82) establishes an ultimate performance bound which can only be achieved in utopian environment. In a dually polarized system with a finite amount of non dispersive coupling that is, K.sub.1, K.sub.2, K.sub.4, K.sub.5 >0, the matched filter bound is degraded somewhat. for K.sub.1 =K.sub.2 =K.sub.4 =k.sub.5 =-20 dB, there was found a small amount of degradation in the ideal MSE.sub.o which is not a function of the dispersive fade depth and only diminishes when there is no cross coupling, that is, in a completely orthogonal system.

It is assumed that the transmit filter is square root Nyquist shaped and the receive filter either matches the overall transmitter and channel or the transmitter only. A Nyquist roll off of 45 percent, both a 40 MHz channel and a 22 MHz channel bandwidth, and a SNR of 63 dB are used in our numerical evaluations.

Referring to FIG. 5, there is depicted the normalized MSE.sub.o as a function of the number of canceler taps, Q, when a 40 dB centered fade over a 22 MHz channel band is applied to both polarized signals. The linear equalizer in this case possesses an infinite number of taps. The case of pure linear equalization, N.sub.1 =N.sub.2 =0, that is, no cancellation, exhibits the largest MSE.sub.o degradation relative to the asymptotic matched filter bound. This is due to the noise enhancement experienced by the linear equalizer during deep fades. When both causal and anticausal canceler taps are present, all the curves rapidly approach the matched filter bound for a finite constant coupling, K.sub.i =-20 dB, i=1,2,4,5. The curve for a decision feedback type canceler starts at ideal decision feedback equalizer normalized MSE.sub.o and approaches the asymptotic value with two anticausal taps. The finite window size canceler curve starts at the linear equalizer case, N.sub.1 =N.sub.2 =0, and reaches the matched filter bound asymptotic value with a total of four causal/anticausal taps. Finally, when no anticausal taps are employed, the curve asymptotically approaches the ideal decision feedback case with only two causal taps.

Referring to FIG. 6, there is depicted similar results as in FIG. 5 for the case when the centered fade notch depth is reduced to 20 dB over a 22 MHz channel. As can be observed the linear equalizer, N.sub.1 =N.sub.2 =0, performance is improved. In both FIGS. 5 and 6, the fade notch is located at the band center. In both cases, because the receiver filter matches the overall channel and transmitter, an offset fade notch does not have a serious impact on the results for the same fade notch depth.

Referring to FIGS. 7 and 8, there is depicted in each, the achievable MSE.sub.o when the linear equalizer has a finite number of taps. The fade notch in FIG. 7 is centered but in FIG. 8 is offset from the band center. For ease of presenting the results, fad notch offset from the band center is expressed in terms of the ratio of the fade notch distance from the band center to the channel equivalent baseband bandwidth in percentage. In FIG. 8, the fade notch is offset by 69 percent over a 22 MHz channel, that is, an offset of 7.6 MHz from the band center. As observed from FIG. 7, a total of nine taps, including the center tap, are required to achieve the asymptotic matches filter bound when decision feedback taps are present. The same asymptotic performance can be achieved no matter how the nine synchronously spaced taps are deployed between the linear equalizer and the canceler as long as the canceler operates in a decision feedback mode. This is because the equalizer and canceler tap windows complement one another; therefore, since the taps do not overlap, for the same number of taps, the performance remains almost the same in the decision feedback cases. An important configuration is seen when the linear equalizer operates only on the main lobe of CPI by means of its center matrix taps, M.sub.1 =M.sub.2 =0. It is clear that as long as the canceler window is sufficiently wide, a main lobe CPI canceler can achieve the asymptotic matched filter bound. The curves indicate that deep fades degrade the linear equalizer, N.sub.1 =N.sub.2 =0, performance significantly.

Previous studies showed that every 3 dB degradation in MSE.sub.o translates into a loss of 1 bit/sec/Hz of data rate efficiency. Hence, linear equalization may not provide adequate rate efficiency in deep fades.

In FIG. 8, similar curves are depicted, as in FIG. 7, but for a 40 dB fade with the notch frequency offset by 69 percent. Improved performance turns out to be due to the particular notch position as will be brought out in the discussion of FIG. 10.

FIG. 9 illustrates a similar set of curves for a 20 dB centered fading of dually polarized signals over both a 22 MHz and a 40 MHz channel. As can be observed, the linear equalizer, N.sub.1 =N.sub.2 =0, performance improves because of the decreased fade depth. Over the wider channel band, however, the degradation over decision feedback is more, as expected. This is due to the wider channel band over which the same fade notch causes more dispersion. The degradation amounts to 2.2 dB loss of MSE.sub.o comparing to matched filter bound, that is, roughly 1 bit/sec/Hz loss of data rate efficiency, and the loss can even be more for offset fades as will be seen in FIG. 10. Hence, even with more typical fades the use of the liner equalizer can be troublesome over a 40 MHz channel.

Finally, to compare some of the techniques described earlier in terms of their sensitivity to fade notch offset, the normalized MSE.sub.o is plotted in FIG. 11 as a function of fade notch position, which, as explained hereinbefore, is expressed here in terms of the ratio of the fade notch distance from the band center to the channel equivalent baseband bandwidth. Consider the following structures:

a. A linear equalizer with

M.sub.1 =M.sub.2 =4

N.sub.1 =N.sub.2 0 (no cancellation)

b. Center tap only linear equalizer/finite window canceler with

M.sub.1 =M.sub.2 =0

N.sub.1 =N.sub.2 =4.

The sensitivity of the linear equalizer to offset fades is quite pronounced. The center tap equalizer with a finite window canceler exhibits a very small sensitivity to offset fades. The degradation of MSE.sub.o for some offset fades can be explained considering the fact that these fades causes cross polarization of the imaginary part of a complex QAM signal into its real part, and for a particular notch offset frequency within the band, the coupling reaches its maximum. Therefore, the MSE.sub.o versus fade notch offset curves exhibit this phenomenon. In dually polarized systems, as in the case of the problem at hand, this is even more pronounced than in single signal transmission, because in the 4.times.4 system under offset fading there is coupling of three interfering data streams into the fourth one. A decision feedback type canceler structure, by canceling the major contributions to CPI and ISI and with a lesser noise enchancement, exhibits an improved performance compared to the linear equalizer. Note that the curves in FIG. 11 have all been obtained under optimum timing conditions.

Referring to FIG. 11, there is shown a normalized MSE.sub.o superimposed by the normalized MS-EC of the received signal curve before equalization or cancellation as a function of sample timing offset from the optimum timing used to investigate the sensitivity of the two structures to timing phase. This is done for a severe fade, namely, a 40 dB fade with a notch frequency offset by 34.5% over a 22 MHz channel. It is clear that the finite linear equalizer is much more sensitive to timing phase than the decision feedback type. The optimum timing reference is established based on minimizing the MS-EC of the received signal in presence of fading, before CPI and ISI cancellation. Hence, after cancellation occurs, this timing reference may not be the one that minimizes the canceler output MSE.sub.o, and indeed the linear equalizer curve of FIG. 11 indicates this fact. The MS-EC curve has a minimum at the optimum timing reference. The sensitivity of the matrix linear equalizer to timing phase can be reduced by applying half-a-baud spaced taps, that is, by deploying fractionally spaced taps. It is to be noted that a decision feedback timing method may prove more robust. But such a method is more complex. The degradation seen in FIG. 11 of the MSE.sub.o is partly attributed to the asymmetric amplitude and delay responses of the fading channel which in the presence of a nonzero roll-off of the shaping filters cause destructive addition of aliases. So, reducing the roll-off of the shaping filters can also improve this situation.