US 4,727,503 AGrant
Systolic Array
Issue Date:1988-02-23
•14 Claims
•4 Drawing Sheets
Abstract
A systolic array of cells for processing a data stream includes an arrangement of nearest-neighbor connected boundary cells, internal cells and a multiplier, arranged as a triangular array and a column. The boundary cells are diagonally interconnected. Each boundary cell evaluates sine and cosine rotation parameters from data received from above for lateral transfer to a neighboring internal cell, and multiplies a diagonal input by the cosine parameter for diagonal output. Each internal cell receives rotation parameters from the left, applies them to data from above to produce an output below, and passes them on laterally. Data input to the column becomes cumulatively rotated before output from the final downstream internal cell. The final downstream boundary cell provides cumulatively multiplied cosine parameters. The multiplier provides the product of the outputs of these final cells. The product is the least squares residual arising from weighted minimization of input signals.
Metadata
Assignee
- The Secretary of State for Defence in Her Britannic Majesty's Government of United Kingdom
Inventor
- John G. McWhirter
Application Information
Application Number:US 6276261
Filing Date:1984-07-03
Priority Date:1983-07-06
Art Unit:231
Classifications
IPC:
G06F 738
Field of Search:
364715;754;736;200 MS File;900 MS File;757-760
Patent Drawings (4 sheets)
Description
Background of the Invention
This invention relates to a systolic array, and more particularly to a systolic array for solving least squares problems.
Systolic arrays are known, the concept being set out by Kung and Leiserson in "Systolic Arrays (for VLSI)" in the text of "Introduction to VLSI Systems" by Mead and Conway", Addison-Wesley (1980). Such an array comprises individual electronic signal processing cells which are interconnected. The operation of the array as a whole depends on the function of individual cells and the interconnection scheme, the only external control required being a clock. The term "systolic" arises from the clock "pumping" the operation of the array. The basic advantage of systolic arrays is that complex operations may be performed by arrays of comparatively simple processing cells having defined functions and appropriate interconnections, preferably nearest-neighbour interconnections only. This approach is highly applicable to the construction of very large scale integrated (VLSI) circuits.
Systolic arrays are particularly suitable for performing pipelined operations. A sequence of operations is said to be pipelined if an element of a data stream can enter the sequence before the preceding element has left it. Pipelining is highly beneficial in VLSI, since it affords the possibility of reducing the number of idle devices awaiting data.
The nomenclature employed in the art of systolic array technology for matrix computations express mathematical relationships rather then physical ones. Arrays implemented as electronic circuits are geometrically arranged on the basis of engineering convenience, since the important factors are processing cell functions and cell interconnections, not the physical positions of electronic components. Accordingly, for the purposes of this specification, geometrical and positional expressions such as triangular, column, nearest neighbour, diagonal, hypotenuse, boundary, internal etc describing array features shall be construed as terms of art expressing mathematical relationships and extending to or including corresponding features of topologically equivalent arrays.
In "Matrix Triangularization by Systolic Arrays", Proc. SPIE., Vol 28, Real-Time Signal Processing IV (1981), Kung and Gentleman showed that systolic arrays might be employed to solve linear least squares problems which arise in a wide range of signal and data processing applications. The particular problem is to determine a p-vector of statistical weights w(N) for which .vertline..vertline.Xw(N)-y.vertline..vertline. is minimized, where y is a given N-vector of data elements and X is a given Nxp design matrix with p.ltoreq.N, the usual Euclidean norm being assumed.
Kung and Gentleman solve this problem by a two stage process employing two coupled systolic arrays. The first systolic array is triangular, and is used to implement a pipelined sequence of Givens rotations. The mathematics of Givens rotations is described by Gentleman, J. Inst. Maths. Applics (1973), 12, pp 329-336. The approach is to carry out a QR decomposition of the matrix X; ie the sequence of Givens rotations operates on the elements of X to build up a unitary matrix Q such that: ##EQU1## where R is a pxp upper triangular matrix (a matrix in which all subdiagonal elements are zero). Each element of R is computed by and stored in a corresponding processing cell of the systolic array as elements x of the matrix X are clocked into it. The approach is to (Givens) rotate each successive row of X with each row of R in turn. The major diagonal of the triangular systolic array is occupied by boundary cells having processing functions appropriate to evaluate sine and cosine Givens rotation parameters. All other (ie above-diagonal) cells are referred to as internal cells, and have processing functions appropriate to apply the rotation parameters to incoming data comprising elements of X. The array may be schematically illustrated as a right isosceles triangle with one shorter side horizontally uppermost and the other vertical. Cell interconnections are between nearest horizontal and vertical neighbours only.
Information or rows of X enters the triangular array via its uppermost row in a temporally skewed order as required to synchronize array operation. This will be described in more detail later. Each boundary or internal cell stores a respective current value r or element of the upper triangular matrix R. Each boundary cell receives input data from above, updates the respective stored value of r, evaluates the rotation parameters and transfers them to the respective lateral nearest neighbour internal cell. Each internal cell receives rotation parameters from one side and input data from above. It applies the rotation parameters to the input, passes on the parameters laterally, provides an output below and updates its stored value of r. When all the elements x of the nxp matrix X have flowed through the triangular systolic array in a pipelined manner, the values of r stored in the cells give the elements of the upper triangular matrix R. An exact QR decomposition or triangularisation of the matrix X has been performed. It should be emphasised that the stored cell values only represent the R matrix when all data has flowed completely through the array. During processing, the stored cell values correspond to data input at different times, in view of the temporal skew applied to input data and the fact that horizontally or vertically successive cells are at any time processing progressively earlier data.
The n-vector of data elements y is fed into a further column of internal cells alongside the triangular array and connected to it in a nearest neighbour fashion. The rotation parameters from the array are passed to this further column for application to y after operation on X. In effect, the vector y is processed as an extra column of the matrix X.
The evaluation of Givens rotation parameters by the boundary cells normally requires calculation of square roots. However, Kung and Gentleman also describe an array for square root free parameter evaluation based on the earlier work of Gentleman, J. Inst. Maths Applics, Vol 2, pp 329-336, 1973. In effect, the Givens rotation is mapped into a different mathematical domain for the purposes of avoiding square root calculation. Different boundary and internal processing cell functions are required, and the boundary cells are connected together along the array diagonal. The values stored by the cells are not equal to the elements of the matrix R, but have a simple relationship thereto. The square root free approach is accordingly mathematically equivalent to the previous technique. It is also possible to employ other forms of processing cells having different but equivalent functions.
The second stage of the Kung and Gentleman procedure to obtain the weight vector w(N) comprises extracting the values stored by each cell of the triangular array and feeding them into a linear systolic array. The linear array performs a back-substitution process which solves the triangular linear system associated with Equation (1) and given by:
where Q.sub.1 is a matrix comprising the first p rows of the matrix Q previously defined. Accordingly, Q.sub.1 y denotes the first p elements of the vector obtained by applying the same series of Givens rotations to the vector y as were employed to generate R from X.
The linear systolic array generates the required weight vector w(N) directly, providing an exact least squares solution. The vector w(N) is then available inter alia for calculating the least squares residual e.sub.N defined by:
where y.sub.N is the Nth element of y, and x.sub.N.sup.T is the Nth or final row of the matrix X. However, the back-substitution process of Kung and Gentleman has a number of disadvantages. The triangular linear system may be ill-conditioned; eg if the Nxp martix X does not have full rank (either N<p or N includes less than p independent rows), the back-substitution process involves division by zero which is undefined. The back-substitution process may also be numerically unstable, ie involve division by small inaccurate quantities. This could be improved by interchanging columns of X, but such a procedure would be inconsistent with the design of a hard-wired systolic array representing a matrix having fixed rows and columns. Furthermore, Kung and Gentleman require both a triangular and a linear systolic array to solve the Equation (2) triangular linear system, and need to compute the vector product x.sub.N.sup.T w(N) in order to obtain the least squares residual e.sub.N.
Summary of the Invention
It is an object of the present invention to provide a modified form of systolic array for solving least squares problems.
The present invention provides a systolic array for processing a data stream flowing through it, the array including nearest neighbour connected processing cells arranged as a triangular array of internal and boundary cells together with a column of internal cells, the boundary and internal cells having processing functions appropriate for evaluating and applying rotation parameters respectively, and processing means arranged to provide recursively the product of each cumulatively rotated data element with cumulatively multiplied cosine rotation parameters. It has been found, surprisingly, that the product of each cumulatively rotated data element with cumulatively multiplied cosine parameters is equal to the recursive least squares residual. The array of the invention therefore has the advantage that least squares residuals are derived recursively without the need to employ a linear systolic array to produce statistical weight vectors by back substitution. This avoids the problems of numerical instability and ill-conditioning and reduces the amount of electronic circuitry required. Moreover, the derivation of recursive residuals is advantageous over the once and for all solution provided by the prior art array.
In a preferred embodiment, the cumulative product of cosine parameters is derived by diagonally connecting the boundary cells, each of which has the additional function of multiplying its diagonal input by the respective evaluated cosine parameter (or its equivalent for non-Givens rotation algorithms) to provide a diagonal output. The output of the final downstream boundary cell is then either equal to the cumulative product of cosine rotation parameters or is related to it according to the rotation algorithm employed. Moreover, the output of the final downstream internal cell of the column is a function of each cumulatively rotated data element. The processing means computes the recursive least squares residual from these two outputs.
In the cases of processing cell functions appropriate for Givens rotation by the square root or square root free algorithm hereinbefore outlined, the processing means comprises a multiplier arranged to multiply together the respective diagonal and vertical outputs of the final downstream boundary and internal cells. The diagonally connected boundary cells have functions to generate cumulative multiplication of Givens rotation cosine parameters or their square root free equivalent. The vertical output of the final downstream internal cell provides data elements to which all evaluated rotation parameters have been applied, and the output product produced by the multiplier provides the required least squares residuals.
An exponential memory may be incorporated in the array of the invention to allow operation in a continuously adaptive mode.
Data for processing by the array may be made subject to linear constraints. For this purpose, the array may be associated with means for subtracting a linear constraint factor from data prior to array entry.
The array of the invention may be employed for linear predictive filtering of images comprising a two dimensional array of data elements or pixels. Each pixel is predicted from the product of associated pixels and a vector of weights which minimizes the prediction error over an ensemble of pixels. The difference between the prediction and the corresponding actual received pixel value may be registered if significant and discarded if not. This provides a means for reducing an image to its significant features only, with consequent reduction in data. The difference corresponds to the least squares residual produced by the invention.
The array of the invention may alternatively be employed for processing signals from a phased array radar having primary and auxiliary antennas and operating as an adaptive digital beamformer. The invention is employed to provide residuals corresponding to differences between the primary antenna signal and a weighted linear combination of the auxiliary antenna signals. This makes it possible to substract noise or jamming signals from the primary antenna signal.
Brief Description of the Drawings
In order that the invention might be more fully understood, one embodiment thereof will now be described, by way of example only, with reference to the accompanying drawings, in which:
FIG. 1 is a schematic drawing of a prior art generalized systolic array,
FIGS. 2 and 3 respectively provide cell function definitions for carrying out square root and square root free Givens rotations with the array of FIG. 1,
FIG. 4 is a schematic drawing of a modification of the FIG. 1 array in accordance with the invention,
FIG. 5 is a schematic drawing of a two dimensional image for processing by the invention.
Detailed Description of the Presently Preferred Exemplary Embodiment
Referring to FIG. 1, a prior art systolic array of processing cells of the kind described by Kung and Gentleman (ibid) is indicated generally by 10. The array 10 comprises four boundary cells 11 indicated by circles 11.sub.11 to 11.sub.44 and ten internal cells 12 indicated by squares 12.sub.12 to 12.sub.45, the first and second suffixes representing row and column positions respectively. The cells 11 and 12 are arranged in the form of a triangular array 13 of boundary and internal cells 11 and 12.sub.12 to 12.sub.34 with an additional column 14 of internal cells 12.sub.15 to 12.sub.45.
Each boundary cell 11 receives input data from vertically above, and evaluates rotation parameters for horizontal output as input to the respective downstream nearest-neighbour internal cell 12 as indicated by arrows 15. Each internal cell 12 receives information from vertically above, applies the rotation parameters thereto, provides an output indicated by arrows 16 to its respective vertical downstream nearest-neighbour cell 11 or 12 below, and passes the rotation parameter horizontally to its respective lateral downstream nearest-neighbout cell (if any) 12 as indicated by arrows 17. Each boundary or internal cell 11 or 12 also stores a respective matrix element which is associated with the triangular matrix R, initially zero and subsequently updated on each cycle of array calculation. The cells 11 and 12 operate in synchronism in equal lengths of time per cycle under the control of a clock (not shown).
The boundary cells 11 may optionally receive an additional data input from diagonally above, perform a further operation upon it and provide a corresponding output to the respective nearest-neighbour boundary cell diagonally below. This optional additional operation is indicated by arrowed chain lines 18.sub.1 to 18.sub.4, and is associated with delay or memory cells indicated by black dots 19 to synchronize array operation. The diagonal input 18.sub.1 to boundary cell 11.sub.11 would be initialized to unity. Two array operation cycles are required for information to pass from one boundary cell 11 to another via an internal cell 12, whereas only one cycle would be required for direct diagonal transfer between neighbouring boundary cells. The memory cells 19 provide a one cycle delay appropriate to synchronize the two inputs received by boundary cells 11.sub.22 to 11.sub.44.
Data for processing by the array 10 is in the form of an Nxp design matrix X of elements x.sub.ij and a column vector y of elements y.sub.i, where i=1 to N, j=1 to p and p=4. The columns of X are fed into the triangular array portion 13, and the column vector y is fed into the additional column 14. Input is carried out in a temporally skewed order to the first or uppermost row of cells 11.sub.11 and 12.sub.12 to 12.sub.15 of the array 10, element x.sub.i1 to cell 11.sub.11, element x.sub.i2 to cell 12.sub.12 and so on to element y.sub.i to cell 12.sub.15. The temporal skew consists of a linearly increasing delay applied across the elements x.sub.i1 to x.sub.i4 and y; ie the inputs of x.sub.i2 to y.sub.i are respectively delayed by one to four array processing cells as compared to x.sub.i1. When boundary cell 11.sub.11 receives an input element say x.sub.ml, it calculates corresponding rotation parameters which subsequently progress across the first or uppermost row of the array 10 in a stepwise fashion each array cycle. By virtue of the temporal skew, the parameters from cell 11.sub.11 reach each of the cells 12.sub.12 to 12.sub.15 in synchronism with the respective input column element X.sub.mj (j=2 to 4) or y.sub.m. Data elements in columns x.sub.i2 to x.sub.i4 experience one, two or three rotation applications at internal cells 12.sub.12, 12.sub.13 and 12.sub.23, and 12.sub.14 to 12.sub.34 respectively, before providing inputs to boundary cells 11.sub.22 to 11.sub.44 for further parameter evaluation and lateral output in the lower array rows. The temporal skew ensures that data elements reach internal cells 12 in synchronism with the relevant rotation parameters to be applied, irrespective of array position.
As the matrix X and column vector y are fed into the array 10, the triangular array 13 receiving the data elements of X builds up and subsequently updates the values stored in cells 11.sub.11 to 11.sub.44 and 12.sub.12 to 12.sub.34. Initially the stored value in each cell is zero. When four rows of X have passed through the triangular array 13, each cell has stored a respective calculated value. Thereafter, successive rows of X update and statistically improve the stored values.
When all data has flowed through the prior art array 10, the stored cell values correspond to the R matrix (triangular array 13) and Q.sub.1 y (column 14) in Equation (2). In order to solve Equation (2) for the weight vector w(N), Kung and Gentleman (ibid) require the stored values to be transferred to a linear systolic array (now shown) for back-substitution. This requires a separate mode of operation of the cells 11 and 12, in which stored values are output from the array 10 as indicated schematically by arrowed chain lines 20.
Referring now to FIG. 2, there are shown the boundary and internal cell functions for applying Givens rotations with square roots as described by Kung and Gentleman. Parts previously mentioned have like references. Each boundary cell 11 has a stored value of r (initially zero), receives an input x.sub.in from vertically above, computes the cosine and sine Givens rotation parameters c, s and updates r as follows: ##EQU2## p The boundary cells 11 output the c, s parameters laterally to the right to the respective downstream nearest-neighbour internal cell 12.
The internal cells 12 each pass on the c, s parameters laterally to the respective nearest neighbour cell, receive inputs x.sub.in from vertically above, calculate outputs x.sub.out and update r as follows:
No diagonal inputs to or outputs from the boundary cells 11 are required. The stored values of r provide the elements of the upper triangular matrix R required for QR decomposition.
Referring now to FIG. 3, there are shown cell functions for the square root free approach described by Gentleman (ibid). The boundary cells 11 each receive inputs x.sub.in from vertically above, .delta..sub.in from diagonally above, compute rotation parameters c, s and z related (but unequal) to the Givens rotation parameters c, s, output c, s and z laterally to the respective lateral nearest neighbour internal cell 12, and update a stored value d and calculate .delta..sub.out. .delta..sub.out is transferred to the respective diagonal downstream nearest-neighbour boundary cell 11. The cell functions are as follows: ##EQU3## .delta..sub.in is initialized to unity for input to the first boundary cell 11.sub.11. The additional function of producing a diagonal output distinguishes the boundary cells 11 of FIG. 3 from those of FIG. 2.
The internal cells 12 each pass on the c, s and z parameters laterally to the respective nearest-neighbour cell, receive inputs x.sub.in, calculate outputs x.sub.out and update a respective stored value r as follows:
Data flow through the array produces d values stored on boundary cells 11 and r values on internal cells 12. The stored values d provide the elements of a diagonal matrix D related to the upper triangular matrix R by:
where R is a triangular matrix having ones on the diagonal and other elements given by the stored values r.
Either of the sets of cell functions shown in FIGS. 2 and 3 may be employed in the array of FIG. 1 in conjunction with a linear systolic array to derive least squares solutions, the linear array receiving stored array values via the array outputs 20. These cell functions may be generalized to deal with complex data in appropriate cases. Referring now to FIG. 4, there is shown a modification to the array of FIG. 1 in accordance with the invention. A diagonal output 30 and a vertical output 31 are taken from the final downstream boundary and internal cells 11.sub.44 and 12.sub.45 in the triangular array 13 and the additional column 14 respectively. The outputs 30 and 31 are fed to processing means 32. In accordance with the invention, the array 10 also requires diagonal connections between the boundary cells 11 as indicated by arrows 18 in FIG. 1. Connections 20 from the array 10 to a linear array are however not required.
The cell functions may either be as indicated in FIG. 3, or as indicated in FIG. 2 with additional diagonal connections 18. Each boundary cell 11 additionally computes the product of its evaluated cosine (FIG. 2) or cosine-like (FIG. 3) rotation parameter and its respective diagonal input 18. The product is output to the respective diagonal nearest neighbour cell 11. An initial value of unity is input to cell 11.sub.11 in either case. This produces cumulative multiplication of the cosine or cosine-like terms at the diagonal output 30 of the final boundary cell 11.sub.44. The processing means 32 is a multiplier which multiplies together the outputs 30 and 31 of the final downstream boundary and internal cells 11.sub.44 and 12.sub.45 respectively. The output 31 of cell 12.sub.45 provides elements of y which have undergone Givens rotation or the square root free equivalent by parameters evaluated at all four boundary cells 11.sub.11 to 11.sub.44. The output M.sub.out of the processing means 32 can be shown (see later proof) to be given by:
Equation (8) represents the recursive least squares residual e.sub.n for the nth element of the vector y and the corresponding nth weighted row of the matrix X, y.sub.n having entered the systolic array 10 four processng cycles previously. The row vector w(n) of weights represents the least squares solution for all elements of X up to row x.sub.n.sup.T. As further elements of y progress through the array, least squares residuals continue to be produced. These residuals are results required in many electronic signal processing applications, and are produced without solving explicitly for the weight vector w(n) as in the prior art. Problems with ill-conditioned or numerically unstable solutions are avoided, and the amount of circuitry needed is reduced since a linear systolic array is not required. There is no need to extract the stored values from the cells 11 and 12 to perform back-substitution. Furthermore, the least squares residuals are produced recursively, as opposed to the once and for all solution provided by the prior art.
In the general case of cell functions for evaluating and applying rotation parameters not necessarily of the Givens or square root free form, the processing means 32 is required to compute an output equal to the least squares residual e.sub.n. In general the product of outputs of cells 11.sub.44 and 12.sub.45 will always have a simple relationship to the residual, which can be extracted by an appropriate processing means 32. Whereas diagonal boundary cell connections 18 provide a particularly elegant means for cumulatively multiplying cosine or cosine-like parameters, other means may be used in achieving the residual e.sub.n. The basic requirement is that appropriate processing means 32 be employed to collect the cosine or cosine-like terms and corresponding cumulatively rotated data elements and to multiply them together.
The proof of Equation (8), that M.sub.out is in fact the recursive least squares residual, is as follows:
Given an nxp matrix X(n) with n.gtoreq.p and an n-element vector y(n), the corresponding n-element least squares residual vector e(n) is defined according to:
where w(n) is the p-element vector of weights which minimizes
and .vertline..vertline...vertline..vertline. denotes the usual Euclidean norm.
Assuming the notation: ##EQU4## the iterative least squares problem may then be stated as follows: For successive values of n=p, p+1 . . . evaluate the least squares residual
The diagonal matrix B(n) given by: ##EQU5## is included for increased generality. It applies an exponential weight factor .beta..sup.n-k (0<.beta..ltoreq.1) to each row x.sub.k.sup.T of the matrix X(n) and this has the effect of progressively weighting against the preceding rows of X(n) in favor of the nth row whose weight factor is unity. The more conventional unweighted least squares pattern (per Kung and Gentleman, ibid.) is obtained by setting .beta.=1, in which case B(n) becomes a simple unit matrix.
For any value of n(.gtoreq.p), this least squares problem may be solved by the method of orthogonal triangularization. This method is numerically well-conditioned and may be described as follows: Generate an nxn unitary matrix Q(n) such that ##EQU6## where R(n) is a pxp upper triangular matrix. Since Q(n) is unitary, it follows that ##EQU7## P(n) and S(n) being the matrices of dimension pxn and (n-p)xn respectively which partition Q(n) in the form ##EQU8## It follows that the weight vector w(n) must satisfy the equation
and hence
Since R(n) is upper triangular, Equation (22) may be solved by a process of back-substitution. The resulting weight vector w(n) could be used to evaluate the iterative least squares residual defined in Equation (14).
The orthogonal triangularization process may be carried out using various techniques such as Gram-Schmidt orthogonalization, Householder transformation or Givens rotations. However, the Givens rotation method is particularly suitable for the iterative least squares problem. It leads to a very efficient algorithm whereby the triangularization process is recursively updated as each new row of data enters the computation.
A Givens rotation is an elementary unitary transformation of the form: ##EQU9## where c.sup.2 +s.sup.2 =1. The elements c and s may be regarded as the cosine and sine respectively of a rotation angle .theta. which is chosen to eliminate the leading element of the lower vector, ie such that:
It follows that c=r.sub.i /r.sub.i ' and s=x.sub.i /r.sub.i ' where r.sub.i '=(r.sub.i.sup.2 +x.sub.i.sup.2).sup.1/2. A sequence of such elimination operations may be used to carry out an othogonal triangularization of the matrix B(n)X(n) in the following recursive manner. Assume that the matrix B(n-1)X(n-1) has already been reduced to triangular form by the unitary transformation: ##EQU10## and define the unitary matrix ##EQU11## then it follows that: ##EQU12## and so the triangularization process may be completed by the following sequence of operations: Rotate the p-element vector x.sub.n.sup.T with the first row of .beta.R(n-1), so that the leading element of x.sub.n.sup.T is eliminated producing a reduced vector x.sub.n.sup.T'. The first row of .beta.R(n-1) will be modified in the process. Then rotate the (p-1)-element reduced vector x.sub.n.sup.T' with the second row of .beta.R(n-1) so that the leading element of x.sub.n.sup.T' is eliminated, and so on until every element of x.sub.n.sup.T has been eliminated. The resulting triangular matrix R(n) then corresponds to a complete triangularization of the matrix B(n)X(n) as defined in Equation (16). The matrix Q(n) is given by the recursive expression
where Q(n) is a unitary matrix representing the sequence of Givens rotation operations described above, ##EQU13## From equations (18) and (29), it also follows that: ##EQU14##
This yields the recursive expression: ##EQU15## Equation 32 demonstrates that the vector U(n) can be updated using the same sequence of Givens rotations. The optimum least squares weight vector w(n) may then be derived by solving Equation (22) by back-substitution. As has been said, Kung and Gentleman (ibid.) employ a triangular systolic array for matrix triangularization to obtain the R matrix, and a separate linear systolic array to perform the back-substitution.
However, for many purposes the weight vector w(n) is not required explicitly. It is rather the least squares residual e.sub.n in Equation (14) which is of interest. Now e.sub.n is the nth element of:
From Equation (16), it follows that ##EQU16## and hence
But the least squares weight vector w(n) must satisfy Equation (22), so Equation (35) may be written in the form:
which does not depend explicitly on the weight vector w(n). Furthermore, since ##EQU17## and thus
From Equation (30) it follows that the recursive update matrix Q(n) must take the form: ##EQU18## where A(n) is a pxp matrix, a(n) and b(n) are p-element vectors, I denotes the (n-p-1).times.(n-p-1) unit matrix and .gamma.(n) is a scalar. It then follows from Equation (32) that: ##EQU19## Similarly, from Equations (21) and (29): ##EQU20## and so finally the expression:
But .alpha.(n) is the result obtained when y.sub.n is rotated with each element in the vector .beta.U(n-1), and is obtained during the triangularization process as the output 31 of the final downstream internal cell 12.sub.45 (FIG. 4). Furthermore, it follows from Equation (42) that .gamma.(n) is the result obtained by applying the same sequence of Givens rotations to rotate a unit input (18.sub.1 in FIG. 1) with each element of the p-element null vector. Its value must therefore be given by the product ##EQU21## where c.sub.i (n) is the cosine parameter associated with the ith Givens rotation in the sequence of operations represented by Q(n). This quantity may be computed during the triangularization procedure by connecting together the boundary cells 11 in FIG. 1 by connections 18, the product ##EQU22## appearing at the output 30 (FIG. 4) of the final downstream boundary cell 11.sub.44.
The foregoing analysis proves that the output of the multiplier or processing means 32 provides the least recursive squares residual e.sub.n without the need for back-substitution, which the prior art requires.
The recursive least squares minimization process described above may also be carried out using the square-root free Givens rotation approach. When matrix triangularization is carried out using this approach, the upper triangular matrix R is represented by a diagonal matrix D and a unit upper triangular matrix R such that R=D.sup.1/2 R. The rotation operation then takes the form: ##EQU23## where x.sub.i and x.sub.k are respectively the inputs to boundary and internal cells, d and r.sub.k are the values stored at boundary and internal cells, the presence or absence of a prime superscript to these quantities represents update or current values respectively, and .delta. and .delta.' are diagonal inputs to and outputs from boundary cells. By analogy with the previous analysis, the update formulae become:
and
c and s being generalized rotation parameters (analogous to the basic Givens rotation parameter c and s) given by:
It is important to appreciate that the basic and square-root free Givens rotation operations are mathematically equivalent despite the fact that they are expressed in terms of different parameters. It follows that the analysis in this section also applies to the square-root free Givens rotation case, and that an orthogonal triangularization of the matrix B(n)X(n) may be carried out using a sequence of square root free operations equivalent to the basic Givens rotation case. In the square root free case, the scaling factor .delta. associated with each data vector x.sub.n.sup.T is initialized to unity whilst the diagonal matrix D(n) is set equal to zero at the outset of the computation.
This latter analysis shows the multiplication by the processing means 32 also provides the recursive least squares residual in the square root free rotation case. The output 30 of boundary cell 11.sub.44 provides a cumulative product of cosine-like terms which is equal to a factor multiplied by the product of Givens rotations cosine terms. The output 31 of internal cell 12.sub.45 provides an output 31 equal to the cumulatively rotated y.sub.n divided by the same factor. On multiplying the outputs 30 and 31 at the processing means 32, the factor cancels out yielding the recursive least squares residual e.sub.n as before.
In general, for rotation algorithms not necessarily of the Givens or square root free varieties, it can be shown that the least squares residual e.sub.n can always be derived from the outputs 30 and 31 by an appropriately arranged processing means 32.
The systolic array of the invention may also be employed to solve least squares problems including constraints. The problem comprises determining a (p+1) vector of weights w for which .vertline..vertline..PHI.w.vertline..vertline. is minimized, where .PHI. is an nx(p+1) matrix with p.ltoreq.n, subject to the constant linear constraint c.sup.T w=.mu., where c is the constraint vector and .mu. is a constant. It is assumed without loss of generality that c.sup.T =[c.sup.T,1], and so the constraint may be expressed alternatively in the form w.sub.p+1 =.mu.-c.sup.T w, where w denotes the first p elements of w. Denoting the first p columns of .PHI. by .PHI. and the (p+1)th column by the vector .rho., the problem may be expressed as follows. Given an nxp matrix and a p-vector .rho., find the p-vector of weights w which minimizes the expression .vertline..vertline..PHI.-.rho.c.sup.T)w+.mu..rho..vertline..vertline.. This expression has the same form as Equation (1), with X replaced by .PHI.-.rho.c.sup.T and y replaced by -.mu..rho.. Making appropriate substitutions in Equation (8), the systolic array of the invention will produce the least squares residual .PHI..sub.n.sup.T w.sub.n. The matrix .PHI.-.rho.c.sup.T may readily be evaluated by subtracting the vector .rho..sub.n c.sup.T or linear constraint factor from each row .PHI..sub.n of the submatrix .PHI. before it enters the systolic array 10. The unconstrained least squares problem to which Equations (1), (2) and (8) relate is in effect a special case of this constrained problem, the special case having the trivial constraint that w.sub.p+1 is equal to unity. It will be apparent that further linear constraints may be incorporated by additional subtraction operations on the matrix .PHI. before it enters the array. Such subtraction operations are electronically straight-forward to implement.
In processing a data system, it may be desirable to give more emphasis to recent data than to earlier data. In the least squares problem discussed with reference to Equation (8), the weight vector w(n) is computed as the best fit to all data received. Necessarily, as the number of data samples builds up, each successive sample has progressively less effect on w(n). To give more emphasis to more recent data, an exponentially decaying memory with a lifetime of approximately (1-.beta.).sup.-1 samples may be implemented in the array of the invention, where 0<.beta..ltoreq.1, as set out in Equation (15) above. This is achieved by ensuring that on every array processing cycle the value of r (see FIG. 2) stored by each cell 11 or 12 in the Givens rotation case is multiplied by .beta. when updated, in addition to the updating requirements of Equations (4) to (7). In the square root free case, it is necessary to multiply by .beta..sup.2 values stored on boundary cells 11 only, values stored in internal cells 12 being unaffected. An additional multiplication operation would accordingly be required in appropriate cells. Incorporation of a memory in this way allows the array of the invention to be used in a continuously adaptive mode.
The processing cells 11 and 12 of FIGS. 2 and 3 may be implemented electronically as a special purpose VLSI circuit comprising the required basic elements (eg a multiplier, square root generator, divider or reciprocal table, adder) together with memory and control units. Two types of circuits would then be required to construct the array of the invention.
Alternatively, the processing cells 11 and 12 may be implemented with appropriately programmed digital signal processing chips. Suitable types are presently commercially available in the form of special purpose microprocessors. The same basic component would then be used throughout the systolic array with the boundary and internal cells having different programs.
The systolic array of the invention may be employed for linear predictive filtering of images. The approach is to use a weighted average of an ensemble of data to predict other data. The residual or difference between the prediction and the received data to which it corresponds need only be recorded if significantly large. In this way only significant features of an image need be registered, resulting in a reduction in the data to be handled and the equipment required. One example of the use of this technique may be stated as follows. Given a two dimensional array of image pixel values, predict each element in a given row of the image using a weighted linear combination of the equivalent elements in the respective four previous rows. A vector of prediction coefficients is defined to minimize the sum squared residual for all data elements or pixel values in the same row up to and including the most recent pixel. In effect an ensemble average along the rows is used to carry out a linear prediction of future data to appear in later rows. An exponential memory may be incorporated as previously described so that the effective region of information averaging is localized, ie more reliance is placed on more recent data. The resulting residuals are employed to build up a filtered or reduced image with useful properties. Large residuals tend to indicate sudden or unpredictable changes within the image, and this type of information regarding discontinuities may be used as an aid to image analysis.
Referring to FIG. 5, an image represented by an array 50 of pixel dots 51 have rows and columns arranged horizontally and vertically. Each of the elements in the (k+5)th row of the image, designated as pixel values y.sub.i (i=1, 2 . . . m), are predicted from the corresponding column elements in the four preceding rows (k+1) to (k+4) respectively. Elements in rows k+j(j=1 to 4) are designated x.sub.1j, x.sub.2j, x.sub.3j, . . . x.sub.mj. The required residual for each element y.sub.i is the difference between it and the weighted x values in the same column of the preceding four rows, the weight vector being calculated to minimize the sum of the squares of the residuals associated with all elements up to y.sub.i. This labelling and the residual correspond exactly to the way in which the matrix X and vector y are fed to the array 10 of FIG. 1 and to the Equation (8) expression for the residual, with rows of image elements x.sub.ij etc corresponding to columns of X. Accordingly, the array of the invention may be employed for linear predictive image filtering without back-substitution as would be required in the prior art.
The systolic array of the invention may also be employed to process the signals from a phased array radar operating as an adaptive digital beamformer. Radar signals may be adulterated by noise such as jamming sources. The phased array radar has primary and auxiliary antenna, and receives the desired signal in the main beam of its primary antenna. Unwanted signals appear in the sidelobes of the primary antenna. To eliminate the unwanted signals, the approach is to form a weighted linear combination of the auxiliary antenna signals in order to produce the best possible match to the noise waveform in the primary antenna channel. The combination may then be subtracted directly from the primary signal to achieve noise cancellation and improve signal to noise ratio. The vector of weights is complex, corresponding to amplitude and phase factors, and in effect generates an amplitude response function which has nulls in the direction of jamming sources.
Referring once more to FIG. 1, the vector y of elements y.sub.1, y.sub.2 etc would in this example represent the sequence of complex or phase and amplitude signal values from the primary antenna, which include contributions from the desired signal and from noise sources. Each column of numbers x.sub.1i, x.sub.2i, . . . x.sub.ni (i=1 to p) represents the sequence of complex signal values from the ith of p auxiliary antenna elements. It is commonly assumed in sidelobe cancellation that the auxiliary antenna elements sample the noise field alone and do not receive the desired signal. The complex signal values are derived from the main and auxiliary antennas by separating the analog signal at intermediate frequency (IF) into its in-phase and quadrature or I and Q channels and passing each channel through an A/D converter.
Assuming that the desired signal is uncorrelated with the various noise signals, noise cancellation from the primary antenna signals is achieved by choosing the vector of complex weights w(i) at the ith sample time such that .vertline..vertline.X(i)w(i)-y(i).vertline..vertline. is minimized. X(i) denotes the ixp matrix of all signal values obtained up to the ith sample time from the p auxiliary antennas, and y(i) denotes the corresponding vector of values from the primary antenna of which the ith value is y(i). The noise cancelled output at time i is then x.sub.i.sup.T w(i)-y.sub.i. This is the residual generated by the systolic array of the invention as demonstrated by Equation (8). The invention is accordingly capable of providing a noise-cancelled output for an antenna array, cell functions being employed which are appropriate for complex amplitude and phase data.
The radar signal processing application of the invention may be made continuously adaptive by incorporating an exponential memory with lifetime.about.(1-.beta.).sup.-1 as previously described. Furthermore, noise-cancellation may be carried out with a general antenna array of (p+1) elements subject to the constraint that the antenna array response in a specific observation direction is constant. This is achieved by incorporating a constant linear constraint of the form c.sup.T w(i)=.mu. as previously described.
Claims
Certificate of Correction present
The claims shown below may not include correction changes. Use the corrected PDF for authoritative text.
I claim:
1. In a systolic array arranged for matrix triangularization of an input stream of data elements, the array including: (1) rows of cells each beginning with a boundary cell and continuing with at least one internal cell, the array rows being also arranged to form columns comprising a first column containing a boundary cell only, a final column containing internal cells only and intervening columns terminating at a boundary cell arranged below at least one internal cell with the number of internal cells increasing from one by one per column to a penultimate column containing one internal cell less than those contained by the final column; (2) processing means in the boundary and internal cells to cause the boundary cells to evaluate S and C rotation parameters from data input thereto, and to cause the internal cells to apply evaluated S and C parameters to data input thereto, the S and C parameters being any one of Givens sine and cosine rotation parameters and non-Givens rotation parameters performing a function related to rotation; (3) nearest neighbor cell interconnection lines arranged to provide for (a) evaluated S and C parameters to pass along rows for application to input data by successive internal cells to produce rotated data, and for (b) rotated data to pass down columns to provide input to adjacent cells; and (4) first row cell inputs arranged to receive the said input stream such that each first row cell receives successive respective data elements; the improvement comprising the array including processing means arranged to multiply successive cumulatively rotated data elements output from the final column's lowermost cell by respective relatively delayed and cumulatively multiplied C parameters output from all boundary cells to generate recursively quantities at least closely related to least square residuals.
2. A systolic array according to claim 1, further including means for emphasising more recent data in the input stream.
3. A systolic array according to claim 2 wherein each boundary cell and each internal cell includes means for multiplying a stored signal by a constant having a value between zero and unity.
4. A systolic array according to claim 2 wherein each boundary cell includes means for multiplying a stored signal by a constant having a value between zero and unity.
5. A systolic array according to claim 1, further including means for substracting a linear constraint factor from the input of said data stream prior to array entry.
6. A systolic array according to claim 1 further including means for inputting image data to the array for linear predictive filtering.
7. A systolic array according to claim 1 further including means for connecting the array to a phased array of radar antennas.
8. In a systolic array arranged for matrix triangularization of an input stream of data elements, the array including: (1) rows of cells each beginning with a boundary cell and continuing with at least one internal cell, the array rows being also arranged to form columns comprising a first column containing a boundary cell only, a final column containing internal cells only and intervening columns terminating at a boundary cell arranged below at least one internal cell with the number of internal cells increasing from one by one per column to a penultimate column containing one internal cell less than those contained by the final column; (2) processing means in the boundary and internal cells to cause the boundary cells to evaluate S and C rotation parameters from data input thereto, and to cause the internal cells to apply evaluated S and C parameters to data input thereto, the S and C parameters being any one of Givens sine and cosine rotation parameters and non-Givens rotation parameters performing a function related to rotation; (3) nearest neighbor cell interconnection lines arranged to provide for (a) evaluated S and C parameters to pass along rows for application to input data by successive internal cells to produce rotated data, and for (b) rotated data to pass down columns to provide input to adjacent cells; and (4) first row cell inputs arranged to receive the said input stream such that each first row cell receives successive respective data elements; the improvement comprising the boundary cells in at least the second to final row having processing means for multiplying C parameter inputs by evaluated C parameters to provide C parameter outputs, each boundary cell other than that in the final row having a C parameter output connected via delaying means to a C parameter input of a respective boundary cell in a preceding row, and the final row boundary and internal cells having respectively a C parameter output and a rotated data output connected to a multiplying means arranged to multiply them together to provide successive products of cumulatively rotated data with cumulatively rotated data and generate recursively quantities at least closely related to least squares residuals.
9. A systolic array according to claim 8 further including means for emphasizing more recent data in the input stream.
10. A systolic array according to claim 9 wherein each boundary cell and each internal cell includes means for multiplying a stored signal by a constant having a value between 0 and unity.
11. A systolic array according to claim 9 wherein each boundary cell includes means for multiplying a stored signal by a constant having a value between 0 and unity.
12. A systolic array according to claim 8 further including means for subtracting a linear constraint factor from the data of said input stream prior to array entry.
13. A systolic array according to claim 8 further including means for inputting image data to the array for linear predictive filtering.
14. A systolic array according to claim 8 further including means for connecting the array to a phase array of radar antennas.
Patent Citations (7)
Non-Patent Literature (3)
- Algorithms for VLSI Processor Arrays, H. T. Kung and Charles Leiserson, pp. 271-293.
- Least Squares Computations by Given Transformations Without Square Roots, W. Morven Gentleman (1973), pp. 329-336.
- Matrix Triangularization by Systolic Arrays, W. M. Gentleman, H. T. Kung (1981), pp. 19-26.