Altering Spatial Characteristics of a Digital Image

This invention relates generally to the display of digital images and more particularly the invention relates to an improved method and means for establishing new pixel values for new pixel locations from an image having established pixel locations and values. The new pixel locations could be chosen, for example, to magnify, minify, translate or distort the original image.

The display of a digital image, for example on a video screen or photographic film, is controlled by a memory which stores values for picture elements or pixels. In a video display, an electron beam is scanned across the phosphor-coated surface of the display screen, and the current of the electron beam is controlled in accordance with the stored values to thus control the light intensity of each pixel element as the electron beam impinges on the phosphor material. Alternatively, the digital image can be transferred to photographic film by scanning a laser beam across the film and modulating the intensity of the laser beam in accordance with the stored pixel values.

In any of the image processing tasks mentioned above, values for pixels at locations different than the original pixel locations must be arrived at. The simplest technique for doing this is to use, for each output pixel, the value of the nearest input pixel. Minification is then performed by selectively deleting pixels while to magnify some or all of the pixel values are replicated. The resulting image quality is often not satisfactory. Values for the new pixel locations can be arrived at using linear interpolation in both directions. However, images magnified using this method are blurry and structured noise is sometimes visible.

Improved fidelity and sharpness can be realized if the interpolation of new pixel values is a higher order polynomial interpolation. For example, in a third order interpolation of a pixel value in a new matrix to be described, the values of sixteen adjacent pixels in the old matrix are used. Generally, the higher the order of the polynomial used, the more fidelity and sharpness the new image will have. However, as the order of the polynomial interpolation increases, more computations are required in determining the new pixel values.

Accordingly, an object of the present invention is an efficient method and apparatus for implementing polynomial interpolation of pixel values in a new matrix using pixel values of an old matrix.

A feature of the invention is the implementation of polynomial interpolation by a weighted combination of the input pixels.

Another feature is the use of a table of coefficients for polynomial interpolation of pixel values in a new matrix which are based on one dimensional (e.g., x or y) incremental spacing from pixels in an old matrix.

Briefly, the pixels in the new pixel matrix are mapped in the old pixel matrix to obtain x and y offsets from old pixels. The offset for each new pixel in one dimension (e.g., x dimension) is determined and values for interim pixels are interpolated or obtained using values of a plurality of old pixels adjacent to each new pixel in the one dimension. The final pixel values are then obtained by polynomial interpolation in the other dimension (e.g., y dimension) using the calculated values of interim pixels.

Importantly, the weighting coefficients for the one dimensional interpolations are precalculated for given incremental values. Thus, the time consuming tasks of calculating polynominal coefficients and evaluating polynomials are not required during the calculation of new pixel values.

The invention and objects and features thereof will be more readily apparent from the following detailed description and appended claims when taken with the drawings, in which:

FIG. 1 is an illustration of a portion of a matrix of known pixel values and a matrix of pixels whose values are to be calculated.

FIG. 2 is a functional block diagram of computer apparatus for determining new pixel values in accordance with the invention.

FIG. 3 is a program flow diagram for calculating new pixel values in accordance with one embodiment of the invention.

FIG. 4 is a schematic diagram of one embodiment of the invention.

Referring now to the drawing, FIG. 1 illustrates a portion of a matrix 10 of pixels of known value such as the pixels 11, 12, 13 and 14 in one horizontal row of the matrix. Assume that a new matrix of pixels is mapped on the old matrix 10 and the new pixel 16 is in the new matrix. As illustrated, the pixel 16 is offset from pixel 12 by a horizontal distance, .DELTA.x, and a vertical distance, .DELTA.y.

In general, the value of a new pixel 16 is arrived at by treating the region of the image in the neighborhood of pixel 16 as some function f (x,y) the details of which depend on the values of neighboring pixels. The value assigned to pixel 16 is then the value of the function f (x,y) at the position of pixel 16. This process is repeated for all the output pixels to arrive at the interpolated image. In polynomial interpolation as used in conjunction with this invention, the function f (x,y) is a polynomial function of x and y. The process of calculating the output pixel value then involves calculating the polynomial coefficients using input pixels neighboring the output pixel and then evaluating the polynomial. Typically, the number of input pixels used is equal to the number of polynomial coefficients of f (x,y) that must be determined. For example, suppose we wish to use as a model the polynomial:

Suppose we want to compute the value for a pixel to be located at (x, y). Coefficients {a.sub.o, a.sub.1 . . . a.sub.8 } such that f (x,y) exactly matches the values of nine input pixels surrounding (x,y) are calculated.

For the model of Equation 1 this is done by solving a system of nine equations (one for each input pixel) with nine unknowns (the coefficients). The coefficients are then used to calculate the output pixel value:

If the two dimensional polynominal model contains all terms up to nth order in either variable, where n is any integer, then the two dimensional interpolation process can be broken down into one-dimensional interpolations. For example, the first step would interpolate in the x direction and arrive at interim pixel values for pixels (x, y.sub.e) where y.sub.e takes on the values of y for rows neighboring the desired output pixel. Then, an interpolation in y is used, using the interim pixel values as inputs, to compute the output pixel value for the desired pixel at (x, y). It will now be shown that for polynomials that include terms to nth order in either coordinate the two dimensional interpolation can be simplified into two one-dimensional interpolation steps. The polynomial model we use is: ##EQU1##

In a (n+1) square array of pixels surrounding the desired output location (x, y) x will take on values {x.sub.o x.sub.1, . . . , x.sub.n } while y will take on values {y.sub.o, y.sub.1 . . . , y.sub.n }. Let v (x.sub.k, y.sub.e) be the value for the pixel at (x.sub.k, y.sub.e). We need to arrive at a value for a new pixel at (x, y). The classical way to do this is to generate a set of coefficients {a.sub.ij } such that ##EQU2## is true for all the (n+1).sup.2 pixels in the square region surrounding (x, y), and then using these to calculate the desired output ##EQU3##

Consider an interim pixel location at (x, y.sub.e). Define b.sub.ie by: ##EQU4##

Note that the coefficients {b.sub.ie } depend on the y coordinate y.sub.e but are indpendent of x. We can now rewrite Equation 4 as: ##EQU5## and Equation 5 for the interim pixel becomes: ##EQU6##

Equations 7 and 8 define an nth order one-dimensional polynomial interpolation in x along the row y=y.sub.e, to arrive at a value for the interim pixel at (x, y.sub.e). It is clear, though, that the value arrived at for this interim pixel by one dimensional interpolation is exactly the value that would be arrived at by the two-dimensional process. This is true for all pixel positions along rows of the old matrix. This can be understood intuitively by noting that for any fixed value of y, Eq. 3 is an nth order polynomial in x, as in the one dimensional model, and if the two agree at (n+1) points along the row, they will agree everywhere along the row.

Suppose that values for interim pixels (x, y.sub.e) for the (n+1) rows surrounding the pixel of interest have been calculated by nth order polynomial interpolation. Again, the interim pixel values agree with what the full two-dimensional method would have given at those locations. Define ##EQU7##

The set {c.sub.j } can be computed by finding the coefficients for which ##EQU8## is true for all the interim pixels. Eq. 5 now becomes: ##EQU9##

Equations 10 and 11 define one dimensional nth order polynomial interpolation in the y direction along the column defined by the interim pixels and which also contains the output pixel (x, y). Again, since both equations 5 and 11 are nth order polynomials and since they agree at (n+1) points (the interim pixels) they must give the same value at (x, y). Thus, the two dimensional interpolation can be implemented using simpler one dimensional interpolation steps. Further, it will be clear to those skilled in the art that for the often used steps of image magnification, minification, and translation, many of the interim pixel values needed to calculate one output pixel can be stored and used to calculate other output pixels.

We can now restrict ourselves to dealing with one dimensional interpolation. It will now be shown that the interpolated output value can be calculated as a linear combination of the input pixel values, where the combination coefficients depend on the relative position of the output point with respect to the input points.

We use as a model: ##EQU10##

Again, the coefficients a.sub.i are arrived at by forcing ##EQU11## to be true for the (n+1) pixels surrounding the output location, where {x.sub.k } and {v.sub.k } are the input pixel locations and values respectively. There are (n+1) equations like Equation 13 which we can write in matrix form:

A is a row vector containing the polynomial coefficients, X is a square matrix containing the x values raised to the appropriate powers, and V is a row vector containing the pixel values. We solve for A by:

and then find the desired output pixel value by

where X is a column vector containing X raised to the appropriate powers. Substituting Equation 15 into Equation 16 and regrouping we get

which shows that the output value is a linear combination of the values of the surrounding input pixels. The combination coefficients are given by (X.sup.-1 X) and depend on the relative position of the output pixel within the input array but are independent of the specific pixel values. Once the combination coefficients are known, the interpolation requires (n+1) multiplications and n additions.

If the input pixels are equally spaced, the combination coefficients depend only on the normalized distance of the output pixel from the pixel to its left. We define this normalized distance by: ##EQU12## where x.sub.k is the pixel location just to the left of x and .DELTA.p is the spacing between pixels.

One advantage of using an odd number for the polynomial order n is that the number of input points used for each output point, (n+1), is even. Thus, the input points can be symmetrically located about the output. Following is a derivation of the combination coefficients for a third order interpolation (n=3).

The model for this third order interpolation is:

Since x.sub.k is the pixel location just to the left of x, the four pixels to be used have values V.sub.k-1, V.sub.k, V.sub.k+1 and V.sub.k+2 and their respective .DELTA.x as values are -1, 0, 1, and 2 respectively. Using Equation 19, the four equations used to calculate the polynomial coefficients a.sub.i are:

Equation 20b defines a.sub.o. Adding Equations 20a and 20c yields:

Substituting Equation 20b and regrouping gives an expression for a.sub.2 :

Subtracting Equation 20a from Equation 20c gives:

Substituting Equations 20b and 22 into Equation 20d and regrouping gives:

Equations 23 and 24 can be solved for a.sub.1 and a.sub.3 :

Substituting Equations 20b, 22, 25 and 26 into Equation 19 and combining like terms yields:

where

Thus, to perform the cubic interpolation when the input pixels are equally spaced, it is not necessary to solve a system of equations for the polynomial coefficients and then to evaluate the cubic polynomial at the desired location. Instead, the combination coefficients for the four points surrounding the output location can be calculated using Equations 18 and 28 and then used with Equation 27 to calculate the output value.

Even with the method, there are many arithmetic operations involved in the interpolation since there are many operations needed to generate the combination coefficients for a given (.DELTA.X). However, this latter computational burden (the operations involved in generating the coefficients) can be eliminated. This is done by precalculating the coefficients for a set of values of .DELTA.x, and storing these in a table. When coefficients for a particular value of .DELTA.x are needed, the coefficents for the nearest table entry are used. Alternatively, linear interpolation between table entries could be used.

It has been found that for a particular application cubic interpolation usig 64 sets of precalculated combination coefficients performs very well. Following is a table of combination coefficients for cubic interpolation at 64 equally spaced values of .DELTA.x:

In this method of interpolation, then, for a desired output pixel location x, the first pixel to the left, x.sub.k, is identified, .DELTA.x is calculated using Equation 18 (note that the pixel spacing .DELTA.p is usually normalized to unity so no division is necessary), the four combination coefficients for the table entry with a value of x closest to the calculated value are found, and these coefficients are used along with the values for the four pixels surrounding x in Equation 27 to calculate the output value.

Referring again to FIG. 1, a value for the interim pixel 15 is first calculated using .DELTA.x and the values of the four adjacent old pixels 11, 12, 13, and 14 (for a third order polynomial). Similarly, values for interim pixels 17, 18 and 19 are calculated. A final pixel value is obtained for pixel 16 from the polynomial interpolation along the y-axis using the values of the pixels 17-19 and the offset .DELTA.y.

This procedure is illustrated in the computer system of FIG. 2. The new pixel 16 from FIG. 1 is located in the matrix map 20 using the matrix 10 of FIG. 1. The x-offset (.DELTA.x) is located at 22, and the y-offset (.DELTA.y) is located at 24. The interim pixel values are then determined at 26 using the x-offset, .DELTA.x, coefficients from the table of coefficients 28, and the values of the sixteen adjacent pixels. The final pixel value of pixel 16 is obtained at 30 using the interim pixel values from 26, the y-offset, .DELTA.y, and coefficients from the table 28.

FIG. 3 is a flow diagram for performing one dimensional interpolation using the present invention. The location x.sub.o of the first output pixel within the input array, the increment .DELTA.x between output points, the number N of outputs desired and the input pixel values {V.sub.k } are provided as inputs at 40. Initialization occurs at 42 wherein the output pixel counter j is set to 1 and the output pixel location x is set to x.sub.o. Next, the index k of the pixel to the left of x is calculated at 44. Here IFIX(x) refers to the largest integer less than or equal to x. The distance .DELTA.x of the output pixel from its left neighbor is also calculated at 44. Coefficients from the table of coefficients 46 are retrieved at 47 and applied in the polynomial equation using the values of the four adjacent input pixels as shown in 48 to obtain the new pixel value OUT.sub.j. At 58, j is compared to N. If j is equal to N then the task is completed. If it is not, the output pixel index j and location x are incremented at 52.

Attached hereto and incorporated by reference is a program listing for creating the table of coefficients and a program listing for image interpolation using a third order polynomial function. Both programs have been implemented in a Data General S 140 computer as emplyed in the General Electric CT 9800 Computerized Tomographic Scanning System.

FIG. 4 is a schematic diagram of one embodiment of apparatus in accordance with the invention which operates as described above. In this embodiment interim pixel values are derived for each column of the new matrix along the pixel rows of the old matrix based on interpolation of old pixel values. Then the final values of pixels in each column are derived based on interpolation of the interim values. The apparatus operates column by column in calculating the interium pixel values and the final pixel values.

As will be described, many of the same elements are used to calculate both interim pixel values and final pixel values. First the x and y coordinates of the upper left-most output pixel (in terms of the input pixel array) are loaded into the next column register 93 and the first y register 94, respectively. Similarly, the distance between output pixels in the x and y directions are loaded into the Dx register 61 and Dy register 62, respectively.

To calculate interim pixels for the first column, switches 91, 92, 95, 89, 79 and 83 are set to their I positions. Switch 90 is set to the INT position to load the input address register 60 with the contents of the next column register and then is moved to the I position. The y register 73 is cleared. The input address register now contains the x coordinate of the first output column which is added to Dx at 64 and stored in the next column register in preparation for the next output column. The input address register is comprised of an integer part and a fractional part since it may lie between two columns of the old matrix.

The fractional portion of the X address is applied through register 66 to address the coefficient memory 68. Assuming an interpolation of four values, four coefficients will be read from memory as the coefficient neighborhood address 69 is clocked through values 0, 1, 2, 3, as will be described below.

The integer portion of the X input is applied through register 70 and combined in sequence at 72 with the address neighborhood (e.g. -1, 0, 1, 2) from 74 and the sum is used along with the y register 73 to address the input image memory 76 for obtaining the four input pixel values for interpolation.

The address neighborhood 74 and coefficient neighborhood 69 are clocked and synchronized to present four pairs of coefficients and pixel values to multiplier 78. The four products are added at 80 and accumulated at 82. The resulting interim pixel value is stored at 84 using the y register 73 as an address. The accumulator 82 is cleared, the y address 73 is incremented and the next interim pixel is calculated.

This process is repeated until all interim pixels for this column have been calculated and stored in 84. The final pixel values for this column are then calculated. Switches 91, 92, 95, 89, 79, and 83 are set to their 0 positions. Switch 90 is set to the INT position and the contents of the FIRST y register 94 are loaded into the input address register 60. Switch 90 is then set to the 0 position. The input image memory 76 and the y register 73 are not used in this step. The accumulator 82 is cleared in order to calculate the first output pixel value.

Again, the value of register 60 is separated into integer and fractional parts. The fractional part is used to select coefficients from table 68 and the integer part is added to the address neighborhood and the sum is used to address the interim pixel memory. By clocking the address and coefficient neighborhood registers through their values (-1, 0, 1, 2) and (0, 1, 2, 3) respectively and summing the product of interim pixel value and coefficient pairs a final pixel value is produced at 82 and stored in the output image memory 86. The accumulator 82 is then cleared, the input address 60 is incremented by DY by adder 64 and the next output pixel is calculated. The process continues until all output pixels for this column are calculated.

Then the processes of calculating interim pixel values and final values in the next column are performed. Switches 91, 92, 95, 89, 79, and 83 are set to their I positions. Switch 90 is used to load register 60 with the x coordinate of this column, the y register 73 is cleared and the x coordinate of the next column is calculated at 64 and stored at 93. The steps that follow are obvious to one skilled in the art.

The device of FIG. 4 as described is best suited for image magnification, minification or stretching in one of the cartesian directions. It can be modified by one skilled in the art to allow a more general alteration of spatial characteristics. The device of FIG. 2 can be used for any of the tasks described above.

While the invention has been described with reference to a specific embodiment, the description is illustrative of the invention and is not to be construed as limiting the invention. Various modifications and applications may occur to those skilled in the art without departing from the true spirit and scope of the invention as defined by the appended claims.