Image Transformation Method and Device

This invention relates to a method of performing image transformation such as enlargement, reduction, and rotation on a rectangular image, and to a device for carrying out this method.

The need to perform such image transformations arises frequently in fields such as computer graphics, computeraided design, and desktop publishing. Such transformations are used, for example, to display letters, symbols, and other images in arbitrary sizes and orientations. Mathematically, these transformations belong to the class of affine transformations, and can be executed in a straightforward manner by matrix multiplication.

Matrix multiplication is a computationally slow process, however, so it is desirable to execute the transformation by a faster method. Faster methods can be achieved by employing well-known digital differential analyzer (DDA) algorithms.

The prior art regarding application of DDA algorithms to these transformations has been described in, for example, a Technical Paper of the Institute of Electronics and Communication Engineers of Japan (Vol. 83, No. 195, EC83-34, Nov. 24, 1983) explaining the processing method of the MN8614 Image Processor.

The prior art disclosed in this reference operates on a source image comprising a rectangular array of pixels, generating a destination image also comprising an array of pixels. The prior art can be summarized as follows.

For a reduction transformation, a DDA algorithm is applied to map a horizontal or vertical source line of M pixels onto a horizontal or vertical destination line of N pixels, where M>N. The mapping is thus many-to-one, each destination pixel corresponding to one or more source pixels. When a destination pixel corresponds to more than one source pixel, the values of those source pixels are combined by OR logic to produce the value of the destination pixel. The reduction ratio is N/M.

For an enlargement transformation, the same procedure is applied in reverse, producing a one-to-many mapping of the source image onto the destination image. The value of each source pixel is transferred to one or more destination pixels.

For a rotation, the reduction procedure is applied twice to a horizontal line of source pixels to generate the x- and y-coordinates of a line of destination pixels. The reduction ratios are the cosine and sine of the angle of rotation.

Problems arise when this method is used to perform a simultaneous rotation and enlargement, however, or a simultaneous rotation and reduction. The basic reason is that the rotation transformation does not generate continuous horizontal or vertical lines of pixels, which could be correctly enlarged or reduced, but instead generates oblique lines which are not suitable for the enlargement or reduction process. The visible results are image defects such as pixel drop-outs.

It is accordingly an object of the present invention to perform image transformations without introducing image defects.

An image transformation method transforms a rectangular source image formed by pixels in a two-axis coordinate system to a destination image formed by pixels disposed on scan lines by mapping vertices of the source image onto vertices of the destination image, then mapping edges of the destination image onto edges of the source image, mapping scan lines in the destination image onto lines in the source image, and assigning to pixel on these scan lines the values of corresponding pixels in the source image. Finally, the destination image is displayed by scanning according to the scan lines.

An image transformation device comprises a source image memory for storing a source image, a destination image memory for storing a destination image, affine transformation means for performing an affine transformation on vertices of the source image to generate vertices of the destination image, edge mapping means for mapping pixels on edges of the destination image to pixels on edges of the source image, scan line mapping means for mapping pixels on scan lines in the destination image to pixels on lines in the source image, controller means for controlling the affine transformation means, edge mapping means, and scan line mapping means and transferring pixel values from the source image memory to the destination image memory as indicated by the scan line mapping means, a data bus for interconnecting the source image memory, destination image memory, and controller means, and display means for displaying images stored in the destination image memory.

FIG. 1 is a general flowchart of a novel image transformation method.

FIG. 2 shows an example of a source image area.

FIG. 3 shows an example of a destination image area.

FIG. 4 is a more detailed flowchart illustrating the first step in FIG. 1.

FIG. 5 is a more detailed flowchart illustrating the second step in FIG. 1.

FIG. 6 is a more detailed flowchart illustrating the third step in FIG. 1.

FIG. 7 is a block diagram of an image transformation device.

A novel image transformation method for transforming a rectangular source image to a destination image will be described below. The source image is formed by pixels in a two-axis coordinate system, the pixels being located at positions with integer coordinates. The pixels having values denoting, for example, different colors or intensity levels. The two axes will be denoted the x-axis and y-axis. The destination image is formed by pixels in a similar two-axis coordinate system. The destination image will be displayed by scanning it along scan lines on which the pixels are disposed. The scan lines are oriented parallel to the x-axis of the destination coordinate system, and perpendicular to the y-axis.

The transformations to be performed are affine in nature; that is, they transform parallel lines into parallel lines. Enlargements, reductions, and rotations are examples of affine transformations. Other examples include reflections, translations, shearing transformations, and product transformations generated by combining any of the above.

First a general description of the method will be given with reference to FIG. 1. As shown in FIG. 1, the method comprises four steps.

The first step 1 is a vertex mapping step for mapping vertices of the source image onto the destination coordinate system, where they become vertices of the destination image. This step will be described in greater detail later with reference to FIG. 4.

The second step 2 is an edge mapping step for mapping edges of the destination image onto edges of the source image. This step will be described in greater detail later with reference to FIG. 5.

The third step 3 is a scan line mapping step for mapping scan lines in the destination image onto lines in the source image, and assigning to pixels on the scan lines the values of corresponding pixels in the source image. This step will be described in greater detail later with reference to FIG. 6.

The fourth step 4 is a display step for displaying the destination image by scanning it according to the scan lines and sending the scanned pixel values to a raster output device such as a cathode-ray-tube display. Methods of display are well known, so further description of this step will be omitted.

In the first three steps, to map a first entity onto a second entity means to generate a list of elements of the second entity corresponding to elements of the first entity. In this invention the elements of interest are pixels defined by x-y coordinates, and a mapping can be characterized as a process that generates a list of coordinates of pixels in the second entity.

An example of a transformation performed by the preceding steps is shown schematically in FIGS. 2 and 3. The source image is shown in FIG. 2; the destination image in FIG. 3. The transformation in this example is the product of a rotation and an enlargement. FIGS. 2 and 3 are schematic in nature, showing only the boundaries of the two images and a line of pixels in each. The image itself, as represented by the pixel values, is not shown.

The first step maps the vertices of the source image (A, B, C, and D in FIG. 2) to the vertices of the destination image (A', B', C', and D' in FIG. 3). That is, given the coordinates of the points A, B, C, and C, it calculates the coordinates of the points A', B', C', and D'.

The second step maps the edges of the destination image onto the edges of the source image. For example, it maps the pixel a.sub.1 in FIG. 3 to the pixel c.sub.1 in FIG. 2, and the pixel a.sub.n to the pixel c.sub.m. The mapping of vertex pixels is known from the preceding step; the mapping of other edge pixels can be calculated by a DDA algorithm, as will be described later. Preferably, for each edge of the destination image, only one pixel is mapped for each scan line meeting the edge, this being the pixel at the end of the scan line within the destination image. For example, although the pixel a.sub.1 is mapped to the pixel c.sub.1, the pixel a.sub.2 is not mapped in this step.

The third step maps pixels disposed on a scan line such as a.sub.1, a.sub.2, . . ., a.sub.n in the destination image to a line of pixels such as c.sub.1, c.sub.2, . . ., c.sub.m in the source image. The mapping of the end pixels a.sub.1 and a.sub.n is already known from the previous step; the mapping of interior pixels can be calculated by a DDA algorithm, as will be described later In this example, since the transformation comprises an enlargement, there are more destination pixels than source pixels, so the mapping is many-to-one. The pixels a.sub.2 and a.sub.3, for example, might both be mapped to the pixel c.sub.2.

The third step also transfers pixel values, the transfer being reverse in direction to the mapping direction. For example, the value of the pixel c.sub.1 is transferred to the pixel a.sub.1, the value of the pixel c.sub.2 is transferred to the pixels a.sub.2 and a.sub.3, and so on.

An advantage of the novel method is that every pixel in the destination image receives the value of a pixel in the source image. Another advantage is that the values of pixels on scan lines in the destination image are derived from lines of pixels in the source image in a natural. continuous way. This avoids image defects such as pixel drop-outs that occurred in the prior art.

A more detailed description of the first three steps of the novel method will next be given with reference to FIGS. 4 to 6. These drawings and the accompanying descriptions should be understood as exemplary and not restrictive in nature.

FIG. 4 is a flowchart illustrating step 1 in FIG. 1.

In the first step 11 in FIG. 4, parameters describing one or more affine transformation are received. The parameters may take the form of elements of matrices describing the transformations.

In the next step 12, the received parameters are combined to generate a product transformation. This can be done, for example, by multiplying the matrices to obtain a product matrix.

In the next step 13 the product transformation is applied to the vertices of the source image. For example, the coordinates of the vertices of the source image are multiplied by the product matrix, thus computing the coordinates of the vertices of the destination image. Methods of matrix multiplication are well known.

FIG. 5 is a flowchart illustrating step 2 in FIG. 1. More specifically, it illustrates the use of a type of DDA algorithm in carrying out step 2 in FIG. 1.

The first step 21 in FIG. 5 computes the coordinates of the ends of the scan lines in the destination image. The computation can be performed for each edge of the destination image by a process of linear interpolation between the two vertices joined by the edge. Details will be apparent to one skilled in the art.

The next step 22 computes an increment value. This computation can also be performed separately for each edge of the destination image. Specifically, the height of the edge is divided into the length the corresponding edge of the source image. For the edge A'B' in FIG. 3, for example, the height Y.sub.AB is divided into the lenght H of the corresponding edge AB in FIG. 2. The relevant formula is:

where y'.sub.A is the y-coordinate of vertex A', y'.sub.B is the y-coordinate of vertex B', and INCR is the increment value. Note that height refers to extent in the y-direction, perpendicular to the scan lines.

The next step 23 initializes a source coordinate. This computation can also be performed once for each edge of the destination image. If the corresponding edge of the source image is parallel to the y-axis, the source coordinate is initialized to the y-coordinate of a vertex of that edge, plus one-half the increment value determined in the preceding step. Again taking the edge A'B' in FIG. 3, which corresponds to the edge AB in FIG. 2, as an example:

where Y.sub.0 is the value to which the source coordinate is initialized and 1 is the y-coordinate of vertex A. If the corresponding edge of the source image is parallel to the x-axis, the x-coordinate is used instead of the y-coordinate.

The next step 24 iteratively updates the source coordinate by adding the increment value, thus generating a series of source coordinates Y.sub.q, where:

The next step 25 takes the integer parts of the updated source coordinates. Using brackets to represent the integer part of an arbitrary number, this can be written as:

The next step 26 obtains the coordinates of corresponding pixels on the edges of the source image according to these integer parts. That is, it supplements the integer parts of the source coordinates generated in the preceding step by supplying the other coordinate, which is constant for each edge of the source image. For the edge AB in FIG. 2 the other coordinate (x-coordinate) is 1, so pixels on the edge A'B' of the destination image at the ends of the scan lines are mapped to the pixels (1, y.sub.q) in source image.

FIG. 6 is a flowchart illustrating step 3 in FIG. 1. This flowchart also illustrates the application of a DDA algorithm.

The first step 31 in FIG. 6 selects an initial destination pixel on a scan lines in the destination image, such as the pixel a.sub.1 in FIG. 3.

The next step 32 calculates a pair of increment values by dividing the length of this scan line in the destination image into the directed lengths along the x-and y-axes of the corresponding line in the source image. In FIG. 3 the scan line under consideration extends from the pixel a.sub.1 to the pixel a.sub.n, and its length is denoted by the letter N. The corresponding line in FIG. 2 extends from the pixel c.sub.1 to the pixel c.sub.m, these two pixels being known from step 2 in FIG. 1. The lenth of this line along the x-axis is denoted by the letter U and is directed in the positive direction, since the pixel c.sub.1 is to the left of the pixel c.sub.m. Its length along the y-axis is denoted by the letter V and is directed in the negative direction, since the pixel c.sub.1 is above the pixel c.sub.m. The pair of increment values, denoted INCRX and INCRY, can be calculated as follows: ##EQU1## where x.sub.U-1 and x.sub.0 are the x-coordinates of the pixels c.sub.m and c.sub.1, respectively; x'.sub.N-1 and x'.sub.0 are the x-coordinates of the pixels a.sub.n and a.sub.1, respectively; and y.sub.V-1 and y.sub.0 are the y-coordinates of the pixels c.sub.1 and c.sub.m, respectively.

The next step 33 initializes a pair of source coordinates by adding half the values of the pair of increment values found in the preceding step to coordinates of a pixel at one end of the corresponding line in the source image. A further value of one is added to the y-coordinate. With x.sub.0 and y.sub.0 as above, a pair of initial source coordinates X.sub.0 and Y.sub.0 can be calculated as follows:

The next step 34 takes the integer parts of the pair of source coordinates. Initially these are [X.sub.0 ] and [Y.sub.0].

The next step 35 assigns to the destination pixel the value of the pixel in the source image at the location specified by the integer parts of the pair of source coordinates. Initially, [X.sub.0 ] and [Y.sub.0 ] are the same as x.sub.0 and y.sub.0, so the value of the pixel c.sub.1 is assigned to the pixel a.sub.1.

The next step 36 decides whether the destination pixel is the last pixel on the scan line in the destination image. For the scan line under consideration in FIG. 3, for example, the x-coordinate of the destination pixel can be compared with the x-coordinate of the pixel a.sub.n to make this decision.

In the next step 37, if the destination pixel is not the last pixel on the scan line, the pair of source coordinates are updated by adding the pair of increment values. The general formula for this is:

Then the next destination pixel a.sub.q on the scan line is selected and the process returns to step 34. Steps 34 and 35 now assign the value of the pixel with coordinates [X.sub.q ] and [Y.sub.q ] to the pixel a.sub.q.

The process from step 34 to 37 is repeated until an affirmative result is obtained in step 36, at which point the mapping of the scan line is completed. The entire process, starting from step 31, can then be repeated for succeeding scan lines, until all scan lines intersecting the destination image have been mapped.

Although FIG. 1 shows step 3 being performed after step 2, it is clear from the foregoing discussion that these two steps have certain parts in common and can be performed in parallel. For example, successive scan lines can be selected from the top of the destination image to the bottom. For a given scan line, the two pixels at the extreme edges of the destination image on that scan line and the corresponding two pixels in the source image can then be found as described FIG. 5, after which the procedure in FIG. 6 can be carried out to map the pixels in this scan line onto the source image and transfer the corresponding pixel values from the source image back to the destination image. Then the same can be done for the next scan line, continuing in this manner until the entire destination image has been mapped.

Finally, a novel image transformation device for carrying out the image transformation method of the invention will be described with refernced to FIG. 7.

The first element of this image transformation device is a source image memory 41 for storing the source image. The source image memory 41 can be, for example, a semiconductor memory device in which pixel values are stored at addresses corresponding to the x-and y-coordinates of the pixel.

The next element is a destination image memory 42 for storing said destination image. The destination image memory 42 can be a separate memory device similar to the source image memory 41. Alternatively, the source image memory 41 and the destination image memory 42 can both be parts of the same physical memory device. The destination image memory 42 should preferably be structured so that the values of pixels on a scan line in the destination image are stored at a consecutive series of addresses.

The next element is an affine transformation means 43 for performing an affine transformation on vertices of the source image to generate vertices of the destination image. The affine transformation means 43 may comprise, for example, a special-purpose arithmetic unit for performing matrix multiplication as discussed in relation to FIG. 4.

The next element is an edge mapping means 44 for mapping pixels on edges of the destination image to pixels on edges of the source image. The edge mapping means 44 may comprise, for example, a special arithmetic unit for executing the DDA algorithm described in FIG. 5.

The next element is a scan line mapping means 45 for mapping pixels on scan lines in the destination image to pixels in the source image. The scan line mapping means 45 may comprise, for example, a special arithmetic unit for executing the DDA algorithm described in FIG. 6.

The next element is a controller means 46 for controlling the affine transformation means 43, the edge mapping means 44, and the scan line mapping means 45, and for transferring values of pixels from the source image memory 41 to the destination image memory 42 as indicated by the scan line mapping means 45. The controller means 46 may comprise, for example, a special-purpose processor executing a microprogram structured as in FIG. 1.

The next element is a data bus 47 for interconnecting the source image memory 41, the destination image memory 42, and the controller means 46. The data bus 47 enables the controller means 46 to read and write pixel data at specified addresses in the source image memory 41 and the destination image memory 42.

The next element is a display means 48 for displaying images stored in the destination image memory 42. The display means 48 may comprise, for example, an interface circuit for reading pixel data from the destination image memory 42 in scan-line order and converting the pixel data to a video signal, and a cathode-ray-tube display for receiving the video signal and displaying an image by raster scanning.

A detailed description of the operation of the image transformation device in FIG. 7 will be omitted as already having been covered in FIGS. 4 to 6. However, a brief description will be given of the operation of the controller means 46.

The controller means initiates the image transformation process when it receives the coordinates of the vertices of the source image and a command specifying the transformation or transformations to be performed. These coordinates and command can be received from an external device such as a computer not shown in the drawing.

The controller means 46 is coupled to the affine transformation means 43, with which it exchanges information. The controller means 46 sends the affine transformation means 43, for example, the coefficients of a series of one or more affine transformation matrices and causes the affine transformation means 43 to multiply these matrices to generate a product matrix; then the controller means 46 sends the affine transformation means 43 the vertex coordinates of the source image, causes the affine transformation means 43 to multiply these by the product matrix, and receives the results, which it stores as the coordinates of the vertices of the destination image in a working memory not shown in the drawing.

The controller means 46 is also coupled to the edge mapping means 44, to which is provides, for example, the coordinates of the vertices of the source and destination images and from which it receives, for example, a list of coordinates of pixels disposed on edges of the destination image and coordinates of corresponding pixels disposed on edges of the source image.

The controller means 46 is also coupled to the scan line mapping means 45, to which it provides, for example, the coordinates of pixels at the ends of a scan line on the destination image, and from which it receives, for example, a list of coordinates of pixels in the source image corresponding to successive pixels on the scan line in the destination image. The controller means 46 then converts the received coordinates to addresses, accesses the source image memory 41 at these addresses, reads pixel values therefrom, and writes these values at successive addresses in the destination image memory 42.

The scope of this invention is not restricted to the specific methods shown in FIG. 4 to 6, but includes many modifications of these methods which will be apparent to one skilled in the art. In addition, the affine transformation means, edge mapping means, scan line mapping means, and controller means in FIG. 7 need not comprise special-purpose arithmetic and processor units, but may be software means comprising programs executed on a general-purpose computer, and the display means may comprise any type of raster output device, such as a laser printer, instead of a cathode-ray-tube display. Finally, although FIGS. 2 and 3 illustrated only a rotation followed by an enlargement, this invention is applicable to any type of affine image transformation.