Correcting for keystone distortion in a digital image displayed by a digital projector

[0002] The present invention relates to digital projection of images and, more particularly, to correcting for keystone distortion in such projected images.

[0003] Image projection systems are used to project images onto a viewing screen. Examples of projection systems include overhead projectors, traditional slide projectors, and digital projectors. FIG. 1 depicts a digital projector 10 projecting an image onto a viewing screen 12, where the optical axis of the projector 14 is normal to the plane of the viewing screen. These systems are typically used by positioning the projector on a level surface (e.g., tabletop, desk, file cabinet, etc.), and projecting the image up at an angle above the plane of the surface, so that the view of the projected image is unobstructed by the audience. Referring to FIG. 2, when the projector 16 is positioned such that the optical axis 18 of the projection lens is at an angle 20 relative to the normal to the viewing screen 22, geometrical distortion is introduced to the projected image. Let the convention be that a positive angle represents a tilt above the normal to the viewing screen (positive tilt), and a negative angle represents a tilt below the normal to the viewing screen (negative tilt). In the case for positive tilt, the distance from the top of the image to the projection lens 24 is longer than the distance from the bottom of the image to the projection lens 26. The path length discrepancy results in a larger magnification at the top of the image than at the bottom, thus yielding an image that has a larger width at the top of the image than the bottom. The projected image is trapezoidal in shape and is referred to as “keystone” distortion since the distorted image resembles the shape of a keystone (FIG. 3a). In the case of negative tilt where the projected image is lower than the projector, e.g., ceiling mounted projectors, the distorted image becomes an inverted keystone shape, where the width of the top of the image is now smaller than the bottom (FIG. 3b).

[0004] There are several methods disclosed in the prior art addressing the correction for keystone distortion in projected images of projection systems. Among these are numerous patents that describe optical keystone correction implementations for fixed projection angles (see U.S. Pat. Nos. 5,283,602, 5,355,188 and 5,706,062). Although some effort has been made towards developing a projector with a variable optical keystone correction (U.S. Pat. No. 5,975,704), optical correction is generally limited to a fixed angle. In addition to the restricted correction angle, the more complicated optical design increases the cost in manufacturing and potentially increases size and weight. An alternative to the optical correction method that is now possible since the availability of digital projection systems is electronic or digital keystone correction.

[0005] Digital keystone correction is implemented by performing a compensation operation on the original digital image prior to projection. If the original digital image is distorted such that the amount of distortion is the inverse of the distortion introduced by the projection setup, then the final projected image will be rectilinear. One of the benefits of digital keystone correction is that additional optical elements are not needed since the correction is performed electronically. In addition, the correction applied to the original digital image can be varied according to the tilt angle and direction of tilt (positive and negative keystone correction). Despite the added flexibility with digital keystone correction, there are some limitations. Interpolation artifacts such as loss in sharpness and aliasing, which can manifest themselves as low frequency banding patterns and edge discontinuities, may be introduced as a result of digital keystone correction.

[0006] There exist in the prior art several patents that disclose different methods of implementing a digital keystone correction process in hardware (U.S. Pat. Nos. 5,764,311 and 5,795,046); however, these methods do not address the reduction of the interpolation artifacts. Kazutaka, et. al., teach a method of reducing edge discontinuity artifacts after keystone correction in Japanese Patent No. 09-247593; however, this method focuses on correcting for artifacts after they have been introduced by the keystone correction process. Once introduced into the original image information, it is very difficult to eliminate these artifacts without further degrading the image.

[0007] It is an object of the present invention to correct for keystoning distortion in digital images projected by a digital projector.

[0008] This object is achieved by a method for correcting for keystone distortion and reducing aliasing defects in a digital image for use by a digital image projector, comprising the steps of:

[0009] (a) receiving an original digital image;

[0010] (b) lowpass filtering the original digital image to provide a lowpass filtered digital image having reduced high frequency spatial components which contribute to aliasing defects introduced by digital keystone correction;

[0011] (c) processing the lowpass filtered digital image to provide for digital keystone correction; and

[0012] (d) sharpening the keystone corrected digital image.

[0013] The present invention provides a method that reduces the occurrence of aliasing defects prior to the digital keystone correction process, while preserving the sharpness of the original digital image.

[0014] It is an advantage of the present invention that it reduces aliasing defects while at the same time correcting for keystone distortion. A second advantage is that the loss of sharpness due to the interpolation process is compensated for in the present invention.

[0015] FIG. 1 is a prior art schematic showing a digital projector which projects an image onto a viewing screen;

[0016] FIG. 2 shows the prior art digital projector of FIG. I being tilted in a vertical direction;

[0017] FIGS. 3A and 3B depict two different prior art types of keystone distortion that are produced by the digital projector of FIG. 2;

[0018] FIG. 4 is a block diagram which depicts the overall operation of the method in accordance with the present invention;

[0019] FIG. 5 is a detail of Block 30 in FIG. 4; and

[0020] FIG. 6 shows an alternative projection configuration to FIG. 2.

[0021] Referring to FIG. 4, Block 28 depicts the application of lowpass filtering on the original digital image to provide a lowpass filtered digital image having reduced high frequency spatial components which contribute to aliasing defects introduced by the digital keystone correction. Aliasing is an artifact related to the high frequency spatial components and caused when the digital sampling is at a too low of a sampling frequency. This step can be represented mathematically as the convolution of the original digital image, I, and a lowpass filter, K_LP:

[0022] I_B=I*K_LP  (1)

[0023] wherein I^Bis the lowpass filtered digital image. A suitable choice for a lowpass filter is:

[0024] {fraction (1/16)}·[−1 4 10 4 −1].  (2)

[0025] The application of a lowpass filter to a digital image is well known to those skilled in the art and therefore need not be described in further detail here.

[0026] Block 30 depicts the application of the digital keystone correction step, and it is expanded in FIG. 5 to show four subsequent processing steps. Referring to FIG. 5, Block 34 depicts the computation of the magnification correction factor as a function of row. This can be accomplished by assuming a simple thin lens model and using the ABCD Matrix ray tracing method to calculate the location of the ideal image plane and the system magnification at the ideal image plane. It is assumed that parameters such as the object size, object-to-lens distance, and the focal length of the projection lens are known. The term object in this discussion refers to the light modulator (e.g. LCD panel) in the projector. It will be understood by one skilled in the art that the use of the thin lens approximation is just one method of modeling the optical system. Alternative approaches to modeling the optical system can be used arrive at the necessary parameters.

[0027] FIG. 6 depicts an alternative representation of the projection configuration shown in FIG. 2. The keystone angle 42 is now represented by the angle formed between the tilted image plane 44 and the ideal image plane 46. Note that angle 42 in FIG. 6 is mathematically equivalent to angle 20 in FIG. 2. The image heights 48 and 50 can be computed using trigonometry. These image heights can then be used to compute the respective magnifications at the tilted plane. The magnification change as a function of row is determined by assuming that the magnification varies linearly from the top to the bottom of the image. The magnification correction factor for a particular row is the inverse of the magnification at that row, normalized to the minimum magnification value for all rows. It can be expressed as: $\begin{array}{cc}{\mathrm{cor\_mag}}_i=\frac{1}{{\mathrm{mag}}_i}·\left(\mathrm{min\_mag}\right)& \left(3\right)\end{array}$

[0028] wherein cor_mag_iis the correction factor at row i, mag_iis the magnification at row i, and min_mag is the minimum magnification over all rows in the keystone distorted image.

[0029] Referring again to FIG. 5, Block 36 depicts the process of padding the lowpass filtered image with columns of zeros at the left and right edges of the image. This step is necessary to avoid edge effects in the interpolation process. A suitable choice for the number of columns of padding is 20 columns per side. The padded, lowpass filtered digital image will hereinafter be referred to as the “padded digital image”.

[0030] Block 38 depicts the computation of the interpolation vector using the magnification correction factors computed in Block 34. The interpolation vector is a vector containing new pixel locations for a row of data that has been appropriately scaled by the corresponding magnification correction factor. The new pixel location for a pixel in a particular column and row can be calculated by the following equation: $\begin{array}{cc}{\mathrm{cor\_xI}}_{\mathrm{ij}}=\left(\frac{1}{{\mathrm{cor\_mag}}_i}\right)·\left({\mathrm{cor\_x}}_j-\mathrm{cor\_zero}\right)+\mathrm{cor\_zero}& \left(4\right)\end{array}$

[0031] wherein cor_xI_ijis the new pixel location for the j-th pixel in the i-th row, cor_mag_iis defined by equation (3), cor_x_jis the column location of the j-th pixel in the padded digital image, and cor_zero is an offset value corresponding to the center of the padded digital image.

[0032] Finally, Block 40 in FIG. 5 depicts the interpolation process using the interpolation vectors computed in Block 38. The padded digital image data is used to compute the new pixel values at the scaled pixel locations specified by the interpolation vector on a row by row basis. Each row of the image data is effectively compressed or expanded in length depending on the value of the magnification correction factor. The padded digital image data is resampled at the spacing determined by the interpolation vectors. The data can be resampled using various methods, e.g., linear, cubic, or cubic spline interpolation. A suitable choice for the resampling is cubic spline interpolation. The output from Block 40 is a keystone corrected digital image.

[0033] The final step in the keystone correction process outlined in FIG. 4 is image sharpening as depicted by Block 32. This step can be represented mathematically by the following expression:

[0034] I_S=I_KC+G·(I_KC*K_HP)  (5)

[0035] wherein I^KCis the keystone corrected digital image, G is a gain factor, and K_HPis a highpass filter. A suitable choice for a highpass filter is: $\begin{array}{cc}\frac{1}{4}·\left[\begin{array}{ccccc}-1& 0& 6& 0& -1\end{array}\right]& \left(6\right)\end{array}$

[0036] and a suitable value for the gain factor is:

[0037] G=1.  (7)

[0038] A look-up table (LUT) can be used to reduce some of the sharpening artifacts such as “ringing” at edges. This LUT is used to remap the values from the second term in equation (5), which will be referred to as the edge_boost term. A suitable table shape is a piecewise linear curve with: $\begin{array}{cc}\mathrm{slope}=\{\begin{array}{cc}\frac{100}{128},& \mathrm{for}\mathrm{edge\_boost}\le 0;\\ \frac{40}{128},& \mathrm{for}\mathrm{edge\_boost}>0;\end{array}& \left(8\right)\end{array}$

[0039] The entire sharpening process can be expressed as:

[0040] I_S=I_KC+LUT[G·(I_KC*K_HP)]  (9)

[0041] wherein LUT is the look-up table, and the remaining terms are defined in equation (5).

[0042] It will be understood by one skilled in the art that the present invention can be extended so that two-dimensional keystone correction can be achieved by two orthogonal, one-dimensional keystone corrections. The lowpass filtering and sharpening steps can also be also be extended to two-dimensions accordingly.

[0043] The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.

[0044] 10 untilted digital projector relative to the normal of the viewing screen

[0045] 12 the viewing screen

[0046] 14 optical axis of the untilted digital projector 10

[0047] 16 tilted digital projector relative to the normal of the viewing screen

[0048] 18 optical axis of the tilted digital projector 16

[0049] 20 angle between the optical axis of the tilted projector and the normal to the viewing screen

[0050] 22 normal to the viewing screen

[0051] 24 distance from the top of the projected image to the tilted projector lens

[0052] 26 distance from the bottom of the projected image to the tilted projector lens

[0053] 28 block depicting application of a lowpass filter to the original digital image

[0054] 30 block depicting application of the keystone correction to the lowpass filtered digital image

[0055] 32 block depicting application of sharpening to the keystone corrected image

[0056] 34 block depicting computation of magnification correction factors

[0057] 36 block depicting application of image padding

[0058] 38 block depicting computation of the interpolation vectors

[0059] 40 block depicting interpolation on a row by row basis using the interpolation vector for keystone correction

[0060] 42 angle between tilted image plane and ideal image plane

[0061] 44 tilted image plane

[0062] 46 ideal image plane

[0063] 48 distance between the bottom of the projected image and the optical axis of the untilted projector

[0064] 50 distance between the top of the projected image and the optical axis of the untilted projector