Method and system for eliminating artifacts in overlapped projections

[0002] 1. Field of the Invention

[0003] The present invention generally relates to a projection system, and more particularly to a method and system for eliminating artifacts in overlapped projections.

[0004] 2. Description of the Related Art

[0005] Recently, large display walls have been constructed using multiple projectors that each project their images onto a large screen. Typically, each projector in such a system provides parts of the image, thus composing the image from several “tiles”.

[0006] To avoid visible seams (e.g., discontinuities) in the composite image, projectors are typically arranged such that their projected images slightly overlap. Pixels in the overlapping areas are attenuated such that their relative contributions add up to 1. Thus, each projector provides a certain percentage (e.g., a relative contribution) of the final pixel color/intensity. Thus, ideally (e.g., in theory), such attenuation makes the overlap areas invisible to a viewer. That is, black pixels (e.g., having a value of 0) must be produced to avoid such discontinuities. Hence, imagine that in the overlap area, one projector provides 100 percent of the intensity. Then, the other projector must generate 0 percent intensity (i.e., an ideal black pixel). Such situation occurs at the edges of the overlap area (e.g., see FIG. 2)).

[0007] Unfortunately, “real-world” (e.g., practical) projectors cannot produce perfectly black pixels. That is, although the input value for a given pixel may be 0, the projector still emits a small amount of light for those pixels. This is due, for example, to imperfections in the light valves used in liquid crystal display (LCD) projectors or residual phosphorescence in cathode ray tube (CRT) projectors.

[0008] As a result of such projections, seams are visible where the “tiles” (e.g., images) generated by different projectors overlap. Such seams are visually distracting and can potentially convey information to the viewer that is not part of the image but purely an artifact of the overlapping tiles.

[0009] FIG. 1 illustrates a concrete example of the above-mentioned problem. That is, a conventional system uses multiple projectors 101-106 that project their respective images 107-112 onto a common projection screen 100. These images 107-112 are also referred to as “tiles”. Separate processors/computers 113-118 provide the pixel information to each of the projectors. Computers 113-118 can be totally independent of each other, or can be in communication with each other over a network, or can be connected to a central controller that determines their operation.

[0010] To ease registration of the tiles, the tiles produced by the projectors typically overlap. Depending on the number of projectors and their overlap, this creates areas that are illuminated by one, two or four projectors 119-121. Each projector modulates the intensities of the pixels in the overlap areas, in order to provide a smooth transition between pixels provided by the contributing projectors.

[0011] FIG. 2 illustrates two out of many possible blending functions (e.g., linear roll-off and cosine-square roll-off). For pixels in the overlap area, the pixel value is multiplied by the blending function. As can be seen from FIG. 2, the blending functions used by neighboring projectors add up to 1, thus creating the visual impression of a smooth transition between tiles.

[0012] FIG. 3 shows an example of pixels positioned along a scanline. The graph 301 shows the computed pixel intensities for the scanline. Graphs 302 and 303 illustrate the pixel intensities after ideal modulation with the blending functions for the two projectors. Finally, graph 304 shows the sum of the modulated pixels. The sum of the modulated pixels is the same as the computed pixel values.

[0013] FIG. 4 shows the relationship between the specified pixel value and the displayed intensity (e.g., computed value vs. actual/real displayed value). Graph 401 shows an ideal mapping of pixel values to intensities, which graph 402 shows a more realistic mapping. It is noted that even an input pixel value of 0 results in a non-zero intensity on the display. Further, real mappings from pixel values to intensities also are frequently non-linear.

[0014] FIG. 5 shows the non-ideal case. FIG. 501 shows again the computed pixel intensities for the scanline. Graphs 502 and 503 show the effect of a non-perfect display of black pixels (e.g., even black pixels have some non-zero intensity in that pixel intensities will not assume a value of 0 in spite of a blending function value of 0. That is, the pixels do not assume a value of 0 because of the imperfection in the projector (e.g., leaking liquid crystal displays (LCDs) or persistence of phosphor in CRTs). Graph 504 shows the effect of summing up the modulated pixel values. It is noted that the pixel intensities in the overlap area are too large, thereby creating the effect of visible seams in the composite image.

[0015] Thus, hitherto the present invention, the conventional systems suffer from the problem of producing visible seams which is annoying and in which the viewer may be given wrong information as a result of such increased intensity.

[0016] In view of the foregoing and other problems of the conventional methods and structures, an object of the present invention is to provide a method and structure in which the artifacts such as seams are eliminated.

[0017] In a first aspect of the invention, a method of eliminating visible artifacts in overlapped projections produced by a plurality of projectors, includes in areas outside of the overlap areas, raising an intensity of first predetermined color pixels to match a level of the first predetermined color in the overlap areas, and scaling values of second predetermined color pixels in the overlap areas such that the values do not exceed the maximum pixel value of a single projector of the plurality of projectors.

[0018] In a second aspect of the present invention, a projection system includes a display screen, a plurality of projectors each for producing a respective image portion for being displayed on the screen, a plurality of image portions forming a composite image having a plurality of pixels, the plurality of image portions having overlapping image areas, and means, coupled to the projectors, for adjusting an intensity of predetermined ones of the pixels across the screen to achieve uniform intensity of the predetermined pixels.

[0019] In another aspect of the present invention, a signal-bearing medium incorporating the inventive method is provided.

[0020] With the unique and unobvious features of the invention, visible seams and artifacts are efficiently and reliably eliminated. Further, while the invention can be implemented in a combination of software and hardware, the present invention also may be advantageously implemented in software only. Thus, in such a case, the present invention would not require any additional hardware.

[0021] The foregoing and other purposes, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:

[0022] FIG. 1 illustrates a concrete example of a conventional system having a problem of visible seams and artifacts;

[0023] FIG. 2 illustrates several exemplary conventional blending functions (e.g., linear roll-off and cosine-square roll-off) for reducing artifacts and seams;

[0024] FIG. 3 shows an example of pixels positioned along a scanline, with graph 301 illustrating computed pixel intensities for the scanline, graphs 302 and 303 illustrating the pixel intensities after ideal modulation with the blending functions for the two projectors, and graph 304 showing the sum of the modulated pixels;

[0025] FIG. 4 illustrates a relationship between the specified pixel value and the displayed intensity, with graph 401 showing an ideal mapping of pixel values to intensities, and graph 402 showing a more realistic mapping;

[0026] FIG. 5 shows a non-ideal case, in which graph 501 shows again the computed pixel intensities for the scanline, graphs 502 and 503 show the effect of a non-perfect display of black pixels, and graph 504 shows the effect of summing up the modulated pixel values;

[0027] FIG. 6 illustrates mapping of input pixel values to output pixel values according to a first preferred embodiment of the method of the present invention;

[0028] FIG. 7A illustrates an exemplary flow diagram of a calibration process according to the method of the first preferred embodiment of the present invention;

[0029] FIG. 7B illustrates an exemplary flow diagram of a pixel mapping process according to the method of the first preferred embodiment

[0030] FIG. 8 illustrates a block diagram of an exemplary system for mapping input pixel values to output pixel values; and

[0031] FIG. 9 illustrates a storage medium for storing steps of the program for eliminating visible artifacts in overlapped projections.

[0032] Referring now to the drawings, and more particularly to FIGS. 1-6, there are shown preferred embodiments of the method and structures according to the present invention.

[0033] To eliminate the above-mentioned problem and other problems of the conventional systems and methods, the invention generally takes the following basic approach.

[0034] First, in areas outside of the overlap areas, the intensity of black pixels is raised to match the black level in the overlap areas. Thereafter, the pixel values are scaled in the overlap areas such that they never exceed the maximum pixel value of a single projector. In practice, the maximum pixel value is usually limited to either 1.0 or 255 (e.g., for 8 bits). It is noted that the above-described sequence may be reversed. That is, the scaling of the white pixels may be performed first, and then in areas outside of the overlap areas, the intensity of black pixels may be raised to match the black level in the overlap areas.

[0035] Hereinbelow, the following conventions and symbols will be used:

[0036] It is noted that “pixel value” is different from “intensity”. That is “pixel value” refers to the input to the system, whereas the intensity is the luminance/brightness of the pixels displayed on the display screen (e.g., what is shown on the display). Pixel values are computed by the application at the graphics subsystem and written into the frame buffer.

[0037] Since up to 4 tiles are overlapping at a given pixel, the maximum intensity of a black pixel is 4B. The present invention aims to establish a uniform intensity of black pixels across the entire screen. Therefore, black pixels that are covered by only one or two tiles, must also be displayed with the intensity 4B.

[0038] In areas covered by only a single tile, the pixel value of black pixels must therefore be set to b₄. For pixels covered by two tiles, black pixels output by a single projector must each have the intensity 2B, i.e. their corresponding pixel values are b₂.

[0039] Similarly, it must be ensured that fully illuminated pixels (e.g., white pixels) have the same intensity everywhere on the projection screen. This intensity is determined by the maximum pixel intensity that a single projector can produce (e.g., pixels with a value of w₁). White pixels in overlap areas must not exceed this intensity.

[0040] Therefore, in overlapping areas, the maximum pixel values must be limited such that the sum of all pixel intensities provided by the projectors does not exceed this maximum.

[0041] For example, in an area of 4 overlapping tiles where one projector displays a white pixel and the other 3 projectors produce black pixels, the white pixel's intensity must not exceed W-3B, because each of the three (3) black pixels contributes light of intensity B.

[0042] The corresponding pixel values are tabulated below:

[0043] Consequently, the range of pixel values from 0 . . . w₁, must be mapped to b₄. . . w₁, b₂. . . w₂, and b₁. . . w₄respectively. Hereinbelow, linear mapping functions m₁, m₂and m₄(e.g., for 1, 2, and 4 overlapping tiles, respectively) are provided that compute an output pixel value for a given input pixel value p: $\begin{array}{cc}{m}_1\left(p\right)=\frac{w_1-b_4}{w_1}p+b_4{m}_2\left(p\right)=\frac{w_2-b_2}{w_1}p+b_2{m}_4\left(p\right)=\frac{w_4-b_1}{w_1}p+b_1& \left(1\right)\end{array}$

[0044] FIG. 6 illustrates the mapping of input pixel values to output pixel values according to the present invention. These mappings ensure that black and white pixels are displayed with uniform intensity across the entire screen, including overlap areas.

[0045] FIG. 7A illustrates an exemplary flow diagram of a calibration process according to the method of the first preferred embodiment of the present invention.

[0046] That is, in the inventive method 700, as shown in FIG. 7, in step 701, the overlap areas and the pixel values of the pixels in the overlap areas and the non-overlapped areas are detected and measured.

[0047] In step 702, in areas outside of the overlap areas, the intensity of black pixels is raised to match the black level in the overlap areas. Afterwards, in step 703, the pixel values are scaled in the overlap areas such that they never exceed the maximum pixel value of a single projector. Once again, it is noted that steps 702 and 703 may be reversed and the calibration effect can still be achieved. Thus, the calibration process is performed in a two-stage process for each projector, as desired and as deemed necessary by the user.

[0048] Referring to FIG. 7B, the process of remapping the pixel values across the entire screen is shown.

[0049] The process ends with a uniform intensity achieved across the display, without visible seams or artifacts.

[0050] Referring to FIG. 7B, an exemplary process is shown for performing a pixel remapping function, after the calibration process of FIG. 7A has been performed. That is, all of the pixel values are remapped across the entire display screen.

[0051] In FIG. 7B, in step 705, for every pixel displayed on the screen, it is determined in what area the displayed pixel is located (e.g., overlap portion or non-overlap portion).

[0052] In step 706, a mapping function is applied. Such a mapping function preferably linear mapping functions including a scaling and an offset operation. Such linear mapping functions include, for example, alpha blending and color tables, as described in further detail below.

[0053] In step 707, the newly-mapped pixels are displayed on the screen.

[0054] Thus, the method according to the present invention includes a calibration process and an actual pixel adjustment. That is, the manipulation of predetermined color (e.g., black/white) values is used during calibration to set up the transfer function. The transfer function is then used to remap any possible pixel value (e.g., not just black or white).

[0055] The above-mentioned steps are described in further detail below.

[0056] FIG. 8 illustrates a block diagram of an exemplary system 800 for mapping input pixel values to output pixel values.

[0057] As shown in FIG. 8, a processor 801 (e.g., any of processors 113—118 shown, for example, in FIG. 1) computes an image, and provides graphics commands to a graphics subsystem 802. For example, the image computed by the processor and provided to the graphics subsystem 802, may be in the form of a plurality of triangles or the like.

[0058] Based on the received graphics commands, the graphics subsystem 802 divides the “triangles” (image) into, for example, a plurality of pixels and associated pixel values, and provides an output representative thereof to a pixel remapper 803.

[0059] The pixel remapper 803 remaps all of the pixels of the image based on the parameters of the remapping function set up during the calibration step (e.g., raising the intensity of black pixels and scaling the white pixels as described above).

[0060] It is noted that the pixel remapper 803 may include therein an alpha blending capability or other similar mapping function (e.g., color tables described below).

[0061] Then, the remapped pixel values are input to a digital-to-analog (D/A) converter 804 which converts the digital remapped pixel values to analog pixel values. The analog pixel values are input to a projector 805, which in turn outputs the pixel intensities onto the image display screen.

[0062] Returning to FIG. 6, the mapping of input pixel values to output pixel values is shown according to the present invention. As mentioned above, these mappings ensure that black and white pixels are displayed with uniform intensity across the entire screen, including the overlap areas.

[0063] However, due to the non-linear transfer functions of physical projectors, there maybe slight variations in intensity for intensities between black and white. Using mapping functions that match the projectors transfer function could alleviate such variations. Another possibility to eliminate such variations is to rely on a gamma correction table in the projector or the graphics adapters driving the projector that linearizes its transfer function.

[0064] The exact transfer function is often not readily available to a user. Therefore, the true values of the parameters w_iand b_imust be determined experimentally. The following describes a simple procedure for such an experimental determination.

[0065] First, the mapping functions (e.g., m_i(p)=p), are disabled. Then, the black pixels are displayed with all projectors (e.g., p=0 for all pixels (x,y)).

[0066] Thereafter, the value of pixels are incremented in areas with two (2) overlapping tiles until their intensity matches the intensity of pixels in areas with four (4) overlapping tiles. This is pixel value b₂.

[0067] The value of pixels are incremented in areas with no overlapping tiles until their intensity matches the intensity of pixels in areas with four (4) overlapping tiles. This is pixel value b₄.

[0068] Then, white pixels are displayed with all projectors (e.g., one projector at a time, while the other projectors are set to black; that is, adjust the overlap areas for one projector, then proceed to a next projector and perform similar operations) (e.g., p=w) for all pixels (x,y), and the value of pixels are decremented in areas with 2 overlapping tiles until their intensity matches the intensity of pixels with no overlapping tiles. This is pixel value w₂.

[0069] Thereafter, the value of pixels are decremented in areas with four (4) overlapping tiles until their intensity matches the intensity of pixels with no overlapping tiles. This is pixel value w₄.

[0070] Finally, using the values w_iand b_i, the mapping functions m_iare established.

[0071] The mapping functions computed by equations (1) and set-up in the final step above, involve both a scaling and an offset operation. There are several ways to implement the mapping functions depending on the facilities offered by the underlying hardware and graphics libraries. Hereinbelow, two possibilities are described (e.g., color tables and alpha blending).

[0072] Provided different color maps can be defined for different screen areas, the use of color tables is a straightforward way to implement the mapping functions (e.g., see J. Foley et al., “Computer Graphics: Principles and Practice”, 2^ndEdition, Addison Wesley, 1990). A color table (e.g., possibly one or each color channel, as described below) is set up such that for each input pixel value it produces the desired output pixel value. Unfortunately, few graphics adapters offer multiple independent color maps.

[0073] Alpha blending is a technique of combining two pixels of different color. Typically, one of the pixels is stored in a frame buffer and the image generation process (e.g., rendering) produces the other pixel (e.g., see “OpenGL Reference Manual”, 2^ndEdition, R. Kempf and C. Frazier (eds.), Addison Wesley, 1998.).

[0074] Each pixel is described by an intensity (or color value, as described below) and an alpha value. The alpha value is in the range 0, . . . 1, and represents (e.g., can be interpreted as) the opacity of a pixel. The new pixel value stored in the frame buffer is computed by combining the incoming pixel intensity and alpha with the stored pixel intensity and alpha: p:=k₁p+k₂q, where p is the stored pixel intensity/alpha and q is the incoming pixel intensity/alpha. The factors k₁and k₂are arbitrary scale factors between 0 and 1. Usually, the resulting incoming pixel value p is clamped to the range 0, . . . w, i.e., all values exceeding the clamp values are limited or cut to the clamp value.

[0075] The process of alpha blending is used to implement the mapping functions as follows. It is noted that each mapping function is in the form ap+b (see equation (1) above).

[0076] First, the image is generated using a rendering algorithm of choice. Now, the frame buffer contains at every pixel the unmapped pixel intensity. Then, the blending function p·α_q+q·1 (e.g., k1=α_pand k₂=1) is set up. This blending function computes the new pixel value by first multiplying the current pixel value with the alpha-value of the incoming pixel and then adding the new pixel value.

[0077] Finally, a rectangle is rendered for every screen region (e.g., one for each region that has 1, 2 or 4 overlapping tiles). For each rectangle, the alpha value is set to the corresponding scale factor a and the color value to the offset value b (e.g., for a region with 2 overlapping tiles α=(w₂−b₂)/w₁and b=b₂). After rendering the rectangles, the frame buffer contains the properly mapped pixel values and can be projected to the screen.

[0078] Regarding color, typically images are represented with color pixels. The description of the method of the present invention thus has been only concerned with single-valued pixels (e.g., grey-scale images). Color images are typically composed of pixels with 3 components or color channels (e.g., red, green, and blue (RGB)). The method described above can be applied by creating the image as three separate grey-scale images and applying to the mapping function to each of the color channels.

[0079] The method of the present invention introduces small color shifts into the image due to the offset in color values introduced by the mapping functions. For small values of b₀, b₂, and b₄, these errors will be small.

[0080] The offset added to the original pixel value by the mapping functions is added uniformly to all color channels. This is equivalent to adding some white to the pixel color (e.g., the pixel color will become more pastel). For example, assume an input pixel with an RGB value of (0.05, 0.0, 0.0), and assume b₄=0.01 and w₁=1.0. Then, m₁(p)=0.99 p+0.01 (e.g., the pixel gets mapped to the RGB value (0.0495, 0.01, 0.01)). The original dark red pixel was mapped into a slightly darker and less saturated red.

[0081] Obviously, this effect will only be significant if the intensity of black pixels B is high. The relative error will be greatest for dark pixels. Fortunately, the human visual system is less sensitive to color differences at low intensities, so that the error will have minimal visual impact.

[0082] As shown in FIG. 9, in addition to the hardware and process environment described above, a different aspect of the invention includes a computer-implemented method for eliminating visible artifacts in overlap projections produced by a plurality of projectors, as described above. As an example, this method may be implemented in the particular hardware environment discussed above.

[0083] Such a method may be implemented, for example, by operating a central processing unit (CPU) included in the processor(s)/computer(s) shown in FIG. 8, to execute a sequence of machine-readable instructions. These instructions may reside in various types of signal-bearing media.

[0084] Thus, this aspect of the present invention is directed to a programmed product, comprising signal-bearing media tangibly embodying a program of machine-readable instructions executable by a digital data processor incorporating the CPU and hardware above, to perform a method of eliminating visible artifacts in overlap projections produced by a plurality of projectors, as described above.

[0085] This signal-bearing media may include, for example, a RAM (not shown) contained within the CPU, as represented by the fast-access storage for example. Alternatively, the instructions may be contained in another signal-bearing media, such as a magnetic data storage diskette 900 (FIG. 9), directly or indirectly accessible by the CPU.

[0086] Whether contained in the diskette 900, the computer/processor 801/CPU, or elsewhere, the instructions may be stored on a variety of machine-readable data storage media, such as DASD storage (e.g., a conventional “hard drive” or a RAID array), magnetic tape, electronic read-only memory (e.g., ROM, EPROM, or EEPROM), an optical storage device (e.g. CD-ROM, WORM, DVD, digital optical tape, etc.), paper “punch” cards, or other suitable signal-bearing media including transmission media such as digital and analog and communication links and wireless. In an illustrative embodiment of the invention, the machine-readable instructions may comprise software object code, compiled from a language such as “C”, etc.

[0087] While the invention has been described in terms of a preferred embodiment, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.