Method and apparatus for reducing flicker in a video image sequence

[0003] 1. Field

[0004] The present invention relates to video image processing, and more particularly, to reducing flicker in a video image sequence.

[0005] 2. Background Information

[0006] FIG. 1 is a schematic diagram illustrating an arrangement for a typical video conferencing system. In this diagram, a user/subject is positioned in front of a camera 110 while looking at a computer monitor 120. The area or room is illuminated by a light source 130. Camera 110 is designed to reduce flicker due to light source 130. This may be accomplished by setting the aperture time of the camera to capture a whole number of illumination cycles for light source 130. In this context, aperture time refers to the length of time during which the camera is sensitive to incident light. Illumination cycle refers to the multiplicative inverse of the frequency of the variation of the intensity of the light source. This may also be accomplished by setting the camera frame rate such that the frame exposure starts at approximately the same point in the illumination cycle for each frame.

[0007] For the situation illustrated in FIG. 1, computer monitor 120 acts as a second light source. The refresh rate on the computer monitor depends on many variables and, quite possibly, does not correspond to one of the frequencies handled by the camera using the previously described methods. Due to this frequency mismatch, some frames, such as in an image sequence displaying the scene illustrated in FIG. 1, may get more illumination than other frames, resulting in flickering in the video sequence that may be annoying to a human observer. A need, therefore, exists for a technique to reduce flicker for video image sequences displaying situations or scenes involving a second light source, such as a computer monitor, for example.

[0008] Briefly, in accordance with one embodiment of the invention, a method of reducing flicker in a video image sequence includes the following. Successive video images are temporally filtered. The filter length is adjusted so as to reduce the amount of filtering across temporal discontinuities in the video image sequence.

[0009] Briefly, in accordance with another embodiment of the invention, a method of reducing flicker in a video image sequence includes the following. Successive video images are temporally filtered. The filter length is dynamically adjusted.

[0010] The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:

[0011] FIG. 1 is a schematic diagram illustrating a typical video conferencing system;

[0012] FIG. 2 is a schematic diagram illustrating an ideal band stop filter;

[0013] FIG. 3 is a plot illustrating a temporal filter that may be employed by an embodiment in accordance with the present invention.

[0014] FIG. 4 is a block diagram of a filter in accordance with one embodiment of the invention.

[0015] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in details so as not to obscure the present invention.

[0016] As previously indicated, FIG. 1 is a schematic diagram illustrating a typical video conferencing system set-up, although the invention is not limited in scope to being employed in connection with video conferencing. As illustrated in FIG. 1, a user/subject is positioned in front of a camera 110 while looking at a monitor 120, such as a computer monitor. The room or area is illuminated by a light source 130. The camera 110 is designed to reduce flicker attributable to the light source 130 in a video image sequence displaying the scene. This may be accomplished by a variety of techniques, as previously described. However, computer monitor 120 acts as a second light source. The refresh rate on the computer monitor may depend on many variables and, quite possibly, does not correspond to one of the frequencies handled by the camera using the previously described methods. Due to this frequency mismatch, some frames in the video image sequence may receive more illumination than other frames, resulting in flickering in the video image sequence that may be annoying to a human watching the video image sequence.

[0017] One embodiment of a method of reducing flicker in a video image sequence in accordance with the present invention includes the following. A succession or sequence of video images are temporally filtered, and the filter length is adjusted, as described in more detail hereinafter. To accomplish this temporal filtering, it may, in some instances, be desirable to determine the flicker frequency to be suppressed. This may be accomplished any one of a number of ways or, alternatively, instead of determining the flicker frequency, a user provided adjustment may be made until the amount of flicker is acceptable to the end user. However, if it is desirable to determine the flicker frequency, this may be determined, for example, from the monitor refresh rate and camera parameters, such as aperture time and frame rate, in some embodiments. This information could be provided by the operating system and/or by the hardware in one example embodiment. Alternatively, the flicker frequency may be determined from a frequency analysis applied to a sequence of sample frames produced by the camera. Techniques to do this are well-known and will not be discussed further here. In yet another alternative, a user interface control may be employed to obtain the flicker frequency from the user or, as previously indicated, without specifically providing a flicker frequency value, the user may modify the filtering capability of the filter and observe the effect on the displayed video image sequence and, through that technique, ultimately arrive at an acceptable amount of flicker.

[0018] In this particular embodiment, although the invention is not limited in scope in this respect, flicker is reduced by implementing a zero-delay band stop filter. FIG. 2 and 4 illustrate a schematic diagram of an idealized band stop filter designed to suppress the flicker frequency. Common techniques are available to generate, for example, a 2N+1 tap, linear phase, finite impulse response (FIR) filter with a stop band at the flicker frequency, where N is a positive integer. The FIR filter may be modified to reduce or eliminate delay by doubling the filter coefficients from previous samples and setting to zero the filter coefficients for future samples. The filter coefficient for the current sample is not changed. The result is an N+1 tap FIR filter with a stop band at the flicker frequency and substantially zero delay. Again, this will not be described in great detail here because techniques for filter design are well-known. Any one of a number of approaches may be employed to produce such a filter and software packages are commercially available to accomplish this. For example, “MatLab” (version 5.2) available from The Math Works, Inc. of Natick, Mass. may be employed, although, clearly, the invention is not limited to employing this commercially available product. It is also noted that it may be desirable to recalibrate the filters if the monitor refresh rate is changed or if the camera parameters, such as frame rate, are modified.

[0019] In one embodiment, the filter length may be adjusted to trade off the amount of observable image flicker and image ghosting, although the invention is not limited in scope in this respect. For example, where an N+1 tap FIR filter is employed, it may be applied to the current video frame and previous video frames on a pixel-by-pixel basis in the temporal direction to produce the current output frame, such as illustrated in FIG. 3, in one embodiment, for example. However, a non-adaptive application of this filter, while suppressing flicker, may also introduce a ringing artifact that appears visually as “ghosting”. The ringing or ghosting may be attributable, at least in part, to object motion which may produce a discontinuity in the temporal direction at a particular pixel location. Therefore, in this particular embodiment, to suppress the flicker without excessive ghosting, a buffer may be employed to store a filter length value for each pixel location. In one embodiment, the filter lengths are initially set to one. Likewise, if the absolute difference between the current frame pixel value and the previous frame pixel value is less than a predetermined threshold, the filter length for that pixel may be incremented. Otherwise, the filter length for that pixel is reset to one, for this embodiment. Thus, the filter length is adjusted to reduce the amount of filtering across temporal discontinuities in the video image sequence. The threshold value provides in this embodiment effectively a trade-off between flicker reduction and ghosting reduction, although the invention is not limited in scope in this respect, and may be adjusted. For example, as previously indicated, this may be a parameter that the end user may adjust in real time to observe the effects on a display or monitor. In another embodiment, it may be presumed that the flicker effect will change a particular pixel value by no more than plus or minus 15%, for example. Applying this value to a pixel value in the range of zero to 255 results in a threshold in the neighborhood of 38. Alternatively, some embodiments may employ the use of a threshold value which is a power of 2. This might, for example, result in a threshold of 32.

[0020] In this particular embodiment, the adaptive filter has a length n on a per pixel basis from one to N+1, where n is a positive integer. In one embodiment, a set of filters with lengths from one to N+1 may be generated with the appropriate filter being selected for a particular pixel based on the filter length value described above. All the filters may be designed for substantially the same stop band, but the longer filters, of course, should provide improved flicker suppression. Alternatively, in another variation, one N+1 tap filter may be generated and, for a filter of length n, the first n coefficients of the N+1 tap filter may be employed. The output signal of the shortened filter may then be appropriately scaled. Many other filter embodiments, of course, are also possible.

[0021] An embodiment of a computing platform, such as a personal computer, for example, may be adapted to implement an embodiment of a method for reducing flicker in a video image sequence in accordance with the invention, such as one of the embodiments previously described. Such a computing platform may be adapted to process video images. For example, a PC may include loaded software that when executed performs this result. The platform may be further adapted to temporally filter a video image sequence and to adjust the length of the temporal filter, such as dynamically. Likewise, in another embodiment, an article of manufacture may comprise a storage medium, such a disk, floppy, CD-ROM, or hard drive, for example, including instructions stored thereon that, when executed by a processor, result in the execution of an embodiment of a method for reducing flicker in a video image sequence in accordance with the invention, such as one of the previously described embodiments.

[0022] While certain features of the invention have been illustrated as described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.