US 6,166,773 AGrant
Method and Apparatus for De-Interlacing Video Fields to Progressive Scan Video Frames
Issue Date:2000-12-26
•4 Claims
•10 Drawing Sheets
Abstract
A method and apparatus are provided for converting interlaced video fields of arbitrary size to progressive scan format video frames of arbitrary size. The method and apparatus of the invention provide consistent conversion quality even while output video dimensions are scaled in real-time. According to a first aspect of the invention the space between output lines is calculated with respect to the input fields. According to a second aspect, the output line spacing information is used to calculate the final line positions in a progressive scan output frame.
Metadata
Assignee
- Genesis Microchip Inc.
Inventors
- Lance Greggain
- James Goel
Application Information
Application Number:US 9110593
Filing Date:1997-08-14
Priority Date:1995-11-08
Art Unit:271
Classifications
IPC:
H04N 701
Field of Search:
348448;458;452;459;1;44;581;714
Patent Drawings (10 sheets)
Description
Background of the Invention
1. Field of the Invention
This invention relates in general to video signal de-interlacing and interpolation. More particularly, the invention relates to a method and apparatus for properly interpolating and spatially positioning video lines from an arbitrarily sized video field to create an arbitrarily resized progressive scan frame.
2. Description of the Prior Art
Current state-of-the-art de-interlacing involves converting an interlaced scanned video signal into a progressive scan (sometimes referred to as sequentially scanned) video signal with exactly double or quadruple the number of output lines. The output size restriction makes it very difficult and expensive to de-interlace video fields for use in an application that requires an arbitrarily sized output progressive scan video window. The original video signal is composed of interlaced scanned odd and even fields. These fields are mapped into progressive scan frames through a process of spatial interpolation and positioning that produces exactly double or quadruple the number of input lines per field.
De-interlacing has many applications including slow motion playback, computer video-in-a-window displays and large screen projection television systems to name but a few. All of these applications require the high quality and performance provided by the generation of progressive scan frames using a de-interlacing process.
Producing high quality de-interlaced video signals is difficult. An object in one video field may be in a different spatial position in the next field which makes it very difficult to properly combined the fields into a single progressive scan frame. In such an application, advanced and expensive methods of temporal and vertical interpolation are required to give satisfactory results. One common prior art method of interpolation uses motion detector circuitry to identify areas of a video field which change over time. These changing areas are interpolated differently from static regions to ensure a high quality image.
The complexity and expense of motion detector circuitry and advanced interpolation circuitry restricts the flexibility and affordability of such prior art de-interlacing systems. These systems have been highly optimized to produce either double or quadruple the number of input lines per field, as discussed above.
Examples of well known prior art de-interlacing systems are described in the following publications and patents:
Publications
Watkinson, John, "The Art of DIGITAL VIDEO--2nd Edition", Focal Press 1994, pp. 180-185.
U.S. Patent Documents
U.S. Pat. No. 5,291,280 March 1994 Faroudja; Yves C.
U.S. Pat. No. 5,070,403 December 1994 Wilkinson, Basingstoke
Summary of the Invention
A general aspect of an object of the present invention is to provide a method and apparatus for creating arbitrarily sized output progressive scan frames of video from arbitrarily sized input interlaced video fields in a manner which overcomes the limitations and drawbacks of the prior art.
The method and apparatus of the present invention de-interlaces video by properly interpolating and spatially positioning each input field line at the correct output progressive scan line position based on the relative size of the input and output video frames. The process of the invention is broken down into two steps. The first step involves calculating the space between output progressive scan lines and then using this information to determine the correct spatial position of these lines. Once the position of the output lines is calculated, an image resizing engine is used to map the input lines to the appropriate output spatial position and store these lines in an external frame buffer.
This two-step process continuously calculates output line positions to provide proper de-interlacing for dynamically resized video signals.
Brief Description of Drawings
FIG. 1 is a schematic representation of a plurality of target lines derived from a plurality of source lines in magnification mode, wherein the number of lines in the input fields is increased to map to the number of the lines in the output frames;
FIG. 2 is a schematic representation of a plurality of target lines derived from a plurality of source lines in bypass mode, wherein the number of lines in the input fields is equal to the number of lines in the output frames;
FIG. 3 is a schematic representation of target lines derived from a plurality of source lines in reduction mode, wherein the number of lines in the input fields is decreased to map to the number of lines in the output frames;
FIGS. 4 and 5 are schematic representations showing details of even field resizing and odd field resizing, respectively, for the de-interlacing system of the preferred embodiment;
FIG. 6 is a block diagram showing the functional components of a first aspect of the de-interlacing system according to the preferred embodiment for calculating a Target Increment (TarInc) value;
FIG. 7 is a block diagram showing the functional components of a second aspect of the de-interlacing system that uses the TarInc value to generate a correct Target Line Spacing;
FIG. 8 is a flow chart showing operation of the de-interlacing system shown in FIGS. 6 and 7; and
FIG. 9 is a memory map diagram showing the placement of the output lines in an external output frame store, according to the preferred embodiment.
Detailed Description of the Preferred Embodiment
As discussed above, video de-interlacing is achieved in accordance with the present invention by continuously calculating the spatial position of the output lines and then mapping interlaced input fields to progressive scan output frames.
FIGS. 1, 2 and 3 each show two input fields (not to scale) which are referred to as the Even Input Field (lines Y0, Y2, Y4, Y6) and the Odd Input Field (lines Y1, Y3, Y5, Y7), and two output frames referred to as the Even Output Frame and the Odd Output Frame. All even input fields are mapped to even output frames and the odd input fields are mapped to the odd output frames. The variables in each of these figures is explained in Table A, below:
According to the present invention, the following relationship is implemented for determining the space between output lines with respect to the interlaced input lines:
The actual spatial position of the output lines depends on whether the input field is even or odd. An even input field has lines positioned in the even line spaces and an odd input field has lines positioned in the odd line spaces. The following relationship is implemented for determining the spatial position of the output lines:
where
P.sub.j=0 =0 for Even Input Fields
P.sub.j=0 =0.5 for Odd Input Fields
Magnification Mode
FIG. 1 shows a single input interlaced video frame containing a total of 8 lines numbered 0 to 7. In actual fact, standard interlaced video frames contain even and odd fields each having 240 active lines, for a total of 480 lines per frame. A reduced number of lines are depicted for ease of illustration. The even lines Y1, Y2, Y4 and Y6 are stored in the even field and the odd lines Y1, Y3, Y5 and Y7 are stored in the odd field.
Each de-interlaced output frame contains a total of 8 lines numbered 0 through 7. To map each input field (4 lines) to each output frame (8 lines) requires a magnification of 2 times. It is important to note that only the first line of the even field (Y0) maps to the first line of the even output frame (F0) and the last line of the odd field (Y7) maps to the last line of the odd output frame (F7). The remaining input lines are mapped to a fractional position with respect to the output frames.
The method and apparatus of the present invention correctly calculates the output line position depending on the user specified resize factor, as shown in greater detail with reference to FIGS. 4 and 5.
Since all of the even and odd field lines are equally spaced (e.g. A=1; S=4 and T=8 in the example of FIGS. 4 and 5) by mapping the input even field to the output even frame, then
and by positioning the input odd field lines between the input even field lines, then
Substituting (2-2) into (2-1) and solving for B: ##EQU1## The variable B is now equal to the space between output lines with respect to the interlaced input lines.
According to the next step in the method of the present invention, the proper spatial location of each output line is determined using 1-2, resulting in the spatial locations set forth in Tables B and C, for the even output frame (FIG. 4) and the odd output frame (FIG. 5), respectively.
The methodology discussed above with respect to magnification mode, can be used for both bypass and reduction modes.
Bypass Mode
Since all of the even and odd field lines are equally spaced (e.g. A=1; S=4 and T=4 from FIG. 2), by mapping the input even field to the output even frame, then
and by positioning the input odd field lines between the input even field lines, then
substituting (2-2) into (2-1) and solving for B: ##EQU2## According to the next step in the method of the present invention, the proper spatial location of each output line is determined using 1-2, resulting in the spatial locations set forth in Tables D and E for the even output frame and the odd output frame, respectively.
Reduction Mode
Since all of the even and odd field lines are equally spaced (e.g. A=1; S=4 and T=3 from FIG. 3), by mapping the input even field to the output even frame, then
and by positioning the input odd field lines between the input even field lines, then
Substituting 2-2 into (2-1) and solving for B ##EQU3##
According to the next step in the method of the present invention, the proper spatial location of each output line is determined using 1-2, resulting in the spatial locations set forth in Tables F and G for the even output frame and the add output frame, respectively.
FIG. 6 depicts a hardware embodiment for executing the process set forth in 1-1. The number of source field lines (S) is multiplied by 2 using a first multiplier 61, and then 1 is subtracted from the product using a first subtractor 63. The value 1 is subtracted from the number of target frame lines using a second subtractor 65, and the sum is multiplied by 2 using a second multiplier 67. The output of subtractor 63 is divided by the output of multiplier 67 using a divider 69, to produce the Output Target Increment (TarInc) value.
FIG. 7 depicts a hardware embodiment for executing the process set forth in 1-2. An accumulator is formed by the combination of register 71 and adder 73 with multiplexer 75 controlling the initial value. As the data from an even field is loaded into image resizing engine 77, the MuxSelect signal ensures that the value 0 is multiplexed into register 71 to initialize the accumulator to zero. As each new input line is loaded into the image resizing engine 77, the value of TarInc is accumulated to produce the target line spacing value required for an even output frame. When data from an odd field is loaded into the image resizing engine 77, the MuxSelect signal ensures that the value -0.5 is multiplexed into register 71 to initialize the accumulator. This ensures that all target line spacing values produced for an output frame are properly positioned. The Clk signal is used to synchronize all data movement in accordance with standard digital system design.
FIG. 8 depicts a flow chart illustrating the control system for the hardware embodiment of FIGS. 6 and 7. After the system is initialized (step 80), a TarInc value is calculated (step 82), as discussed in greater detail above. If the apparatus is processing an even field (step 803), the MuxSelect signal loads a 0 value into register 71 (step 804) and then loads the Target Line Spacing to the Image Resizing Engine 77 (step 806). If more lines must be processed (i.e. a NO decision at step 808), Adder 73 adds TarInc and Target Line Spacing from register 71 (step 810). MuxSelect now selects input A of multiplexer 75 (step 812) and the new Target Line Spacing is fed to the Image Resizing Engine (step 808). The odd field is processed the same way with respect to steps 803 and 814-822, except that an initial starting value of -0.5 is loaded in register 71 (step 814).
After both the even and odd fields have been processed, (step 824), then the system waits to process the next pair of input video fields relative to the next calculated Target-Line spacing.
Output frame lines must be stored at different locations in an external frame buffer. FIG. 8 shows storage of video lines from the output even frame starting at the top of the frame buffer. Lines from the output odd frame are stored in such a way that the last line is at the bottom of the frame buffer. These different storage requirements greatly simplify the top and bottom boundary conditions encountered while spatially positioning the output.
All of the hardware components depicted in FIGS. 6 and 7, and the frame buffer whose operation is discussed in FIG. 9, are of well known design. The image resizing engine is preferably as set forth in U.S. patent application Ser. No. 125,530 filed Sep. 22, 1993, the contents of which are incorporated herein by reference.
According to an alternative embodiment of the present invention, the steps of calculating the output target increment and output line spacing as set forth in the hardware embodiment of FIGS. 6 and 7, with reference to FIG. 8, may be implemented by a computer implementing appropriate software such as set forth in Appendix A to the present disclosure.
Other embodiments and variations of the invention are possible. For example, rather than having an integer number of lines per field or per frame, the apparatus of the present invention will operate equally well where a fractional number of lines are provided per field or per frame. All such embodiments are believed to be within sphere and scope as set forth in the claims appended hereto.
Claims
What is claimed is:
1. A video de-interlacing system for mapping one of odd or even input video fields each having a predetermined number of lines per field to a single output video frame having a predetermined number of lines per frame, comprising: a) a target increment calculator for receiving said predetermined number of lines per field and said predetermined number of lines per frame and in response generating a target increment value representing space between respective lines of said output video frame based on space between lines of said input video fields; b) a target spacing calculator for receiving said target increment value and for successive ones of said lines of said output video frame generating respective target line spacing values associated with said odd or even input video fields; and c) an image resizing engine for receiving successive ones of said lines of said odd or even input video fields and respective target line spacing values and in response generating successive lines of said output video frame, wherein said target spacing calculator introduces a spatial offset between said target line spacing values associated with said odd field and said even field; and wherein said spatial offset is equivalent to one-half of said space between lines of said input video fields.
2. The system of claim 1, wherein said target increment calculator further comprises: d) a first multiplier for multiplying said number of lines per field by two and in response generating a first product value; e) a first subtractor for subtracting one from said first product value and in response generating a first difference value; f) a second subtractor for subtracting one from said number of lines per frame and in response generating a second difference value; g) a second multiplier for multiplying said second difference value by two and in response generating a second product value; and h) a divider for dividing said first difference value by said second product value and in response generating said target increment value.
3. The system of claim 1, wherein said target spacing calculator further comprises an accumulator for receiving said target increment value and generating said respective target line spacing values as a running accumulated sum of said target increment value.
4. The system of claim 3, wherein said accumulator further comprises an adder, a multiplexer and a storage register, a first input of said adder being provided for receiving said target increment value, an output of said adder being applied to a first input of said multiplexer, a second input of said multiplexer being provided for receiving a first accumulator initialization value for even ones of said pair of input video fields, a third input of said multiplexer being provided for receiving a second accumulator initialization value for odd ones of said pair of input video fields, an output of said multiplexer being connected to an input of said storage register, and an output of said storage register being connected to a second input of said adder.
Patent Citations (22)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4698674 | 1987-10-01 | Bloom | |
| US4872054 | 1989-10-01 | Gray et al. | |
| US4937667 | 1990-06-01 | Choquet et al. | |
| US4947251 | 1990-08-01 | Hentschel | |
| US5070403 | 1991-12-01 | Wilkinson | |
| US5134480 | 1992-07-01 | Wang et al. | |
| US5289305 | 1994-02-01 | Lake, Jr. | |
| US5291280 | 1994-03-01 | Faroudja et al. | |
| US5305104 | 1994-04-01 | Jensen et al. | |
| US5349385 | 1994-09-01 | Glenn | |
| US5473382 | 1995-12-01 | Nohmi et al. | |
| US5532750 | 1996-07-01 | De Haan et al. | |
| US5532751 | 1996-07-01 | Lui | |
| US5559905 | 1996-09-01 | Greggain et al. | |
| US5579054 | 1996-11-01 | Sezan et al. | |
| US5610661 | 1997-03-01 | Bhatt | |
| US5619272 | 1997-04-01 | Salmon et al. | |
| EPX266079A2 | 1988-05-01 | ||
| GBX2151431A | 1985-07-01 | ||
| GBX2184628A | 1987-06-01 | ||
| GBX2197152A | 1988-05-01 | ||
| WOXWO95/08814 | 1995-03-01 |
Non-Patent Literature (3)
- John Watkinson, "The Art of Digital Video", Second Edition, 1990, pp. 180-185.
- Search Report for International Application No. PCT/CA 96/00699.
- Patent Abstracts of Japan, vol. 18, No. 674 (P-1846), Dec. 19, 1994 & JP,A,06 266319 (Hitachi), Sep. 22, 1994.