Motion-adaptive interpolation apparatus and method thereof

[0001] 1. Field of the Invention

[0002] The present invention relates to a de-interlacing of an image sequence, and more particularly, to a motion-adaptive interpolation apparatus and method for accurately performing a de-interlacing.

[0003] 2. Description of the Related Art

[0004] In general, an interlaced scan method has an advantage that a resolution in a vertical direction can be doubled even without raising a refresh rate of a screen, so that the resolution can be effectively improved with a limited frequency band.

[0005] However, the interlaced scan method has a disadvantage that it causes a screen jitter or a line crawl phenomenon that the screen moves up and down slowly.

[0006] Especially, in a still picture without a motion, the trembling of screen is very unpleasant to a user. A progressive scan method has been introduced to solves this problem.

[0007] For example, the NTSC interlaced scan method is performed by scanning an electron beam 60 times a second over the entire screen having 525 lines, whereby the electron beam is scanned 30 times to create 262.5 even lines, and remaining 30 times to create 262.5 odd lines. Accordingly, in terms of fields, 60 fields are reproduced in 1 second, while in terms of frames, 30 frames are reproduced in 1 second. On the other hand, the progressive scan method reproduces 60 frames in a second by scanning 525 scan lines at once.

[0008] A technique for converting an image sequence of an interlaced scan method to an image sequence of a progressive scan method is called a de-interlacing technique.

[0009] Conventional de-interlacing techniques are disclosed in U.S. Pat. No. 4,876,596 (Film-to-video converter with scan line doubling: Faroudja), U.S. Pat. No. 5,550,592 (Film mode progressive scan conversion; Markandey et al.), U.S. Pat. No. 5,563,651 (Method and apparatus for identifying video fields produced by film sources employing 2—2 and 3-2 pull down sequences; Christopher et al.), U.S. Pat. No. 5,596,371 (Film-mode video line-doubler motion detectors; Pakhchyan et al.), U.S. Pat. No. 5,689,301 (Method and apparatus for identifying video fields produced by film sources; Christopher et al.). Interpolation methods of the above techniques can be summarized by five methods as follows.

[0010] A first method is to interpolate by repeatedly using line information of the current field itself.

[0011] A second method is an interfield interpolation method that a very previous field data is inserted as it is between the current field line data without performing a motion compensation.

[0012] A third method is an intrafield interpolation method for interpolating by using a linear line interpolation of the current field itself.

[0013] A fourth method is a motion compensation interpolation method for interpolating by searching a motion vector.

[0014] A fifth method is a motion-adaptive interpolation method for interpolating based upon a motion by estimating a motion amount.

[0015] The first to third interpolation methods can be implemented by a simple hardware, but its picture quality is degraded after interpolation.

[0016] Referring to the fourth interpolation method, a picture quality is improved after interpolation, but its hardware construction is considerably complicated.

[0017] Meanwhile, the fifth method is implemented with a comparatively simple hardware, and its picture quality is improved after interpolation.

[0018] However, even if the fifth interpolation method is adopted, in case that only the motion between frames is detected, a motion of an object moving instantly or an object moving at a certain speed is hardly detected. In addition, even in case that a motion between simple fields is detected; it causes an excessive motion detection. Because of such an excessive motion, a resolution of an image after interpolation is degraded and overall picture quality is also degraded.

[0019] Accordingly, in order to more accurately detect a motion of an object, a plurality of memory units or a complicated processing procedure is required. It is inevitable that a complicated structure of the overall hardware is required and a manufacturing expense of a circuit for the construction is increased.

[0020] Accordingly, the present invention is directed to a motion-adaptive interpolation apparatus and a method therefore that substantially obviates one or more problems due to limitations and disadvantages of the related art.

[0021] One object of the present invention is to provide a motion-adaptive interpolation method that is capable of improving a picture quality after interpolation by performing a suitable compensation according to motion information.

[0022] Another object of the present invention is to provide a motion-adaptive interpolation apparatus that is capable of improving a picture quality with a simple hardware structure compared to an existing device.

[0023] To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described herein, there is provided a motion-adaptive interpolation apparatus and method in which horizontal directional motion information of a field to be currently interpolated is accurately estimated from information of past fields and future fields on the basis of the field to be currently interpolated, and a suitable interpolation is performed according to the estimated motion information.

[0024] According to one aspect of the present invention, there is further provided a motion-adaptive interpolation apparatus including: a horizontal directional motion estimating means for receiving a continuous field data and setting a basic unit image region by estimating a horizontal directional motion, obtaining a block matching error (BME) by moving the basic unit image region at certain intervals in a horizontal direction of mutually opposite direction, and detecting a linearly interpolated pixel value by using the block matching error and outputting the detected pixel value; and a line interpolating means for calculating a final interpolation value by using a rule and a filtering by using the linearly interpolated pixel value and outputting the final interpolation value.

[0025] According to another aspect of the present invention, there is provided a motion-adaptive interpolation method including: receiving an image signal and outputting a continuous field data; receiving the field data and estimating and outputting horizontal directional inter-frame motion information of a field to be currently interpolated; and calculating a line interpolation value by applying a rule and filtering based upon the estimated motion information.

[0026] The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and appended claims of the present invention, taken in conjunction with the accompanying drawings.

[0033] The following detailed description of the embodiments of the present invention, as represented in FIGS. 1–4, is not intended to limit the scope of the invention, as claimed, but is merely representative of the presently preferred embodiments of the invention. In the description, same drawing reference numerals are used for the same elements even in different drawings. The matters defined in the description are nothing but the ones provided to assist in a comprehensive understanding of the invention. Thus, it is apparent that the present invention can be carried out without those defined matters. Also, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

[0034] FIG. 1 is a block diagram of a motion-adaptive interpolation apparatus in accordance with a preferred embodiment of the present invention.

[0035] As shown in FIG. 1, a motion-adaptive interpolation apparatus of the present invention includes: a field data providing unit 11 for receiving an input image signal (Vin), and storing or outputting a continuous field data; a motion detecting unit 12 for receiving the continuous field data and detecting an inter-frame motion amount and a inter-field motion amount; a post processing unit 13 for filtering the detected motion and extending the filtered signal to output a precise motion amount value; a horizontal directional motion estimating unit 14 for receiving the field data to estimate a horizontal directional motion and set a basic unit image region, obtaining a block matching error (BME) by moving the basic unit image region at certain intervals in a horizontal direction of the mutually opposite direction, and detecting a linearly interpolated pixel value and outputting the pixel value; an edge direction detecting unit 15 for receiving the field data, estimating a direction of edges of a field image to be interpolated, and performing an interpolation according to the direction; and a line interpolating unit 16 for receiving output information of the post-processing unit, the horizontal direction motion estimating unit and the edge direction detecting unit, and obtaining a final interpolation value by using a rule and filtering in consideration of an influence of the block matching error value and the pixel value used for the temporal motion compensation.

[0036] The operation of the motion-adaptive interpolation apparatus constructed as described above will now be explained.

[0037] The field data providing unit 11 sequentially stores the input image signal (Vin) and provides four pieces of field data for a motion detection and interpolation, by using a three serially connected field memory units 11A˜11C and line memory units 11D and 11E The motion detecting unit 12 includes an inter-frame motion detecting unit for receiving data of four continuous fields, that is, one piece of reference (current) field (n), two pieces of previous (past) field (n−1) and (n−2) and one piece of post (future)−field (n+1), and detecting a motion amount between fields existing at the same phase at time intervals of one frame or a plurality of frames; and an inter-field motion detecting unit for detecting a motion between fields existing at different phases at time intervals of one field.

[0038] The post-processing unit 13 performs various functions to detect a precise motion amount. That is, for reliable motion detection, the post-processing unit 13 low-pass filters a bright difference signal outputted from the motion detecting unit 12 and maps the filtered brightness difference signal to a level determined for motion amount discrimination.

[0039] And then, in order to group the portion with motion, the post-processing unit 13 median-filters the mapped brightness difference signal, extends the filtered signal and outputs a precise motion amount value.

[0040] The edge direction detecting unit 15 estimates a direction of edges of the field image to be interpolated and allows interpolation according to the interpolation.

[0041] The horizontal directional motion detecting unit 14 detects inter-frame motion information of the field data, precisely estimates a horizontal directional motion from the motion information, and allows a temporal compensation even if there is a motion in the direction.

[0042] The operation of the horizontal directional motion detecting unit 14 will now be described.

[0043] FIG. 2 is an exemplary view of an image field and a frame block for a horizontal directional motion estimation and interpolation in accordance with the preferred embodiment of the present invention.

[0044] As shown in FIG. 2, two lines of the (n−2)th field and one line of the (n−1)th field used for a motion detection in the horizontal directional motion detecting unit 14 are combined to form one frame image block (‘A’ frame block), and two lines of the nth field and one line of the (n+1)th field to form a different frame image block (B’ frame block), to thereby set a basic unit image region for the ‘A’ frame block and ‘B’ frame block on the basis of pixels to be currently interpolated.

[0045] At this time, the number of pixels in the vertical direction of the basic unit image region as set is ‘3’, and the number of pixels in the horizontal direction may be variably set according to an applied method.

[0046] That is, an image region of a basic unit for a motion estimation is 3×N (the number of vertical directional pixels×the number of horizontal directional pixels).

[0047] And then, a block matching error (BME) is obtained by moving the basic unit image region at certain intervals in the horizontal direction of the mutually opposite direction.

[0048] At this time, a movement interval of the basic unit image region may be variably set according to application, and generally, a movement interval of ¼˜1 pixel is mostly used.

[0049] The process of the block matching is continuously performed with a unit of pixel to be currently interpolated, which can be expressed in the following expression (1). $\begin{array}{cc}\begin{array}{c}\mathrm{BME}\left(i,j,k\right)=1/\left(3\times N\right)\sum _{a=-1}^1\sum _{b=-\left(N-1\right)/2}^{\left(N-1\right)/2}p(i+\\ a,j+b-k/2,n-1)-\\ p(i+a,j+b+k/2,n+1)\end{array}& \left(1\right)\end{array}$

[0050] In this respect, with reference to FIG. 2, the pixel to be interpolated is one in the j-th position in a vertical direction and j-th position in a horizontal direction of the nth field, and ‘k’ is a horizontal directional movement interval between the basic unit image regions on the basis of the pixel to be currently interpolated.

[0051] Accordingly, the BME (i, j, k) indicates a block matching error (BME) according to the ‘k’, the horizontal directional interval of the pixel to be currently interpolated.

[0052] Thereafter, a BME having a minimum value is selected from BMEs obtained as described above for each pixel to be interpolated. And, the BME with the minimum value, the temporally linear-interpolated value according to the direction according to the position of the pixel to be currently interpolated, and the pixel values of the previous and the next fields used for the linear interpolation are outputted to the line interpolating unit.

[0053] In this respect, if the BME has a minimum value in the ‘K’ movement interval, a temporally linear-interpolated pixel value in this direction can be expressed by equation (2):
P_temporal(i,j,n)=(p(i,j−k/2,n−1)+p(i,j+k/2,n+1))/2  (2)

[0054] Therefore, in the motion estimating method of the present invention, since the motion is estimated for an image with a horizontal directional motion without additionally installing hardware such as a memory unit, a picture quality can be improved without adding hardware compared to the conventional motion-adaptive interlacing method.

[0055] FIG. 3 is an exemplary view of an input data of a line interpolating unit in accordance with the preferred embodiment of the present invention.

[0056] As shown in FIG. 3, a data value inputted to the line interpolating unit 16 includes a motion amount value (M), an output of the post-processing unit 13; a pixel positioned at a very upper side (p(i−1), j, n)), the output of the field data providing unit 11, and a lower side (p(i+1, j, n)) of the pixel to be currently interpolated spatially, and a pixel of the previous (p(i, j, n−1) and next (p(i, j, n+1) fields existing at the same position spatially as that of the pixel being currently interpolated; spatial linear interpolation value (P_spatial(i, j, n)), the output of the edge direction detecting unit 15, according to an edge direction as the below equation (3), the then spatial block matching error (BME_spatial), and pixels (p(i+1, j−a/2, n), p(i−1, j+a/2, n)) used for the spatial linear interpolation;
P_spatial(i,j,n)=(p(i+1,j−α/2,n)+p(i−1,j+α/2,n))/2;  (3)
and a temporally motion-compensated interpolation value (P_temporal(i, j, n)), the output of the horizontal directional motion estimating unit 14, and the then temporal block matching error (BME_spatial), and pixel values (p(i, j−k/2, n−1), p(i, j+k/2, n+1) used for the temporal motion compensation.

[0057] Thus, the line interpolating unit 16 receives the output values, applies a rule and a filtering, calculates an interpolation value (P_moving(i, j, n)), and outputs a final line interpolated pixel value P_{interpolating}(i, j, n).

[0058] FIG. 4 is an exemplary view of a rule adopted for outputting a final interpolation value and a filtering algorithm in accordance with the preferred embodiment of the present invention.

[0059] The rule and filtering as shown in FIG. 4 shows one example, and the interpolation value calculation method in the line interpolating unit 16 is not limited thereto.

[0060] The concept of the rule and filtering will now be described.

[0061] In case of accurate motion estimation, temporal motion compensation is performed. In case of vague motion estimation, a pixel value blended according to temporal motion estimation and spatial edge direction estimation is used. Meanwhile, in case of inaccurate motion estimation, a pixel value interpolated according to spatial edge direction estimation is used.

[0062] Accordingly, a suitable filtering can be performed according to a slope in the estimated edge direction.

[0063] Consequently, a line interpolation pixel value (P_{interpolating}(i, j, n)) to be finally obtained is expressed by below equation (5): $\begin{array}{cc}{P}_{\mathrm{stationary}}\left(i,j,n\right)=\left(p\left(i,j,n-1\right)+p\left(i,j,n+1\right)\right)/2& \left(4\right)\\ P_{\mathrm{interpolating}}\left(i,j,n\right)=\left(1-M\right)·p_{\mathrm{stationary}}\left(i,j,n\right)+M·P_{\mathrm{moving}}\left(i,j,n\right),\left(0\le M\le 1\right)& \left(5\right)\end{array}$
wherein, P_stationary(i, j, n) is a pixel value of stationary interpolation, P_moving(i, j, n) is a pixel value of moving interpolation, and ‘M’ is a motion amount value, the output of the post-processing unit.

[0064] As so far described, the motion-adaptive interpolation apparatus and method of the present invention has the following advantages.

[0065] That is, after an image signal is received, horizontal directional inter-frame motion information of a field to be currently interpolated is estimated, and interpolating is suitably performed according to the motion information. Thus, a picture quality is improved after de-interlacing, and a circuit for implementation of the operation is simplified, so that a unit cost can be reduced.

[0066] As the present invention may be embodied in several forms without departing from the spirit or essential characteristics thereof, it should also be understood that the above-described embodiments are not limited by any of the details of the foregoing description, unless otherwise specified, but rather should be construed broadly within its spirit and scope as defined in the appended claims, and therefore all changes and modifications that fall within the meets and bounds of the claims, or equivalence of such meets and bounds are therefore intended to be embraced by the appended claims.