Motion determining apparatus and determining method

[0002] This is a continuation of copending International Application PCT/JP99/05449 having an international filing date of Oct. 4, 1999.

[0003] The present invention relates to a motion determining apparatus and a motion determining method for accurately determining the motion of a picture that is interlace-scanned.

[0004] When an interlace-scanned picture (hereinafter referred to as interlace picture) is converted into a non-interlace-scanned picture that is a progressively scanned picture (hereinafter this picture is referred to as progressive picture), coefficients such as tap coefficients of a filter used for the process are varied depending on whether or not the interlace picture moves so as to properly process the interlace picture. There are several types of converting processes such as a process for converting an interlace picture of one standard into an interlace picture of another standard; a process for converting a standard resolution interlace picture into a high resolution picture; and a process for controlling tap coefficients of a digital filter for detecting the motion of an interlace picture. In these converting processes, as a pre-process, it may be determined whether or not an interlace picture to be processed moves.

[0005] However, in a conventional motion determining apparatus, when only part of a considered field of an interlace picture is largely different from adjacent fields (for example, an interlace picture to be processed shows a jet plane that flies fast and disappears from the screen), although the part moves, it may be improperly determined that the part stops.

[0006] In this case, a process suitable for a picture that stops (for example, an inter-frame process for interpolating data of the current frame with data at the same position of the preceding frame) is performed for a picture that moves. The processed result is not a picture that smoothly moves. In addition, the process may fail in the middle.

[0007] On the other hand, although a picture that stops, when it is determined that the picture moves, a process suitable for a picture that moves (for example, an intra-field process) is performed. Thus, the picture that stops blurs.

[0008] Thus, the determined result of whether or not a picture moves largely affects a picture that is processed. Consequently, the motion determining process should be accurately performed (without error).

[0009] In addition to the process for an interlace picture, the motion determining process should be correctly performed for a progressive picture. A recursive noise reducing apparatus using the phenomenon of which a picture signal correlates on time axis but noise does not is known. With such a noise reducing apparatus, noise of a progressive picture can be reduced. The noise reducing apparatus determines whether or not a particular picture moves. When the motion of the picture is detected, the recursive process should be stopped. In addition, the contents of the frame memory should be rewritten. Thus, even in the noise reducing apparatus, unless the motion can be correctly determined, the above-described problem will take place.

[0010] Therefore, the present invention is made from the above-described point of view. An object of the present invention is to more accurately determine the motion of a picture than before.

[0011] The present invention is to solve the above-described problem. The invention of claim 1 is a motion determining apparatus for determining a motion of an object contained in an input picture signal, comprising a space variation detecting means for detecting a space variation of the input picture signal, a time variation detecting means for detecting a time variation of the input picture signal, a ratio detecting means for detecting the ratio of the space variation to the time variation, and a motion determining means for determining the motion of the object contained in the input picture signal according to the space variation, the time variation, and the ratio.

[0012] Preferably, the space variation detecting means detects a space variation of the input picture signal of the considered field corresponding to an input picture signal of a considered field and input picture signals of at least the fields that precede and follow the considered field. In the progressive system, a field is equivalent to a frame.

[0013] The invention of claim 11 is a motion determining method for determining the motion of an object contained in an input picture signal, comprising the steps of detecting a space variation of the input picture signal, detecting a time variation of the input picture signal, detecting the ratio of the space variation to the time variation, and determining the motion of the object contained in the input picture signal according to the space variation, the time variation, and the ratio.

[0014] FIG. 1 is a block diagram showing an example of the structure of a motion determining apparatus according to an embodiment of the present invention;

[0015] FIG. 2 is a schematic diagram for explaining a process of a tap structuring portion;

[0016] FIG. 3 is a schematic diagram for explaining a process of a time variation calculating portion;

[0017] FIG. 4 is a schematic diagram for explaining a motion determining process according to the embodiment of the present invention; and

[0018] FIG. 5 is a flow chart for explaining a process of the motion determining apparatus shown in FIG. 1.

[0019] FIG. 1 is a block diagram showing an example of the structure of a motion determining apparatus according to an embodiment of the present invention. The motion determining apparatus according to the embodiment of the present invention determines whether or not a particular interlace picture moves.

[0020] In other words, each field that composes an interlace picture is supplied in a time sequence to a field memory 1. The field memory 1 successively stores pictures of individual fields. The field memory 1 has a storage capacity for at least pictures of three fields.

[0021] The field memory 1 is connected to a tap structuring portion 2. The tap structuring portion 2 reads predetermined pixels (pixel values) from the field memory 1 and structures taps to be processed. In other words, when a field at time t (this field is referred to as t-th field) is considered, a particular pixel of the t-th field is considered. The particular pixel is referred to as considered pixel. In this case, the considered pixel may be a real pixel that composes the picture of the t-th field.

[0022] It should be noted that the present invention can be applied to the case that a progressive signal of which frames are successively arranged is processed.

[0023] The tap structuring circuit 2 selects 50 pixels shown in FIG. 2 as taps used for determining the motion of a picture. The tap structuring circuit 2 outputs the selected taps. FIG. 2 shows partial pictures at the spatially the same positions of successive three fields at time t−1, t, and t+1. In FIG. 2, dots represent pixels sampled at a predetermined frequency. For example, one pixel is a luminance value composed of eight bits.

[0024] In the field at time t−1, five taps are selected from each of chronologically successive three lines in such a manner that the positions of the five taps accord on each line. Likewise, in the field at time t+1, five taps are selected from each of chronologically successive three lines. Thus, the spatial positions of the taps of these fields are the same. In contrast, in the present field (at time t), five taps are selected from each of chronologically successive four lines in such a manner that the positions of the five taps accord on each line. Due to the interlace picture, the taps of the four lines of the current field at time t spatially deviate by one line from those of the fields at time t−1 and time t+1. In FIG. 2, x represents a considered pixel. With respect to the considered pixel, the motion of the picture is determined.

[0025] The tap structuring portion 2 is connected to a space variation calculating portion 3 and a time variation calculating portion 4. The space variation calculating portion 3 calculates space variation amounts SG (t), SG (t−1), and SG (t+1) of the taps of the (t−1)-th field, the t-th field, and the (t+1)-th field. The space variation amounts are supplied to a time variation to space variation calculating portion 5 and a first condition determining portion 6.

[0026] The space variation amount of each field is obtained by calculating the absolute value of the difference between each pixel and each of the adjacent pixels thereof and summing the obtained absolute values. When the values of the pixels as the taps are defined as shown in FIG. 2, the space variation amounts SG (t), SG (t−1), and SG (t+1) are calculated as follows.

[0027] SG(t)=[|b00−b01|+|b01−b02|+|b02−b03|

[0028] +|b03−b04|+|b10−b11|+|b11−b12|

[0029] +|b12−b13|+|b13−b14|+|b20−b21|

[0030] +|b21−b22|+|b22−b23|+|b23−b24|

[0031] +|b30−b31|+|b31−b32|+|b32−b33|

[0032] +|b33−b34|+|b00−b10|+|b01−b11|

[0033] +|b21−b02|+|b12−b03|+|b13−b14|

[0034] +|b10−b20|+|b11−b21|+|b12−b22|

[0035] +|b13−b23|+|b14−b24|+|b20−b30|

[0036] +|b21−b31|+|b22−b32|+|b23−b33|

[0037] +|b24−b34|]/31

[0038] SG(t−1)=[|a00−a01|+|a01−a02|+|a02−a03|

[0039] +|a10−a11|+|a11−a12|+|a12−a13|

[0040] +|a20−a21|+|a21−a22|+|a22−a23|

[0041] +|a00−a10|+|a01−a11|+|a02−a12|

[0042] +|a03−a13|

[0043] +|a10−a20|+|a11−a21|+|a12−a22|

[0044] +|a13−a23|]/17

[0045] SG(t+1)=[|c00−c01|+|c01−c02|+|c02−c03|

[0046] +|c10−c11|+|c11−c12|+|c12−c13|

[0047] +|c20−c21|+|c21−c22|+|c22−c23|

[0048] +|c00−c10|+|c01−c11|+|c02−c12|

[0049] +|c03−c13|

[0050] +|c10−c20|+|c11−a21|+|c12−a22|

[0051] +|c13−c23|]/17

[0052] The time variation calculating portion 4 calculates an inter-face difference FrG as a first time variation and inter-field differences FiGu (t−1), FiGu (t), FiGd (t−1), and FiGd (t) as second time variations. The inter-frame difference FrG (t) is obtained by calculating the difference between each of pixel values of 15 taps of the (t−1)-th field and that of the (t+1)-th field in such a manner that the position of the former accords with the position of the latter, obtaining the absolute values of the obtained 15 inter-frame differences, and obtaining the average value. In other words, FrG (t) is obtained as follows.

[0053] FrG(t)=[|a00−c00|+|a01−c01|+|a02−c02|/

[0054] +|a03−c03|+|a04−c04|+|a10−c10|

[0055] +|a11−c11|+|a12−c12|+|a13−c13|

[0056] +|a14−c14|+|a20−c20|+|a21−c21|

[0057] +|a22−c22|+|a23−c23|+|a24−c24|]/15

[0058] Generally, it is determined that the motion of two adjacent frames is proportional to the inter-frame difference FrG (t).

[0059] FIG. 3 is a schematic diagram for explaining the inter-frame difference. In the interlace system, the vertical line positions of adjacent fields deviate. Thus, there are two types of inter-field differences: a difference between a line of the current field and a line of an adjacent field thereof in the case that the line of the current field is higher than the line of the adjacent field (the difference is denoted by a suffix u) and a difference between a line of the current field and a line of an adjacent field thereof in the case that the line of the current field is lower than the line of the adjacent field (the difference is denoted by a suffix d).

[0060] As is clear from FIG. 3, FiGu (t−1) represents an upward inter-field difference between the (t−1)-th field and the t-th field. FiGd (t−1) represents a downward inter-field difference between the (t−1)-th field and the t-th field. Likewise, FiGu (t) represents an upward inter-field difference between the t-th field and the (t+1)-th field. FiGd (t) represents a downward field difference between the t-th field and the (t+1)-th field. As with the frame difference FrG (t), the average value of the sums of the absolute values of the difference values of (5×3) pixels of one frame and (5×4) pixels of the adjacent frame is detected as the inter-field difference.

[0061] The detected inter-frame difference FrG (t) is supplied to the time variation to space variation calculating portion 5. The time variation to space variation calculating portion 5 is composed of a dividing device. The time variation to space variation calculating portion 5 obtains the ratio of the inter-frame difference FrG (t) of the t-th field received from the time variation calculating portion 4 to the space variation amount SG (t) of the taps of the t-th field received from the space variation amount calculating portion 3 (hereinafter, the ratio is referred to as time variation to space variation gfr (t) (=FrG (t)/SG (t))).

[0062] The time variation to space variation gfr (t) is supplied to a first condition determining portion 6, a second condition determining portion 7, and a third condition determining portion 8. The first condition determining portion 6 determines the motion of a picture corresponding to the variation of the space variation amount between adjacent fields. The second condition determining portion 7 determines the motion of a picture corresponding to the inter-field difference. The third condition determining portion 8 determines the motion of a picture in the same manner as gradient method. When each of the condition determining portions 6, 7, and 8 determines that the picture moves, it outputs data “1” (logical 1). When each of the condition determining portions 6, 7, and 8 determines that the picture stops, it outputs data “0” (logical 0). The output data of each of the condition determining portions 6, 7, and 8 is supplied to an OR gate 9. The OR gate 9 outputs the determined result of the motion of the picture. Thus, when one of the condition determining portions 6, 7, and 8 determines that the picture moves, it outputs data “1” as the determined result. Otherwise, it outputs data “0” as the determined result.

[0063] When the following three conditions are satisfied, the first condition determining portion 6 determines that the picture moves. Otherwise, the first condition determining portion 6 determines that the picture stops.

[0064] Condition C1: The space variation amount SG (t) of the current field (t-th field) should be equal to or larger than threshold value th1.

[0065] Condition C2: The variation between the space variation amount SG (t) of the current field and the space variation amount of the past field or the future field should be equal to or larger than threshold value th2.

[0066] Condition C3: The ratio of the inter-frame difference FrG and the space variation amount SG (t) of the current field should be equal to or larger than threshold value th3.

[0067] When all the three conditions are satisfied, the first condition determining portion 6 determines that the picture moves and outputs data “1” to the OR gate 9. Otherwise, the first condition determining portion 6 determines that the picture stops and outputs data “0” to the OR gate 9. These conditions can be expressed by the following formulas.

[0068] C1: SG(t)≧th1

[0069] C2: min {|SG(t)−SG(t−1)|, |SG(t)−SG(t+1)|}≧th2

[0070] C3: gfr(t)=FrG/SG(t)≧th3

[0071] The second condition determining portion 7 uses the following conditions.

[0072] Condition C1: The space variation amount SG (t) of the current field (t-th field) should be equal to or larger than threshold value thl.

[0073] Condition C4: The maximum value of the inter-field differences of individual directions should be equal to or larger than threshold value th4.

[0074] Condition C5: The ratio of the inter-frame difference FrG to the space variation amount SG (t) of the current field should be equal to or larger than threshold value th5.

[0075] When all these conditions are satisfied, the second condition determining portion 7 determines that the picture moves and outputs data “1” to the OR gate 9. Otherwise, the second condition determining portion 7 determines that the picture stops and outputs data “0” to the OR gate 9. These conditions can be expressed by the following formulas.

[0076] C1: SG(t)≧th1

[0077] C4: max {FiGu(t−1), FiGu(t), FiGd(t−1),FiGd(t)}≧th4

[0078] C5: gfr(t)=FrG/SG(t)≧th5

[0079] The third condition determining portion 8 uses the following condition.

[0080] Condition C6: The ratio of the inter-frame difference FrG to the space variation amount SG (t) should be equal to or larger than threshold value th6.

[0081] When the condition is satisfied, the third condition determining portion 8 determines that the picture moves and outputs data “1” to the OR gate 9. Otherwise, the third condition determining portion 8 determines that the picture stops and outputs data “0” to the OR gate 9. The condition C6 can be expressed by the following formula.

[0082] C6: gfr(t)=FrG/SG(t)≧th6

[0083] where threshold value th6 is larger than threshold values th3 and th5.

[0084] In this case, the third condition is a value of which the inter-frame difference FrG is normalized with the space variation amount SG (t). The threshold values with respect to the ratio gfr (t) are th3, th5, and th6. As shown in FIG. 4, the threshold values are proportional to the space variation amount SG (t) and the inter-frame difference FrG. In FIG. 4, the horizontal axis and the vertical axis represent the space variation amount SG (t) and the inter-frame difference FrG, respectively. Threshold value th6 is larger than threshold values th3 and th5. Thus, there are an area 1 that is smaller than threshold values th3 and th5, an area 2 surrounded by threshold values th3, th5, and th6, and an area 3 that is equal to or larger than threshold value th6.

[0085] Only with the third condition that uses the ratio gfr (t) and the threshold value th6, it is determined that the picture stops in the areas 1 and 2.

[0086] In addition, it is determined that the picture in the area 3 that is equal to or larger than threshold value th6. However, according to the embodiment of the present invention, with threshold values th3 and th5, it is determined that the picture moves in the area 2. Only with threshold value th6, although the picture moves, it is determined that the pictures stops in the area 2. In such a manner, the accuracy of the motion determining process can be improved.

[0087] The OR gate 9 ORes the determined results of the first condition determining portion 6, the second condition determining portion 7, and the third condition determining portion 8. In other words, when it is determined that a considered pixel of a considered field (t-th field) moves, the OR gate 9 outputs data “1”. In contrast, when it is determined that the considered pixel stops, the OR gate 9 outputs data “0 ”.

[0088] Next, with reference to a flow chart shown in FIG. 5, the motion determining process of the motion determining apparatus shown in FIG. 1 will be described. The motion determining process that follows can be accomplished by a software process of a computer rather than the motion determining apparatus shown in FIG. 1.

[0089] At step S1, taps of a considered pixel are structured. The tap structuring portion 2 extracts predetermined pixels in the vicinity of the considered pixel from the (t−1)th field, the t-th field, and the (t+1)-th field as shown in FIG. 2. The extracted taps are supplied to the space variation calculating portion 3 and the time variation calculating portion 4.

[0090] At step S2, the space variation calculating portion 3 calculates space variation amounts SG (t), SG (t−1), and SG (t+1). The obtained space variation amounts SG (t), SG (t−1), and SG (t+1) are supplied to the first condition determining portion 6. In addition, the space variation amount SG (t) is supplied to the time variation to space variation calculating portion 5. The time variation to space variation calculating portion 5 calculates the ratio of the inter-frame difference FrG to the space variation amount SG (t).

[0091] Moreover, at step S2, the time variation calculating portion 4 calculates the inter-frame difference FrG that is the first time variation amount. In addition, the time variation calculating portion 4 calculates the inter-field differences FiGu (t−1), FiGd (t−1), FiGu (t), and FiGd (t) that are the second time variation amounts.

[0092] At step S3, the time variation to space variation calculating portion 5 calculates the ratio gfr (t) of the time variation to the space variation (=FrG/SG (t)). The ratio gfr (t) is supplied to the first condition determining portion 6, the second condition determining portion 7, and the third condition determining portion 8.

[0093] At step S4, the first condition determining portion 6, the second condition determining portion 7, and the third condition determining portion 8 determine which of the conditions is satisfied. When one of the first to third conditions is satisfied, it is determined that the considered pixel moves. Thus, the OR gate 9 outputs data “1” (at step S5). On the other hand, when all the first to third conditions are not satisfied, it is determined that the considered pixel stops. Thus, the OR gate 9 outputs data “0” (at step S6). In such a manner, the motion determining process is completed.

[0094] Instead of the sum of absolute values, the sum of squares may be obtained.

[0095] In the embodiment of the present invention, a considered field and adjacent fields thereof are used. Alternatively, the considered field and adjacent fields that are apart therefrom by two fields may be used.

[0096] According to the embodiment of the present invention, as shown in FIG. 2, a picture contained in a rectangular block is extracted as taps. Alternatively, pixels contained in a circular block or a cross shaped block may be extracted as taps.

[0097] According to the present invention, when one of the first condition determination corresponding to the inter-field space variation amount, the second condition determination corresponding to the inter-field difference, and the third condition determination corresponding to the ratio of the inter-frame difference to the space variation amount is satisfied, it is determined that a considered pixel moves. Thus, the motion of the considered pixel can be accurately determined. Consequently, corresponding to the determined result of the motion determining process, a picture that moves, a picture that stops, and a picture that slightly moves can be properly processed.