Improvements in or Relating to Video Image Keying Systems and Methods

This invention relates to video image keying systems and methods.

Keying systems are widely used for picture editing and composition in television and are used to key a foreground image into a background image, For example a keying system may be used for keying letters or other characters forming a caption into a picture to provide information relating to the subject matter of the picture. In such a case video signals representing the characters, as selected by the operator, are generated by a source such as a character generator, for example the character generator sold by the assignee hereof Quantel, Ltd. of Newbury, England under the trade mark "CYPHER". When a character is selected, the generator not only provides the video signals representing the character which may be regarded as foreground signals F, but it also provides internal control signals K which are used to key the character or foreground video signals over other video signals representing a background matte B. Normally the background matte B is a uniform black. The resulting combined signal is output from the source for further processing by other television image apparatus. The control signals K are also used within the source to key video signals representing white (value 1) over video signals representing black (value 0) to produce a key signal .alpha. which is output from the source for external use together with the combined signal. Thus, the output of the character generator comprises the video signal representation V of the foreground F keyed over the background B together with the external key . The external key .alpha. is used subsequently in external circuitry as will be described in greater detail hereinafter.

At present it is usual to perform the keying of the foreground F over the background matte B within the source character generator using a linear interpolating circuit in which the internal control signals K function as an interpolating coefficient. The video V which is the result of the interpolation can then be represented by

It is also usual for the control signals K to have a maximum value, which may be 1 or less than 1, for pixels lying within the character represented by the video signals F and to have value 0 for pixels within the background matte B, except for a zone round the boundary of the character over which the control signals K decline gradually from the maximum value, e.g. 1, to 0 to produce a soft edge between the character F and the background matte B. As is well known, this so-called soft edge keying is employed to reduce aliasing which would otherwise arise between character and background. The maximum value of K will be less than 1 where a foreground image F is to be combined transparently with the background matte B. Thus the coefficient K in the above expression for V may have values in the range:

and it will be understood that similar soft edging will also appear in the external key signal .alpha.. In the accompanying drawings FIG. 1(A) and 1(B) together illustrate a character generator or source of the above described form, conditioned to generate an output video as represented by the above expression for V and also to generate the above mentioned external key .alpha.. All the Figures in the drawings are in the form of schematic functional diagrams in which different circuits elements are represented by blocks identified by functional symbols denoting the nature of the respective circuit elements. Specific explanation of individual functional units in the drawings is considered to be unnecessary for an understanding of the invention by those possessed of the appropriate skills.

Returning to FIG. 1(A) it will be appreciated that where K=0 pixels in the video image V will relate solely to the matte background B, where K=1 pixels in the video image V will relate solely to the foreground character F and that for pixels within the soft edges, where K is between 1 and 0, the output video V will include a contribution both from F and from the background B. In the special case where the background matte is black of value B=0, then the term B(1-K) will remain 0 regardless of the value of K, but the term F.K will diminish with K at the soft edges, thereby "darkening" the character, in effect increasing the contribution from black. It will also be appreciated from FIG. 1(B) that the value of .alpha. will track that of K, being non-zero where K is non-zero.

The external key .alpha. is used by external circuitry to key the video signal V output from the source into video signals representing a picture which may be regarded as arbitrary external background Bg. In this way a resultant picture R is produced comprising the background picture Bg and the foreground character F.

The external keying of the video V into the arbitrary external background Bg has hitherto usually been performed by another linear interpolation process as illustrated in FIG. 1(c) to produce the result:

Bearing in mind that from equation (1) hereinabove V =F.K+B(1-K) then the above equation for R shows that any contribution of the original background matte B in the signal V is multiplied by .alpha.. As mentioned hereinabove pixels within the soft edges are darkened towards black by the keying performed within the character source. In other words pixels in the soft edge output from the character source include an inseparable contribution from the black background of the matte. Therefore, the result signal R from the external interpolator includes the black background contribution in the soft edges thereof in addition to the external background Bg and this black contribution is therefore a spurious signal which may be seen as halos or shadows in the result image R. While these unwanted artifacts are usually feint they are nevertheless visible and can be disturbing. Similar problems can occur in other systems where two or more keying operations are required over the same area of a picture, for example in systems such as described in commonly assigned British Patent No. 2,113,950 (corresponding U.S. Pat. No. 4,602,286), or in systems in which different foreground objects can be internally keyed over two copies of the same background picture and an output produced by internally cross fading between one stamped copy and the other.

The object of the present invention is to reduce the problem of spurious signals such as illustrated above. According to one aspect of the present invention there is provided an image processing system in which first image data, representing at least one foreground image keyed to a portion of a first background in accordance with respective keying data, is combined with second image data, representing a second background, to produce data defining a combined image comprising said at least one foreground image keyed to a portion of said second background, the system being arranged such that the second image data is weighted by control data derived from said keying data before being combined with the first image data such that data representing the first background is replaced by the second image data.

According to another aspect of the invention there is provided a keying system for television image processing, the system comprising: a source of first video signals V representing a foreground object F keyed over a first background B; a source of second video signals representing an arbitrary background Bg; a source of key signals (.alpha.) for keying the foreground object over the arbitrary background Bg; and a keying circuit responsive to said first and second video signals and said key signals to produce the result

On the condition that:

the above result R is:

which yields the required blend for soft edging between the foreground and the arbitrary background, Bg, with no contribution in the Result from the background B. That is to say, the first background B is replaced by the new arbitrary background Bg in such a way as to avoid the introduction of spurious artifacts such as halos or shadows in the resultant image. According to another aspect of the invention there is provided an image processing system comprising a first data source for supplying first image data representing a first image formed from at least one foreground image keyed to a first background image by interpolation in accordance with at least one respective first interpolation coefficient and for supplying a second interpolation coefficient derived from the or each first interpolation coefficient, and a divider for dividing the first image data by the second interpolation coefficient.

In a further aspect the invention provides a method of processing image data, the method comprising supplying first image data representing a first image formed form first image data representing at least one foreground image keyed to a first background image by interpolation in accordance with at least one respective first interpolation coefficient, deriving a second interpolation coefficient from the or each first interpolation coefficient, and dividing the first image data by the second interpolation coefficient. The above and further features of the invention are set forth with particularity in the appended claims and together with advantages thereof will become clearer from consideration of the following detailed description of exemplary embodiments of the invention given with reference to the accompanying drawings.

FIG. 1(A) and 1(B) illustrate a simple character generator and FIG. 1(C) illustrates a prior art external keying process as previously discussed hereinabove;

FIGS. 2(A), 2(B) and 2(C) illustrate a simple external keying system according to the present invention;

FIGS. 3(A), 3(B) and 3(C) illustrate a system including an external keyer for keying overlapped foreground objects onto the same background;

FIG. 4 illustrates a system including an external keyer for keying two different foreground objects over respective copies of the same background and for cross fading between the two; and

FIG. 5 illustrates a system in which a foreground image is pre-processed before being delivered externally to a conventional keying circuit.

Further reference will now be made to FIGS. 2(A) to 5 of the accompanying drawings, which illustrate several different exemplary embodiments of the invention.

Turning first to FIG. 2(A), there is illustrated a system including a simple external or so-called downstream keying circuit according to the invention. The keying circuit of FIG. 2(A) operates on the video V output from the character source of Figure 1(A) under control of the key .alpha. output from the source shown in FIG. 1(B). There is a second video input in FIG. 2(A) namely a picture which can be regarded as an external arbitrary background Bg. It will be noted that in the downstream keying circuit of FIG. 2(A) there is no multiplying circuit in the input path for V, as there would be if the keying circuit were a linear interpolating circuit such as used hitherto and illustrated in FIG. 1(C) of the accompanying drawings. The Result which is output from the FIG. 2(A) downstream keyer is given by the equations:

and if B=0 in equations (1), then substituting for V and .alpha. gives

That is to say, the result R output from the external keyer circuit to FIG. 2(A) is the foreground F keyed onto the external background Bg with no contribution from the original background B, which is the desired result.

The condition that B=0 implies that the "internal" background B in the video represented by the first video V signal is black matte. However, the CCIR specification for broadcast television signals specifies a luminance value 16 for black, and the second video signals representing the "external" arbitrary background might have black equal to sixteen in the scale of luminance signal values. In this event as illustrated in FIG. 2(B) of the accompanying drawings the black "offset" value of 16 would, in practice, be subtracted from both video signals V and Bg before applying them to the keying circuit, and would then be re-added to the Result. The circuit of FIG. 2(B) can be simplified to the circuit of FIG. 2(C).

The invention would usually be applied to color television signals in digital form comprising different channels for different color components e.g. YUV or RGB. In this specification the description is confined to one channel, assumed to be the Y channel of a YUV system. For such a system the Y component to represent black has the value 16 (in the scale 0-255), as already discussed. For U and V components, the CCIR specification prescribes the value 128 (again on the scale 0 to 255) for zero chroma, and appropriate measures for dealing with the offset are required in the U and V channels also. Similar considerations apply to the generation of the external keying signal .alpha. from the internal key signals K. FIGS. 3(A), 3(B) and 3(C) illustrate a system for keying overlapping foreground objects F1 and F2 onto the same background B, followed by subsequent keying of the output over an external arbitrary background Bg. FIG. 3(A) shows the circuit components for keying F1 and F2 and B, FIG. 3(B) shows the circuit components for generating the external key .alpha., while FIG. 3(C) shows the external keying circuit. The video output from FIG. 3(A) is represented by: ##EQU1##

The key signal output from FIG. 3(B) is represented by:

It will be noted that K.sub.1 is the only output from the first of the two multiplying stages in FIG. 3(B) when the video signal for black is B=0.

It will also be noted that the external or downstream keyer circuit of FIG. 3(C) is exactly the same as that shown in FIG. 2(A) and that similar considerations regarding the offset values of black as discussed hereinabove in relation to FIGS. 2(B) and (C). Since the FIG. 3(C) downstream keyer is the same as that in FIG. 2(A) it follows that the result R of the external keying operation performed in the FIG. 3(C) circuit is again represented by the equation:

If B=0 in equation (3) then substituting: ##EQU2##

It should be noted that Bg (1-K.sub.1)(1-K.sub.2) is the same term as was applied to B in equation (3) and is therefore free from contributions from the original background B, as required.

The arrangement illustrated by FIGS. 3(A), 3(B) and 3(C) can be used in an image composition system such as shown in commonly assigned British Patent 2,113,950 and U.S. Pat. No. 4,602,286.

FIGS. 4(A), 4(B) and 4(C) illustrate an arrangement for keying two different foreground objects F1 and F2 over two respective copies of the same background B in which arrangement the two results are cross faded, as are the two internal keys. Subsequently the cross faded foregrounds F1 and F2 are keyed by a downstream keyer into an external arbitrary background Bg. The initial background B is again assumed to be a black matte and thus to have a value notionally of B=0. As an illustration of the use to which the form of the invention may be put, the foreground objects F1 and F2 may be symbols denoting different sports, for example a cricket bat and tennis racket keyed into the top left hand corner of a black matte. The external background Bg may be a sports commentary of a cricket match and a tennis match, with the commentator switching at will between the two. At any one time the appropriate symbol is keyed into the external background Bg, and when the commentator switches from one match to the other, the two symbols F1 and F2 are cross faded. As represented in FIG. 4(A), cross fading of the foregrounds F1 and F2 can be affected under control of the operation by a dissolve signal D, which determines the "profile" of the fade. The FIG. 4(A) arrangement provides an output video V, while FIG. 4(B) shows the circuit arrangement for producing the corresponding external key .alpha. derived from the two internal keys K.sub.1 and K.sub.2. The expressions for the signals V and .alpha. are given by the following expression: ##EQU3##

Subsequently V and .alpha. are applied to an external downstream keying circuit as illustrated in FIG. 4(C). Again it should be noted that the circuit of FIG. 4(C) is identical to that of FIG. 2(A) and similar considerations regarding offset values apply. When V and .alpha. are applied to the external keying circuit shown in FIG. 4(C) together with the external background Bg, the following expresses the result R output therefrom.

From equations (6) ##EQU4##

If B is 0 in equation 5 then

Now R=V+BG (1-.alpha.), and so substituting both for V (with B=B=0) and for (1 -.alpha.) gives:

Result=Bg [D(K.sub.2- K.sub.1)+(1-K.sub.2)]+F.sub.1 K.sub.1 D+F.sub.2 K.sub.2 (1-D)

This is the same term for Bg as was applied to B in equation 5 and is therefore the required result.

That is to say, in the result R the original internal background B is replaced with the external background Bg with no residual contribution from the original background.

In FIGS. 2(A) through 4(C) various video signals V, which have previously been created by keying one or more foreground signals F onto a background matte signal B, are supplied to a downstream keyer together with an external keying signal .alpha. derived from the internal keying signals K used to produce the video signal. The derived external keying signal .alpha. is used to weight an arbitrary background signal Bg in the downstream keyer and the weighted background signal is then combined with the video signal V, unweighted, to produce the desired result. An alternative arrangement allows a conventional keyer similar to that shown in FIG. 1(C) of the accompanying drawings to be used. The keyer shown in FIG. 1(C) is a widely available apparatus and by using such a keyer the costs of the system can be kept to a reasonable level. The alternative arrangement is shown in FIG. 5 of the accompanying drawings and comprises a conventional source, such as a system available from the assignee hereof under the trade name CYPHER, i.e., a character generator which provides a video signal V and a keying signal .alpha. similar to those signals already discussed hereinabove in relation to FIGS. 3(A), 3(B), 4(A) and 4(B) of the drawings, and a conventional keyer. The video signal comprises a foreground F keyed onto a black matte background B by a key K. The output video V and the internally derived external key .alpha. are output to a look up table LUT which divides V by .alpha.. In setting up the look up table consideration must be given to the black offset in the foreground data F and the key data (see FIGS. 2(B) and 2(C)). The output from the look up table is delivered together with the external key .alpha. and an arbitrary background Bg to the conventional keyer which keys the .alpha.-divided video V onto the background using the key .alpha.. It will be noted that net effect of the look up table is to precondition the foreground signal F such that it is supplied unchanged to the conventional keyer. Once inside the keyer the foreground signal is multiplied by the external key .alpha. and added to the background weighted by (1-.alpha.) to produce the result R=V-Bg(1-.alpha.) as before. Thus, the look up table performs the inverse operation to that performed by the multipliers within the source. Thus, the look up table, which may be a stand-alone unit but preferably is built into the source, serves to precompensate the foreground video V so that it is not processed for a second time by the same key data in the conventional keyer thereby ensuring that the background matte B is replaced completely by the external background Bg in the conventional keyer without the introduction of unwanted artifacts.

In each of the above described embodiments the video signal is created by keying one or two foreground signals over a black matte background using a respective internal key for the or each foreground signal. An external key signal is derived from the internal keys and is subsequently used to combine the video signal with an external background signal of arbitrary content. The embodiments are not limited to the use of a video signal created from only one or two foreground signals and can be applied equally to a video signal formed from multiple foreground images so long as the keying signal for each foreground signal is available for use in deriving the external keying signal.

Having thus described the present invention by reference to preferred embodiments it is to be understood that the embodiments in question are exemplary only and that modifications and variations such as will occur to those possessed of appropriate knowledge and skills may be made without departure from the spirit and scope of the invention as set forth in the appended claims and equivalents thereof.