Managing Letterbox Signals With Logos and Closed Captions

1. Field of the Invention

This invention relates to the field of automatic detectors for letterbox video sources, for example as would be useful in wide screen televisions.

2. Description of the Art

In "letterbox" format, an image in wide format display ratio, for example 16.times.9 (also designated 16:9), is displayed across the full width of a narrower format display ratio medium, for example 4.times.3 (also designated 4:3). Movies in a wide format to be broadcast commercially without distortion or cropping are converted by the broadcaster to letterbox format, to enable display on a conventional receiver screen, for example 4.times.3. The broadcaster effectively adds blank or matte top and bottom borders, and broadcasts the combined picture and borders in a 4.times.3 composite image signal. Assuming standard 16.times.9 and 4.times.3 ratios, and standard NTSC signals, only 181 horizontal lines in each field are devoted to the main video, the remaining lines being the matte, gray or black borders. The useful picture information, or main video, is generally referred to as active video, whereas the borders are generally referred to as inactive video, even if the borders are flat fields having some luminarice value and/or chrominance component. This distinction and designation is followed herein. The borders may be used, if at all, to display different types of auxiliary information. A first type is generally text, such as storm warnings, news alerts and subtitles. A second type includes station and network logos, which are often graphic or a combination of text and graphics. Both types of auxiliary information might be present together, or separately. Both types of auxiliary information are also generally referred to as active video. A third type of auxiliary information which can interfere with correct identification of a signal in letterbox format is closed captioning, which is usually transmitted during the vertical blanking interval.

When such letterbox displays (in 4.times.3 format) are shown on a wide screen television, for example 16.times.9 format, the video signal can be vertically zoomed, that is enlarged, by a factor which both increases the portion of active video to substantially fill the wide screen display, and at the same time, substantially eliminates the upper and lower borders. This can be accomplished without incurring image aspect ratio distortion. It is a useful feature to automatically detect letterbox sources and vertically enlarge the active portion of the video source signal so as to substantially fill the display screen.

An automatic letterbox detector in a wide screen television is disclosed in PCT/US91/03739 (WO 91/19390). As described therein, the uppermost and lowermost active video areas are detected by calculating two gradients for each line in the video field. Four values are required to calculate the two gradients: maximum and minimum values of the current line, and maximum and minimum values of the previous line. The first gradient, designated the positive gradient, is formed by subtracting the minimum value of the previous line from the maximum value of the current line. The second gradient, designated the negative gradient, is formed by subtracting the minimum value of the current line from the maximum value of the previous line. Either of the gradients may have positive or negative values depending on scene content, but the negative values of both gradients may be ignored. This is because only one gradient may be negative at a time, and the magnitude of the gradient with the positive value will always be greater than or equal to the magnitude of the gradient with the negative value. This simplifies the circuitry by eliminating the need to calculate an absolute value of the gradients. If either gradient has a positive value which exceeds a programmable threshold, video is considered to be present on either the current line or on the previous line.

These values can be used by a microprocessor to make a determination of whether or not the video source is in the letterbox format and to determine the first and last line numbers of active video. The calculation can be performed only for the top (or the bottom) of the image, provided means are provided for centering the image vertically. Inasmuch as the detection of active video is a function of gradient rather than luma absolute value, the device can detect inactive borders of a predetermined matte color, provided the borders have a substantial (but unchanging) luminance level. The luminance level detector is enabled for only about 70% of each line in order to improve signal to noise ratios.

However, if the borders are used to display auxiliary material, such as storm warnings, news alerts, subtitles, station or network logos, and the like, the automatic letterbox detector will return operation to the standard, non-zoomed operation. This may be desirable for storm warnings, news alerts and subtitles, but can be a significant nuisance for logos. Moreover, continuous transmission of a logo in the border area of a letterbox source can prevent automatic detection of the source as being letterbox in the first instance.

Interference with detection of signals in letterbox format can also be caused by any VCR of the helical scan type, for example video cassette recorders utilizing the VHS format, which will insert a head switching transient anywhere from 5-8 lines prior to the vertical synchronizing signal. Head switching transients can be mistaken for active video, causing incorrect letterbox detection. False detections based on VCR head switching transients must be avoided.

The invention overcomes the problem of automatic letterbox detectors erroneously responding to the presence of station and network logos, and the like, head switching transients and interference during vertical blanking, while still permitting automatic response to such information as storm warnings, news alerts and subtitles.

A video display control system which overcomes this problem comprises: a video display having a wide format display ratio; a letterbox detector for sampling video information in pictures represented by input video signals having a letterbox format and generating a control signal for enlarging the pictures for substantially filling the video display means with active video, the letterbox format pictures sometimes having auxiliary information disposed in a border area; and, a circuit for preventing the letterbox detector from sampling the video information in any portion of the border area in which auxiliary information such as logos or closed captioning can be expected to be disposed or transmitted, or in which head switching transients can be expected to occur, when present. In the case of logos, for example, which are generally disposed near or at the right side of the lower border, a horizontal sampling range will exclude at least the right side of each horizontal line. In the case of head switching transients and vertical blanking, a circuit will define a vertical sampling range which excludes a number of horizontal lines at the end and at the beginning of each field.

In practice, the letterbox format pictures sometimes have auxiliary information of first and/or second types, as defined above, disposed in a border area. Operation of the letterbox detector is controlled, so as to ignore the second type of information (e.g. logos), if present, but not so as to ignore the first type of information if present, by defining a horizontal sampling range which excludes the range in which logos normally appear but includes at least some of the range in which storm warnings, news alerts, subtitles normally appear. Such a range can be substantially centered with respect to the active video of each horizontal line, but small enough to exclude logos, or any other information, at either end of each horizontal line. Information of the third type, associated with vertical blanking, will occur, for example, from three lines before the vertical synchronizing signal to 14 lines after the vertical synchronizing signal. Head switching transients will occur, for example, anywhere from 5-8 lines prior to the vertical synchronizing signal.

An advantageous embodiment of the invention recognizes that in the majority of instances where logos are present in the border regions of a letterbox signal, such logos are located in the lower right hand corner of the picture, that is, on the right side of the lower border area. Accordingly, the luminance level detector is enabled for examining each line only in sections which are not the right hand portions in which the logos are found. This can be accomplished by sampling approximately 70% of each line, for example, substantially in the middle of each line. In terms of pixels, and assuming that each horizontal line includes 1024 pixels counting from the horizontal synchronizing pulse, 860 of which are active, sampling from approximately pixel 272 through pixel 872 corresponds approximately to 37.2 microseconds of each video line. This will avoid responding to the presence of logos disposed to the left of pixel 272 or to the right of pixel 872, in either the upper or lower border areas. At the same time, less than all horizontal lines will be sampled to avoid interference during head switching transients and vertical blanking. Nevertheless, there is still enough information being sampled to reliably identify letterbox source material, and still improve performance by approximately 1 decibel over results obtained from sampling all lines, and each horizontal line entirely.

In the illustrated embodiment, circuits inhibit operation of the letterbox detecting circuit during certain vertical ranges and during certain horizontal ranges. One of the circuits comprises: first and second comparators, responsive to first and second predetermined pixel counts respectively, and both responsive to a running pixel count; and, a flip/flop responsive to each of the comparators, for initiating a letterbox detection enable signal when the running pixel count reaches the first predetermined pixel count and terminating the enable signal when the running pixel count reaches the second predetermined pixel count. The other of the circuits comprises: first and second comparators, responsive to first and second predetermined line counts respectively, and both responsive to a running line count; and, a flip/flop responsive to each of the comparators, for initiating a letterbox detection enable signal when the running line count reaches the first predetermined line count and terminating the enable signal when the running line count reaches the second predetermined line count.

An automatic letterbox detector in accordance with the principles of this invention is explained in connection with the drawings, wherein:

FIG. 1 is a diagram illustrating how a conventional video signal in letterbox format, and including a network logo, can be fully enlarged for display on a wide screen television;

FIGS. 2 and 2(a) are diagrams illustrating how a conventional video signal in letterbox format, and including a storm warning, can be enlarged to a lesser degree for display on a wide screen television;

FIG. 3 is a diagram illustrating how a conventional video signal in letterbox format, and including a storm warning, can be displayed on a wide screen television without being enlarged;

FIG. 4 is a block diagram of an automatic letterbox detection circuit for initiating the fully enlarged display shown in FIG. 1;

FIG. 5 is a diagram useful for explaining the horizontal range of operation of the letterbox detector;

FIGS. 6(a), 6(b) and 6(c) are useful for explaining the vertical range of operation of the letterbox detector in a first of successive video fields; and,

FIGS. 7(a), 7(b) and 7(c) are useful for explaining the vertical range of operation of the letterbox detector in a second of successive video fields .

An automatic letterbox detector can be used to detect a conventional video signal 12 in letterbox format, as shown in FIG. 1. An automatic letterbox detector can be used to detect a 1. A signal in letterbox format, also referred to as a letterbox signal or letterboxed signal, has a central area 14 with active video, an upper border area 16 and a lower border area 18. The border areas are typically flat video fields, often black in color. The first line 20 of active video represents a transition from the upper border 16 to the active video area 14. The last line 22 of active video represents a transition from the active video area 14 to the lower border 18. If the video signal 12 has a format display ratio of 4.times.3, if the active video area has a format display ratio of 16.times.9, and if a wide screen television has a video display screen 10 with a format display ratio of 16.times.9, then the active video area 14 can be mapped precisely onto the wide screen display 10 by enlarging the active video area 14 by a factor of 4/3 both horizontally and vertically. Under these circumstances, the flat field border regions 16 and 18 will be effectively eliminated from view. A number of ways are known to implement such enlargement. The actual format display ratio of the active video area 14 may be different in practice from 16.times.9. In these cases, the mapping will not be precise. A decision must be made, either automatically or responsive to manual control, to fill the wide screen display, even if material is laterally cropped, or to fill the wide screen display vertically, even if lateral borders on each side of the enlarged active video area are visible on the wide screen display.

Letterbox detectors generally detect the absence of active video in one or both to the upper and lower border regions 16 and 18. Such circuit are generally fooled by information which appears in the border areas, such as storm warnings, news alerts, subtitles, station or network logos, and the like. Detection of a storm warning is illustrated in FIG. 2. A video signal 40 in letterbox format has a central area 42 with active video, an upper border area 44 and a lower border area 46. A storm warning 52 is transmitted in the lower border area 46. The first line 48 of active video represents a transition from the upper border 16 to the active video area 14, as in FIG. 1. However, the last line 50 of active video does not represent a transition from the active video area 14 to the lower border 18. Instead, line 50 represents the last line of video in the storm warning, as shown more clearly in the enlarged portion of the storm warning FIG. 2(a). A letterbox detector will determine that the area of active video includes not only area 42, but most of border area 46 as well. This has the effect of making the active video area have an effective format display ratio less than 16.times.9. In other words, the detected active video area is not as wide, compared to height, as is area 42 alone. In accordance with one of the alternatives noted above, the enlarged picture on the wide screen television 30 has lateral border areas 32 and 34, and the border area 46 which includes the storm warning 50 is displayed as a lower border area 36 having the storm warning in area 38.

Another display for storm warnings and the like is shown in FIG. 3. In this case, video signal 40 is displayed on wide screen television 60 without any enlargement at all, showing the border areas of the letterbox signal and lateral borders 62 and 64 as well.

In the case of storm warnings and other useful messages, either of the alternatives shown in FIGS. 2 and 3, or others, may be appropriate. However, it may not be desirable to operate as shown in FIGS. 2 and 3 when the only information transmitted in the border area is a station or network logo 24, depicted as a lambda (.lambda.) in FIG. 1. A full enlargement is desirable, notwithstanding the logo 24. Most useful information which appears in the border areas of letterbox signals extends horizontally across a substantial portion of the border, whereas most logos are much smaller, and usually placed near the far right hand edge of the lower border, as illustrated by the placement of logo 24. An automatic letterbox detector, which might otherwise undesirably respond to the presence of a logo, can be provided with a mode of operation in which such logos are usually ignored. An automatic letterbox detector which is insensitive to logos is shown in FIG. 4.

Generally, two gradients are calculated for each line in the video field to determine lines of active video. Four values are required to calculate the two gradients: (1) maximum and minimum values of the current line; and (2) maximum and minimum values of the previous line. The first gradient, designated the positive gradient, is formed by subtracting the minimum value of the previous line from the maximum value of the current line. The second gradient, designated the negative gradient, is formed by subtracting the minimum value of the current line from the maximum value of the previous line. Either of the gradients may have positive or negative values depending on scene content, but the negative values of both gradients may be ignored. This is because only one gradient may be negative at a time, and the magnitude of the gradient with the positive value will always be greater than or equal to the magnitude of the gradient with the negative value. This simplifies the circuitry by eliminating the need to calculate an absolute value of the gradients. If either gradient has a positive value which exceeds a programmable threshold, video is considered to be present on either the current line or on the previous line. These values can be used by a microprocessor to determine the first and last lines of active video and thereby to determine the particular aspect ratio of the active area of the signal. This determination defines whether or not the video source is in the letterbox format, and enables the further calculation of the extent of change to the deflection circuit and the resolution circuits needed to convert the displayed signal as required.

A circuit 100 for implementing a letterbox detector which is insensitive to the presences of logos is shown in block diagram form in FIG. 4. The circuit 100 comprises a luminance input filter, a line maximum (LINE MAX) detector 120, a line minimum (LINE MIN) detector 122, and an output section 124. The luminance input filter comprises finite impulse response (FIR) stages 112 and 114, as well as adders 116 and 118. The letterbox detection circuit 100 operates on digitized luminance data Y.sub.-- IN. The input filter is utilized in order to improve noise performance and make detection more reliable. The filter is essentially two cascaded FIR stages, having a transfer function as follows: ##EQU1## The output of each stage is truncated to eight bits (divided by two) to maintain a DC gain of one.

The LINE MAX detector 120 includes two registers. The first register contains the maximum pixel value (MAX PIX) at the current point in the line period. It is initialized at the beginning of every line period to a value of 80h. The value of 80h represents the minimum possible value for an eight bit number in two's complement format (the most significant bit being the sign). The circuit is enabled by a signal, designated LTRBX.sub.-- EN, which goes high for approximately 70% of the active video line. The second register contains the maximum pixel value (MAX LINE) for the entire previous line, and is updated once per line period. Incoming luminance data Y.sub.-- IN is compared to the current maximum pixel value stored in the MAX PIX register. If it exceeds the register value, the MAX PIX register is updated on the next clock cycle. At the end of the video line, MAX PIX register will contain the maximum value over the entire portion of the line for which it was enabled. At the beginning of the next video line, the value of the MAX PIX register is loaded into the MAX LINE register and the MAX PIX register is reloaded with 80h.

The line minimum detector 122 works in an identical manner except that a MIN LINE register will contain the minimum pixel value for the previous line. A MIN PIX value is initialized to a value of 7Fh, which is the maximum possible pixel value for an eight bit number in the two's complement format.

The output section 124 will take the MAX LINE register value and the MIN LINE register value, and store them in eight-bit latches that are updated once per line. Two gradients are then calculated, namely the positive gradient and the negative gradient. On the first line in a field where either of these gradients is positive and greater than a programmable threshold, an enable signal is generated which allows a first line register to be loaded with the current line count value. On every line where either of the gradients is positive and exceeds the programmable threshold, another enable signal is generated which allows a last line register to be loaded with the current line count value. In this manner the last line register will contain the last line in the field where the threshold was exceeded. At the beginning of every field, the circuit is reinitialized, and the values in the first line and last line registers are loaded into respective letterbox end registers. The LTRBX.sub.-- BEG and LTRBX.sub.-- END signals represent those line numbers which mark the beginning and end respectively of a letterbox signal, and can be used as control signals for enlarging the picture, either by signal processing techniques, manipulation of the deflection circuits or some combination thereof.

Generation of the LTRBX.sub.-- EN signal, which enables operation of the line maximum and line minimum detectors 120 and 122 is controlled to avoid responding to logos in the border areas. The LTRBX.sub.-- EN signal is generated by comparators 102 and 104 and J-K flip/flop 106. Each horizontal line is deemed to have 1024 pixels, for purposes of illustration, as shown in FIG. 5. The first 150 pixels, counted from the horizontal synchronizing pulse, occur during horizontal blanking. The next 860 pixels are active pixels, so that the active video extends from pixel 151 through pixel 1010. Pixels 1011 through 1024 represent the front porch of the video signal, and are also not deemed active video. Moreover, each pixel or pixel position is assumed to be represented by a ten-bit number. Comparator 102 has an A input on line 103 which represents a first pixel. Comparator 104 has an A input on line 105 which represents a second pixel. The B input of each of the comparators 102 and 104, on line 107 is a signal representing a running pixel count H.sub.-- COUNT, which represents the output of a pixel counter, not shown, operating for example at 1024f.sub.H. In order to simplify hardware requirements, the signal H.sub.-- COUNT is the eight most significant bits of a ten-bit number, which is the equivalent of dividing the ten-bit number by four. The fixed pixel count inputs to the comparators must therefore also be one-fourth of the desired value. Accordingly, number "68" on line 103 actually represents a pixel with four times that value, namely pixel "272". Similarly, number "218" on line 105 also represents a pixel with four times that value, namely pixel "872". As a result, the sampled or tested portion of each line is substantially centered, ignoring the portion of each line corresponding to horizontal blanking, with approximately 150 pixels on each side not being tested. In each horizontal line tested, approximately 600 pixels out of 860 pixels of active video are sampled, approximately 70% of those pixels having active video information.

When H.sub.-- COUNT reaches "68" an output on line 109 sets the J-K flip/flop 106 and the Q output goes high, generating the LTRBX.sub.-- EN signal to permit operation of detectors 120 and 122. When H.sub.-- COUNT reaches "218" an output on line 111 resets the J-K flip/flop 106 and the Q output goes low, inhibiting the LTRBX.sub.-- EN signal and preventing the operation of detectors 120 and 122. Letterbox detection is thereby enabled for each video line only after a delay of approximately 16.8 microseconds from the horizontal synchronizing signal. Letterbox detection for each video line is disabled approximately 9.4 microseconds before the end of each video line. This provides enough video data processing for reliable detection of letterbox signals, and at the same time, ignores all video information in the areas most likely to contain only logos. Storm warnings, subtitles and other useful information will still be detected, for the purpose of modifying the otherwise substantially fully enlarged wide screen display.

Video cassette recorders utilizing the VHS format, or any VCR of the helical scan type, will insert a head switching transient, which might be mistaken for active video, anywhere from 5-8 lines prior to the vertical synchronizing signal (V.sub.-- SYNC), causing incorrect letterbox detection. In addition, it is necessary to determine the full extent of any interference which may be caused by the vertical blanking interval and any auxiliary information which might be contained therein, such as closed captioning information. In order to avoid false detections based on closed captioning information and on VCR head switching transients, the outputs of the LINE MAX and LINE MIN registers 120 and 122 are prevented from being loaded into the nine-bit latches of the output section 124 by an enable signal which is allowed to occur only in a range of horizontal lines in each field which avoids the regions of head switching transients and vertical blanking, for example between horizontal lines 24 and 250 of each field.

A circuit for controlling a signal, designated VERT.sub.-- EN in FIG. 4, for enabling loading of the nine-bit latches in the output section 124, is formed by comparators 130 and 132, a J-K flip/flop 134, and a horizontal line counter, not shown. The line counter can be cleared in such a way that successive fields are quantized into 262 and 263 lines respectively. In other words, line 0 begins on the horizontal synchronizing signal immediately following the leading edge of the vertical synchronizing signal V.sub.-- SYNC. The beginning value of VERT.sub.-- EN can be set to a line count of 24 and the end value can be set to a line count of 250. This eliminates interference during the vertical blanking interval, which can extend from three lines before V.sub.-- SYNC, or a line count of 250, to 14 lines after V.sub.-- SYNC, or a line count of 16. This also eliminates any interference from head switching transients, which can occur from five to eight line before V.sub.-- SYNC, or a line count of 254 to 257.

The foregoing relationships are illustrated in the drawings. FIGS. 6(a), 6(b) and 6(c) show the VERT.sub.-- EN signal, the V.sub.-- SYNC signal and the horizontal line count for a first of two successive fields, quantized to 262 horizontal lines designated "0" through "261". The VERT.sub.-- EN signal is disabled from line 250 until line 24, 36 lines in all. FIGS. 7(a), 7(b) and 7(c) show the VERT.sub.-- EN signal, the V.sub.-- SYNC signal and the horizontal line count for a second of two successive fields, quantized to 263 horizontal lines designated "0" through "262". The VERT.sub.-- EN signal is also disabled from line 250 until line 24, but for 37 lines, because the second field is one line longer. The additional line, designated line "262", occurs during vertical blanking.

Referring again to FIG. 4, comparator 130 has an A input on line 131 which represents a first horizontal line count, for example "24". Comparator 132 has an A input on line 133 which represents a second horizontal line count, for example "250". The B input of each of the comparators 102 and 104, on line 107, is a signal representing a running horizontal line count L.sub.-- COUNT, which represents the output of a horizontal line counter, not shown.

When L.sub.-- COUNT reaches "24" an output on line 137 sets the J-K flip/flop 134 and the Q output goes high, generating the VERT.sub.-- EN signal to permit loading of the latches in the output circuit 124. When L.sub.-- COUNT reaches "250" an output on line 139 resets the J-K flip/flop 134 and the Q output goes low, preventing the latches in output circuit 124 from being loaded Letterbox detection is thereby enabled only for video lines not having head switching transients or during vertical blanking. Letterbox detection is enabled by this part of the circuit 100 for a majority of the active video lines in each field, providing enough lines of video data for reliable detection of letterbox signals.

The automatic letterbox detector described herein may be operable at all times, or may be operable whenever automatic letterbox detection is selected manually as a mode of operation, or may be available as one of a plurality of letterbox detection modes of operation, when automatic letterbox detection in general is selected.