Method and Apparatus for Generating a High Definition Electronic Signal From a Line Scan of a Color Original

1. Technical Field

This invention pertains to the field of electronic image generation and, more particularly, to scanning apparatus for producing electronic picture signals from a color original and to signal processing techniques for use therewith.

2. Background Art

Although generally useful in the electronic imaging art, this invention has special application to a linear array film scanner used in a telecine machine for producing a television signal from a motion picture film. A linear array film scanner typically uses a light-sensitive linear charge-coupled device (CCD), which provides a serial output representing a line of a television raster. For color television, a film scanner usually includes an assembly of three separate CCD arrays, one each for red, green and blue. The film is driven at a uniform rate between the linear array assembly and a light source in a direction perpendicular to the linear dimension of the sensor arrays, and a beam splitting optical system images an illuminated section of film on each CCD array. The film motion provides the vertical (frame) scan and the linear cycling of the CCD arrays provides the horizontal (line) scan. Such a scanner is described in U.S. Pat. No. 4,205,337.

Instead of using separate linear devices, U.S. Pat. No. 4,736,251 discloses three CCD line sensors formed on a single solid state substrate. Because different illuminated sections of the film are imaged on the respective line sensors, the signals output from the sensors must be corrected by using shift registers or memory to obtain identical timing in the vertical direction. The '251 patent also discloses a CCD array incorporating four line sensors--three color sensors and a fourth line sensor for deriving a luminance signal. According to the patent disclosure, it is then possible to obtain a luminance signal having a higher resolution than that composed from the color signals, thereby allowing the capacity of memories for storing the color signals to be smaller than that for storing the luminance signal.

The extension of linear array techniques directly to high definition television, which requires more lines of much higher resolution than in conventional television, is difficult for several reasons. To begin with, existing arrays will not work fast enough to read out at the data rates required for high definition scanning. For example, a high definition frame of 1920 pixels/line for 1035 lines/frame requires a very high output data rate of at least 60 mHz--partly due to higher resolution in the horizontal (line) scanning direction and partly due to increased line scans in the vertical (frame) scanning direction. (Indeed, the data rate of 60 mHz would only be adequate if the full frame height could be used to provide data. In practice, because the high definition aspect ratio is 16:9 whereas the conventional motion picture film aspect ratio is 4:3, only about 62% of the film frame height is used. As a consequence, a much higher data rate of about 100 mHz or more is needed.) This problem can be addressed, albeit with additional complexity, by interleaving the output of the photosites to two or more output registers to lower the output clock frequency for a given data rate (which is useful in the line scanning direction, see U.S. Pat. No. 4,330,793) or by simultaneously scanning paired line sensors oriented to adjacent lines to double the number of lines for a given line rate (which is useful in the frame scanning direction, see published U.K. Patent Application GB 2191061).

More importantly, considering the very short linear scanning times involved in reaching the required resolution, the sensitivity of existing devices is not high enough to provide good signal-to-noise performance. Providing a plurality of output registers or paired sensors is a way of accommodating the shorter line times and higher data rates with a reachable clock frequency; by themselves, such techniques do nothing to increase sensitivity of the sensors. Thus a high definition film scanner operating linear arrays according to known techniques is plagued with a complex read-out architecture yielding a relatively inadequate signal-to-noise performance.

The invention departs from the prior art in several aspects. First, the desired high definition output signal is obtained from a high definition detail component and a plurality of lower definition color components. A luminance array produces a full resolution luminance signal including high definition detail, i.e., a signal meeting the requirements of the high definition standard in use. A plurality of color arrays produce a like plurality of lower definition color signals. The high definition output signal is obtained by combining high definition detail (extracted from the full resolution luminance signal) with the low resolution color signals, which are registered with the detail and interpolated both horizontally and vertically to match the high definition standard.

As the color arrays produce lower definition signals, the problem of a high read-out rate is restricted to the luminance array. The luminance array is divided into substantially like-sized segments and multiple registers are then used, one for each segment, to reach the data rate required for high definition scanning. Furthermore, by originating the detail separate from the color, a significant noise improvement can be obtained from the color arrays. Since each color signal is a lower definition, unsharp representation of an image in the horizontal (line) scanning direction, a larger photosensitive area is devoted to each photosite in the horizontal direction. In addition, by reading the color signal for every other line and thereby decreasing resolution in the vertical direction, the color photosites then can be larger in both dimensions in proportion to the ratio of the numbers of luminance to color photosites. For example, if there are twice as many luminance photosites as color photosites, the color photosites have linear dimensions twice those of the luminance photosites. Insofar as integration time is doubled (every other line scanned) for a quadrupled area (linear dimensions doubled), an eight-fold signal-to-noise improvement is realized.

Inasmuch as a larger light collection period (integration time) is possible with each color array, and these arrays are not read out on every line of the film scan, a single readout register for each color array provides an adequate rate for an achievable clock frequency. By additionally selecting the photosites in each color array to be an integral multiple of the photosites in a luminance segment, a common clock frequency can be used for driving all the output registers . . . leading to a considerable simplification in the processing associated with the sensor assembly. With such a common clock, and, e.g., twice the photosites in each color array for the number in a luminance segment, two luminance lines are read out for each line of color, thereby directly providing lesser resolution in the vertical direction without the complexity of a different clocking scheme for that purpose.

The invention will be described in relation to the drawings, in which

FIG. 1 is a block diagram of a film scanner for generating a high definition electronic signal according to the invention;

FIG. 2 is a detailed diagram of the unsharp color sensors and the high resolution luminance sensor shown in FIG. 1;

FIG. 3 is a detailed diagram of an alternative embodiment for the sensors shown in FIG. 2; and

FIG. 4 (A-F) are several abbreviated views of the color and luminance sensors, showing in particular exemplary photosite arrays proportioned according to an aspect of the invention.

Referring first to FIG. 1, a film transport 10 advances a motion picture film 12 at a substantially uniform speed through a film gate 14 from a supply reel 16 to a take up reel 18. A light source 20 generates a light beam that is directed through a circle to line converter 22 and focussed upon a linear section of the film 12 in the film gate 14. The light is modulated by the image in the film 12 and transmitted through an objective lens 24 to a beam splitter 26, which transmits one portion of the modulated light to an unsharp color sensor 28 and reflects the other portion to a high resolution luminance sensor 30. A sprocket wheel 32 engages perforations in the motion picture film 12 to provide a scanner controller 34 with film speed information, particularly in relation to speed variations thereof.

The color sensor 28 comprises three color sensors respectively sensitive to red, green and blue light. The color sensor structure, which is better shown in FIG. 2, includes a red-sensitive linear CCD array 36r including photosites R1, R2, . . . , a green-sensitive linear CCD array 36g including photosites G1, G2, . . . , and a blue-sensitive linear CCD array 36b including photosites B1, B2, . . . . In this embodiment, each array 36r, 36g, 36b includes 960 active photosites. The spectral sensitization of the photosites is provided by linear color filter strips (not shown) applied to the sensor 28 and overlying the arrays 36r, 36g and 36b. Each linear array has associated therewith a respective transfer gate 38r, 38g, 38b separating each array from a respective output shift register 40r, 40g, 40b. Image charge accumulated in the charge wells of the respective arrays is transferred to the respective shift registers 40r, 40g, 40b by dropping the appropriate transfer gate 38r, 38g, 38b low. A sensor clock generator 42 (FIG. 1) provides an appropriate gating signal to the transfer gates 38r, 38g, 38b to effect charge transfer. In addition, the sensor clock generator 42 provides a clock signal of predetermined frequency for shifting the respective image signals from the output registers 40r, 40g, and 40b.

As further shown in FIG. 2, the luminance sensor 30 includes a linear array 44 sensitized to light having a spectral composition approximating a luminance function. In addition, the signal level from the high resolution luminance sensor 30 benefits from the wide spectral response of the luminance sensitivity function to which it is responsive. The linear array 44 produces a full resolution signal according to the requirements of the high definition standard in use, that is, the luminance array 44 includes a sufficient number of active photosites so as to correspond to the line resolution of the high definition standard. In this embodiment, the luminance array includes 1920 active photosites. As the clocking rate for such a large number of photosites poses a problem in relation to available output devices, the array 44 is divided into four like-sized segments 44a, 44b, 44c and 44d--each including a subset P1, P2, . . . of photosites; in this embodiment, there are 480 active photosites in each segment. A transfer gate 46, connected to the sensor clock generator 42, is juxtaposed between the segments 44a, 44b, 44c, 44d and a corresponding plurality of output shift registers 48a, 48b, 48c, 48d, which are also connected to the sensor clock generator 42. With such an architecture, the image charge in all photosites in the array 44 are simultaneously gated to the output registers 48a, 48b, 48c, 48d and simultaneously shifted therefrom at one-quarter the clock frequency ordinarily needed for the high definition data rate, i.e., the data output rate of the luminance sensor 30 is four times the clock frequency applied to the individual output registers 48a, 48b, 48c and 48d.

In the preferred embodiment, the film may run at a speed slightly in excess of 30 frames per second (actually, 31.5 f.p.s.), therefore suggesting a data rate of 120 mHz for the full high definition image. The sensor clock frequency for shifting the image signals from the output registers 40r, 40g, 40b, 48a, 48b, 48c, and 48d, consequently, is fixed at 30 mHz, i.e., one-quarter of the high definition data rate. The sensor clock generator 42, furthermore, is responsive to film speed variation via the scanner controller 34, accordingly varying the duration of a "dead period" at the end of each line to accommodate the change in line integration time needed if the film speed is varied. (The sensor clock frequency is therefore selected to accommodate the highest film speed that will be encountered.)

The sensors 28 and 30 are illustrated next to each other in FIG. 2 to emphasize the relative arrangement of the linear arrays 36r, 36g, 36b and 44. It is of particular note that the color and luminance sensors 28 and 30 cover substantially the same linear dimension, but with different resolution. Low resolution, or unsharp, color is provided from the color arrays 36r, 36g, 36b by fewer photosites (960) than for luminance (1920). This provides lower color resolution in the horizontal scanning direction and allows the color photosites to be accordingly larger, which has the advantageous affect of increasing the signal-to-noise performance. In addition, color resolution in the vertical scanning direction is reduced by scanning one line of color for every two lines of luminance, thereby allowing the vertical color dimension to be increased (doubled) in relation to the luminance photosites. The total area of the color photosites is accordingly four times that of the luminance photosites. Taking further into account that the integration time of each color photosite is twice that of a luminance photosite (because each color line is read out half as frequently), the signal from the color photosites realizes an eight-fold noise improvement.

The three color arrays 36r, 36g and 36b are offset from each other by an integral number of lines so that, at any instant, three separate lines from the film 12 are imaged on the sensor 28. With the color arrays collecting light over a two (high definition) line period for each color line read out, the color arrays are spaced by an integral multiple (m) of twice the high definition line spacing (2n). This is shown in FIG. 2. The luminance sensor 30 may be aligned to a fourth, separate line or, preferably, is aligned via the beam splitter 26 to one of the color lines, say the line imaged upon the green-sensitive array 36g. The horizontal (line) scan is provided by transferring image charge from the linear arrays to the output registers, and accordingly clocking the signals from the registers. The vertical (frame) scan is provided by the motion imparted to the film 12 by the film transport 10 (FIG. 1), and typically would allow for more than a thousand (say, 1035) lines in a frame.

A further advantage of the invention is obtained by structuring the luminance segments 44a, 44b, 44c, 44d relative to the color arrays 36r, 36g, 36b so that the number of photosites in a color array is an integral multiple of the number of photosites in a luminance segment. For the preferred embodiment shown in FIG. 2, each color array has twice as many photosites as a luminance segment; specifically, each color array 36r, 36g, 36b has 960 photosites in relation to 480 photosites for each luminance segment 44a, 44b, 44c, 44d. The luminance registers 48a, 48b, 48c, 48d are read in parallel once for each line while the color registers 40r, 40g, 40b are read in parallel once for every two lines of luminance. (Such a benefit is obtained when the number of photosites in each segment comprises a submultiple of the number of photosites in a color array, i.e., which is a special case of the "integral multiple" relationship.) Since there are twice as many photosites per register in color as in luminance and the color is read half as frequently, the numbers of photosites read per second is the same for color and luminance; consequently, only a single clock frequency is required to read out all of the registers 40r, 40g, 40b, 48a, 48b, 48c, 48d. A single clock frequency thus interrogates both color and luminance, providing both the necessary data rates and the desired vertical and horizontal resolution.

Referring again to FIG. 1, the unsharp color sensors 28 provide three channels of color data to an analog-to-digital (A/D) converter 50 and the luminance sensor 30 provides four channels of luminance data, one from each register 48a, 48b, 48c, 48d (FIG. 2), to another analog-to-digital (A/D) converter 52. Because the three color arrays 36r, 36g, 36b are spaced on the sensor 28 in the direction of film motion, the photosites being read at a given time correspond to different vertical locations on the film frame. This fixed mis-registration is corrected by a color registration circuit 54 shown in FIG. 1, which includes suitable line delays for registering the color lines with each other and with the luminance line. For instance, for film moving as shown in FIG. 2, a 2(m.times.2n) delay is imparted to the red signal from the register 40r, a (m.times.2n) delay is imparted to the green signal from the register 40g, and no delay is given to the blue signal. In addition, depending on the registration of the luminance line relative to the color lines, a line delay may be inserted into the luminance channel(s) to register the luminance lines with a selected color line.

Since the digital color values represent lower resolution data than the digital luminance values, additional color values are generated in a color interpolation circuit 56 in both the horizontal and vertical directions. Various conventional interpolation techniques will work. For example, the additional values for the extra high definition locations, which fall between the existing color values, may be a simple repeat of the existing color value before it in the horizontal direction and a repeat of the thus-interpolated line values in the vertical direction. Alternatively, the existing values can be averaged in the horizontal direction and then further averaged in the vertical direction. In either case, the number of color values are expanded to match the number of luminance values in a high definition line.

Meanwhile, high frequency detail is extracted from the four channels of luminance data by a detail extraction circuit 58, which includes an array of high pass filters (not shown). Various conventional filtering techniques can be used; for instance, the signal value of each photosite can be subtracted from an average of the neighboring photosites, either in one or two dimensions. This can be repeated for differently-sized neighborhoods to give a set of spatial frequencies representative of various levels of detail in the luminance signal. The signals in the four luminance channels are aligned "end-to-end" to correspond to a physical line in a reformat circuit 60, which may, for example, be a conventional multiplexer triggered in quarter-line sequence to output a continuous detail signal. In addition, depending on the orientation of the luminance line scan in relation to the color line scan, additional delay elements may be included prior to the reformat circuit 60 to register the detail signal with the registered color signals.

The detail is added to each channel of color in an addition circuit 62, forming thereafter a full resolution, high definition color output signal. This high definition output signal is applied to an image store 64, which includes a first framestore 64a and a second framestore 64b. Recalling now that the high definition signal to this point is a sequential signal, an interlace controller 66 loads a video frame sequentially into one framestore while extracting video fields (of a previously loaded frame) in interlace format from the other framestore. A digital red, green, blue high definition field signal is thus provided at the output of the image store 64 for further use, which may include immediate broadcast transmission or recording, e.g., on video tape (after suitable standards conversion or encoding, as necessary). In any event, such further use is not to be part of the present invention.

Instead of the separate luminance sensor 30 and color sensor 28 as shown in FIG. 2, an alternative architecture based on a quad-linear array 70 is shown in FIG. 3. Similar reference characters are used inasmuch as the linear arrays for color and luminance, the transfer gates and the output registers are basically the same for either embodiment. The difference in FIG. 3 is that all the sensor components are on one solid-state substrate 72. This configuration eliminates the beam splitter 26 in FIG. 1 and has each array--the luminance array 44 as well as the color arrays 36r, 36g and 36b--registered to a different vertical location on the film frame. The spacing of the luminance and color arrays is shown in FIG. 3 to be equivalent to (pn+n/2), where n is the high definition line spacing and p is an integer. As the color arrays 36r, 36g and 36b collect light over a two (high definition) line period and are read at half the rate as the luminance array 44, the half-line offset n/2 is needed to ensure that when luminance and color are read out the spacing between them is an integral number of luminance lines. The integer p provides additional luminance lines of spacing as determined by physical requirements in the integrated circuit chip design. As before, the color-to-color spacing is an integral multiple of twice the high definition line spacing. The color registration circuit 54 is shown with the necessary delays to correct the mis-registration of the color channels relative to the luminance channel. An additional delay block 74 is shown in the luminance channels to provide the delay required by the chip layout for the luminance array 44 to register with the last array scanned, that is, the blue color array 36b.

FIG. 4 shows a number of alternative sensor structures for the luminance (1) array and the color (c) array, which together utilize a common clock frequency in combination with multiple register readout (the numbers refer to an exemplary number of photosites in each segment). FIG. 4A illustrates multiple register readout of high resolution luminance in a structure that uses a double frequency clock (60 mHz) in comparison to the clock of the preferred embodiment (FIG. 4C). The arrangement of FIG. 4A provides for reading color on every line, albeit vertical color resolution (which is controlled by the pixel size) is still only half that of luminance. FIGS. 4B and 4D show the same horizontal resolution in color and luminance, instead using the segment approach and a common clock frequency to obtain lowered vertical resolution for color--one line of color for two lines of luminance in the case of FIG. 4B and one line of color for three lines of luminance in the case of FIG. 4D. FIG. 4E illustrates a segment approach in both luminance and color to allow a single clock (e.g., 30 mHz) to read out both the high resolution luminance and the unsharp color, but with the same vertical resolution as outlined in connection with FIG. 4A. FIG. 4F adds to the structure of FIG. 4E by further segmenting the luminance to additionally allow a yet lower clock frequency (e.g., 15 mHz) with continued unsharp vertical resolution in color . . . in this case, one line of color for every two lines of luminance. FIG. 4C is the preferred embodiment, as previously discussed and shown here for comparison. In each case the photosites in a color array are an integral multiple of the photosites in a luminance segment, that is, for these examples, an integral multiple of one (FIG. 4A, 4E), two (FIGS. 4B, 4C, 4F) and three (FIG. 4D). In some of these examples (for instance, FIGS. 4A and 4E) vertical color resolution is predominantly controlled by photosite size while in other examples (FIGS. 4B, 4C, 4D and 4F) it is controlled by the integration period. It is seen, moreover, that the term "color array" includes one of several color arrays so that, in FIGS. 4(E) and 4(F), the integral multiple comparison is between one color array (of two) and one luminance segment (of four, or eight, respectively).

While the invention has been disclosed for use with a telecine machine that generates a television picture signal from a motion picture film, it is intended that the disclosed film scanner could be used with other types of electronic imaging machines. One example is a color copier, which also may be used to illustrate a number of modifications that are included within the concept of the invention. For example, rather than moving a color original past the linear arrays, the color copier may instead provide means, such as an oscillating mirror, for moving the scanning beam relative to the linear array (i.e., both the original and the sensors are stationary). Alternatively, the linear arrays can be designed to move across the original. Moreover, the image modulation imparted to the scanning beam can originate from transmission through the original or reflection from the original. Finally, while digital processing is preferred, the concepts of the invention can be implemented in a wholly, or partially, analog processing system.

The invention has been described in detail with particular reference to a presently preferred embodiment, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.