Solid State Television Camera

1. Field of the Invention

This invention relates to a solid state imaging apparatus, wherein solid state imaging devices such as COD (charge-coupled device) imagers are employed.

2. Brief Description of the Prior Art

Recently, the development of solid state television cameras has been very active. They employ solid state imaging devices such as photo-diode arrays, CCD imagers, BBD (bucket brigade device) imagers, and so on, instead of vidicon tubes. These solid state imaging devices have limited number of picture elements, therefore, it is very difficult to improve resolution of an image. If a plurality of imaging devices are employed in one camera system, the resolution of picked up images can be improved by suitable displacement of imagers and signal processings as shown in U.S. Pat. No. 3,975,760, which was assigned to the same assignee as the present application.

That is, three imaging chips or devices 1G, 1R and 1B are disposed with respect to a single focused image as shown in FIG. 1. In this case, with the focused image on the imaging device 1G being as a reference, the imaging device 1R is shifted therefrom in the horizontal direction by a distance corresponding to a phase .theta..sub.12 and the imaging device 1b is shifted therefrom in the horizontal direction by a distance corresponding to a phase .theta..sub.13. FIG. 1 is an example such as being arranged as a color camera system and hence color filters FR, FB and FG (not shown) are respectively disposed in front of the corresponding imaging devices 1R, 1B and 1G. Accordingly, a signal corresponding to red color is obtained from the imaging device 1R, and similarly a signal corresponding to blue color from the imaging device 1B and a signal corresponding to green color from the imaging device 1G, respectively.

In FIG. 1, if the alignment pitch of horizontally arranged picture elements 2 is taken as x and scanning time of this pitch x is .tau..sub.H, sampling frequency f.sub.c for the focused image can be expressed as f.sub.c =1/.tau..sub.H. In FIG. 1, y indicates an aperture width of each picture element 2, and .tau..sub.0 a scanning time of this aperture width y. Further, the phases .theta..sub.12 and .theta..sub.13 correspond to the distance of 1/3 x and 2/3 x, respectively.

In the case that the spatial positioning of the focused image and the imaging devices is selected in a manner as mentioned above, if the respective imaging devices 1G, 1R and 1B are read out at proper sampling timings corresponding to the above positioning, the output levels and the phase relation as shown in FIG. 2 are obtained. FIG. 2 shows levels and phase relation of respective components of a composite output, in which Y.sub.B designates a base-band component and Y.sub.S a side-band component. Carriers of the side-band component Y.sub.S (sampling pulses) C.sub.G, C.sub.R and C.sub.B have phase relation as shown in FIG. 2. It is assumed that the spatial phase relation can be correctly selected as .theta..sub.12 =2/3.pi. and .theta..sub.13 =4/3.pi.. In this case, upon picking up a black and white image, if outputs of three imaging devices are added together with their levels being made equal so as to produce a luminance signal, the side-band components Y.sub.S are cancelled by taking vector sums, while only the base-band components Y.sub.B remain.

As described above, when the alignment adjustment among the devices 1G, 1R and 1B is perfect, the side-band components are cancelled out so that a desired aim can be achieved. Meanwhile, if the alignment adjustment is imperfect, the side-band components are not cancelled but remain in the base-band components Y.sub.B. Therefore, the quality of pictures can not be improved and also the frequency band can not be widened. As a matter of fact, however, each imaging device is small in size. Besides, with a device incorporated with several hundreds of picture elements in the horizontal direction, it is quite difficult to correctly provide a mechanical positioning corresponding to a time 1/3.tau..sub.H.

A first object of this invention is to provide a television camera system using a plurality of image sensors.

A second object of this invention is to provide a solid state camera which uses a plurality of image sensor chips to exhibit good resolution.

A third object of this invention is to provide a solid state color television camera which uses a plurality of image sensor chips with limited number of picture elements to produce a luminance signal having a high resolution.

As disclosed in U.S. Pat. No. 3,975,760 which has been previously filed and patented, signal processing with a high resolution can be effected by precise chip alignment and suitable signal handling. In the present invention, however, the precise chip alignment is not required. For this reason, an error signal compensating circuit is newly provided.

Each CCD imager provides an output containing base-band components and side-band components as mentioned above. The phases of the side-band components are much influenced by the chip alignment and read-out clocks, while the base-band components are not. Therefore, the original base-band components from respective CCD chips are cancelled to eliminate undesired signals from the resultant output signal. In the practical embodiment, a frequency converter is employed to interchange respective frequencies of side-band and base-band components. The original base-band components can be cancelled out either before the frequency conversion or after the frequency conversion.

This invention is applicable not only to a black and white television camera using a plurality of CCD imagers but also to a color camera system. When a luminance signal with high resolution is produced by the color camera, it is necessary for the levels of signals from respective chips to be properly adjusted to cancel base-band components upon taking a black and white image.

The other objects, features and advantages of this invention will be apparent from the following description taken in conjunction with the accompanying drawings.

FIG. 1 is a view showing an arrangement of imaging devices used for explaining the invention;

FIG. 2 is a graph showing frequency characteristics of outputs from the imaging devices shown in FIG. 1;

FIGS. 3A through 3C and FIGS. 4A through 4E are views showing phase relations which are used for explaining the invention;

FIG. 5 is a view showing another arrangement of imaging devices according to this invention;

FIG. 6 is a graph showing frequency characteristics of outputs from the imaging devices shown in FIG. 5;

FIG. 7 is a block diagram showing one embodiment of this invention;

FIG. 8 is a view showing frequency characteristics used for explaining an operation of the embodiment shown in FIG. 7;

FIG. 9 is a block diagram showing another embodiment of this invention;

FIGS. 10A and 10B are views showing frequency characteristics of filters used in this invention;

FIG. 11 is a partial view of the main system showing a further embodiment of this invention; and

FIG. 12 is an illustration of the optical means for projecting the image of the object on the CCD imagers.

Before describing constructive arrangements of the invention, theoretical analysis will be given in conjunction with FIGS. 1 to 4.

Now, if a focused image is assumed to have only a single frequency component of p cycles, the focused image i(x) of spatial expression is given as follows:

where L' is brightness level. Therefore, an output l.sub.G (t) from the device 1G is expressed as follows: ##EQU1## Further, R.sub.G (f) (f=p, 1/.tau..sub.H -p, 1/.tau..sub.H +p, . . . ) indicates overall frequency characteristics after spatial sampling by the picture elements 2 contained in the imaging device 1G.

Outputs l.sub.R (t) and l.sub.B (t) of the other devices 1R and 1B can also be expressed in the same manner as the equation (2). In this case, if the spatial positioning from the reference phase is taken as .theta. as mentioned above, a time deviation from the reference phase is taken as .PSI., and further the device 1G is selected to be a reference as shown in FIG. 1, the device 1R is adapted to be shifted by .theta..sub.12 in space and .omega..sub.12 in time. Similarly, the device 1B is adapted to be shifted by .theta..sub.13 in space and .PSI..sub.13 in time, respectively, as a result, outputs l.sub.R (t) and l.sub.B (t) are expressed as follows: ##EQU2## The phase relations of these outputs l.sub.G (t), l.sub.R (t) and l.sub.B (t) are shown in FIGS. 3A, 3B and 3C.

In the case that, for example, the spatial phase .theta..sub.12 of the intermediate device 1R is not a proper phase corresponding to 2/3.pi. relative to the output l.sub.G (t), the phase .PSI..sub.12 has to be also changed that much in accordance with the phase .theta..sub.12 containing the above deviation, so that the following relation is obtained.

Even in such a case, as will be apparent from the equations (4) and (5), the phase .PSI..sub.12 deviated from the proper state exists in the phase terms of side-band components. Accordingly even if the outputs l.sub.G (5), l.sub.R (t) and l.sub.B (t) are added together, side-band components will still exist.

That is, in a composite signal of outputs l.sub.R (t) and l.sub.B (t) as shown in FIG. 4B, the phases of side-band components are not correctly opposite to those of side-band components of the output l.sub.G (t) shown in FIG. 4A, so that vestigial components exist in the composite signal of outputs l.sub.G (t), l.sub.R (t) and l.sub.B (t) as shown in FIG. 4C. In this case, however, since the base-band components of respective signals are coincident in phase, if the composition such as l.sub.G (t)-[l.sub.R (t)+l.sub.B (t)] is carried out, the respective base-band components are cancelled by each other, resulting in zero, while the side-band components are obtained as a vector sum as shown in FIG. 4D. Accordingly, if the side-band components and base-band components are interchanged with each other as shown in FIG. 4E, converted side-band components can be utilized as a luminance signal which is free from undesired signals.

Thus, even in a case that the mechanical arrangement of respective imaging devices is not proper, it is possible to produce a luminance signal in which an error signal is cancelled. When a solid state color camera system is constructed, imaging devices 1G, 1R and 1B can be arranged as shown in FIG. 5, so that a relation of .theta..sub.12 =.theta..sub.13 =.pi. is established and the levels of a composite signal may satisfy the relation G.sub.L =R.sub.L +B.sub.L upon picking-up a black and white image, where G.sub.L, R.sub.L and B.sub.L are levels of respective outputs of the devices 1G, 1R and 1B. In this case, even though .theta..sub.12 and .theta..sub.13 are not properly shifted by a distance corresponding to .pi. radian, a luminance signal having no noise can be obtained according to this invention. FIG. 6 shows levels and phase relation of signals derived from the devices 1G, 1R and 1B arranged as shown in FIG. 5.

One embodiment of this invention is shown in FIG. 7. This embodiment is used in a color television camera system having the device arrangement as shown in FIG. 5.

In FIG. 12, taken with FIG. 7, an image of an object 44 focused through a lens 45 is divided into three portions which are fed, by means of mirrors 49, 50 and 50 through color filters 46, 47 and 48, respectively to imaging devices 1G, 1R and 1B. Color filters are disposed in front of the respective devices. Horizontal shift registers (not shown) provided in the devices 1G, 1R and 1B are applied with a sampling (reading-out) clock pulse P.sub.C with a frequency of, for example, 1/.tau..sub.H through respective terminals 3G, 3R and 3B. The clock pulse P.sub.C fed to the respective chips can be arranged to have the phases equal to their spatial phases. In this case, however, the clock pulse P.sub.C having the same phase is supplied to the respective devices 1G, 1R and 1B, and after sampling, the output signals therefrom are caused to match with the spatial phases.

An output S.sub.G from the chip 1G is supplied through a sample-and-hold circuit 5G for waveform shaping to another sample-and-hold circuit 7G for phase matching. The clock pulse P.sub.C is used as each sampling signal for these circuits 5G and 7G. However, the phases of respective sampling pulses to be fed to the circuits 5G and 7G are a little different from that of the original clock pulse P.sub.C because the correct output S.sub.G can not be sampled with a sampling signal in phase with the pulse P.sub.C. To this end, a phase shifter 6G is used for the circuit 5G and a phase controller 8G is used for the circuit 7G. The phase controller 8G may be omitted because the phase-shifted pulse from the phase shifter 6G can be theoretically applied to the circuit 7G with the same effect.

The other outputs S.sub.R and S.sub.B from the imagers 1R and 1B are similarly shaped in waveform before being adjusted in phase. The sampling phases .PSI..sub.12 and .PSI..sub.13 are adjusted so as to coincide with the spatial phases .theta..sub.12 and .theta..sub.13. As a result, the following relation is obtained.

Thus, the spatial phases and the time phases are completely coincident with each other.

The phase-adjusted imager outputs S.sub.G, S.sub.R and S.sub.B from the respective circuits 7G, 7R and 7B are fed to an error compensating circuit 20 where undesired components contained therein are eliminated. The error compensating circuit 20 includes a cancelling circuit 21 for cancelling base-band components in the outputs S.sub.G, S.sub.R and S.sub.B and a balanced modulator 22 for frequency-converting the output from the cancelling circuit 21. The cancelling circuit 21 is practically formed of a subtraction circuit.

In order to cancel the base-band components, the following condition is required.

where a and b are gain ratios of the imager outputs S.sub.R and S.sub.B to the imager output S.sub.G, respectively, and L.sub.G, L.sub.R and L.sub.B are brightness of green, red and blue colors of the focused image. Of course, it is sufficient if the above condition is applicable for a black and white image to be focused. Further, when the black and white image is picked up, the circuit adjustment is made so as to establish the following relation:

Accordingly, if the values of a and b are selected in a predetermined manner, the base-band components can be completely cancelled. In other words, if S.sub.G, S.sub.B and S.sub.R are expressed by l.sub.G (t), l.sub.B (t) and l.sub.R (t) as the equations (2), (4) and (5), the base-band components are cancelled and the side-band components are obtained as the vector sum as shown in FIG. 4D.

In the case of FIG. 5, a and b are selected to be 0.5 and .theta..sub.12 and .theta..sub.13 should be .pi.. But actually .theta..sub.12 and .theta..sub.13 may be different from .pi. due to the alignment difficulty.

Now, special attention will be paid to the relation between the phases of FIG. 4C, resulting from the prior art signal processing and those of FIG. 4D resulting from the above-mentioned signal processing. At first, in FIG. 4C, the desired base-band components having frequency p and the undesired side-band components are both obtained. On the other hand, at the cancelling circuit 21, only the side-band components existing at the frequencies (1/.tau..sub.H -p) and (1/.tau..sub.H +p) are obtained in contrast with the base-band components of FIG. 4C. The phase relation shown in FIG. 4D is obtained so far as the equations (7) and (8) are satisfied. The side-band components shown in FIG. 4D are originally undesired signals. However, if these components are frequency-converted into the base-band frequency range, the above undesired signals themselves can be used as the base-band components. In this case, the frequency-converted base-band components are considered as undesired signals, but these undesired signals are completely cancelled and hence there is no influence to be exerted.

Accordingly, in this invention the balanced modulator 22 for frequency conversion is provided at the rear stage of the cancelling circuit 21 after a low pass filter 23. A modulating pulse Pm for the balanced modulator 22 is obtained from the clock pulse Pc through a phase shifter 24 so that the phase of the modulating pulse Pm is selected in accordance with a phase .PSI. (refer to FIG. 4D) which is caused by the alignment error. A modulated output from the balanced modulator 22 becomes as shown in FIG. 4E. In other words, the modulation is carried out without changing the phase of the modulating pulse Pm.

The filter characteristics of the low pass filter 23 is selected in the following manner. At first, the output l(t) of the cancelling circuit 21 is expressed as follows: ##EQU3##

When the focused image is a black and white one, the aforesaid equations (8) and (9) are established, so that the first term of the equation (1) becomes zero. In this case, when the output l(t) of the cancelling circuit 21 is directly fed to the balanced modulator 22 without passing through the filter 23, the modulated output l.sub.0 (t) from the modulator 22 is expressed as follows: ##EQU4##

In the above equation (11), the signal components of the first and second terms are base-band components converted to the high frequency range, but the signal components of and after the third term are undesired signals. Now, when the following relation is satisfied:

the components of and after the third term of the equation (11) must be kept always zero. Therefore, in order to make these undesired components zero, the frequency characteristics G(f) of the transmission path is necessary to be determined as follows: ##EQU5## Accordingly, the filter 23 to be used is selected with low-pass filter characteristics with a cut-off frequency of 1/.tau..sub.H.

As a result, the modulated output l.sub.0 '(t) of the balanced modulator 22 in the case of using the filter 23 includes only the signal components of the first term of the of the equation (11) as follows: ##EQU6##

To the output stage of the balanced modulator 22 is further connected a low pass filter 25, which is similar to the low-pass filter 23 at the front stage, to eliminate the base-band components which are modulated to (1/.tau..sub.H +p). With the signal process as mentioned above, the finally obtainable output S.sub.0 contains no side-band component in its high frequency range with the result that the alignment error is compensated for.

The frequency response of the composite output S.sub.0 is shown in FIG. 8 by a solid line. On the transmission paths of imager outputs S.sub.G, S.sub.R and S.sub.B are provided low pass filters 28G, 28R and 28B, respectively, the cut-off frequency of each being selected to be about 1 MHz, so that low frequency outputs are obtained therefrom as shown by a dotted line in FIG. 8. Thus, the low frequency outputs from the filters 28G, 28R and 28B are added with the composite output S.sub.0 at adders 29G, 29R and 29B, respectively, to produce respective outputs S.sub.0G, S.sub.0R and S.sub.0B of the imaging devices 1G, 1R and 1B. These outputs S.sub.0G, S.sub.0R and S.sub.0B are supplied to a color encorder 30 as well known, and, for example, an NTSC system color television signal is obtained from a terminal 30a. In FIG. 7, 27 designates a delay circuit.

As described above, according to the circuit arrangement of this invention, the alignment error can be completely compensated for by an electric process only, so that no influence is exerted on a picture image and the frequency range of base-band components can be enlarged.

According to my experiments, when the spatial phases as described in the embodiment of FIG. 7 are used, it is ascertained that the alignment error can be absorbed almost positively, even though they are deviated from the proper phase by about 50%. Even in the case of using the spatial arrangement as shown in FIG. 1, an allowable error which is about five times as large as the prior art can be absorbed. In other words, when two micro meters (.mu.m) is assumed to be the maximum allowable value in the prior art, the devices with alignment error up to about ten .mu.m can be fully used in a practical manner in the present invention.

In the above mentioned embodiment, the base-band components of the imager outputs S.sub.G, S.sub.R and S.sub.B are cancelled before the outputs are subjected to conversion. However, it is also possible to cancel the imager outputs S.sub.G, S.sub.R and S.sub.B after being subjected to balanced modulation. Further, at least two imaging devices are required, but they are not limited to three.

The above example is described with respect to the compensation of alignment error when the arrangement of imaging devices is positively changed. However, this invention is also applicable to the compensation of positional error in the case when the imager chips are arranged at the same position.

FIG. 9 shows another embodiment of this invention in the case of producing a luminance signal Y. In this embodiment, there are newly provided adders 31 and 32. The outputs from the low pass filters 28G, 28R and 28B are respectively applied to the adder 31 to produce a low frequency component Y.sub.L of the imager outputs S.sub.G, S.sub.R and S.sub.B. While the composite output S.sub.0 from the error compensating circuit 20 is utilized as a high frequency component Y.sub.H of the luminance signal Y. These components Y.sub.L and Y.sub.H are added to each other at the adder 32 to produce the luminance signal Y.

In the example of FIG. 7, the cut-off frequency of the filter 23 is selected to be 1/.tau..sub.H. However, when using a low-pass filter including a higher frequency portion than 1/.tau..sub.H as shown in FIG. 10A, the higher frequency portion contains side-band components of higher harmonics of (2/.tau..sub.H -p), so that after the signal conversion these undesired components will exist at the lower frequency side. For this reason, a circuit 40 for eliminating the lower frequency components, which contain undesired signals, can be provided at the rear stage of the balanced modulator 22 as shown in FIG. 11. The eliminating circuit 40 comprises a low-pass filter 41 with filter characteristic as shown in FIG. 10B by a dotted line, a delay circuit 42 and an adder 43. Thus, modulated output S.sub.0 with a band shown in FIG. 10B by a solid line is produced, in which the undesired signals are eliminated. The signal process after the modulated output S.sub.0 being obtained is the same as in FIG. 7, so that its description will be omitted.

FIG. 12 is a diagrammatic showing of the object 44, the image of which is being projected through a lens system which includes a lens 4t, via mirrors 49, 50, 51 and 52, through color filters 46, 47 and 48 respectively, onto the CCD imagers 1G, 1R and 1B.

It will be apparent that many modifications and variations may be effected without departing from the spirit and scope of the novel concepts of the present invention.