Image Pickup Apparatus With Excess-Charge Control

1. Field of the Invention

The present invention relates to an image pickup apparatus using a solid state image pickup device.

2. Description of the Prior Art

In the case of using a solid state image pickup device, for example, a CCD (charge coupled device), in e.g., a television camera, it has conventionally been proposed that the accumulating time of the image pickup device can be reduced to less than the field (or frame) period of a television signal by changing the method of driving the solid state image pickup device.

For instance, in the U.S. patent application Ser. No. 596,404 filed on Apr. 3, 1984, there has been proposed by the assignee of this application the driving method whereby, for example, in solid state image pickup devices of the frame transfer type, the charges in the photosensing unit to perform the photoelectric conversion and the accumulation of the charges are drained by once vertically transferring them within the vertical interval and the remaining interval of the vertical interval is further used as the substantial accumulating time.

According to this method, in the television camera of, e.g., the NTSC system, although the accumulating time is ordinarly 1/60 second, this time can be set to 1/120 second, 1/500 second, or the like; therefore, there are effects such that the iris need not be reduced even in the case of a large amount of incident light, an image of an object moving at a high speed is not moved and the like. However, while the foregoing operation is being performed, in particular, if the substantial accumulating time is reduced when the charges which are accumulated for the period of time within the vertical interval are drained without taking them out as a signal as mentioned above, the amount of charges to be drained will have considerably increased. For example, when the substantial accumulating time (t.sub.1) is set to 1/500 second, the ratio of the accumulating time t.sub.2 of the charges to be drained to the time t.sub.1 becomes

Namely, the ratio becomes about 7.3. Assuming that the level of charges which are accumulated for the period of time t.sub.1 is the normal level, charges which are 7.3 times larger than normal level are accumulated for the time t.sub.2. In the case where such a large number of charges are accumulated in this manner, it is very difficult to drain all of these charges and prevent their having an influence on the substantial accumulating time. Particularly, a great quantity of charges are generated in what is called a highlight portion or the like on the display screen and when the vertical transfer is performed within the vertical interval to drain the charges, a large amount of charges remain, so that an image which is displayed on the screen is adversely influenced.

In the output of the image pickup device, the amount of those residual charges becomes four to ten times (nearly the level of the saturation signal of the image pickup device) as large as the signal level. Thus, such a high signal level causes abnormal response of the signal processor and malfunction of the gain control or iris control.

In addition, an image pickup apparatus in which the charges in both the photosensing unit and the storage unit are drained within the vertical interval has also been proposed. According to this apparatus, no output signal is obtained for the interval before the transfer within the vertical interval, so that the iris control, gain control and the like become inaccurate.

It is an object of the present invention to solve the above-mentioned conventional drawbacks.

A second object of the invention is to provide an image pickup apparatus in which the iris control and gain control operations are properly performed even in the case where the substantial accumulating time is reduced due to the draining of a signal from an attached photoelectric converting device within the photoelectric converting and charge accumulating intervals.

Another object of the invention is to provide an image pickup apparatus which performs the iris control and gain control operations in response to an output of a photoelectric converting device, wherein even in the case where an unnecessary noise component is time-sequentially generated in the output of the photoelectric converting device, the image pickup apparatus is not influenced by this noise component.

Still another object of the invention is to provide an image pickup apparatus in which the iris control and gain control operations of the image pickup apparatus, in particular, are properly carried out irrespective of the noise component mentioned above.

According to the present invention, these objects are attained by provision of an image pickup apparatus having an image pickup device which converts an image into an electrical signal and also outputs an Robstacle" or unusable signal, i.e., a signal whose presence it is desired to eliminate. A control device is provided in conjunction with the image pickup device, to control the apparatus responsive to the electrical signal, in such a manner that the pickup apparatus generates a proper signal representing the image. A device is also provided for adjusting the control such that the latter is not influenced by the obstacle signal output by the image pickup device. The control device may for example include an iris for limiting the amount of image which enters the pickup device. The control may for example include an amplifier for amplifying the electrical signal outputted from the image pickup device and a control circuit for controlling the amplifier gain. According to one preferred embodiment, the pickup device may include an image sensing portion for producing the electrical image signal and the readout portion for reading out that signal, and a drive means may be provided for driving the pickup device. In this embodiment, the drive means has a first mode for reading the image signal out from the pickup device and a second mode for clearing the signal from the sensing portion of the pickup device during readout of the signal. First and second control means are also provided in this embodiment, the first being responsive to the image signal and being structured and arranged for effecting control of a signal level of the image signal, and the second being responsive to a noise component due to the pickup means when the latter is driven in the second mode and being structured and arranged to respond to the noise component to effect control of the first control means.

The above and other objects and features of the present invention will be more fully appreciated from the following detailed description with reference to the accompanying drawings.

FIG. 1 is an explanatory diagram of a frame transfer type CCD;

FIG. 2A is an explanatory diagram of the short time mode;

FIG. 2B is an explanatory diagram of the normal mode;

FIG. 3 is an explanatory diagram of an unnecessary signal which is generated in the short time mode;

FIG. 4 is a block diagram showing an arrangement of a first embodiment of an image pickup apparatus of the present invention;

FIG. 5 is a timing chart of a second embodiment of the invention;

FIG. 6 is a block diagram showing an arrangement of a third embodiment of the invention;

FIG. 7 is a block diagram showing an arrangement of the third embodiment of an image pickup apparatus of the invention;

FIGS. 8A to 8D are timing charts of the third embodiment of the invention;

FIG. 9 is an output diagram of a sample/hold circuit of the third embodiment of the invention;

FIG. 10 is a diagram showing an example of an arrangement of a sync signal generator 127 shown in FIG. 7;

FIG. 11 is a block diagram of an embodiment in which a signal which is generated for the interval shown at T.sub.3 in FIG. 3 is replaced by another signal; and

FIG. 12 is a timing chart of another embodiment of the short time mode shown in FIG. 2A.

The preferred embodiments of the present invention will now be described in detail hereinbelow with reference to the drawings.

FIG. 1 is an arrangement diagram of a CCD solid state image pickup device of the frame transfer type which can be applied to the present invention. In the diagram, reference numeral 1 denotes a frame transfer type CCD solid state image pickup device; 2 is a pickup unit serving as a photosensing part for photoelectrically converting an incident light from an image pickup optical system (not shown) and accumulating charges; 3 a storage unit for transferring the charges in the pickup unit 2 and further reading them out at every horizontal interval; 4 a horizontal register for horizontally transferring, in response to a pulse .phi.T, the charges which are transferred from the storage unit 3 at every horizontal interval; 5 a charge-voltage converting unit for converting the charges from the horizontal register 4 into a voltage and outputting this voltage; DG a gate or barrier provided along the register 4; and DR an overflow drain.

FIG. 2A is a drive waveform diagram in a short time mode of the image pickup device 1. In this diagram, .phi.I, .phi.S, .phi.T and .phi.R indicate drive waveforms of the pickup unit 2, storage unit 3, register 4, and converting unit 5, respectively.

Reference numeral IV represents a vertical interval of a television signal and T.sub.2 is a substantial accumulating time.

At the start of the vertical interval, the charges in the pickup unit 2 are vertically transferred to the storage unit 3 in response to vertical transfer pulses indicated at .circle.1 and .circle.4 . Thereafter, those charges are transferred to the horizontal register 4 on a line by line basis for one horizontal interval by a pulse shown at .circle.5 and then horizontally transferred to the converting unit 5 for the horizontal interval by a pulse .circle.7 . Those charges are converted to a voltage by a pulse .circle.8 and taken out as an output.

After the charges have been photoelectrically converted and accumulated for a period of time T.sub.1 in the pickup unit 2, the charges only in the pickup unit are vertically transferred by a pulse .circle.2 and the charges accumulated for the period T.sub.1 are drained from the pickup unit. Thereafter, the charges picked up and accumulated for the period T.sub.2 are vertically transferred to the storage unit 3 by pulses .circle.3 and .circle.6 . The subsequent operation is executed in a manner similar to the above. As shown in FIG. 2B, the pulse .circle.2 in FIG. 2 is omitted in the normal mode in that about one field period is ordinarily the accumulating time. In the short time mode, the charges only in the pickup unit 2 are vertically transferred by the pulse .circle.2 and when the charges accumulated in the period T.sub.1 are drained, the amount of charges which are accumulated for the period T.sub.1 is larger than that for the period T.sub.2, so that all of them are not always drained but some are left.

FIG. 3 shows an output waveform of the image derived as described above.

In FIG. 3, V.sub.sat denotes a saturation output voltage of the image pickup device, V.sub.av is a normal average output voltage of the image pickup device, and T.sub.BLK is a vertical blanking interval. As mentioned above, the residual charges which could not be drained by the vertical transfer before the substantial accumulating time T.sub.2 are outputted, so that an output of the V.sub.sat level is generated in an initial period T.sub.3 of the vertical interval. When the period T.sub.3 is short, this output does not appear on the display screen; however, V.sub.sat is as much as about three to five times larger than V.sub.av, so that the signal process, iris control, gain control, and the like are largely adversely influenced.

Practically speaking, in general, the iris control, gain control, or the like is carried out in a manner such that an image output, shown in FIG. 3, is transmitted through a low-pass filter and its mean value is obtained. When this mean value is small, the iris is largely opened, in execution of the iris control or the gain is increased in execution of the gain control, or the like. Therefore, in the case of performing iris control or gain control, the mean value becomes higher then the inherent level due to the V.sub.sat as shown in FIG. 3. Thus, hitherto, there has been a problem that the iris is likely to be closed or the gain to be controlled decreases due to such a variation of the mean value.

FIG. 4 shows the first preferred embodiment of the present invention to solve such a problem. Reference numeral 101 denotes an image pickup optical system including an image pickup lens, an optical filter, and the like; 102 an iris; 103 a sample/hold circuit ("S/H") for sequencing a dispersive output from the image pickup device 1; 104 a signal processor for processing an output signal of the sample/hold circuit 103 and synthesizing a television signal; 105 a filter circuit for filtering a luminance signal which is outputted from the processor 104; 106 an error or differential amplifier for comparing an output of the filter circuit 105 with a reference level; 107 and 108 voltage dividers for generating reference levels; 109 a switch serving as correcting means for swithcing between the reference levels formed by the dividers 107 and 108; 110 an iris driver serving as control means for controlling an amount of photo charges produced in the image pickup device 1 by driving the iris; 111 a driver for driving the device 1; 112 a sync signal generator serving as accumulating time control means for generating a drive pulse of the device 1, a drive pulse of the sample/hold circuit 103, a sync pulse for signal processing, a television sync signal, and the like as shown in FIGS. 2A and 2B; and 113 a switch for switching the output modes of the generator 112 and thereby switching the operating mode of the device 1. The switches 109 and 113 operate in an interlocking manner. The operating modes of the image pickup device 1 include the normal mode, similar to that of ordinary video cameras, in which the accumulating time is used as a vertical interval of a television signal, and the short time mode, in which the charges only in the pickup unit are vertically transferred within the vertical interval. Those operating modes are switched by changing over the outputs of the sync signal generator 112 by the switch 113.

An object image is transmitted through the image pickup optical system 101 and iris 102 and photoelectrically converted to an electrical signal by the device 1. This electrical signal is outputted synchronously with the sync signal and sequenced by the sample/hold circuit 103 and processed by the signal processor 104. At the same time, a television signal is synthesized together with the sync signal from the generator 112 and outputted to a television signal output terminal 114. On the other hand, a luminance signal for the iris control is outputted from the signal processor 104, filtered by the filter 105 and compared with the reference level by the error or differential amplifier 106. Then, the luminance signal is inputted to the iris driver 110, thereby controlling the iris 102.

Due to this control, the feedback control is performed in a manner such that a mean value of the output voltage of the filter 105, namely, the output voltage of the device 1, becomes a constant voltage which is set by the voltage driver 107 or 108.

In this embodiment, two kinds of reference voltages are provided for the error or differential amplifier 106 and switched by the switch 109, which operates interlockingly with the switch 113 as mentioned above. Namely, the reference voltages are set in accordance with the normal and short time modes as the operating modes of the image pickup device. Image pickup means is constituted by the components 1, 101 to 108, 110 to 114, and the like.

In the case of performing the operation in the short time mode in accordance with the timing as shown in FIG. 2A, the output of the voltage divider 108 in the short time mode is set to a voltage higher than that of the output of the voltage divider 107 in the normal mode by only about an amount of product of the time period T.sub.3 and V.sub.sat in FIG. 3. Due to this, it was confirmed that in the television signal which is obtained as the output of the image pickup device 1, the luminance of the object image on the display screen can be kept constant irrespective of the operating mode of the device 1.

Although the reference levels can be switched by the switch 109 in this embodiment, the substantial accumulating time may be varied sequentially or in a stepwise manner and further the reference voltage may be changed sequentially or in a stepwise manner in response to the variation of the accumulating time. The invention also incorporates the above-mentioned constitution. On one hand, the invention can be also applied even to the constitution such that the level of the video signal of the image pickup means is controlled by a shutter in place of the iris.

In this case, the shutter may include not only mechanical or physical shutters but also electronic shutters, namely, means for controlling the accumulating time.

The invention can be also useful for the constitution such that the amount of photocharges which are produced is controlled using a combination of two or three of the iris, shutter, and means for controlling the accumulating time in accordance with the video signal level and thereby controlling the video signal level.

The apparatus may be also constituted in a manner such as to prevent the output of the image pickup device in the period T.sub.3 in FIG. 3 from being inputted to the input side of the signal processor 104 by way of the method whereby the operation of the sample/hold circuit 103 is stopped for the period T.sub.3 in FIG. 3, or a gate circuit is interposed in the signal system and the operation of the S/H circuit 103 is interrupted for the period T.sub.3, or the like. Even in this case as well, the invention can be embodied in a manner similar to the above. However, in this case, since the video signal becomes a black level for the period T.sub.3, the reference level is needed to be reduced in the short time mode.

Although the charges only in the pickup unit are vertically transferred within the vertical interval in the short time mode of the image pickup device in the above embodiment, the invention can be also applied to the system in which the charges in both of the pickup unit and the storage unit are drained within the vertical interval. In this case, the reference voltage in the short time mode is set to be lower than that in the normal mode.

FIG. 5 is a diagram showing the second embodiment of the invention in the case as mentioned above, namely, showing a modification in the short time mode. In this embodiment, the charges in the storage unit are also vertically transferred at a high speed as indicated at .circle.9 synchronously with the high-speed vertical transfer of the charges in the pickup unit as indicated at .circle.2 . Therefore, the charges in the pickup and storage units are collected to the horizontal shift register 4 and overflow. These overflow charges flow into the drain DR through the gate or barrier DG and are drained.

Thus, the output of the charge-voltage converting unit 5 lacks when the overflow of charges occurs in the shift register 4 as shown by the portion .circle.10 in FIG. 5. In this case, as mentioned above, the reference voltage in the short time mode is corrected so as to become a value lower than that in the normal mode by correcting means by way of a switch or the like.

In addition, among image pickup devices of other systems, for example, CCD or MOS image pickup devices of the interline system, or the like, various kinds of devices capable of controlling the accumulating time have been also considered. The invention can be also applied to the case where the output characteristics of the image pickup devices in the short time mode and normal mode differ.

Although the foregoing embodiments have been described with respect to the case where the invention was applied to the iris control system, the invention can be also applied to the gain control system which operates such as to make the output level constant by changing the gain in accordance with the voltage of the video signal.

That is, the invention also includes the constitution in which the video signal level is controlled by the output system of the image pickup device.

FIG. 6 is a diagram showing the third embodiment of the invention in the above-mentioned case, in which the same parts and components as those shown in FIG. 4 are designated by the same reference numerals.

Reference numeral 115 denotes an amplifier; 116 a filter circuit for filtering an output of the amplifier 115; 117 an error or differential amplifier for comparing an output of the filter 116 with a reference level and thereby forming an error or difference signal; and 118 a transistor an impedance of which varies in repsonse to an output of the amplifier 117. The video input level to the amplifier 115 is changed by the transistor 118, thereby controlling the gain.

Voltage dividers 120 and 121 generate reference voltages, respectively, and either one of them is inputted as a reference level to the error or differential amplifier 117. The output level of the divider 120 is higher than the output level of the divider 121.

A switch 119 serves as correcting means for switching those reference levels and is switched interlockingly with the change-over of the switch 113 to switch the operating modes of the image pickup device.

Namely, the switch 119 is switched such as to input the output of the voltage divider 120 to the error amplifier 117 when the switch 113 is switched to the side of the short time mode and to input the output of the voltage divider 121 to the error amplifier 117 when the switch 113 is switched to the side of the normal mode.

As another example of the constitution in which the video signal level of the image pickup means is controlled due to the gain control as mentioned above, it is also possible to control the video signal level of the image pickup means by controlling the sensitivity of the image pickup device itself.

According to those methods by the gain control, there is an effect such that the response speed is very high as compared with the method whereby the video signal level is changed using the iris, shutter, or the like.

The occurrence of the malfunction in the control of the iris and gain control systems due to the signal which is generated for the period T.sub.3 in FIG. 3 can be solved by the foregoing embodiments. However, there is a problem such that when the signal shown in FIG. 3 is reproduced by the monitor display, the signal which is generated for the period T.sub.3 in FIG. 3 appears in a part of the screen and an indistinct image is displayed as mentioned above. An embodiment to solve such a problem will then be described with reference to FIGS. 7 to 12.

FIG. 7 shows an embodiment to solve the above-mentioned problem of the invention. In FIG. 7, reference numeral 122 denotes a sample/hold circuit serving as mask means for eliminating a clock signal from the output of the image pickup device 1 and also setting the signal for a partial period of the video signal to the signal of a predetermined level; 123 is a luminance signal Y processor including a low-pass filter, clamping circuit, a gamma circuit, and a clipping circuit; 124 a color separating circuit for separating chrominance signals of red R, blue B, and green G from an output of the sample/hold circuit 122; 125 a color processor including low-pass filters, clamping circuits, gamma circuits, clipping circuits, and matrix circuits of those R, G and B signals; 126 an encoder for synthesizing a television signal by the Y, R-Y, and B-Y signals and a sync signal; 127 a sync signal generator including a reference oscillation frequency divider and a decoder; 128 a mode switch for switching the operating modes of the generator 127; and 129 a drive circuit for producing a drive pulse of the image pickup device by an output of the generator 127.

The output of the image pickup device 1 is sequenced after removing the clock signal therefrom by the sample/hold circuit 122. The luminance signal Y is formed by the Y processor 123 and inputted to the encoder 126. The output of the S/H circuit 122 is also inputted to the color separating circuit 124, by which the color components in the video signal are separated in accordance with a pattern of a color separating filter (not shown) sticked on the image pickup device 1, thereby forming the R, B and G signals. These R, G and B signals are respectively processed by the color processor 125 and further subjected to the matrix operation to form the color difference signals R-Y and B-Y. The encoder 126 receives the Y signal from the Y processor 123, the R-Y and B-Y signals from the color processor 125, and the sync signal from the generator 127 and forms a television signal of, for example, the NTSC system or the like and outputs this TV signal. The mode switch 128 switches the operating modes of the image pickup device 1 to either of the normal mode and the short time mode.

The operation of the drive circuit 129 shown in FIG. 7 will then be described with reference to timing charts shown in FIGS. 8A to 8D.

FIG. 8A shows a pulse .phi.I in the normal mode and the vertical transfer is performed within the vertical blanking interval in response to this pulse .phi.I. FIG. 8B shows a pulse P.sub.SH in the normal mode which is continuously generated for the video interval. FIG. 8C shows a pulse .phi.I in the short time mode and a vertical transfer pulse is generated within the video interval, thereby reducing the substantial accumulating time. FIG. 8D shows a pulse P.sub.SH in the short time mode and the pulse P.sub.SH is stopped for a period of time T.sub.4 which is slightly (for example, one to a few horizontal interval) longer than the time corresponding to T.sub.3 in FIG. 3 in the video period.

The output of the sample/hold circuit 122 in the short time mode is shown in FIG. 9. It will be appreciated that there is an effect such that the signal due to the charges which cannot be drained for the period T.sub.3 does not appear in the output of the S/H circuit 122, so that the following circuits are not adversely affected. Further, in this case, even when the video signal is displayed on the monitor screen as well, the display portion corresponding to the period T.sub.3 is muted and becomes black; therefore, the foregoing problem such that an indistinct image is displayed on the monitor screen is solved.

FIG. 10 is a block diagram showing an arrangement of the main part of the sync signal generator 127. In FIG. 10, reference numeral 201 denotes a quartz resonator; 202 a reference oscillator; 203 a counter-1; 204 a decoder of a pulse of the H system; 205 a counter-2; 206 a decoder of a pulse of the V system; 207 a P.sub.I switching circuit; and 208 a P.sub.SH switching circuit. When an output of the mode switch 128 is "1", the pulses P.sub.I and P.sub.SH as shown in FIGS. 8A and 8B are formed from pulses I.sub.N and SH.sub.N in the normal mode. When the output of the mode switch 128 is "0", the pulses P.sub.I and P.sub.SH as shown in FIGS. 8C and 8D are formed from pulses I.sub.S and SH.sub.S in the short time mode.

With reference to FIG. 11, explanation will then be made with regard to an embodiment in which the signal which is generated for the period T.sub.3 in FIG. 3 is replaced by, for example, a character pattern signal to solve the problem such that the signal which is generated for the period T.sub.3 in FIG. 3 appears on a part of the display screen and causes the display image to become indistinct.

FIG. 11 is a block diagram showing an embodiment in such a case.

In FIG. 11, reference numeral 301 denotes an adder and 302 is a character signal generator. When the mode switch 128 is switched to the side of the normal mode, a character signal generator 302 does not operate. On the contrary, when the switch 128 is switched to the side of the short time mode, the generator 302 operates and a character signal is generated and the character data is mixed to the luminance signal by the adder 301. The character signal is constituted such as to insert characters for the period T.sub.4 in FIG. 9 in response to the sync pulse from the sync signal generator 106. The signal which is generated is not limited to the character signal but may be a symbol signal or the like.

In FIG. 11, reference numeral 130 denoted an example of display on the monitor when characters were inserted in such a case. According to this embodiment, when the substantial accumulating time is shorter than the ordinaray time, the situation in which the photographing operation suitable for "SPORTS ACTION" using a short accumulating time is being executed can be displayed on the monitor and informed to the operator who is watching the monitor 130.

According to this embodiment, even when the substantial accumulating time of the image pickup device is set to be short, it is possible to eliminate an adverse influence to the display screen due to the unnecessary charges when the video signal is displayed on the monitor.

Although the above embodiment relates to the system in which the charges only in the pickup unit are vertically transferred within the vertical interval in the short time mode of the image pickup device, even in the system in which the charges in both the pickup and storage units are drained within the vertical interval as well, a part of the readout image becomes indistinct; therefore, the invention can be applied to such a system.

FIG. 12 shows a timing chart of an embodiment in the case where the invention is applied to the system for performing the vertical transfer as mentioned above; namely, a modified form of the short time mode is shown. In this embodiment, the charges in the storage unit are also vertically transferred at a high speed in the portion .circle.9 synchronously with the high-speed vertical transfer in the portion .circle.2 in the pickup unit. Therefore, the charges in the pickup and storage units are collected to the horizontal shift register 4 and overflow. These overflow charges flow into the drain DR through the gate or barrier DG and are drained.

Thus, the output of the charge-voltage converting unit 5 lacks in the portion .circle.10 and an indistinct image is displayed on the screen.

To solve this drawback, the methods shown in the embodiments of FIGS. 7 to 12 can be applied.

On one hand, among image pickup devices of other systems, for example, CCD or MOS image pickup devices of the interline system, or the like, various kinds of devices capable of controlling the accumulating time have been considered. For instance, in the case of controlling the accumulating time using the interline system CCD, a noise component is added to the screen at near the end of the vertical interval. Therefore, it is sufficient to control the video signal level in this portion so as to become a predetermined level.

Although the substantial accumulating time T.sub.2 is constant in the above embodiment, it may be variable continuously or in a multistepwise manner. In such a case, the amount of charges which cannot be drained depends on the period T.sub.2. For instance, since the amount of charges which cannot be drained increases as the period T.sub.2 becomes short, the apparatus may be constituted such as to change the period T.sub.4 shown in FIGS. 8 and 9 in accordance with the duration of the period T.sub.2.

In addition, although the constitution to stop the operation of the sample/hold circuit was used as the mask means in the foregoing embodiments, the apparatus may be constituted such that a gate circuit is provided in front of or after the sample/hold circuit and the gate is closed for the period T.sub.4. The invention also includes the constitution in which the signal for a partial interval of the video signal is replaced by another signal pattern of a predetermined level by a switching circuit.

As described above, according to the embodiments, even when the substantial accumulating time of the image pickup device is set to be short, an influence to the display screen due to the unnecessary charges can be eliminated.