Defect Correction in Solid State Imaging

1. Field of the Invention

This invention relates in general to electronic imaging and in particular to a technique for identifying and/or correcting for defects in a solid state imager.

2. Description Relative to the Prior Art

Solid state imaging, such as by means of a charge coupled device (CCD), has found wide acceptance in the video arts for a variety of reasons, among which are lower power consumption, higher sensitivity, and potential cost saving as the technology for producing such devices matures. A typical CCD imager, as representative of the general class of imagers in question, comprises an array of a large number of evenly distributed elemental photosensitive picture elements, i.e. pixels: the greater the number of pixels per imager, the greater the image resolution of the imager. While there is a trend toward employing more and more pixels per imager for image resolution purposes, it is recognized that the greater the number of pixels that an imager has, the greater will be the chance that one or more of the pixels will be bad, thereby possibly rendering the imager useless. Two kinds of pixel defects are generally known to exist in a solid state imager: a "dead" pixel in which little or no image representative photocharge can exist; and a "hot" pixel in which too much photocharge gets accumulated due, for example, to electrical dark current leakage at the pixel. Thus, despite the trend toward more and more pixels per imager, practical design considerations often militate against using more pixels per imager than are absolutely necessary.

Various relatively straightforward techniques are known for obviating the effect of one or more defective pixels in a CCD imager. See, for example, U.S. Pat. No. 4,253,120, issued Feb. 24, 1981, which teaches that, if lower resolving optics for imaging onto a high resolution imager are employed, any given image spot will overlap more than one pixel; and, attendantly, when a pixel is bad, interpolation can be employed to obscure, in real time, the effect of such bad pixel. It is also known to memorize, by use, for example, of a read-only-memory (ROM), the locations of defective pixels of an imager, thereby to produce appropriate correction signals (typically signals corresponding to neutral gray) as substitutes for signals produced by dead and hot pixels . . . this correction technique being, however, one that does not occur in real time. In any event, regardless of which of the aforementioned schemes are employed, correction for defective pixels is usually less than perfect. The latter scheme merely works to dull the effect of a defective pixel; and the former works in a way that, in essence, guesses at what occurs between pixels.

Other art which may be of interest in relation to the present invention may be found in an article from "Television Technology in the 80's" by A. A. Goldberg and J. P. Rossi, pp. 80-88 presented at the SMPTE Conference held Feb. 6-7, 1981.

In accordance with the present invention, correction for a defective pixel occurs in real time and in an amount precisely dependent on the discrete optical image signal applied to the defective pixel. To this end, the invention draws on principles outlined by Nyquist, viz. that--to avoid signal intermodulation and image aliasing--it is essential when sampling a baseband video signal to sample at least twice the uppermost frequency within the base-band signal. As is known, when sampling in this manner, a power-vacant window will occur in the frequency spectrum between the uppermost frequency of the baseband signal and the lowermost frequency of the lower sideband of the sample-modulated signal. Pursuant to the invention, the base bandwidth of any optical image that falls upon an imager is customized so that its uppermost spatial frequency is less than half the sampling frequency associated with the imager, thereby creating a power-vacant frequency window in the signal output of the imager. Since the edges of dead and hot pixels correspond with very high spatial frequency components, the very existence of a dead or hot pixel will result in signal power within the aforementioned window of the sampled output of the imager . . . and the extent of such power will be directly dependent on the magnitude of the discrete optical image that falls in the vicinity of the dead or hot pixel.

For convenience, apparatus according to the invention detects the exsistence of power at a frequency of one-half the imager sampling frequency (i.e. at the center of the aforenoted power-vacant window); thereafter, such power is transformed into a correction signal for properly timed algebraic addition to the baseband signal output of the imager.

Two distinct advantages are provided by the practice outlined above:

(a) Rather than work sensor design to employ just enough pixels for a given application, the density of pixels may be as large as possible, the optical bandwidth being qualified to negate the effect of any bad pixels, should they exist. In this way, perfect imagers with high densities of pixels can be saved for special applicatons; and less than perfect--but otherwise high density--imagers can be employed as would imagers of lower resolution.

(b) Sensors, heretofore considered as scrap because of bad pixels, may be salvaged for use in lower cost apparatus by appropriately qualifying the optics associated with such sensors.

With the above as background, reference should now be had to the figures identified below, and to the accompanying detailed description of the invention in its presently preferred form:

FIG. 1a and FIG. 1b are diagrams that illustrate analog signals corresponding to the scanning of a line of an imager in which, respectively, there is a dead pixel and in which there is a hot pixel,

FIG. 2a is a diagram that illustrates the bandwidth of a scanned imager adapted to accommodate the invention, and Fig. 2b is a diagram that illustrates what occurs to the bandwidth of FIG. 2a when the scanning operation encounters a bad pixel,

FIG. 3 is a schematic block diagram illustrating apparatus according to the teaching of the invention, and

FIG. 4 is a schematic diagram illustrating a presently preferred implementation of the invention.

Referring to FIG. 1a, the kind of analog signal waveform resulting from scanning the samples of a line of pixels of a solid state imager is depicted--there being, by way of example, a dead #13 pixel in the line. Similarly, in FIG. 1b, the kind of analog signal waveform resulting from scanning the samples of a line of pixels in which one of the pixels (#10) is a hot pixel is depicted. Needless to say, were the signal waveform of either FIG. 1a or FIG. 1b to be converted to its corresponding visual display, the display in question would be faulty . . . either because of a pinpoint dark spot (FIG. 1a), or a pinpoint bright spot (FIG. 1b), along the displayed line. Whereas the pinpoint dark spot defect is a bit objectionable, a pinpoint bright spot is especially bad because it appears as a highlight. It will be appreciated further that in the event--and as may often be the case--successive analog line signals all have dead, or hot, similarly numbered pixels, vertical dark or bright lines will appear in the corresponding video display; and, in this case, a dark line may be as objectionable as a bright one.

In accordance with the practice of the invention, correction signals for properly timed algebraic addition to the analog signals are artificially produced. Thus, in the case of a FIG. 1a kind of defect, the produced correction signal will serve to "fill in" the cross-hatched part of the FIG. 1a analog signal; in the case, however, of a FIG. 1b type defect, the correction signal will serve to cancel the effect of the cross-hatched part of the FIG. 1b analog signal.

FIG. 2a depicts the spatial frequency baseband I of an optical image customized according to the invention . . . which is to say that the optical spatial frequency cutoff f.sub.c is less than one-half the sampling frequency f.sub.s effected by means of the imager in question. Note the power-vacant window that exists between the lower sideband SB.sub.L and the baseband I. ("SB.sub.U " stands for "upper side band".) Given the existence of a dead or hot imager pixel, high frequency signal components greater than f.sub.c are generated as the dead or hot pixel is read during sampling. Attendantly, the upper frequency edge on the baseband signal shifts (see dashed line 1) toward the frequency (f.sub.s -f.sub.c); and, corollarily, significant signal power (dotted line P) suddenly appears in the window as the lower sideband SB.sub.L reaches toward the baseband signal I (see dashed line 2). As will be appreciated, the phase of the signal at f.sub.s /2 will depend on whether a dead or hot pixel had been encountered during sampling, i.e. whether the leading and trailing edges of the FIG. 1a and FIG. 1b cross-hatched signal aberrations were "down-and-then-up", or "up-and-then-down". In accordance with the invention, use is made of this phase characteristic to discern the direction of correction to be effected for a bad imager pixel.

Reference should now be had to FIG. 3 for a general arrangement of apparatus for implementing the invention: Given that a sensor 10 has a distribution of pixels which is such that an image received thereby gets sampled by the imager at a frequency f.sub.s, the invention calls for optics 12 to limit the optical spatial frequency bandpass to less than the frequency f.sub.s /2, whereby the imager-and-optics assembly will operate pursuant to the teaching of Nyquist. Note the baseband optical frequency spectrum applied to the sensor 10; and the "windowed" electrical frequency spectrum exiting the sensor 10. In the event signal power, of frequency f.sub.s /2, exists at the sensor 10 output (evidencing a "defect" associated with the baseband signal), such power is detected by a detector 14 . . . which in response thereto causes a correction signal generator 16 to produce a correction signal for algebraic addition (18) to the baseband signal . . . such baseband signal being produced by a sample-and-hold operation on the signal appearing at the output of the sensor 10. Correction signals are as depicted in phantom in FIGS. 1a , 1b; and may be produced in a manner described later in connection with the apparatus of FIG. 4. It suffices to say at this point, however, that the correction signals are proportioned according to the peak power occuring at the frequency f.sub.s /2 and, attendantly, precisely compensate for any signal distortion effected by a bad pixel. (In other words, the height of the signal rise, or the depth of the signal decline, at the occurrence of a defective pixel is determined and employed to correct for the occurrence.) The output of the adder 18 may then be processed (20), say, into the NTSC signal format for display on a television set, or processed into a suitable form for recording on a magnetic medium or the like.

With the above as background, consideration should now be given to the circuit of FIG. 4: The analog video signal produced by the sensor 10 is applied to a narrow band filter 22 tuned to the frequency f.sub.s /2 which occurs at the center of what should be a power-vacant window in the spectrum of the sensor output. In the event signal power exists in the spectrum window, such power is synchronously detected by a detector 24, which may be of any wellknown type. Synchronous detection is employed in this instance, as noted above, so that the phase characteristic of the defect-identifying f.sub.s /2 power may be discerned . . . and whereby the proper direction of correction may be implemented. Thus, appearing at the output of the detector 24 will be a pulsating signal of one polarity for the signal correction indicated according to the showing of FIG. 1a . . . or a pulsating signal of the opposite polarity for the signal correction indicated according to the showing of FIG. 1b. The polarized output of the synchronous detector 24 is applied to a sample-and-hold circuit 26 and to a full-wave rectifier 28, the latter serving to produce a signal 30 (irrespective of the polarity of the signal applied to the full-wave rectifier), and the former serving to capture the peak amplitude of the high frequency pulses produced by the synchronous detector 24. The rectified signal 30 is differentiated by a circuit 32 to produce a signal that "zero-crosses" at the instant peak power exists in the spectrum window, thereby identifying the particular high frequency pulse output of the synchronous detector 24 which is to serve for precise signal correction.

The output of the differentiator 32 is applied to a zero-crossing detector 34 (which may for example be a comparison circuit referenced to zero volts) and, at the instant the peak of the power signal applied to the differentiator 32 occurs, the zero-crossing detector 34 pulses the sample-and-hold circuit 26. Attendantly, a high speed, high current FET input buffer 36 (e.g. an LH0032 amplifier) samples the polarized high frequency pulsed output of the synchronous detector 24 (at the occurrence of the peak pulse thereof); and this sampled pulse is gated (38) for "holding" by a capacitor 40. Having identified the proper correction signal, i.e. the sampled signal, signal correction is effected by switching (42) the correction signal via a high speed, high current FET buffer amplifier 44 (e.g. LH0032) to a video difference amplifier 46. Since the baseband defect to be corrected will inherently correspond with the duration of a single pixel during the scanning thereof, a single clock pulse generator 48 is employed to turn on a gating FET 50 (e.g. an SD211 gate) for the scan duration of a single pixel.

As indicated above in connection with FIG. 3, the windowed signal spectrum appearing at the output of the sensor is sampled and held (52) to recover the baseband signal; and by means of a delay device 54, which compensates for the signal delay associated with the circuit components 22-50, the baseband signal is applied to the difference amplifier 46. There, it is algebraically added to the correction signal to provide the baseband signal with the form it would have had were the sensor in question to have been perfect in the first place.

The invention has been described in detail with particular reference to preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.