Three-color imaging on each pixel for increased resolution

[0002] 1. Field of the Invention

[0003] The present invention relates to methods and apparatus for eliminating the need for color interpolation in single-array digital imaging systems by detecting modulated red, green and blue light at each pixel, and band pass filtering the detected light to extract signals representing red, green and blue.

[0004] 2. Description of the Prior Art

[0005] In digital cameras, there are three generally accepted methods for acquiring color images using a light detector array. The first method is to put a color filter array (CFA) over the detector array, such that certain pixels detect certain colors. For example, some of the pixels would detect red light, some of the pixels would detect green light, and some of the pixels would detect blue light. The result of using such a color filter is that each pixel in the array sees only one color, so that to obtain a full color image, interpolation must be used to estimate the other two colors at each pixel location. FIGS. 1A and 1B (prior art) show how this method is used. The image obtained by the detector array through the color filter array has an arrangement of red pixels, blue pixels, and green pixels, for example in the Bayer CFA pattern, as shown in FIG. 1A. The intensity at each pixel indicates how much of the selected color of light was detected. In FIG. 1B all of the pixels have all three colors of light, because the green light is interpolated between pixel G1 and pixel G4 to determine the intensity of green light in the upper left hand pixel, and so on. Thus, each three color pixel is only really accurate with respect to one of the three colors. In addition, each color array is sparsely filled. This decreases the sample rate at each color and causes increased aliasing in the image.

[0006] The second method is more expensive, and more accurate with respect to color. Three separate detector arrays are used, one for each color. Thus three images, one for each color, are formed. Then the three images are overlapped to give a composite full color image. The grey scale resolution is no better than the first method, but the accuracy of the colors is greater. FIG. 2 (prior art) indicates how the three images are overlapped to generate an accurate three-color image. In addition, each array has a 100% fill factor (ratio of pixel size to pixel spacing), meaning there is little or no dead space between pixels. Consequently, aliasing is minimized.

[0007] The third method is to use a color filter wheel so that only one color illuminates the object at a time. This has the disadvantage that the field rate must be tripled to give the same frame rate. In addition, if the object moves during the exposure, the image acquires color artifacts. For example, a black/white edge appears as a color edge.

[0008] A need remains in the art for methods and apparatus for increasing resolution and color accuracy in digital imaging systems without the need for employing multiple detector arrays, by temporally modulating red, green and blue light, detecting all three colors at each pixel, and band pass filtering the detected light to extract signals related to red, green and blue at each pixel.

[0009] It is an object of the present invention to provide methods and apparatus for increasing resolution and color accuracy in digital imaging systems without the need for employing multiple detector arrays, by temporally modulating red, green and blue light, detecting all three colors at each pixel, and band pass filtering the detected light to extract values for red, green and blue at each pixel.

[0010] This is accomplished by sinusoidally modulating red, green and blue light sources at different frequencies. The modulation can be performed, for example, by modulating the current of the light source. The detector array detects all three colors of light at each pixel. A detector which preserves the modulation, such as a complementary metal oxide semiconductor detector is used. Three band pass filters, centered at the three modulating frequencies, are used to extract the signal associated with each color.

[0011] Alternatively, if an integrating detector array such as a charge coupled device (CCD) is used, the three illumination sources can be switched on and off sequentially, and the output read out for each color, at different times. The switching must be at three times the frame rate.

[0012] The present invention is useful in devices where artificial illumination is used, such as microscopes, endoscopes, color scanners, and manufacturing inspection cameras.

[0013] FIG. 1 (prior art) shows an image obtained by a detector array through the color filter array.

[0014] FIG. 2 (prior art) shows the reconstructed three-color image from interpolating the pixels of the image of FIG. 1A.

[0015] FIG. 3 (prior art) indicates how the three one-color images are overlapped to reconstruct a three-color image.

[0016] FIG. 4 is a block diagram showing three-color imaging at each pixel according to a first embodiment of the present invention wherein the colors of light is modulated.

[0017] FIG. 5 is a flow diagram showing the process of three-color imaging at each pixel according to the embodiment of FIG. 4.

[0018] FIG. 6 is a block diagram showing three-color imaging at each pixel according to a second embodiment of the present invention wherein the colors of light are sequentially switched during one frame.

[0019] FIG. 7 is a flow diagram showing the process of three-color imaging at each pixel according to the embodiment of FIG. 6.

[0020] FIG. 8 is a block diagram showing the embodiment of FIG. 4, further including a first embodiment of ranging apparatus.

[0021] FIG. 9 is a block diagram showing the embodiment of FIG. 4, further including a second embodiment of ranging apparatus.

[0022] FIG. 4 is a block diagram showing three-color imaging at each pixel according to a first embodiment of the present invention wherein each color of light is modulated at a different frequency. Modulated light source 402 modulates red, blue and green light at three different frequencies. The modulated light 404, 406, 408 is reflected off object 410, imaged by lens 411, and detected by detector array 412. Detector array 412 detects all three colors of light at each pixel 414. A detector which preserves the modulation, such as a complementary metal oxide semiconductor detector is used. A band pass filter array 416 is used to extract the signal associated with each color at each pixel. Each filter element 418 comprises three band pass filters tuned to the three frequencies used to modulate the three colors of light. The signal passed by each filter element then represents the amount of light of a particular color seen by that pixel. CMOS detectors have the advantage (in addition to preserving the modulation) that the filter circuits could be built on the same semiconductor chip as the detectors. Image forming element 420 combines values for each color at each pixel and forms a three-color image of the object.

[0023] FIG. 5 is a flow diagram showing the process of three-color imaging at each pixel according to the embodiment of FIG. 4. In step 502, the different colors of light are modulated by different frequencies (in general three colors of light, red, green, and blue, are used). The modulated light is reflected off of the object to be imaged in step 504. The reflected light is is imaged by lens 411 in step 505 and is detected at each pixel of the detector array 412 in step 506. In step 508, the signal from each pixel is passed through band pass filters 416 tuned to the modulating frequencies in order to extract the signal related to each color of light. The image is formed with values for all of the colors at each pixel in step 510.

[0024] FIG. 6 is a block diagram showing three-color imaging at each pixel according to a second embodiment of the present invention, wherein each color of light is switched sequentially. Switching light source 602 switches between providing red, blue and green light sequentially. The light is reflected off object 610, imaged by lens 611, and detected by detector array 612. Detector array 612 is an integrating type array, so in order to preserve the information as to which color was detected as a particular time, three integrators or storage elements 615 are selected as the light source switches between colors. Image forming element 620 forms an image of the object, which has values for each color at each pixel from the three integrator arrays.

[0025] FIG. 7 is a flow diagram showing the process of three-color imaging at each pixel according to the embodiment of FIG. 6. In step 702, the different colors of light are switched during one frame. The light is reflected off of the object to be imaged in step 704, and is imaged onto the detector array in step 705. The reflected light is detected at each pixel of the detector array in step 706. In step 708, the signals from the detector array are switched to the correct integrator array for the color of light provided. The signals are integrated in steps 710. The image is formed with values for all of the colors at each pixel in step 712.

[0026] FIG. 8 shows a modification to the embodiment of FIG. 4 which allows the system to also detect object range, using the modulation phase of at least one color of light. This feature is useful in endoscopes, manufacturing inspection, and the like. Modulated light source 802 now also provides the modulation phase of at least one of the three modulated colors as signal 804. Detector 812 also extracts the modulation phase of the same color(s) detected in each pixel and provides it as signal 806. Range computation array 810 detects the offset in modulation phase between light source 802 and after reflection off object 410, and computes the range to each imaged segment of object 410. While detecting offset in the modulation phase of only one color is necessary, detecting offset in modulation phase of more that one color may be useful for double checking the range or in resolving range ambiguities. If the phase has been offset by more than one wavelength of the modulation, there is an n2π phase ambiguity. This ambiguity can be resolved by making measurements of the phase at different modulation wavelengths, as long as the modulation wavelengths are not integral multiples of each other.

[0027] FIG. 9 shows the measurement of range by modulating a nonvisible portion of the spectrum (e.g. infrared) with a modulation wavelength to form ranging signal 904. Measurement of the phase offset of signal 904 is then accomplished as described with respect to FIG. 8. Measurement of offset in one of the colors (or another modulated infrared signal) could be used to remove range ambiguities.

[0028] Those skilled in the art will appreciate that many other configurations fall within the spirit and scope of the present invention.