Color LiDAR scanner

[0001] This application claims the benefit of prior provisional application Ser. No. 61/183,265 filed on Jun. 2, 2009, the contents of which are hereby incorporated by reference.

[0002] This invention relates generally to systems and methods for laser imaging detection and ranging, particularly including such systems that are capable of processing both ranging and color information.

[0003] There are many laser imaging detection and ranging (LiDAR) range-finding systems that can capture a pixilated image in 2D and 3D format. These sensors use Infrared (IR) lasers to issue a pulse of light that travels to a target that then bounces off and returns to a detector. By tracking the time of flight of the light pulse, a distance can be calculated. This distance, combined with the position and directional angle of the sensing device, can be converted to a pixel in 3D space. Multiple pixels are combined to form images, including three-dimensional images. Contrast can be obtained by measuring the intensity of the return and rendering accordingly.

[0004] LiDAR systems are often preferable since they offer exact distance measurements for each pixel in the image, and rely on their own generated light to illuminate the target and to capture distance pixels. This is useful for navigation, 3D mapping (including low light conditions), object identification, and many other applications. In addition to distance information, IR-based LiDAR systems also report intensity information, useful for identifying contrast features of the surrounding environment (such as road markings), and for adjusting the power applied to the laser beam for subsequent pulses.

[0005] While LiDAR is a useful tool for 3D imaging, current LiDAR systems typically render images in grayscale rather than color. Some have tried to produce color versions of LiDAR imaging systems, but such systems are deficient in one respect or another. An effective color LiDAR system in accordance with preferred versions of the present invention helps provide realism for 3D maps, aids in object identification, and provides the ability to capture a full color image using the LiDAR's own supplied light, thus eliminating the need for daylight or other artificial light sources, and eliminating the problems caused by uneven light conditions (such as shadows) at the time of capture.

[0006] In general terms, the present invention is a color LiDAR scanner device. In one example of the invention, three color laser diodes (red, green, blue) and three avalanche photodiodes (red, green, blue) illuminate and then detect the color light intensity returned from a target. In a preferred version these color laser/detectors are coupled with a single infrared laser/detector that detects the range and infrared light intensity from the same target. The combined range and color intensity information is combined to produce a single colored pixel in space. Multiple illuminations are used to create multiple pixels which can be combined to produce images.

[0007] The scanning device is contained within a housing that is positioned and configured so that it can be driven by a motor (not shown) to pan, tilt, or rotate about a vertical axis. Accordingly, when the device rotates it scans across a horizontal field of view, rendering multiple pixels as it scans.

[0008] The composite signal and data points can be used to create a 360-degree, one pixel thick (z axis slice) color and range representation of the surrounding environment within a desired radius of the device. In an exemplary version, the maximum is a 120 meter radius about the device. If the device is moved with respect to the z axis, or if multiple sets of laser/detectors are employed, a 3D color composite image can be rendered from the data points.

[0015] As shown in FIG. 1, a preferred version of a LiDAR scanning device is contained within a housing 10. The device in the illustrated version uses four laser diode emitters mounted to be able to send signals at a face of the housing: one each for infrared 20, blue 22, red 24, and green 26. Each emitter is fixed-aligned with a corresponding Avalanche Photodiode (APD) detector, 30, 32, 34, and 36 positioned to receive signals returned to the face of the housing. As shown, the four emitter/detector pairs (20-26 and 30-36) are aligned in a horizontal row that is generally perpendicular to a vertical axis extending through the housing. In a preferred version of the invention, one or more lenses is mounted in front of the emitter/detectors in order to focus the outgoing laser light and the returned light.

[0016] The housing 10 is positioned and configured so that it can be driven by a motor (not shown) to rotate about the vertical axis that is perpendicular to the horizontal row formed by the emitter/detector pairs. Accordingly, when the device spins the four emitter/detector pairs each pass across a target along a horizontal plane. Although the four emitter/detector pairs are aligned in a horizontal row, they may be oriented to transmit and detect light at an angle that is angularly offset above or below a plane that is perpendicular to the vertical axis. In other versions of the invention, however, the housing need not rotate. In order to obtain an image of the environment some versions may include a housing that tilts, pans, scans or otherwise moves along a path that may not amount to rotation.

[0017] For each pixel that is to be rendered to create an image, the system combines a distance value from the IR emitter/detector with the color values from the RGB emitter/detectors. Preferably the IR emitter detector is pulsed to enable the system to measure return signal timing and thereby calculate distance as a function of signal transit time. The color emitters may be steady state rather than pulsed in versions in which they are not used to determine distance.

[0018] In one embodiment, there are 32 color pixels generated for every IR pixel generated. The IR pixel is used for distance measurement, and is needed to understand the distance to the target. The red, green, and blue lasers are used for intensity information and are not used for distance measurement. The IR measurement is crucial not only for placement of the colorized pixel in 3D space, but in also adjusting the intensity of the color return by virtue of the formula I/R², R=range. In the versions in which the housing rotates, the color intensity return values are each slightly offset horizontally from one another and with respect to the IR value being returned. Accordingly, the horizontal positions of the 32 color intensity returns are interpolated with respect to location between IR distance returns in order to produce a single color intensity value for each of the RGB emitters to match with the IR distance measurement.

[0019] Once the IR, blue, red, and green emitter/detector pairs have each passed across the target and have collected their respective information, the color pixel at that particular location can be assembled and rendered.

[0020] In one embodiment the device uses four Laser Diodes with the following output specifications:

[0021] The IR laser diode emitter 20 preferably operates at 904 nm, in the Infrared spectrum, operating at 70 mW, and is pulsed at 5 ns intervals.

[0022] The red laser diode emitter 24 preferably operates at 650 nm, in the Red spectrum, operating at 50 mW, and has a constant output.

[0023] The green laser diode emitter 26 preferably operates at 532 nm, in the Green spectrum, operating at 50 mW, and has a constant output.

[0024] The blue laser diode emitter 22 preferably operates at 400 nm, in the Blue spectrum, operating at 50 nW, and has a constant output.

[0025] In the above example there are four APD Detectors 30, 32, 34, 36 that detect laser radiation reflected from the output lasers 20, 22, 24, 26. The preferred specifications of the four APD Detectors are as follows:

[0026] The IR APD detector 30 operating at 904 nm, within the Infrared spectrum.

[0027] The red APD detector 34 operating at 650 nm, within the Red spectrum, detecting intensity only.

[0028] The green APD detector 36, operating at 532 nm, within the Green spectrum, detecting intensity only.

[0029] The blue APD detector 32, operating at 400 nm, within the Blue spectrum, detecting intensity only.

[0030] In a preferred version, each APD detector is the same component but individually filtered to detect a desired portion of the light spectrum. For IR detection the detector 30 is unfiltered, but each color detector 32, 34, 36 is fitted with an optical glass bandpass filter specific to its light spectrum. For example, the 532 nm Green detector 36 has a filter that passes “green” light (within the green portion of the light spectrum) and removes all other light.

[0031] As shown in one embodiment in FIG. 2, the housing may be connected to a motor configured to allow the device to rotate 360 degrees about the Z axis. The horizontal row of emitter/detectors may be oriented to be within a plane defined by the X-Y axes, or may be oriented upward or downward at an angle above or below the X-Y plane.

[0032] In a preferred example, the housing rotates at a rate of up to 900 revolutions per minute (15 Hz), with all lasers and detectors mechanically linked and set at a fixed angle with respect to the Z axis. In other versions, the housing may rotate at a faster or slower rate, as desired. The data sampling rate (number of data points received per second) is preferably constant and independent of the rate of rotation, although it may be adjusted as a function of the rate of rotation.

[0033] Reflected laser radiation is detected by the respective APDs 30, 32, 34, 36 and the analog intensity signal is digitized. In one version, the digitization uses 16 bits per color detector and 8 bits for the infrared detector. As desired, however, a greater or lesser number of bits may be used for intensity.

[0034] The infrared detector determines range by the well-established method of peak-to-peak time measurement from laser pulse to return signal detection. Range data, as provided by the IR emitter/detector pair, is used to calibrate the intensity of the color return in accordance with the formula I/R². Three 16-bit values (16 bits per R, G, or B) combine to form a 48-bit color point. In other versions, a greater or lesser number of bits may be used for color intensity.

[0035] The sampled RGB signals are processed by the Digital Signal Processor (DSP) 40 to create a 2D, 360 degree color/range pixel point cloud that represents a fixed, Z axis position in space. Although a DSP is used in the preferred version, a Field Programmable Gate Array (FPGA) or other processor may alternatively be used. Rendering software stored in a memory and operable by the processor then interprets the point cloud and assembles each RGB pixel in 3D space according to position. If the device is moved with respect to the Z axis (using forward motion, tilting, nodding, or other means), a 3D representation of the physical environment can be created by using appropriate processing and image render algorithms.

[0036] FIGS. 1 and 2 further show the preferred laser diode and APD Detector layout on a face of the housing 10. Three Feedback Diodes 52, 54, 56 are used to adjust the color laser output. Each of the components described above are housed in a sealed enclosure.

[0037] In the RGB Receiver Circuit as shown in FIG. 3, a variable high voltage 60 of 100-200V is applied to the Avalanche Photodiode Detector (APD) 62. Laser energy falling onto (that is, detected by) the APD 62 creates a signal that is first buffered by buffer 64 and then digitized by a 16-bit A/D converter 66, preferably operating at 1 MHz. The A/D converter 66 outputs data to a 48-bit serial bus 68 where each color intensity signal occupies 16 bits. Although one RGB receiver circuit is shown, a circuit such as that shown in FIG. 3 is provided for each of the colors red, blue, and green and then coupled to the bus 68. Thus, the APD 62 as shown in FIG. 3 is intended to represent each of the individual APDs 32, 34, and 36 as illustrated and described above.

[0038] The 8-bit IR intensity signal is processed in a manner similar to that described for the color APDs and is provided on a parallel bus.

[0039] In addition to bathing the target with color laser signal, each red, green, and blue laser diode signal is also presented to a Receiver Feedback Circuit. This receiver circuit reports the actual laser energy issued by the color lasers to the processor. Thus, as illustrated in FIG. 1, each of the blue, red, and green laser diodes 22, 24, 26 is coupled to a digital signal processor (DSP) 40. The DSP also receives return signals from blue, red, and green feedback detectors 52, 54, 56.

[0040] This actual energy issued from the laser emitters is compared to the returned signal for two reasons. First, it provides a benchmark signal that eliminates the variance in the laser diode drive circuit components and always reports the correct departing laser intensity for calibration purposes. Secondly, it allows the DSP to adjust the power to the laser diode to minimize the energy needed to illuminate the target, thus preventing saturation of the APD detectors, minimizing laser diode heat and extending laser diode life. A servo circuit from the Feedback Photodiodes 52, 54, 56 monitors RGB intensity and sends a signal to the DSP 40 where software adjusts the output of the Laser Diodes so as not to saturate the APD detectors. The laser intensity signals are then driven by the RGB DAC output.

[0041] The RGB DAC Circuit is illustrated in FIG. 4 and holds the intensity value of the laser at a constant value. Thus, the Photodiode Feedback circuit 70 adjusts each color laser's output intensity by monitoring environmental conditions and detector sampling variations. The DSP 40 adjusts the drive signal based on the signal strength from the respective feedback diode. The drive signal is then converted by the DAC 71 to the appropriate level to drive the laser diode 72. A similar circuit is provided for each of the laser diode emitters. Thus, the laser diode 72 in FIG. 4 represents each of the laser diodes 22, 24, 26 in FIG. 1.

[0042] An alternate embodiment includes multiple IR/R/G/B emitter detector sets located along vertical planes to form multiple rows of emitter/detector pairs, each row being substantially the same as described above. Thus, as shown in FIG. 1 the system preferably includes a single set of emitters and detectors positioned along a horizontal line. In an alternate version, multiple sets of such emitters and detectors may be positioned in similar horizontal lines, with separate sets of emitters and detectors being positioned above and below one another to form a column.

[0043] In such an alternate version of the device, as shown in FIG. 5, multiple sets of IR, Red, Blue, and Green emitter/detector pairs are aligned and oriented to emit and detect laser radiation at multiple defined angles along the Z axis. Each of the sets preferably has a constant angular offset 80 from each other set of emitter/detectors. This enables the scanner/device to create an image having a desired height without any movement along the Z axis. There is no theoretical limit to the number of laser sets that can be aligned in this manner. One exemplary version has 12 sets of lasers across a 24 degree vertical field of view, thus having each laser set aligned a ½ degree increments as shown in FIG. 5.

[0044] In another version, enough IR/R/G/B sets are combined together to provide a full image, and/or the color scanner is actuated so as to capture a full 3D image such as that captured by a movie camera. This color movie camera device would be entirely reliant on its own light so would operate without any concern for lighting conditions. It would also provide a full 3D image since all the pixels would have precise distances associated with them. Two such cameras positioned on either side of a moving object would give the effect of a moving hologram of the object in the camera's field of view.

[0045] In yet another alternate version of the invention, the IR emitter 20 and detector 30 are omitted. As a substitute for the IR emitter/detectors in order to obtain ranging information, one of the color emitter detector pairs is used to obtain ranging information. Preferably, the green emitter 26 and detector 36 are used for this purpose. Accordingly, the green emitter in this version operates in a pulsed fashion in the manner of the IR emitter of the first version described above. The red and blue emitters continue to operate in a steady state. The detected radiation from the green emitter is then used to provide both ranging and intensity information that was provided by the IR emitter, as well as to provide color information in the green spectrum.

[0046] While the preferred embodiment of the invention has been illustrated and described, as noted above, many changes can be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention is not limited by the disclosure of the preferred embodiment. Instead, the invention should be determined entirely by reference to the claims that follow.