Time-resolving sensor for range measurement and 2D greyscale imaging

[0001] This patent application claims the priority benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application No. 62/820,833, filed on Mar. 19, 2019, the disclosure of which is incorporated herein by reference in its entirety.

[0002] The subject matter disclosed herein generally relates to an apparatus and a method for range measurement and a greyscale intensity measurement and, more particularly, to an apparatus and a method for direct time-of-flight (TOF) range measurement and a greyscale intensity measurement in the same pixel of an image sensory array.

[0003] Existing single-photo single-photon avalanche diode (SPAD) sensors for Light Detection and Ranging (LiDAR) have a low spatial resolution, a low fill factor and a high power consumption if a time-to-digital counter (TDC) is placed inside each pixel of a sensor array. If a single-ended time-to-analog converter (TAC) is used instead of a TDC, then a low range accuracy results. Additionally, existing SPAD sensors do not provide an intensity imaging capability together with range information.

[0004] An example embodiment provides a pixel of an image sensor that may include a pinned photodiode (PPD), a switching device, and an output circuit. The PPD may have a first terminal and a second terminal in which the second terminal may be coupled to a ground voltage. The switching device may have first, second and third terminals in which the first terminal may be coupled to the first terminal of the PPD, the second terminal may be coupled to a floating diffusion, and the third terminal may be coupled to a first enable signal and a second enable signal. The switching device may be responsive to the first enable signal to transfer a first charge on the PPD to the floating diffusion, and may be responsive to the second enable signal to transfer a second charge on the PPD to the floating diffusion, the second enable signal being subsequent to the first enable signal. The output circuit may output a first voltage that is based on the first charge on the floating diffusion and may output a second voltage that is based on the second charge on the floating diffusion in which the first voltage may correspond to a time of flight of one or more detected photons and the second voltage may correspond to a greyscale image.

[0005] Another example embodiment provides an image sensor that may include an array of a plurality of pixels. At least one pixel of the array of pixel may include a pinned photodiode (PPD), a switching device, and an output circuit. The PPD may have a first terminal and a second terminal in which the second terminal may be coupled to a ground voltage. The switching device may have first, second and third terminals in which the first terminal may be coupled to the first terminal of the PPD, the second terminal may be coupled to a floating diffusion, and the third terminal may be coupled to a first enable signal and a second enable signal. The switching device may be responsive to the first enable signal to transfer a first charge on the PPD to the floating diffusion, and may be responsive to the second enable signal to transfer a second charge on the PPD to the floating diffusion, the second enable signal being subsequent to the first enable signal. The output circuit may output a first voltage that is based on the first charge on the floating diffusion and may output a second voltage that is based on the second charge on the floating diffusion in which the first voltage may correspond to a time of flight of one or more detected photons and the second voltage may correspond to a greyscale image.

[0006] Still another example embodiment provides an image sensor that may include an array of pixels. At least one pixel of the array of pixels may include a capacitive device, a switching device, and an output circuit. The switching device may be responsive to a first enable signal to transfer a first charge on the capacitive device to a floating diffusion, and may be responsive to a second enable signal to transfer a second charge on the capacitive device to the floating diffusion in which the second enable signal may be subsequent to the first enable signal. The output circuit may output a first voltage that is based on the first charge on the floating diffusion and may output a second voltage that is based on the second charge on the floating diffusion in which the first voltage may correspond to a time of flight of one or more detected photons and the second voltage may correspond to a greyscale image.

[0016] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the disclosure. It will be understood, however, by those skilled in the art that the disclosed aspects may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail not to obscure the subject matter disclosed herein.

[0017] Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment disclosed herein. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” or “according to one embodiment” (or other phrases having similar import) in various places throughout this specification may not be necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In this regard, as used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not to be construed as necessarily preferred or advantageous over other embodiments. Also, depending on the context of discussion herein, a singular term may include the corresponding plural forms and a plural term may include the corresponding singular form. It is further noted that various figures (including component diagrams) shown and discussed herein are for illustrative purpose only, and are not drawn to scale. Similarly, various waveforms and timing diagrams are shown for illustrative purpose only. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, if considered appropriate, reference numerals have been repeated among the figures to indicate corresponding and/or analogous elements.

[0018] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting of the claimed subject matter. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The terms “first,” “second,” etc., as used herein, are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly defined as such. Furthermore, the same reference numerals may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules having the same or similar functionality. Such usage is, however, for simplicity of illustration and ease of discussion only; it does not imply that the construction or architectural details of such components or units are the same across all embodiments or such commonly-referenced parts/modules are the only way to implement the teachings of particular embodiments disclosed herein.

[0019] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0020] The disclosures of U.S. patent application Ser. No. 15/878,392, filed Jan. 23, 2018, entitled “Time-Resolving Sensor Using SPAD+PPD or Capacitors in Pixel For Range Measurement,” and Ser. No. 16/140,529, filed Sep. 24, 2018, entitled “Time-Resolving Image Sensor for Range Measurement and 2D Greyscale Imaging,” are incorporated by reference in their entirety.

[0021] One example embodiment of the subject matter disclosed herein may be used to provide improved vision for drivers under difficult conditions, such as, but not limited to, low light, bad weather, or strong ambient light, and provides an improvement to the performance of an autonomous navigation system by combining a single-photon avalanche diode (SPAD) circuit, a logic circuit, and a pinned photodiode (PPD) circuit in a pixel of an time-of-flight system. Such a system may provide range measurements that are based on a controlled charge transferring, photon counting and a single-ended-to-differential time-to charge conversion. In one embodiment, the PPD circuit may include more than one transfer gate and more than one storage node. The PPD circuit also outputs a greyscale intensity image. Because the same PPD circuit captures the range information and the greyscale intensity information, no image fusion of the two images is needed because both images are aligned.

[0022] In one embodiment, each pixel of an array of pixels of a LiDAR image sensor includes a SPAD and PPD-based time-to-charge converter (TCC). Three-dimensional (3D) TOF information may be obtained using a synchronized laser source. After the synchronized laser source has been flashed, TOF information stored at a floating diffusion (FD) node of a PPD circuit in each pixel of an array of pixels is read out in, for example, a raster-scan manner. While the TOF information is being read out, the PPD may be reset and is then used to integrate ambient light to capture a two-dimensional (2D) image for a low ambient-light environment. The 2D integrated ambient light signal detected by the PPD may be transferred to the FD node for read out after the TOF information has been read out from the FD. In one embodiment, a second transfer gate and a second FD node may be used for the 2D image readout instead of sharing the same output path that is used by the TOF information readout. In another embodiment, a capacitor-based TCC circuit may be used for TOF information, and a separate PPD circuit may be used for 2D image information. This example embodiment uses separate output paths for the 3D and the 2D image data.

[0023] FIG. 1 depicts a block diagram of an example embodiment of a pixel 100 that generates a ranging measurement output and a greyscale 2D image output according to the subject matter disclosed herein. The pixel 100 may be one of a plurality of other similar pixels in an image array (not shown). The pixel 100 may include one or more SPAD circuits 101a-101n, a logic circuit 103, and a PPD circuit 105.

[0024] In one embodiment, each of the one or more SPAD circuits 101 may include a SPAD 111, a resistor 113, a capacitor 115, a p-type MOSFET transistor 117 and a buffer 119. The SPAD 111 may include an anode connected to a ground potential, and a cathode. The resistor 113 may include a first terminal for receiving a VS PAD voltage and a second terminal connected to the cathode of the SPAD 111. In another embodiment, the positions of the SPAD 111 and the resistor 113 may be swapped. The SPAD 111 may respond to a photon. In response to receiving a photon, the SPAD 111 outputs a pulse signal that goes rapidly from the VSPAD voltage to below a breakdown voltage of the SPAD 111, and then more gradually returns to the VSPAD voltage.

[0025] The capacitor 115 may include a first terminal connected to the cathode of the SPAD 111 and a second terminal. In an alternative embodiment, the capacitor 115 may be omitted. The p-type MOSFET 117 may include a first S/D terminal connected to the second terminal of the capacitor 115, a gate for receiving a SHUTTER signal, and a second S/D terminal for receiving a VPIX voltage (V_DD). The buffer 119 may include an input connected to the second terminal of the capacitor 115, and an inverting output, which may output a DE signal corresponding to the output of the SPAD circuit 101. In an alternative embodiment, the buffer 117 may be noninverting.

[0026] The logic circuit 103 may include inputs connected to the DE signal of each of the one or more SPAD circuit 101a-101n, and outputs a TXEN signal and a TXENB signal, which may be an inversion of the TXEN signal.

[0027] The PPD circuit 105 may include a pinned photo diode (PPD) 151, a first transistor 152, a second transistor 153, a third transistor 154, a fourth transistor 155, a fifth transistor 156, a sixth transistor 157, and a seventh transistor 158.

[0028] The PPD 151 may be exposed to light and may include a first terminal connected to a ground potential, and a second terminal. The PPD 151 may store a charge in a manner that is similar to a capacitor and in some cases be referred to as a capacitive device.

[0029] The first transistor 152 may include a gate terminal connected to an RSTPD signal, a first S/D terminal connected to a VPD potential, and a second S/D terminal connected to the second terminal of the PPD 151.

[0030] The second transistor 153 may include a gate terminal connected to a TX signal, a first S/D terminal connected to a floating diffusion FD node, and a second S/D terminal connected to the second S/D terminal of the first transistor 152 and the second terminal of the PPD 151. The floating diffusion FD node is represented in FIG. 1 with a capacitor symbol. There may be a parasitic capacitance between the FD node and ground, which is not indicated in FIG. 1. In one embodiment, a physical capacitance may also be connected between the FD node and ground.

[0031] The third transistor 154 may include a gate terminal connected to the FD node and the first S/D terminal of the second transistor 153, a first S/D terminal connected to a VPIX voltage, and a second S/D terminal. The third transistor 154 may operate to convert a charge on the FD node to a voltage at the second S/D terminal of the third transistor 154. The fourth transistor 155 may include a gate terminal connected to a RST signal, a first S/D terminal connected to the VPIX voltage, and a second S/D terminal connected to the first S/D terminal of the first transistor 152 and the FD node.

[0032] The fifth transistor 156 may include a gate terminal connected to the TXEN signal, a first S/D terminal connected to a VTX signal, and a second S/D terminal connected to the gate terminal of the second transistor 153. The sixth transistor 157 may include a gate terminal connected to the TXENB signal, a first S/D terminal connected to a ground potential, and a second S/D terminal connected to the gate terminal of the second transistor 153 and the second S/D terminal of the fifth transistor 156. The seventh transistor 158 may include a gate terminal connected to an SEL signal, a first S/D terminal connected to the second S/D terminal of the third transistor 154, and a second S/D terminal connected to a pixel output line PIXOUT1.

[0033] FIG. 2 depicts an example relative signal timing diagram 200 for the pixel 100 that generates a ranging measurement and a greyscale 2D image according to the subject matter disclosed herein. Initially, the RST signal also goes high, which allows the fourth transistor 155 to conduct the VPIX voltage to the FD node, resetting the FD node. The PPD 151 is reset to 0V.

[0034] After the SHUTTER signal goes high, a light pulse is transmitted a delay of T_dly later. When the SHUTTER signal is active high, the TXEN signal is active and the VTX signal is passed through the fifth transistor 156, thereby making TX signal active. The TX signal modulates the voltage applied to the gate of the second transistor 153 to make the charge that is transferred from the PPD to the floating diffusion FD node a function of time. When a detection event DE occurs, the TXEN signal becomes inactive and the TX signal also become inactive. The time between when the light pulse is transmitted and the detection event DE is the time of flight T_tof. The charge that was being transferred from the PPD 151 to the FD node stops. When the TXENB signal becomes active, a ground voltage is connected to the gate of the second transistor 153.

[0035] When the SHUTTER signal ends, the TX signal becomes inactive and the voltage associated with the charge level on the FD node corresponds to the T_tof of the light pulse, which may be read out from the PIXOUT1 output line. The time the SHUTTER signal is active is T_sh. The voltage corresponding to the charge level for the time-of-flight is output from the PIXOUT 1 output line. Variations in the slope of the VTX signal from pixel to pixel do not cause range measurement errors as long as the second transistor 153 operates in a linear mode during an active SHUTTER signal.

[0036] After the T_tof voltage has been read out from the PIXOUT1 output line, the RSTPD signal goes high, resetting the charge on the PPD 151 to be high (3 V). The RST signal also goes high resetting the charge level on the FD node. The TXEN signal goes high a greyscale-integration time T_gs after the PPD 151 has been reset causing the charge level on the PPD 151, which has been integrating the received light during the integration time T_gs, to be transferred to the FD node. The charge on the FD node corresponds to a greyscale image intensity received at the pixel, which may be read out from the PIXOUT1 output line as a regular 2D sensor.

[0037] FIG. 3 depicts a flow diagram of a method 300 to operate a pixel to generate a ranging measurement and a greyscale 2D image according to the subject matter disclosed herein. The method starts at 301. At 302, an active shutter signal is generated. At 303, one or more photons that are incident on at least one SPAD circuit 101 are detected (a detection event DE) during the active shutter signal in which the one or more detected photons have been reflected from an object. At 304, an output signal is generated based on the detection event DE. At 305, a first enable signal TXEN and a second enable signal TXENB are generated based on the output signal for the detection event DE. In one embodiment, the first enable signal becomes active in response to a beginning of the active shutter signal and becomes inactive in response to the output signal, and the second enable signal becomes active in response to the output signal and becomes inactive in response to an end of the active shutter signal.

[0038] At 306, a charge on a capacitor device SC is transferred to a floating diffusion FD node to form a first charge on the floating diffusion FD node based on the first enable signal being active. At 307, the first charge is output on the PIXOUT output line as a voltage corresponding to a time-of-flight of the one or more detected photons. At 308, the PPD 151 is reset. At 309, the ambient light received by the PPD 151 is integrated for a greyscale-integration time T_gs to generate a second charge level on the PPD 151 that corresponds to the greyscale intensity of the image received at the pixel. At 310, the second charge level is transferred to the floating diffusion FD and output on the PIXOUT output line as a voltage corresponding to the greyscale intensity of the image received at the pixel. The method ends at 311.

[0039] FIG. 4 depicts a block diagram of a second example embodiment of a PPD circuit 405 that generates a ranging measurement and a greyscale 2D image according to the subject matter disclosed herein. The PPD circuit 405 differs from the PPD circuit 105 of FIG. 1 by including a separate path for reading out the greyscale intensity value. Rather than being output from the PIXOUT1 output line, the greyscale intensity value is output through a separate PIXOUT2 output line. Also, a TX2 signal, which is applied to the gate of a transistor 401, goes high to transfer the greyscale charge level on the PPD 151, which has been integrating the received light during the integration time T_gs, to a second floating diffusion FD2 node. The second floating diffusion FD2 node is represented in FIG. 4 with a capacitor symbol. The charge on the second floating diffusion FD2 node corresponds to a greyscale image intensity received at the pixel, which may be read out from the PIXOUT2 output line using transistors 402 and 403. Operation of the PPD circuit 405 is similar to the timing diagram shown in FIG. 2.

[0040] FIG. 5 depicts a third example embodiment of a PPD circuit 505 that generates a ranging measurement and a greyscale 2D image according to the subject matter disclosed herein. The PPD circuit 505 differs from the PPD circuit 105 of FIG. 1 by including a storage diffusion SD node and a transistor 501. FIG. 6 shows an example relative signal timing diagram 600 for the PPD circuit 500 that generates a ranging measurement and a greyscale 2D image according to the subject matter disclosed herein.

[0041] FIG. 7 depicts a fourth example embodiment of a PPD circuit 705 that generates a ranging measurement and a greyscale 2D image according to the subject matter disclosed herein. The fourth PPD circuit 705 may include a capacitor device SC, a first transistor 751, a second transistor 753, a third transistor 755, a fourth transistor 757, a fifth transistor 759, a sixth transistor 761, a seventh transistor 763, an eighth transistor 765, a ninth transistor 767, a tenth transistor 769, an eleventh transistor 771, a twelfth transistor 773, and a thirteenth transistor 775.

[0042] The capacitor device SC may include a first terminal connected to a ground potential, and a second terminal. The capacitor device SC may store a charge in a manner that is similar to a capacitor. In one embodiment, the capacitor device may be a capacitor. In another embodiment, the capacitor device may be a PPD that may be covered so that it does not respond to light. In either embodiment, the capacitor device SC may be used as part of a time-to-charge converter.

[0043] The first transistor 751 may include a gate terminal connected to an RST signal, a first S/D terminal connected to a ground potential, and a second S/D terminal connected to the second terminal of the capacitor device SC. The second transistor 753 may include a gate terminal connected to a TXA signal, a first S/D terminal connected to a first floating diffusion FD1 node, and a second S/D terminal connected to the second S/D terminal of the first transistor 751 and the second terminal of the capacitor device SC. The first floating diffusion FD1 node is represented in FIG. 7 with a capacitor symbol. There may be a parasitic capacitance between the FD1 node and ground, which is not indicated in FIG. 7. In one embodiment, a physical capacitance may also be connected between the FD1 node and ground.

[0044] The third transistor 755 may include a gate terminal connected to the FD1 node and the first S/D terminal of the second transistor 753, a first S/D terminal connected to a VPIX voltage, and a second S/D terminal. The third transistor 755 may operate to convert a charge on the FD1 node to a voltage at the second S/D terminal of the third transistor 755. The fourth transistor 757 may include a gate terminal connected to the RST signal, a first S/D terminal connected to the VPIX voltage, and a second S/D terminal connected to the second S/D terminal of the first transistor 751 and the second terminal of the capacitor device SC.

[0045] The fifth transistor 759 may include a gate terminal connected to the TXEN signal, a first S/D terminal connected to a VTX signal, and a second S/D terminal connected to the gate terminal of the second transistor 753. The sixth transistor 761 may include a gate terminal connected to the TXENB signal, a first S/D terminal connected to a ground potential, and a second S/D terminal connected to the gate terminal of the second transistor 753 and the second S/D terminal of the fifth transistor 759.

[0046] The seventh transistor 763 may include a gate terminal connected to an SEL signal, a first S/D terminal connected to the second S/D terminal of the third transistor 755, and a second S/D terminal connected to a pixel output line PIXA. The eighth transistor 765 may include a gate terminal connected to a TXB signal, a first S/D terminal connected to a second floating diffusion FD2 node, and a second S/D terminal connected to the second S/D terminal of the first transistor 751, the second terminal of the capacitor device SC, and the second terminal of the second transistor 753. The second floating diffusion FD2 node is represented in FIG. 7 with a capacitor symbol. There may be a parasitic capacitance between the FD2 node and ground, which is not indicated in FIG. 7. In one embodiment, a physical capacitance may also be connected between the FD2 node and ground.

[0047] The ninth transistor 767 may include a gate terminal connected to the FD2 node and the first S/D terminal of the eighth transistor 765, a first S/D terminal connected to the VPIX voltage, and a second S/D terminal. The ninth transistor 767 may operate to convert a charge on the FD1 node to a voltage at the second S/D terminal of the ninth transistor 767. The tenth transistor 769 may include a gate terminal connected to the RST signal, a first S/D terminal connected to the VPIX voltage, and a second S/D terminal connected to the second S/D terminal of the first transistor 751, the second terminal of the capacitor device SC, and the second S/D terminal of the eighth transistor 765.

[0048] The eleventh transistor 771 may include a gate terminal connected to the TXENB signal, a first S/D terminal connected to the VTX signal, and a second S/D terminal connected to the gate terminal of the eighth transistor 765. The twelfth transistor 773 may include a gate terminal connected to the TXEN signal, a first S/D terminal connected to a ground potential, and a second S/D terminal connected to the gate terminal of the eighth transistor 765 and the second S/D terminal of the eleventh transistor 771. The thirteenth transistor 765 may include a gate terminal connected to the SEL signal, a first S/D terminal connected to the second S/D terminal of the ninth transistor 767, and a second S/D terminal connected to a pixel output line PIXB.

[0049] The PPD circuit 705 also includes a separate PPD for the greyscale intensity measurement. That is, the PPD circuit 705 may also include most of the components of the PPD circuit 105 in FIG. 1, and operates in a similar manner to measure a greyscale intensity. The PPD 151 of the PPD circuit 705 is separate from the capacitor device SC.

[0050] FIG. 8 depicts an example relative signal timing diagram 800 for the operation of time-of-flight portion of the PPD circuit 700 according to the subject matter disclosed herein. In the signal timing diagram of FIG. 8, the TXENB signal is an inversion of the TXEN signal. When the SHUTTER signal is active high, the TXEN signal is active and the VTX signal is passed through the fifth transistor 759, thereby making the TXA signal is active. The charge on the capacitor device SC is transferred through the second transistor 753 to the FD1 node. Meanwhile, the ground potential is passed through the twelfth transistor 773, which makes the TXB signal inactive.

[0051] When a detection event DE occurs, the TXEN signal becomes inactive and the TXENB signal becomes active. When the TXEN signal becomes inactive, the TXA signal also become inactive and charge is stopped from being transferred through the second transistor 753 from the capacitor device SC to the FD1 node. When the TXENB signal becomes active, the TXB signal becomes active and charge is transferred from the capacitor device SC through the eighth transistor 765 to the FD2 node.

[0052] When the SHUTTER signal ends, the TXB signal becomes inactive and charge is stopped from being transferred through the eighth transistor 765 from the capacitor device SC to the FD2 node. The respective voltages associated with the charges on the FD1 node and the FD2 node are read out from the PIXA and PIXB output lines.

[0053] It should be noted that variations in the slope of the VTX signal and variations in the capacitance of the capacitor device SC from pixel to pixel do not cause range measurement errors, as long as the second transistor 753 (TXA) and the eighth transistor 765 (TXB) operate in a linear mode during an active SHUTTER signal.

[0054] As will be recognized by those skilled in the art, the innovative concepts described herein can be modified and varied over a wide range of applications. Accordingly, the scope of claimed subject matter should not be limited to any of the specific exemplary teachings discussed above, but is instead defined by the following claims.