Three-dimensional reconstruction of a millimeter-wave scene by optical up-conversion and cross-correlation detection

[0001] The present application claims priority to U.S. Provisional Patent Application No. 61/750,865, filed on Jan. 10, 2013, and incorporated herein by reference in its entirety.

[0002] In existing approaches to millimeter wave (mmW) based imaging, an array of optical fibers carrying the up-converted mmW signal may be arranged to mimic an array of antennas capturing the mmW radiation from the scene. The optical beams from the fibers may then be allowed to propagate in free space, and interfere to form an optical image corresponding to the mmW scene on a CCD array. Ideally, each pixel of the CCD receives optical beams from all the fibers. The drawback to this approach is that it does not readily provide a way to obtain three-dimensional (3D) images.

[0003] Embodiments of the invention may include a device and associated method for three-dimensional imaging of a millimeter-wave (mmW) scene that may use sparse-aperture capture of the radiation, its up-conversion to optical domain, optical measurement of cross-correlation terms, and digital reconstruction of the original mmW scene from the cross-correlation terms.

[0013] As noted above, in prior arrangements for mmW imaging, optical beams from the various fibers carrying upconverted mmW signals may be allowed to propagate in free space, and may interfere to form an optical image corresponding to the mmW scene on a CCD array. In contrast, in embodiments of the present invention, the beams from the fibers may be allowed to interfere only pairwise; that is, each detector may receive beams from as few as two fibers. As a result, the intensities measured by individual detectors do not correspond directly to the mmW intensities of the scene as in the conventional configuration. Instead, the mmW scene may be reconstructed, digitally from the measured intensities. This may permit recovery of 3D information.

[0014] We begin here by describing sparse-aperture millimeter-wave imaging based on optical upconversion in some detail to lay the groundwork for the system improvement that is the subject of embodiments of this invention. FIG. 1 shows an overview of an example of a distributed-aperture mmW system. Millimeter-wave radiation originating at the scene may be captured by a sparsely distributed array of antennas 10 (see, also, FIG. 1A), 21 (see FIG. 2A), which may couple their detected radiation to respective electro-optic (EO) modulators 22, as shown in FIG. 2A. An EO modulator 22 may convert the RF energy to the optical domain. It may do so by modulating an optical beam (see FIG. 2B) produced by an optical source 20, such as, but not limited to, a laser. The time-variant modulation may manifest itself in the frequency domain as a set of sidebands flanking the original carrier frequency (or wavelength), at which the optical source operates, an example of which is shown in FIG. 2C. As a result, the RF energy radiated in the mmW domain may appear in the optical domain as sidebands of the carrier frequency. This up-conversion of the mmW signal into the optical domain may be coherent in the sense that all the phase and amplitude information present in the mmW is preserved in the optical sidebands. This property of coherence preservation in optical up-conversion may allow the recovery of the mmW image using optical means.

[0015] Returning to FIG. 1, the optical beams containing the laser carrier wavelength and the sidebands with imprinted mmW signal may be conveyed by optical fibers to a lenslet array 12 that may mimic the array of the RF antennas, at a reduced scale. The lenslet array 12 and other components may be part of an optical processing unit 11. Following lenslet array 12, the beams may propagate in free space, no longer guided by optical fibers. The propagation of the individual beams in free space, where they may overlap to form a large combined beam 13, may allow the individual beams to interfere with one another. Part of the combined beam 13 may be split off, e.g., using a splitter 15, combined with a reference beam 14, and sent to an array of detectors 18, in order to detect, and allow for the compensation of, optical phase variation originating in the individual fibers due to environmental conditions, such as vibrations and acoustics. This may ensure that the resulting image comes from the mmW scene as opposed to vibrating fibers. A band-pass optical filter, 16 in FIG. 1 or 23 in FIG. 2A, may be used to strip off the carrier wavelength and may allow only one of the sidebands to pass through, e.g., as shown in FIG. 2D. The overlapping beams that now carry only a single sideband may be projected onto a charge-coupled device (CCD) array 17 (shown as a “Camera” in FIG. 1), where they may interfere to form a representation of the mmW image in the optical domain. In other words, the optical image formed by the overlapping beams may correspond to a replica of a mmW image as seen by the sparse aperture of the antenna array.

[0016] As described above, the mmW image may be reconstructed directly on the CCD array 17 by the interference of light emanating from individual fibers. As such, the imaging may be limited to a two-dimensional reconstruction of the mmW scene, just as a point-end-shoot camera captures only a two-dimensional representation of a three-dimensional world. Yet, the fibers may generally carry all information available to the sparse aperture—including the depth information of the scene. This information is encoded in cross-correlations between the individual channels. To access this information, cross-correlations between channels may be measured as shown in the example embodiments of the present invention.

[0017] To aid in elucidating the concepts of various embodiments of the invention, a configuration of an imaging system with an emphasis on the optical layer is presented in FIG. 3. A single laser source may be split M ways 30, and the beams may be routed through modulators coupled to antennas capturing the mmW radiation 31. The (optical) outputs of the modulators may be filtered 32 to allow only a single sideband corresponding to the captured mmW radiation to pass. The correlation engine 33 may then allow the interference of the optical beams among the different channels, and the result of the interference may be measured with an array of detectors 34.

[0018] In abstract mathematical terms, the mmW imager may be thought of as corresponding to a linear operator from the scene to the detector array. An arbitrary mmW scene can be represented as a vector of mmW intensities emanating from each point of the considered volume. Similarly, the intensities detected by the detectors in the array can be arranged in a vector. The imager may serve to convert the mmW (emission) intensities to optical intensities that may be detected by the elements of the detector array. This “conversion” may be a linear process in the sense of a linear operation from the (linear) space of all possible mmW scenes to the (linear) space of all possible optical detections. The linear operator corresponding to this conversion process may depend on the details of the imager design—in particular, it may depend on the antenna configuration and on the chosen optical correlation engine. Under some circumstances, an (approximate) inverse of this operator can be found, which may allow the reconstruction of the full mmW scene from the measured optical intensities.

[0019] In the context of some embodiments of the present invention, this operator may be explicitly calculated for the case of an arbitrary sparse-aperture antenna array and a pairwise cross-correlation, engine. In this case, every pair of the M channels may be routed through a 50/50 beam splitter, and two intensities may be measured, as reflected in FIG. 4.

[0020] The following is an example of mathematical analysis to illustrate how the 3D mmW scene may be recovered from pairwise cross-correlations. It is noted that, as stated in general terms above, the concepts discussed here can also be readily applied to the case where more than two beams are allowed to interfere at any given detector (i.e., cross-correlations among more than two beams). Therefore, the analysis for mmW image recovery from pairwise cross-correlations presented below should be construed as an illustrative example rather than limiting the scope of the invention.

[0021] The time variation of the electric field in an optical fiber before the modulator may generally correspond to monochromatic radiation and may thus be expressed as

[0022] $\begin{array}{cc}\frac{1}{\sqrt{2}}{B}_m{e}^{i\omega t}+c.c.,& \left(1\right)\end{array}$

[0023] where B_m is the amplitude of the optical field in the m-th channel, ω is the optical frequency, and c.c. signifies the presence of the complex-conjugate term as required to make expression (1) real. The modulation with the mmW radiation captured from the scene may modify (1) by introducing an extra phase φ_m, which may result in the following time-variable electric field:

[0024] $\begin{array}{cc}{A}_m=\frac{1}{\sqrt{2}}{B}_m{e}^{i\left(\omega t+{\phi }_m\right)}+c.c.& \left(2\right)\end{array}$

[0025] Phase φ_m may generally be time-dependent. It may also depend on the placement of the particular antenna in the array, and/or on the intensities of the sources. Assuming that the scene consists of a discrete set of monochromatic mmW sources, the phase φ_m may be written in the form

[0026] $\begin{array}{cc}{\phi }_m=\sum _k^{}\frac{S_k}{r_{\mathrm{km}}}\cos \left(\Omega t+{\varphi }_{\mathrm{km}}\right),& \left(3\right)\end{array}$

[0027] where k enumerates the mmW sources, S_k is the amplitude of radiation at the k-th source sealed by the antenna gain and by the modulation efficiency of the modulator, r_km is the distance between the k-th source and the m-th antenna, Ω is the frequency of the mmW radiation, and φ_km is a phase that the mmW radiation may pick up on the way from the source to the antenna (which is=r_kmΩ/c for free-space propagation).

[0028] Assuming that the mmW sources in the scene are uncorrelated, as may be typical for passive imaging, the power detected at the outputs of the 50/50 splitters in FIG. 4 may be found as

[0029] $\begin{array}{cc}{P}_{{\mathrm{mm}}^{\prime }}=\frac{1}{8M}\sum _k^{}{S}_k^2\left[\frac{B_m^2}{r_{\mathrm{km}}^2}+\frac{B_{m^{\prime }}^2}{r_{{\mathrm{km}}^{\prime }}^2}-2\frac{B_{\mathrm{km}}{B}_{{\mathrm{km}}^{\prime }}}{r_{\mathrm{km}}{r}_{{\mathrm{km}}^{\prime }}}\sin \left({\varphi }_{\mathrm{km}}-{\varphi }_{{\mathrm{km}}^{\prime }}\right)\right],& \left(4\right)\end{array}$

[0030] which shows that the detected optical power P_mm′ is related by a linear operation to the mmW power emitted by the scene S_k². Inverting this relation may allow digital reconstruction of the full 3D mmW scene from the measured cross-correlation terms.

[0031] Stated another way, the entire mmW scene (defined by terms S_k²) can be, in principle, reconstructed from the measurement of cross-correlation terms P_mm′, as long as there is a sufficient number of antennas distributed throughout the aperture. With M channels, there are

[0032] $\begin{array}{cc}\frac{\left(M-1\right)M}{2}& \left(5\right)\end{array}$

[0033] pairs and, correspondingly, (M−1)M measurements to be made for the cross-correlations. For a large number of channels, using a 50/50 beam splitter as in FIG. 4 for each pair may become cumbersome. As an alternative, a somewhat integrated approach may be used instead. An example of such an integrated approach is illustrated in FIG. 5, where first the beam carried by each fiber carrying a modulated optical signal may be split into two 50, 51, and then the output of each fiber may be split M ways 53. The fibers in each of the two bundles may be arranged in a linear array in such a way as to produce a square array of free-propagating beams. The square arrays may then be combined using a beam splitter/combiner 54 to yield two square arrays of spots 55, where each spot may represent a combination of two beams carried in two channels. Alternatively, the beam splitting and combining may be achieved in free space, or dielectric medium, starting with a single, for example, linear, array of fibers, and using optical elements known in the art, including, e.g., but not limited to, (free-space) splitter/combiners, mirrors, lenses, and/or wave-plates.

[0034] In order to ensure proper interference at the detector array(s), the polarization of the component, and combined beams may be managed using, for example, means that may include polarizers, beam splitters/combiners, wave-plates, and/or polarization rotators.

[0035] Embodiments of the present invention may provide the unique ability to fully reconstruct a three-dimensional mmW scene without the use of active illumination—i.e., by using only passive imaging and relying on the natural radiation of millimeter waves by objects at terrestrial temperatures. As such, embodiments of the invention can be applied to improve the imaging capabilities of mmW imagers based on sparse-aperture and optical up-conversion.

[0036] Numerical simulations have been carried out that confirm the validity of the approach. In order to verify the approach presented above, a series of computational experiments were performed in which a certain antenna configuration was assumed and was presented with a simple geometrical pattern of mmW emission. Cross-correlation terms were then computed using Eq. (1), and the simulations computationally reversed the relation to see what a recovered scene would look like.

[0037] In this preliminary study, the size of the computational space was kept to a minimum. Also, for ease of visualizations the dimension of the imaged space was reduced from three to two. To this end, the two transverse dimensions were collapsed to one, and the depth dimension was maintained in order to test the ability of the system to recover the scene in this dimension.

[0038] Schematically, the scene set-up was as shown in FIG. 6. FIG. 6 shows an antenna array (shown here as a two-dimensional (2D) array; a one-dimensional (1D) array was used in simulations), on the left, and a simple geometrical pattern of a mmW source, on the right. The rest of the scene was non-emitting, and is shown as black. The choice of the mmW scene and of the radiation pattern was dictated by the desire of simplicity on the one hand, and by the need to extract useful information from the tests, on the other hand.

[0039] For the simulations, two different distances of the object from the aperture were used, along with three different populations of the antenna array. Also, given that the problem being simulated is scale-invariant, the dimensions were expressed in terms of the wavelength λ of the mmW radiation used for imaging. The aperture was 154λ across, and the distance from the aperture to the nearest part of the object was 450λ or 900λ for the two tests. The imaged object was 124.5λ high and 28.5λ deep.

[0040] FIGS. 7A-7F show the results of the simulated scene reconstruction. The images shown in FIGS. 7A-7F corroborate the analytical findings and demonstrate that the depth information of a mmW scene can be recovered computationally from cross-correlation terms. The images also provide some insight as to which system parameters may play an important role in the quality of the recovered image. The fidelity of the recovery may depend on the number of antennas used in the array. However, the effect of the antenna number may be considerably more pronounced for objects lying closer to the aperture than those farther away. Also, in general, the image quality may improve for objects closer to the aperture.

[0041] FIG. 8 shows an example flow-chart of a method that incorporates some of the above-described concepts. Received RF signals 80 may be modulated onto optical signals 81. Cross-correlations of the resulting modulated optical signals may then be detected 82 in the optical domain. Finally, the detected cross-correlations may be processed to obtain a reconstructed image that may correspond to image data carried by the received RF signals 83.

[0042] It is noted that a processing device to implement the processing 83 may be integrated with detectors shown and described above or may be implemented as a separate processing device. Such a processing device may include a computer or other general-purpose or application-specific processing hardware, which, particularly in the case of general-purpose processing hardware, may be programmed with appropriate software as needed to perform the processing. Application-specific processing hardware may be implemented in many forms, for example, but not limited to, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), programmable logic arrays (PLAs), etc.

[0043] It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. Rather the scope of the present invention includes both combinations an sub-combinations of various features described hereinabove as well as modifications and variations which would occur to persons skilled in the art upon reading the foregoing description and which are not in the prior art.