Super-resolving imaging system

[0002] 1. Field of the Invention

[0003] This invention relates to an imaging system. More particularly this invention relates to a super-resolving imaging apparatus having increased depth of field which can be employed in collision avoidance systems for vehicles, and for distance estimation applications in general.

[0004] 2. Description of the Related Art

[0005] Nowadays distance estimation is mainly performed using two basic techniques: active sensing and image processing. In the active sensing category are included radar-type devices which illuminate the region of interest by scanning and then estimating the distance by measuring the time between the transmitted and received signals. Although such devices are relatively simple, they are presently too expensive to be practical in private vehicles. These devices have limited resolution, and are generally too slow for real-time collision avoidance applications for terrestrial vehicles.

[0006] The other major category distance estimation devices are passive systems based on detector arrays, such as CCD, CMOS, and ICCD gated CCD, which employ advanced image processing algorithms. Such systems, in order to obtain algorithmic convergence, require high resolution and thus have a restricted field of view. These systems also suffer from a limited depth of focus. In addition, the postprocessing algorithms that perform the time-to-impact evaluation require heavy computation. Despite their present limitations, the passive systems are constantly being upgraded in technological performance, and are being dramatically reduced in price.

[0007] Technical problems in the field of optical scanning are similar to those of the immediate field of the invention. Imaging based scanners are required to convert spatial degrees of freedom in order to obtain high horizontal and vertical resolution, and adequate depth of focus. Here are several related patents that deal with high resolution and extended depth of field scanning of objects as bar codes. Several disclosures are known to deal with similar problems. U.S. Pat. No. 5,657,147 discloses an optical scanner which includes a light source and optical deflector for scanning a light flux from the light source. Beams with extended confinement for scanning purposes, as proposed in U.S. Pat. No. 5,315,095, may be the basis for scanners usable, for example in a bar code reader, by generating a multitude of Gaussian beams derived from a common light source. A scanner using this technique has an extended working range. An optical scanner for reading barcodes detected within a large depth of field is disclosed in U.S. Pat. No. 4,748,316, which proposes the use of a plurality of scan lines having different focal lengths. If at least one of the scan lines is small relative to irregularities of the surface on which the label appears, an extended depth of scanning can be achieved. U.S. Pat. No. 4,820,911, describes an apparatus which is adapted to be hand-held and utilizes anamorphic optics for scanning and reading a bar code. The optics provides a beam having an oblong cross-section in the nominal plane of the bar code. U.S. Pat. No. 5,680,253 discloses a light beam scanning apparatus comprising a first and second diffraction grating plate which provides a high resolution scanning apparatus utilizing mass producible holograms instead of utilizing auxiliary optical systems. An optical system for a large depth of field barcode scanner was suggested in U.S. Pat. No. 4,978,860 to Bayley et al. The working range of this device is determined not by the lens aperture, but by the dimension and orientation of the detector array using a modified Scheimpflug arrangement.

[0008] Nevertheless, the existing prior art devices do not fully solve the technical problems required for imaging systems to be used for distance estimation and collision warning for vehicles.

[0009] The invention provides super-resolving devices and their applications. More particularly, it provides geometrical super-resolution, i.e. resolution that is better than the resolution defined by the geometry of the detector array. In addition, the disclosed technology also provides apparatus to extend the depth of field of the system.

[0010] It is a primary object of some aspects of the present invention to improve imaging systems in the fields of distance estimation and collision warning for vehicles.

[0011] It is another object of some aspects of the present invention to provide apparatus for estimating distance that affords wide depth of focus and a large horizontal field of view.

[0012] It is yet another object of some aspects of the present invention to improve the speed of distance estimation in collision warning systems.

[0013] These and other objects of the present invention are attained by imaging apparatus in which special optical elements placed on the imaging lens and the detector array allow the conversion of degrees of freedom in one axis of a field of view to a larger degree of freedom in another axis in order to obtain a high resolution image with a wide depth of focus and large field of view suitable for performing distance estimation on an object within the field of view.

[0014] The invention employs a numerical algorithm which is simple and can be rapidly performed to yield the desired result.

[0015] According to some aspects of the invention a detector array with a specially designed optical element is directed toward a large volume of space in which a target may be found. In certain applications, such as vehicular warning systems, the vertical field of view required is significantly smaller than the actual vertical field of view of the apparatus. The special optical element converts the unneeded vertical pixels (degrees of freedom) into an increased depth of focus resolution. The apparatus provides a view comprising several horizontal “stripes” wherein each stripe represents a planar object and corresponds to an optical element which has both a unique focal length and thus a unique depth of focus region. In a preferred embodiment of the invention a distance evaluation is performed by evaluating the amount of misfocus of each of the various stripes.

[0016] In an alternate preferred embodiment of the invention the distance estimation is performed by employing two cameras, each having similar structure to that of the first embodiment. A triangulation calculation is performed in order to extract the distance of the object seen by both cameras.

[0017] In another aspect of the invention, a diffractive optical element that is attached to the aperture of the system further improves the geometrical resolution of the imaged scene by converting vertical degrees of freedom into horizontal resolution. A system including a camera employing the apparatus according to the invention provides high resolution performance both in the vertical and in the depth axes together enabling determination of the desired distance estimation.

[0018] In yet another aspect of the invention an optimal filtering approach is applied in order to perform the recognition of the high-resolution image.

[0019] The invention provides a method of imaging, which is performed by tilting an image plane with respect to an object plane, defining pixel units on the image plane, and defining a field of view on the object plane, wherein a vertical dimension of the field of view differs from a horizontal dimension thereof. A plurality of images of the field of view are formed on the image plane, the images being spatially interrelated by subpixel shifts. The formed images are then super-resolved into an enhanced image.

[0020] According to an aspect of the invention the step of super-resolving is geometrically performed by transforming the images according to a Gabor transform.

[0021] According to another aspect of the invention the step of super-resolving also has applying a spatial mask to the image plane.

[0022] According to still another aspect of the invention the images are oriented according to the larger of the horizontal dimension and the vertical dimension.

[0023] In a further aspect of the invention there is a further step of deconvolving out-of-focus point spread functions of the images, preferably using a Wiener filter.

[0024] According to an aspect of the invention the images are formed by time multiplexing.

[0025] According to another aspect of the invention the images are formed by wavelength multiplexing.

[0026] According to yet another aspect of the invention the images are formed by diffractive gratings.

[0027] According to still another aspect of the invention the images are transformed according to a Mellin transform, and time-to-impact analysis is performed on the transformed images.

[0028] In a further aspect of the invention the images are transformed such that their impulse responses are diagonally displaced from one another on the image plane with respect to horizontal and vertical axes of the images defined thereon.

[0029] The invention provides a method of imaging which is performed by simultaneously viewing a plurality of elongated fields of view on a target, wherein a focus of each field of view has a different optical displacement from an imaging surface, diffracting beams of radiant energy that travel along a path extending between the fields of view and the imaging surface to form elongated replicas of each elongated field of view, the replicas being mutually shifted in a direction of elongation thereof, and being mutually displaced in a direction that is substantially transverse to the direction of elongation. The method is further performed by combining the replicas of each field of view into corresponding enhanced images, each enhanced image having a higher resolution than the resolutions of its associated replicas.

[0030] Preferably the image plane is tilted with respect to planes of the fields of view, and the method further comprises determining defocus of the fields of view.

[0031] According to another aspect of the invention the method further comprises performing a Mellin transform on an image formed from all of the replicas.

[0032] In a further aspect of the invention the method comprises conducting the beams through a spatial mask that is constructed such that zero values of a Fourier transform of a function representing spatial responsivity of pixels on the image plane are eliminated.

[0033] The invention provides a method of imaging, which is performed by tilting an image plane with respect to first and second object planes intersecting a target, wherein a displacement of the second object plane from the image plane is greater than the displacement of the first object plane from the image plane. The method includes defining pixel units on the image plane, defining a first field of view on the first object plane, and defining a second field of view on the second object plane, wherein vertical dimensions of the first and second fields of view differ from respective horizontal dimensions thereof. The method further comprises forming a plurality of first images of the first field of view on the image plane, the first images being spatially interrelated by subpixel shifts, and simultaneously forming a plurality of second images of the second field of view on the image plane, the second images being spatially interrelated by subpixel shifts. The method further comprises super-resolving the first images into a first enhanced image, and super-resolving the second images into a second enhanced image, and comparing a defocus magnitude of the first field of view with a defocus magnitude of the second field of view.

[0034] Preferably the first field of view is obtained through a first region of a lens, and the second field of view is obtained through a second region of the lens.

[0035] According to an aspect of the invention the step of super-resolving is geometrically performed by transforming the first images and the second images according to a Gabor transform.

[0036] According to another aspect of the invention the step of super-resolving further comprises applying a spatial mask to the image plane.

[0037] According to still another aspect of the invention the plurality of first images and the plurality of the second images are oriented according to the larger of the horizontal dimension and the vertical dimension.

[0038] According to an aspect of the invention the plurality of images are formed by spatially transforming coordinates to establish a panoramic field of view, in which has a horizontal field differs in magnitude from the vertical field of view.

[0039] In a further aspect of the invention the method further comprises determining a displacement between the image plane and the target, responsive to the step of comparing defocus magnitudes.

[0040] In another aspect of the invention the first images and the second images are spaced apart, and a position of the target is determined with respect to the first and second images by triangulation.

[0041] The invention provides an imaging apparatus, including a lens which has a field of view that includes an object plane, wherein a horizontal dimension of the field of view differs from a vertical dimension thereof. A detector of radiant energy has an image formed thereon by the lens, wherein the detector has a plurality of pixel elements, and a spatial mask is disposed proximate the detector. The mask has a plurality of subpixel apertures formed therein. The apparatus further includes a diffractive element disposed proximate the lens, wherein a diffracted light beam forms the image on the detector, and the image comprises a plurality of focused replicas of the field of view, the focused replicas being offset from one another by subpixel shifts, and a prism is disposed between the lens and the detector for refracting radiant energy passing therethrough.

[0042] According to an aspect of the invention the spatial mask is formed so as to eliminate zero values from a Fourier transform of a response function of the pixel elements.

[0043] According to another aspect of the invention the diffractive element is a one dimensional zone plate strip.

[0044] According to yet another aspect of the invention the replicas are mutually shifted in a direction of elongation of the Barfield of view, and are spaced apart from one another in a direction that is substantially transverse to the direction of elongation.

[0045] According to still another aspect of the invention the apparatus further includes a signal processing unit which is responsive to the detector, and is programmed to combine the replicas into a single image which has a greater resolution than the repilicas.

[0046] In a further aspect of the invention the signal processing unit performs a Gabor transform on a signal output of the detector.

[0047] In another aspect of the invention the signal processing unit performs a Mellin transform on a signal output of the detector in conjunction with time-to-impact analysis.

[0048] According to an aspect of the invention there are first and second lenses spaced apart from one another.

[0049] According to another aspect of the invention the signal processing unit is programmed to determine a displacement by triangulation of a target using a first image thereof obtained from the first lens and a second image thereof obtained from the second lens.

[0050] According to still another aspect of the invention an aperture of the lens is divided into a first region and a second region, wherein the signal processing unit is programmed to determine a displacement by triangulation of a target, using a first image thereof obtained from the first region and a second image thereof obtained from the second region of the lens.

[0051] The invention provides an imaging apparatus, including a plurality of lenses each having a corresponding field of view that includes an object plane, wherein a horizontal dimension of the field of view differs from a vertical dimension thereof, the lenses having different focal lengths. The apparatus further includes a detector of radiant energy having a plurality of corresponding images formed thereon by the lenses, a spatial mask disposed proximate the detector, the mask having a plurality of subpixel apertures formed therein. The apparatus further includes a diffractive element disposed proximate each lens, wherein each image on the detector has a plurality of focused replicas of the corresponding field of view, the focus replicas are offset from one another by subpixel shifts, and a prism is disposed between the lens and the detector for refracting radiant energy passing therethrough onto the detector.

[0052] In a further aspect of the invention the prism has a plurality of prisms, each prism refracting radiant energy that is received from a corresponding lens.

[0053] Preferably the spatial mask is formed so as to eliminate zero values from a Fourier transform of a response function of the pixel elements.

[0054] According to an aspect of the invention the diffractive element is a one dimensional zone plate strip.

[0055] According to another aspect of the invention the replicas are mutually shifted in a direction of elongation of the corresponding field of view, and are spaced apart from one another in a direction that is substantially transverse to the direction of elongation.

[0056] According to yet another aspect of the invention the apparatus further includes a signal processing unit which is responsive to the detector, and is programmed to combine the replicas of each the field of view into a corresponding single image which has a greater resolution than the replicas.

[0057] According to another aspect of the invention the lens includes means for spatially transforming the plurality of focused replicas to establish a panoramic field of view, in which has a horizontal field differs in magnitude from the vertical field of view.

[0058] According to still another aspect of the invention the signal processing unit performs a Gabor transform on a signal output of the detector.

[0059] In a further aspect of the invention the signal processing unit performs a Mellin transform on a signal output of the detector.

[0060] Preferably the signal processing unit deconvolves out-of-focus point spread functions of the images.

[0061] According to an aspect of the invention the signal processing unit employs a Wiener filter for deconvolving the point spread functions.

[0062] For a better understanding of these and other objects of the present invention, reference is made to the detailed description of the invention, by way of example, which is to be read in conjunction with the following drawings, wherein:

[0063] FIG. 1 is a schematic illustration of a lens in accordance with some aspects of the invention;

[0064] FIG. 2 is a partially schematic sectional view of an imaging apparatus in accordance with a preferred embodiment of the invention;

[0065] FIG. 3 schematically illustrates a Scheimpflug arrangement in accordance with the prior art;

[0066] FIG. 4 is a schematic view of an imaging apparatus having a plurality of lenses in accordance with an alternate embodiment of the invention;

[0067] FIG. 5 is a schematic view of an imaging apparatus having a plurality of prisms in accordance with an another alternate embodiment of the invention;

[0068] FIG. 6 is a schematic illustration of an optical arrangement for creating a panoramic field of view in accordance with an alternate embodiment of the invention;

[0069] FIG. 7 is a schematic illustration of another optical arrangement for creating a panoramic field of view in accordance with an alternate embodiment of the invention;

[0070] FIG. 8 is a block diagram of a system that includes the embodiment illustrated in FIG. 4;

[0071] FIG. 9 illustrates subpixel shifting of replicas produced by the embodiment illustrated in FIG. 8;

[0072] FIG. 10 illustrates a spatial mask applied to a detector in an imaging apparatus according to a preferred embodiment of the invention;

[0073] FIG. 11 illustrates a bar code suitable for imaging in accordance with the invention;

[0074] FIGS. 12, 13, 14, 15, 16, and 17 are plots of computer simulations that demonstrate the performance of preferred embodiments of the invention;

[0075] FIG. 18 is a sectional view of a stepped prism and detector array in accordance with an alternate embodiment of the invention;

[0076] FIG. 19 schematically illustrates a collision avoidance system in accordance with the some aspects of the invention;

[0077] FIG. 20 is a schematic illustration of an alternate embodiment of the invention employing two lenses; and

[0078] FIG. 21 is a schematic illustration of an alternate embodiment of the invention wherein a single lens is divided into two regions which are separately processed.

[0079] In the following description, numerous specific details are set forth in order to provide a through understanding of the present invention. It will be apparent however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances well known circuits, control logic, and the details of computer program instructions for conventional algorithms and processes have not been shown in detail in order not to unnecessarily obscure the present invention.

[0080] Creation of Wide Depth of Focus

[0081] Turning now to the drawings, and more particularly to FIG. 1 thereof, an optical lens 10 is shown as a component of a preferred embodiment of the invention suitable for distance estimation and vehicular collision avoidance systems. It will be appreciated that the exact longitudinal position of the target object is generally unknown, and accordingly a large depth of focus is required from the capturing device or camera. It is desirable to detect the target detection within a wide working range of distances between the camera and the target object. In the preferred embodiment the vertical dimension of the field of view of the imaging device is smaller than the horizontal dimension. Although the terms “horizontal” and “vertical” are employed in the explanations herein, those skilled in the art will appreciate that these terms can be exchanged as required for a particular application without departing from the spirit of the invention.

[0082] As shown schematically in FIG. 2 a special diffractive optical element 12 is attached to an imaging lens (not shown) of the camera. the optical element 12 is constructed as several strips wherein each strip includes a cylindrical lens 14, 16, 18, 20, 22, each lens having a different focal length, and consequently its own depth of focus region. While cylindrical lenses are employed in the preferred embodiment, the invention can be practiced with non-cylindrical lenses. The optical element 12 is realized using a plurality of one dimensional lenses, or one-dimensional zone plate strips. The focal lengths of the cylindrical lenses of the strips are designed such that, in cooperation with the imaging lens, the combined working ranges include a target 24. The combined working range of the cylindrical lenses 14, 16, 18, 20, 22 not only provides the required depth of focus to image the target 24, but the field of view of each cylindrical lens also has sufficient width to permit the operation of a sub-pixel algorithm to improve resolution. A set of prisms 26 is also disposed between the cylindrical lenses 14, 16, 18, 20, 22 and the detector plane 28 in order to achieve a modified Scheimpflug arrangement and to obtain the images in a proper position on the detector plane.

[0083] The sub-pixel algorithm is preferably the technique disclosed in our copending Israel Application No. 129258, which is hereby incorporated by reference.

[0084] The depth of focus is achieved using several one dimensional (“1-D”) zone plate strips. Each strip corresponds to a different region of space where optimal focusing resolution is obtained. The combination of several such 1-D zone plates provides a wide range of focus depth. A 1-D zone plate can be described by $\begin{array}{cc}g\left(x,y\right)=e^{-\frac{i\pi x^2}{\lambda f}}& \left(1\right)\end{array}$

[0085] where i represents the square root of −1, x is the horizontal coordinate, lambda is the wavelength, and f is the focal length. Note that when two lenses having focal lengths of f1 and f2 respectively are attached to one another, the total focal length becomes:

[0086] 1/ft=1/f1+1/f2  (2)

[0087] Referring again to FIG. 1, one may write the following equation set which apply to the lens 10: $\begin{array}{cc}\begin{array}{c}\frac{D}{y_2}=\frac{w}{a-y_2}\\ \frac{D}{y_1}=\frac{w}{y_1-a}\end{array}& \left(3\right)\end{array}$

[0088] where D is the aperture of the imaging lens 10, F is its focal length and w is the maximal size of the focus spot 30 to be obtained over the detector plane 28. The distance a indicates the position of the detector plane 28.

[0089] Now, using the imaging relations, one may obtain: $\begin{array}{cc}\begin{array}{c}\frac{1}{y_1}+\frac{1}{x_1}=\frac{1}{F}\\ \frac{1}{y_2}+\frac{1}{x_2}=\frac{1}{F}\end{array}& \left(4\right)\end{array}$

[0090] After simple mathematical manipulations one may obtain the expression for the depth of focus region: $\begin{array}{cc}\begin{array}{c}{x}_1=\frac{1}{\frac{1}{F}+\frac{w-D}{\mathrm{Da}}}=\frac{\mathrm{FDa}}{\mathrm{Da}+\mathrm{wF}-\mathrm{DF}}\\ x_2=\frac{1}{\frac{1}{F}-\frac{w+D}{\mathrm{Da}}}=\frac{\mathrm{FDa}}{\mathrm{Da}-\mathrm{wF}-\mathrm{DF}}\end{array}& \left(5\right)\end{array}$

[0091] and $\begin{array}{cc}\begin{array}{c}\Delta =x_2-x_1\\ =\frac{2{\mathrm{awDF}}^2}{a^2{D}^2-2{\mathrm{aD}}^{2}F+D^2{F}^2-w^2{F}^2}\end{array}& \left(6\right)\end{array}$

[0092] According to Scheimpflug, if one tilts the object plane as well as the imaging plane (detector array plane) away from a conventional transverse orientation, a wide depth of focus may be obtained. As disclosed hereinbelow, the arrangement of Scheimpflug may be improved by using a prism arrangement, in which the detector and the lens plane remain parallel. FIG. 3 presents the basic concept introduced by Scheimpflug, in which an object point 32 is captured as an image point 34. The Scheimpflug arrangement 36 may be formulated with the following equations, wherein the quantities a, b, d, and f are distances, and quantities A, B, and C are angles, as indicated in FIG. 3. Line 38 is parallel to the lens axis 40 of the lens 42. Line 44 is parallel to line 46.

[0093] For the image side 48 of the Scheimpflug arrangement 36:

[0094] a(tanB−tanC)=d

[0095] f(tanB+tanA)=d  (7)

[0096] and for the object side 50:

[0097] b(tanA+tanC)=d  (8)

[0098] Thus, $\begin{array}{cc}\begin{array}{c}\frac{1}{a}+\frac{1}{b}=\frac{1}{d}\left[\left(\tan B-\tan C\right)+\left(\tan A+\tan C\right)\right]\\ =\frac{1}{d}\left(\tan B+\tan A\right)\\ =\frac{1}{f}\end{array}& \left(9\right)\end{array}$

[0099] FIG. 4 illustrates a modification of the Scheimpflug arrangement which is used in the preferred embodiment. A prism 52 is inserted between the detector 54 and the lens 56 to create the required phase delay. Here, instead of tilting the image plane, the path difference is obtained using a prism. Thus, if an object 58 is to be placed somewhere on the object plane, it will be focused on the plane of the detector 54. It will be understood that in practice the object has a depth dimension that is perpendicular to the plane of the page, and that the field of view of the lens 56 also has a depth dimension.

[0100] The magnification is: $\begin{array}{cc}M=\frac{b}{a}& \left(10\right)\end{array}$

[0101] and since $\begin{array}{cc}a=\frac{1}{\frac{1}{f}-\frac{1}{b}}& \left(11\right)\end{array}$

[0102] Let b_min, b_n, b_max, a_min, a_n, a_maxrepresent the minimal, nominal and the maximal values for the distances b and a in FIG. 4 respectively, wherein a=a1+a2, and f is the focal length of the lens 56. If L is the size of the detector then given the requirement for the nominal and maximal values of b, one may obtain the following relations: $\begin{array}{cc}\begin{array}{c}{a}_{\max }=\frac{d+L\sin B-a_{\min }\tan A+\frac{a_{\min }d}{b_{\max }}}{\tan B}\\ a_{\min }=\frac{1}{\frac{1}{f}-\frac{1}{b_{\max }}}\\ b_{\min }=\frac{1}{\frac{1}{f}-\frac{1}{a_{\max }}}\end{array}& \left(12\right)\end{array}$

[0103] Knowing the size of the target and the size of the detector's pixels, and calculating the range of the magnification M, one may obtain the required geometrical super-resolution of the improvement.

[0104] Note that a rough estimation for the distance d may be obtained according to: $\begin{array}{cc}d=\frac{{\mathrm{Lb}}_{\min }}{f}& \left(13\right)\end{array}$

[0105] It is important to see that various considerations must be taken into account when implementing the depth of focus configuration based upon the Scheimpflug approach combined with geometrical super-resolution. First, replications are obtained on the array plane of the detector 54 due to a diffractive optical element 60 attached to the lens aperture. As is disclosed in further detail hereinbelow, the diffractive optical element 60 is used to obtain the geometrical super-resolution, and the gratings of this element must be sufficiently close to each other so that all of them will stay on focus on the detector plane, which is effectively tilted with respect to the aperture of the lens. Second, due to the replication of the imaged object, the geometrical super-resolution approach to obtain improved resolution requires a sensitive detector, since an increase in the number of replicas will reduce accordingly the energy of each replica.

[0106] The robustness of the arrangement illustrated in FIG. 4 may be increased by attaching several prisms to the detector array plane, as shown in FIG. 5. In the system 62, there are a plurality of prisms 64, 66, 68 adjacent the plane of the detector array 70 which achieve functional optical tilting of the image plane with respect to the object plane in accordance with the modified Scheimpflug arrangement discussed above. Each prism is a constituent of a different horizontal strip, and each lies in an optical path leading from a different depth of focus region through the lenses 69. In addition it performs vertical replication of the imaged scene 72, the replicas being directed to the locations of each one of the prism stripes. Once again this approach is possible due to the fact that the vertical field of view of the optical system 62 is much narrower than the horizontal one.

[0107] Panoramic Field of View:

[0108] As previously mentioned a high ratio exists between the required horizontal and vertical field of views. The required horizontal field of view is much larger. Since most of the detectors are squared the required ratio may be obtained by using special optical elements. There are several possibilities for obtaining the desired panoramic field of view that is actually a sophisticated coordinate transformation.

[0109] One approach uses the optical setup of FIG. 6 that contains a single optical element, which can be either of the optical elements 74, 76. This element has an impulse response of several spots 78, 80, 82 having diagonal locations on the detector plane 84. When an input image is entered it is replicated as seen in image point 34 (FIG. 3) and a panoramic field of view is obtained. This approach involves only one element. The element is attached to one of the imaging lenses 86, 88 and it consists out of several strips each having a prism that tilts the light horizontally and vertically and resulting in an impulse response that produce spots with diagonal locations. Here the term “diagonal” is referenced to the horizontal and vertical orientations of the field of view. In practice a squared detector array is used to obtain unequal ratio between the horizontal and the vertical fields of view as required. If for instance the impulse response is three diagonal spots, then the ratio between the horizontal and the vertical field of view (FOV) in the panoramic image is 9 (FOV/3 for the vertical dimension and FOV*3 for the horizontal dimension).

[0110] The disadvantage of such an approach is that since each strip is itself narrower than the lens aperture. Thus the vertical resolution of the image is impaired. The resolution reduction is proportional to the number of strips. The main advantage of this approach is that since it contains prisms it is not sensitive to wide wavelengths band illumination.

[0111] An alternate approach also utilizes only one element attached to the lens, which has a similar impulse response as the first approach. However, in this case the element does not contain strips of prisms but it is one unified element, still having a diagonally located impulse response. It may be for instance a Dammann grating, which is known from the publications “High-Efficiency In-Line Multiple Imaging By Means Of Multiple Phase Holograms,” H. Dammann and K. Gortler, Opt. Commun. 3, 312-315 (1971); and “Coherent Optical Generation And Inspection Of Two Dimensional Periodic Structures”, H. Dammann and E. Klotz, Opt. Acta 24, 505-515 (1977). The main advantage of this configuration is that the vertical resolution is unimpaired, since the element covers the entire lens plane and it is not divided into stripes. Its disadvantage is that it may be sensitive to wide band wavelength illumination since it does not consist of prisms. This sensitivity may be reduced if the element is produced with etching depths that are much longer than one wave-length.

[0112] A second alternate approach does not reduce the resolution of the obtained panoramic image and it is not sensitive to wide band illumination. However, it requires two special elements instead of one and it reduces the F number of the optical system. The arrangement is illustrated in FIG. 7. As mentioned it includes a first imaging lens 90, a second imaging lens 92, and two special elements. The first element, a prism 94, is placed in the intermediate image plane and the second element, a prism 96, is attached to the second imaging lens 92. The prism 94 is divided into regions that are to be relocated in the panoramic image that is created on a detector 98. In each region a prism is positioned such that its information is directed into different zones of the second imaging lens 92. Since we allow a reduction of the F number, the area of this lens will allow multiplexing of all the regions. In the lens plane another element is attached. It also consists out of prisms 100, 102 that correct the propagation direction of the spatial information arriving to the zones on the lens aperture. FIG. 7 illustrates an example for the case where the ratio between the horizontal and the vertical field of views is 9. The optical parameters of the arrangement are indicated in FIG. 7. The arrangement allows for sufficient space to perform spatial multiplexing.

[0113] Geometrical Super Resolution

[0114] Resolution of optical systems is restricted by various parameters. For example, optical imaging systems in a free space medium are limited by diffraction. The geometrical resolution of an imaging device depends on the pitch of its pixels and its focal length. This resolution may be improved if the image to be captured is recorded several times, provided that prior to each capture, a well controlled subpixel shift is executed. The resolution improvement relates to the number of captured images undergoing the sub pixel shift. The mathematical operation used for obtaining the super-resolved image out of the sub pixel-shifted images is a Gabor-like transform. A fully detailed description of this issue is provided in the above noted Israel patent application No. 129258; however in the interest of clarity, a brief discussion is now presented.

[0115] According to the preferred embodiment of the invention, the sensing device samples the object K times, each time with a subpixel shift in the captured object relative to the capture device by an amount Δx/K. Letting u_k[n] denoted the detected energy of the n^thpixel in the k^thsampling, the detected energy is given by $\begin{array}{cc}{u}_k\left[n\right]={\int }_{-\infty }^{\infty }u\left(x+n\Delta x+k\frac{\Delta x}{K}\right)g\left(x\right)dx,n=0,1,\dots ,N-1;k=0,1,\dots ,K-1.& \left(14\right)\end{array}$

[0116] Together, the various samples u_k[n] provide KN samples given $\begin{array}{cc}y\left[n\right]={\int }_{-\infty }^{\infty }u\left(x+n\frac{\Delta x}{K}\right)g\left(x\right)dx,n=0,1,\dots ,\mathrm{KN}-1.& \left(6\right)\end{array}$

[0117] This integral may be replaced by a discrete Gabor transform over u with window fu namely, $\begin{array}{cc}y\left[n\right]=\sum _{p}g\left(p\frac{\Delta x}{K}\right)u\left(\left(m-p\right)\frac{\Delta x}{K}\right).& \left(15\right)\end{array}$

[0118] In the preferred embodiment, geometrical super-resolution is achieved by using a special grating-like diffractive optical element. This element, which is attached to the imaging lens, creates replicas by spatial multiplication of the imaged scene on the detector array plane.

[0119] Referring to FIGS. 8 and 9, there is schematically illustrated an aspect of the preferred embodiment, wherein an array of cylindrical lenses 104, 106 having gratings attached thereto are directed to a plurality of corresponding horizontally oriented strips 108, 110 of a scene. When the scene is captured multiple times, each image replica, for example the replicas 112, 114, captured in strip 108, have an appropriate horizontal subpixel shift with respect to their neighbors.

[0120] FIGS. 8 and 9 demonstrate the optical element and its effect on the detector array plane for the approach where the depth of focus is achieved with several strips each having a cylindrical lens with different focal lengths.

[0121] A digital signal processor 116 extracts the high resolution image, applying the Gabor-like operation over all the shifted replicas. For obtaining the desired Gabor-like operation a mask is attached to the sensor plane. Its aim is to avoid zeros in the spectral domain due to the non ideal sampling of information by the pixels of the detector. The practical advantages of applying the mask include the avoidance of division by zero operations, and the addition of frequencies in the spectral domain, which would otherwise not be seen. The details of this approach appear in the above noted Israel patent application No. 129258. Briefly, the mask is constructed in the form of a plurality of optically transparent slits using well known methods. The mask function is a spatial function m(x), preferably taking values 0 and 1, and its effect is to modify the pixel sensitivity function from g(x) to p(x)=g(x)m(x). $\begin{array}{cc}m\left(x\right)=1-\sum _{i=1}^M\mathrm{rect}\left(\frac{x}{\delta x_i}\right)\otimes \delta \left(x-x_i\right),& \left(16\right)\end{array}$

[0122] where M is the total number of slits, X₁, X₂, . . . , X_M, are the locations of the slits, δx_iis the width of the i^thslit, and {circle around (x)} denotes convolution. The function rect() is given by $\begin{array}{cc}\mathrm{rect}\left(z\right)=\{\begin{array}{cc}1,& -1/2<z<1/2,\\ 0,& \mathrm{otherwise}\end{array}& \left(17\right)\end{array}$

[0123] Preferably, the value of M is a number between zero and the number of replicas, K. The values δx are each approximately equal to the sub-pixel size Δx/K, where Δx is the width of a pixel. The values of the mask m(x) are 0 inside the slits and 1 outside the slits, The total width of the mask is Δx. Although the spectral mask has been described in terms of a single dimension for charity of presentation, preferably the mask is a two-dimensional mask which covers a two dimensional array of detectors.

[0124] FIG. 10 is a simplified illustration of a mask attached to a sensor panel of array detectors in an image sensing device, in accordance with a preferred embodiment of the present invention. A sensor panel 118 in an image sensing device contains a two-dimensional array of detectors 120. Detectors 120 are depicted as individual cells within sensor panel 118. The array of detectors 120 is periodic, with periodic spacings Δx and Δy in the horizontal and vertical dimensions, respectively. The spacings Δx and Δy correspond to the pixel spacings in the sampled digital image obtained from the image sensing device. A masking pattern 122 in the form of a fine sub-pixel transmission grating is used to generate a doubly periodic mask, with respective periods Δx and Δy in the horizontal and vertical dimensions. In a preferred embodiment of the present invention, the doubly periodic mask is attached to sensor panel 118. Each of the detectors 120 is masked by the same masking pattern 122. It is possible in some applications to vary the focal power of the lenses such that the replicas will be properly focused in the image plane, utilizing the Scheimpflug arrangement 36 (FIG. 3) or the variants thereof which are disclosed hereinabove. If this is done, it is not necessary to use the digital signal processor 116.

[0125] Computer simulation:

[0126] The effectiveness of the preferred embodiment disclosed above is shown by a computer simulation. The software has taken into account the diffraction resolution, as limited by the F number of the imaging lens, the geometrical resolution, as a function of the size of the detector's pixels and the defocus problem created in the replicas due to the fact the detector plane is tilted with respect to the object plane, as seen schematically in FIGS. 4 and 5. In order to solve the defocus problem of the various replicas, the applied numerical algorithm modeled not only the geometrical super resolution technique discussed above, but also computed the deconvolution of the out-of-focus point spread functions of the various replicas. The deconvolution was performed in the form of Wiener filtering in the Fourier plane. Since the defocusing distortion may be computed according to the formula: $\begin{array}{cc}H\left(f_x\right)=\mathrm{trian}\left(f_x\right)\mathrm{sinc}\left[\left(\frac{8w}{\lambda }\frac{f_x}{2f_0}\right)\left(1-\left|\frac{f_x}{2f_0}\right|\right)\right]& \left(18\right)\end{array}$

[0127] to where: $\begin{array}{cc}\begin{array}{c}{f}_0=\frac{F}{2\lambda {\mathrm{bF}}_{\#}}\\ w=e\frac{F^2}{8F_{\#}^2}\\ e=\frac{1}{b}+\frac{1}{a+\delta s}-\frac{1}{F}\end{array}& \left(19\right)\end{array}$

[0128] The correction filter has an inverse form, wherein F_#is the F number, F is the focal length of the imaging lens and δs is the amount of defocus obtained on the detector plane. Notating by dx the size of each pixel on the detector, and assuming that the vertical distance between each two replicas is 5 pixels, then:

[0129] δs=5dx cos(B)  (20)

[0130] In the following examples bar code imaging was simulated, using a resolution improvement of 4 times. In the simulation the camera had 512 pixels with size of 12 microns. A=20 degrees, F=20 mm. , F₁₉₀=2 and the wavelength was 0.68 microns. FIG. 11 demonstrates the input barcode 124, a 15 mil barcode with d=55 mm. FIG. 12 shows a plot of the captured image, without application of the super-resolving algorithm. FIG. 13 illustrates conventional reconstruction, in which a threshold is applied to the defocused image obtained in the specific plane. FIG. 14 shows the result obtained after applying the algorithm combining geometrical-super resolution together with the Wiener defocusing correction. Those simulations were obtained at a range of b=1.6 m. One may see that the combined algorithm manages to cope with all the possible resolution distortions, including diffraction, geometrical distortion, and defocus.

[0131] The algorithm can easily cope with close distance where super-resolution is much less required. For instance, in FIG. 15, where the simulation was performed under the condition that the range b=35 cm. and d=55 mm., a successful reconstruction may be seen.

[0132] Decreasing the distance d decreases the angle B and make the defocus problem more severe. In FIG. 16 one may see the obtained results for b=1.2 m and d=40 mm. For comparison FIG. 17 presents the results obtained without applying the super-resolving algorithm.

[0133] In the above noted simulations, the out of focus problem of each replica is solved by the use of a numerical algorithm, the Wiener defocusing correction. In an alternate embodiment of the invention, instead of employing the numerical algorithm, proper focus for the replicas is obtained by designing the diffractive optical element such that a small optical power is added to each replica created therein. In this embodiment all the elements together create the desired depth of focus range. In this manner all the replicas are obtained in proper focus.

[0134] Referring now to FIG. 18, in yet another alternate embodiment of the invention stairs 126 are formed in a prism 128 attached to detector array 130, so that all the replicas will have optical paths of equivalent length, and will be properly focused on the detector plane.

[0135] The technique according to the invention may be used in various applications such as vehicle detection and warning system and barcode scanners. The detector preferably contains an imaging device, such as a CCD array, which images desired horizontal and vertical fields of view. A high horizontal spatial resolution is extracted using a geometrical super resolution concept obtained using a special diffractive optical element which is attached to the lens and creates a replica of the imaged horizontal axis. In addition, as described in the above noted Israel patent application No. 129258, a special mask is attached to the sensor to prevent spectral zeros. Due to this mask, an efficient inverse filtering algorithm (Gabor like) is possible to be applied for extracting information at a high spatial resolution. A wide depth of focus is obtained using a set of strips with varied focal lengths or by a prism placed between the imaging plane and the imaging lens according to the modified Scheimpflug arrangement. In addition, instead of using a single prism, several prismatic strips may be used to obtain several traces with different focus depths. In this embodiment the special optical element attached to the imaging lens is also performs the vertical replication of a portion of the imaged scene that corresponds with the respective fields of view of each of the strips. This arrangement increases the system's robustness.

[0136] FIG. 19 illustrates an application of the imaging apparatus according to the invention in a collision avoidance system for motor vehicles. A vehicle 132, in which a collision avoidance system 134 is installed, is trailing a preceding vehicle 136 at a distance u. The collision avoidance system 134 employs one of the preferred embodiments of the invention as discussed above, having either several cylindrical lenses or several prisms. Each imaged strip 138, 140, 142, 144 presents information focused at a different depth. Thus, for small values of the distance u, focused information appears only in one of the stripes. The rear surface of the preceding vehicle 136 is properly focused by a particular strip 142. By identifying the strip in which information is focused, for instance by spatially deriving the information, one may approximate the distance u. Where v is the distance between the lens and the detector plane, one obtains: $\begin{array}{cc}\frac{1}{u}+\frac{1}{v}=\frac{1}{F}& \left(21\right)\end{array}$

[0137] and $\begin{array}{cc}\delta v=-\frac{v^2}{u^2}\delta u& \left(22\right)\end{array}$

[0138] where F is the focal length. Δu and Δv are the variation in the distances. The extent of defocusing (variation in v) correlates with the distance estimation resolution using this approach (the variation in u).

[0139] The collision avoidance system 134 includes a computer 146, preferably employing a digital signal processor 148. The digital signal processor 148 receives signals from detector 150, responsive to the optical energy impinging thereon via the optics 152. The computer 146 is programmed by known techniques to respond to the output of the digital signal processor 148, and to compute the distance u. Responsive to the distance u, or to changes thereof, signals are transmitted via bus 154 from the computer 146 to a controller 156. If appropriate conditions of the distance u are fulfilled, the controller 156 may alert the operator of the vehicle 132 by activating an alarm 158, or in more sophisticated systems, by automatically actuating a control 160 of the vehicle 132 to vary its speed or direction.

[0140] In another alternate embodiment of the invention, the distance from the target may be extracted using two cameras and the well known triangulation relation, as shown in FIG. 20. Optical elements 162 are each constructed according to one of the embodiments illustrated in FIG. 5 and FIG. 8. $\begin{array}{cc}\begin{array}{c}\frac{x_1}{x}=\frac{F_1}{H-\Delta H}\\ \frac{x_2}{L-x}=-\frac{F_2}{H-\Delta H}\\ H-\Delta H=\frac{x\left(F_1-F_2\right)+F_{2}L}{x_1-x_2}\end{array}& \left(23\right)\end{array}$

[0141] Assuming F₁=F₂=F one obtains: $\begin{array}{cc}\delta \left(H-\Delta H\right)=\left(\delta x_2-\delta x_1\right)\frac{{\left(H-\Delta H\right)}^2}{\mathrm{LF}}& \left(24\right)\end{array}$

[0142] where δ(H−ΔH) is the depth resolution and δx_{1, 2}is the spatial position detection resolution on the detector plane. For instance, let us assume that the detector has pixels whose size is Δx=12 μm and the size of the detector plane is D=1.28 cm. If we wish to obtain a vertical field of view (“FOV”) of 86 degrees (1.5 rad) it requires a focal length of: $\begin{array}{cc}F=\frac{D}{\mathrm{FOV}}=\frac{1.28\left[\mathrm{cm}\right]}{1.5\left[\mathrm{rad}\right]}=8.54\left[\mathrm{mm}\right]& \left(25\right)\end{array}$

[0143] In this example, a resolution improvement of N=5 times is obtained, and it is also assumed that after applying a center of gravity finding algorithm, the position resolution is improved by another factor of 10: $\begin{array}{cc}\delta x=\frac{\Delta x/N}{10}=\frac{12\mathrm{\mu m}/5}{10}=0.25\mathrm{\mu m}& \left(26\right)\end{array}$

[0144] Thus

[0145] δx₂−δx₁≦2×0.25 μm=0.5 μm  (27)

[0146] In yet another alternate embodiment of the invention triangulation may be obtained be accomplished by dividing the aperture plane of a single optical element. As shown in FIG. f17 an optical elements 164 is constructed according to one of the embodiments illustrated in FIG. 5 and FIG. 8, but is divided into two identical halves 166. The triangulation calculation is performed between the images captured on the two halves of the detector array panel 168.

[0147] The distance between the two images is half the detector plane size: L=D/2=6.4 mm. For targets located 25 meters from the camera the distance estimation resolution becomes: $\begin{array}{cc}\delta \left(H-\Delta H\right)=\frac{0.5\mathrm{\mu m}\times {\left(25m\right)}^2}{6.4\mathrm{mm}\times 8.54\mathrm{mm}}=5.5m& \left(28\right)\end{array}$

[0148] A decrease in the fields of view will improve the depth estimation resolution.

[0149] The solution presented in FIG. 21 may be combined with the Scheimpflug based concept disclosed above with respect to the first embodiment.

[0150] Computer simulations have been performed as discussed above. The simulations have taken into account the aperture's size and the diffraction limitations of propagating light, the geometrical limitations of the detector due to the finite size of its pixels, the replicas obtained on the detector plane, which are required for the geometrical super resolution approach, and the defocus effects resulting from the fact that different optical paths of the replicas lead to different areas of the tilted detector plane. The simulations have verified the suggested concept.

[0151] Another important issue to be discussed is related to computer vision algorithms which may be combined with optical data processing for extraction of the desired distance estimation. If organ optical Mellin coordinate transformation of (lnr, θ) are applied over the Cartesian input image, and a correlation operation is used for estimation, the obtained result yields scale and rotation invariant recognition. Such a property may be well used for extracting the TTI (time to impact) parameter. The suggested coordinate transformation may be implemented optically using the special optical elements disclosed herein. The transformations and computer vision algorithms are more fully described in the documents “Shift and Scale Invariant Pattern Recognition using Mellin Radial Harmonics,” D. Mendlovic, E. Marom and N. Konforti, Opt. Commun. 67, 172-177 (1988); and “Optical Coordinate Transformation Methods And Optical Interconnection Architectures,” D. Mendlovic and H. M. Ozaktas, Appl. Opt. 32, 5119-5124 (1993).

[0152] It is also possible to achieve the effect of diffraction to create replicas by techniques unrelated to diffraction—time multiplexing and wavelength multiplexing, in which case the imaging lens is made to vibrate on a platform. As previously mentioned, in order to obtain geometric super resolution a relative subpixel shift is required between the images. As disclosed above this shift was achieved by spatial multiplexing, that is by creation of multiple replicas at the expense of vertical pixels in the detector array. Instead one may, for instance use time multiplexing to obtain the set of images which are relatively shifted from one another by subpixel amounts. For example, if the camera is placed on a vibrating platform the relative shifts may be obtained due to the vibrations and, the set of the required images is accumulated with time. Another option is to encode each shifted image in a different wavelength and then to obtain all the images simultaneously in time and space. In this case it is the detector's responsibility to be able to distinguish between various wavelengths used for the multiplexing.

[0153] The above embodiments have been explained in terms of light energy. However the techniques disclosed herein are applicable with suitable modification to other forms of radiant energy, such as ultraviolet light, infrared, microwave energy, x-rays and radioactive emissions.

[0154] While this invention has been explained with reference to the structure disclosed herein, it is not confined to the details set forth and this application is intended to cover any modifications and changes as may come within the scope of the following claims.