Spatially Multiplexed Optical Signal Processor

The present invention relates to the field of electro-optical signal processors. The inherent parallelism associated with optical signal processing techniques allows such systems to process a large image just as rapidly as a small one; whereas, for digital methods, the processing time tends to increase exponentially with image size. For example, a single liquid crystal television can be driven at 60 frames per second. Thus, 60 2D Fourier transforms (Fraunhofer diffraction patterns) could be formed per second. This is much faster than current state-of-the-art digital techniques. Moreover, other spatial light modulators such as ferro-electric devices, which are more expensive, can be driven at even faster rates of 2,000 frames per second.

Current optical signal processors, using spatial light modulators as programmable transparencies, process a single image at a time. Since very high resolution spatial light modulators are commercially available, often the images to be processed do not require the full TV screen. Hence the "single image" optical systems discussed above don't take full advantage of the inherent parallelism associated with optical processing signal processing. It is thus desirable to partition the TV screen or other SLM into separate regions so that several Fourier transforms can be optically computed in parallel to significantly increase the throughput rates for many optical signal processing applications.

Using lenslet arrays has been suggested for spatial multiplexing. However, using lenslet arrays suffer from serious disadvantages. Most importantly, for high resolution spatial light modulators, small diameter lenses must be used. Such small lenslets tend to be poor in quality and consequently have aberrations that significantly degrade the diffraction pattern. Moreover, even if perfect small lenslets could be manufactured, their small diameter, relative to the partitioned region to be optically processed, also causes serious degradation in the resulting diffraction patterns. In order to avoid this problem, the lenslet diameter would need to be several times larger than the partitioned region; but this severely limits the number of regions that can be partitioned and thus reduces the number of Fourier Transforms that can be computed in parallel. Secondly, each of the lenslets must be aligned and spaced very accurately to within a few microns. No such lenslet arrays exist and consequently, haven't been used to perform spatial multiplexing. Thirdly, lenslet arrays are very expensive. The present invention does not suffer from any of these drawbacks.

In accordance with the invention, a two-dimensional diffraction grating is illuminated by a laser and a focussing lens converts the interference pattern produced by the grating into a phased array of point sources, each illuminating an associated sub-region on an image display device such as a spatial light modulator (SLM) or a liquid crystal TV (LCTV).

The processor of the present invention can be used to measure the degree of correlation of several pairs of input and reference images simultaneously in a single SLM which offers advantages in speed, cost, size and power consumption relative to existing digital methods.

FIG. 1 discloses the spatially multiplex optical signal processor being employed as a joint transform correlator;and

FIGS. 2 and 3 illustrate aspects of an exemplary use of the invention.

In FIG. 1, a laser 101 is used as a coherent illumination source. The laser beam 103 is expanded by a beam expander 105, and the resulting expanded beam 107 is used to illuminate a two-dimensional diffraction grating or cross grating 109. The diffraction grating 109 is used to generate, for example, a five by five rectangular two-dimensional array of twenty-five point sources 115 (five such point sources s.sub.1, s.sub.2 . . . s.sub.5 are shown in FIG. 1). This diffraction grating is a conventional grating, in an optical sense, as defined in Jenkins and White's well known text, "Fundamentals of Optics", McGraw Hill, Inc., New York, 1950, on page 320. Such a grating has parallel, equidistant slits of the same width, as is well understood by skilled workers in the art. The interference pattern produced by the grating is directed at focussing lens 111 which focusses the bright regions of the interference pattern via path 113 to form the phased array point sources 115. The diffraction grating 109 can be implemented in several ways. If the spacing of the point sources never changes, then a fixed grating such as a Ronchi ruling, or Damann grating can be conveniently used. However, if a variable diffraction grating is desired, a spatial light modulator could be used as a real-time programmable diffraction grating controlled by processor 127. For example, a rectangular grating g(x,y) can be mathematically expressed as ##EQU1## where a(x,y) is the effective aperture of the grating; b.sub.x and b.sub.y are the dimensions of the sampling element and g.sub.x and g.sub.y are the sampling interval of the grating in the x and y directions, respectively. Each of the bright spots generated by the diffraction grating expressed by equation (1) consists of a narrow ##EQU2## function (due to the relatively large aperture function a(x,y)). The bright spots are separated from each other by ##EQU3## in the x and y directions respectively, where Z.sub.g is the distance from the exit pupil (the grating g(x,y) is the aperture stop) to the plane containing the array of point sources and A(.xi.,.eta.) is the Fourier Transform of the aperture function a(x,y).

Using either the fixed or variable diffraction grating has several advantages over using a lenslet array or using an array of laser diodes as point sources. As discussed in the background section, the diameter of each lenslet cannot be neglected if it is approximately the same size as the multiplexed sub-regions. Using lenslets wherein the diameter is about the same size as the multiplexed sub-regions severely degrades the correlator systems performance. In addition, alignment tolerances required in an optical correlation system tend to preclude lenslets from being used. Using either the fixed or variable diffraction grating also has several advantages over using an array of laser diodes as point sources. First, the spacing of the point sources using a diffraction grating is more accurate. Second, a diffraction grating is inexpensive. Third, the diffraction gratings can be custom designed to control the intensity of each individual point source. Fourth, and perhaps most importantly, the point sources produced by a diffraction grating only radiate in a fixed solid angle dictated by the size of the aperture on the diffraction grating 109. Consequently, the point sources produced by the diffraction grating will only illuminate an associated sub-region or sub-area of the spatial light modulator 117 (t1, t2 . . . t5 shown in FIG. 1). If an array of laser diodes were used, each diode would radiate equally in all directions and specially designed lenses would have to be designed to overcome this.

The optional use of a programmable diffraction grating at 109 enables the flexibility to change the point source spacing, intensity, and angle of illuminatating in real-time thus allowing it to be used for a wider variety of applications. A particularly novel method for using a liquid crystal TV as a programmable point source generator was used in the protype joint transform correlator system to track blood flow and tissue motion in ultrasound images. This allows the LCTV to become a sinusoidal grating coupled with a variable aperture, whereby the grating may be dynamically programmed to adjust both the point source spacing and individual point source intensities in real time if desired. For further details regarding a suitable prior art dynamic programming technique for an SLM, see "Dynamic Holographic Interconnects that use Ferro-electric Liquid-crystal Spatial Light Modulators", Applied Optics, vol. 33, No. 14, May 1994; D.C. O'Brien et al. In addition, this technique allows the aperture of the gratings to be varied. The grating aperture is that portion of the grating that is transmitting light. Varying the aperture size in turn controls the sub-regions or sub-areas each point source illuminates on SLM 117.

Lens 121 is the Fourier transform lens. It produces an array of images, (T.sub.1, T.sub.2, . . . T.sub.5 in FIG. 1) that correspond to 2-dimensional Fourier Transforms 123 of each individual sub-region on the spatial light modulator 117; t.sub.1, t.sub.2, . . . t.sub.5 in FIG. 1.

The spatially muliplexed optical signal processor shown in FIG. 1 was incorporated into a joint transform correlator (JTC) system. A conventional JTC system is well known to those skilled in the art and is fully described in U.S. Pat. No. 5,119,443, entitled SINGLE SLM JOINT TRANSFORM CORRELATOR, and incorporated by reference herein. The irradiance pattern (magnitude squared) of the aforesaid Fourier transforms are recorded by the CCD camera 125. They can be recorded for future use or can, if desired, be promptly digitized by processor 127 and sent to the SLM 117 for a second pass through the system, and the inverse Fourier transforms of the fresh signals now in the SLM may be recorded by the CCD camera to determine the degree of correlation between the input and reference images of each image pair being displayed in each of the aforesaid sub-areas of the SLM 117 which thus may be termed an image display means. This process is described in the aforesaid patent. The JTC constructed in accordance with the invention, used an Epson LCTV for the SLM 117 and a 1000 line/inch two-dimensional cross grating was generated therein. Focussing lens 111 was arbitrarily chosen having an approximate 100 mm focal length and was positioned to intercept the interference pattern produced by the grating and to produce the rectangular array of point sources, separated by 5 mm from each other. The aperture on the grating was then adjusted to ensure that the resulting illuminated areas didn't overlap. The irradiance or light intensity illuminating each sub-area of the LCTV was equalized by using small pieces of film to appropriately attenuate the individual illuminated regions. The array of point sources were imaged by the second Fourier transform lens 121 upon the CCD camera. A 2.times.2 image spatially multiplexed JTC prototype was built. The JTC system was used experimentally to track blood flow and tissue motion in ultrasound speckle images. The size of the subregions of the input LCTV depends upon the size of the kernel and search regions k and s respectively, shown in FIG. 3, required for accurate tracking. The kernel regions were rectangular with dimensions w.sub.k.sup.x =15 and w.sub.k.sup.y =17 pixels. The size of the kernels used were the same as that used by Duke University to measure the axial tracking accuracy of their digital system. The larger search region was also rectangular with dimensions w.sub.s.sup.x =17 and w.sub.s.sup.y =22 pixels. To illustrate the spatial multiplexing technique, four subregions, denoted t.sub.00, t.sub.01, t.sub.10, t.sub.11, are illustrated in FIG. 2. The separation between the kernel and search region, denoted by d in FIG. 3, should be sufficient to ensure that the desired cross-correlations are formed.

The region surrounding the kernel and search regions, can be chosen to add a little dead space between subregions to avoid aliasing between the illumination sources.

A LCTV was used as a programmable sinusoid diffraction grating to generate equally bright point sources. The aperture on the sinusoidal grating was programmed to ensure only 64.times.64 pixel (approximately 5.times.5 mm) regions on the spatial light modulator 117 (which was also a LCTV). The kernel and search regions were separated by 32 pixels.

The choice of the second lens 121, having a focal length f.sub.2, is more critical than the choice of lens 111 having a focal length of f.sub.1. The array of point sources 115 were imaged by the second lens 121 onto the CCD camera 125. The active area of the CCD camera used in the spatially multiplexed system was approximately 1.times.1 cm which corresponded to size of the four point sources used to illuminate the Epson LCTV. The point sources were placed a distance 2f.sub.2 in front of the second lens (Z.sub.35 =2f.sub.2). Moreover, the resulting Fraunhofer diffraction pattern form each subregion of the input LCTV must be large enough for the CCD camera to resolve the fringe pattern. In addition, if methods to remove the repeated orders are not used, the scale Z.sub.45 must be chosen to ensure the first order terms do not interfere with any of the zero order diffraction patterns. Consequently, Z.sub.45 <f.sub.2. A 100 mm focal length lens was used to do this. The LCTV was positioned against the lens. The scale Z.sub.45 was chosen to produce first order terms which were space 3/4 the distance between the image of the point sources formed at Z.sub.5.

Thus it should now be appreciated that this invention allows the inherent parallelism associated with optical signal processing systems to be optimized. The increased throughput is accomplished by using a diffraction grating to generate a phased array of point sources which in turn are used to illuminate individual regions of an image display device. The remaining components of this invention ensure accurate and appropriately scaled diffraction patterns are formed.

Since this invention can be used to optically compute tens of thousands of 2D Fourier transforms per second it can be used in a wide variety of real-time pattern recognition and tracking applications with an accuracy comparable to digital methods (which are much slower). In addition, this invention offers advantages in cost, size, and power consumption over other techniques.

Since other embodiments of the invention will occur to the skilled worker in the art, the scope of the invention is to be measured by the terms in the claims and art recognized equivalents thereof.