US 5,068,801 AGrant
Optical Interconnector and Highly Interconnected, Learning Neural Network Incorporating Optical Interconnector Therein
Issue Date:1991-11-26
•23 Claims
•4 Drawing Sheets
Abstract
A variable weight optical interconnector is disclosed to include a projecting device and an interconnection weighting device remote from the projecting device. The projecting device projects a distribution of interconnecting light beams when illuminated by a spatially-modulated light pattern. The weighting device includes a photosensitive screen provided in optical alignment with the projecting device to independently control the intensity of each projected interconnecting beam to thereby assign an interconnection weight to each such beam. Further in accordance with the present invention, a highly-interconnected optical neural network having learning capability is disclosed as including a spatial light modulator, a detecting device, an interconnector according to the present invention, and a device responsive to detection signals generated by the detecting device to modify the interconnection weights assigned by the photosensitive screen of the interconnector.
Metadata
Assignee
- Teledyne Industries, Inc.
Inventors
- Rodney L. Clark
- Charles F. Hester
Application Information
Application Number:US 4318994
Filing Date:1989-11-06
Priority Date:1989-11-06
Art Unit:238
Classifications
IPC:
G06E 300
Field of Search:
364350365513;807;841;845;200;8223.78;96.14;162 SF99
Patent Drawings (4 sheets)
Description
Background of the Invention
The present invention pertains to an optical interconnector and an optical learning system or network incorporating the interconnector therein. More particularly, the present invention pertains to an optical interconnector including an interconnection beam projection means which projects a distribution of light beams to interconnect a detecting means with an input means, and a weighting means, located in an optical path between the projection means and either the input or the detecting means, which controls a parameter of each of the interconnecting beams to set the interconnection strength or weight thereof. The optical learning system according to the invention incorporates the interconnector to provide a highly interconnected neural network having learning capability.
The present invention relates to an optical interconnector and an optical learning system incorporating such an interconnector to provide what has become known in the art as a "learning neural network". By way of background, generally a neural network comprises devices that simulate the responses of biological neurons. A simplistic model for a neuron N.sub.0 is shown in FIG. 1 to receive three inputs X.sub.1, X.sub.2, and X.sub.3 at a device which sums the inputs according to the simple equation S=X.sub.1 +X.sub.2 +X.sub.3. Positive X's may be defined as excitatory and tend to make the model neuron "fire", that is provide a nonzero output. Negative X's, defined as inhibitory, tend to prevent the model neuron from firing. A nonlinear operator changes an output signal from the output S to the summing device into a new signal according to a nonlinear, threshold response curve. A low input signal to the nonlinear operator, that is a signal below some threshold, S.sub.0, results in a zero output at the nonlinear operator. A high input signal gives a fixed axial output. An intermediate input results in an intermediate output.
Output S' from the nonlinear operator is applied to still other neurons after multiplication by a weighting factor W by a distributor. All signals W.sub.1 S', W.sub.2 S', and W.sub.3 S' are proportional to S'. The weighting factor W controls the strength or weight of the connections between neuron N.sub.0 and the summing elements of the three other neurons N.sub.1, N.sub.2, and N.sub.3 shown in FIG. 1. If, for example, the weighting factor W.sub.1 is small, the interconnection between neurons N.sub.0 and N.sub.1 is said to be weak and thus the signal S.sub.1 from neuron N.sub.0 to neuron N.sub.1 is attenuated. Conversely, if the weighting factor is large the signal S.sub.1 is amplified and the interconnection between the neurons is said to be strong. It is seen that the information, memory, and problem solving methods characteristic of a neural network are determined by the interconnections in network, that is what is interconnected to what and with what strength.
Turning to the prior art, one recognizes U.S. Pat. No. 4,660,166 to Hopfield as disclosing an earlier network which electronically simulates neural activity to provide a system capable of retrieving particular information from a system memory in response to an interrogation thereof. The patentee describes such a retrieval system as an associative memory, that is a memory that provides an output which is particularly associated with a particular input applied to the system. In the Hopfield device an interconnected network of electronic amplifiers provides the "neurons".
To provide a neural network with capability to learn, the interconnection strengths between the "neurons" in the network must be modifiable. The neurons must be modifiable to provide a desired output when the network is presented with an input. To provide modifiable neurons in an electrical network greatly complicates the overall electronics. Moreover, electronic implementations appear to be inherently limited in the number of interconnections that can be made without regard to how difficult it is to change the strengths of the interconnections. It is believed unlikely that an electronic circuit providing for more that about 1,000,000 i.e. 1.times.10.sup.6 interconnections is feasible as cross-talk problems and problems with power requirements become overwhelming. In view of the shortcomings of electronic implementations, optical learning networks have been developed. Such a network is described in an article by K. Wagner and S. Psaltis which discloses a learning network utilizing a volume hologram comprising photorefractive crystals to interconnect nonlinear optical devices known in the art as Fabry Perot etalons. The interconnections are both made and weighted by the interference patterns in the photorefractive crystal. Learning commences with the presentation of an optical input to the network. The network initially will display an output which greatly differs from the desired output. To correct the variance between the actual and desired outputs, i.e. to induce the network to "learn", error signals are determined by taking the difference between the actual and desired outputs. The error signals are sent backwards through the photorefractive material as light rays to modify the interference patterns and thus the weight of the interconnections made by the hologram. Such modification is executed continuously until the generated output matches or very nearly matches the desired output whereafter the error signals are no longer permitted to propagate through the network.
The interconnection capacity of photorefractive material holograms also is inherently limited. The density of the interference patterns in the photorefractive material increases in proportion to the number of interconnections made by the hologram. As this density is increased, it becomes increasingly difficult to modify the weight of some interconnections without undesirably changing the strengths of other interconnections. These cross-talk problems in the interconnection capacity of photorefractive interconnecting holograms thereby inherently limit the number of modifiable interconnections which can be made in a neural learning network which incorporates such.
Summary of the Invention
The optical interconnector according to the present invention overcomes limitations inherent in prior art systems by providing a first element dedicated to making optical interconnections and a separate, second element dedicated to assigning weights to the interconnections made by the first element. The second element can be controlled to change the assigned interconnection weights or strengths during operation of the interconnector to provide learning capacity in an optical neural network incorporating such an interconnector. The active area of either the first or the second elements of the interconnector in accordance with the present invention is not limited by crystal size or uniformity. Accordingly, the interconnector overcomes unavoidable limitations in interconnection capacity in for example photoreactive crystal, volume hologram interconnectors.
The "first element" in the optical interconnector in accordance with the present invention comprises a projection means that projects a distribution of interconnecting light beams. In the preferred embodiments, the projecting means is provided by a holographic means comprising an array of spatially-localized holograms. The "second element", located remotely of the projecting means, amplitude modulates each interconnection beam to encode the strength of each interconnection made by the beams. In the preferred embodiments, this "second element" comprises a photosensitive screen. The photosensitive screen controls the intensity of each interconnecting beam and may be completely transparent, partially transparent or substantially opaque to any particular interconnecting beam. In the preferred embodiments, the screen could comprise a photochromic, or a photodichromic material, a photodichroic material, or a material having both photodichroic and photochromic properties.
In a preferred embodiment of the interconnector where the photosensitive screen comprises a photochromic material, the screen will change its absorption spectrum in response to exposure by light of a certain threshold intensity. The screen is imaginarily divided into partitions or cells in one-to-one correspondence with the holograms of the holographic means. Each cell or partition can be exposed to light having the requisite threshold intensity to locally change its absorption spectrum to make such partition opaque, without affecting the optical transitivity of adjacent partitions. The screen partitions become progressively less transmissive with continued exposure to light of the threshold intensity. On the other hand, light having the same wavelength, but an intensity below the threshold can pass through the screen without appreciably affecting the transitivity of the screen.
In an alternative embodiment of the interconnector, the photosensitive screen is formed of a photodichroic material. Modification of the photodichroic screen can be carried out by several known techniques including direct writing or suppressive writing, and direct or suppressive extinction writing.
In use, the interconnector in accordance with the present invention is positioned to optically interconnect an input device, such as a spatial light modulator and an output device such as a photodetector. The pbotosensitive screen can be positioned in the optical path between the holographic means and the input device or between the holographic means and the output device. When illuminated by an input spatial pattern from an input spatial modulator, the holographic means projects interconnection beams onto the output photodetector to fully interconnect the input and output devices. The projected interconnecting beams are modulated according to the input spatial pattern. The holographic means only switches the direction of the modulated light from the input device and does not change the relative intensity of the switched light. The photosensitive screen however does amplitude modulate the interconnecting beams to change their relative intensities and thereby encode their interconnection strengths.
The disclosed optical, learning neural network which includes an interconnection apparatus according to the present invention has an interconnection capacity attaining 10.sup.1 2 interconnections. The preferred learning network utilizes a page-oriented holographic means comprising an N.sup.2 .times.N.sup.2 array of spatially localized holograms arranged on a single holographic plate or substrate. Each hologram projects one interconnecting beam so that the entire array provides for up to N.sup.4 interconnections. The number N is contemplated to be very large and to equal or exceed 10.sup.3.
In the preferred network according to the present invention, a transmissive spatial light modulator (TSLM), defined to have N.sup.2 pixels, modulates laser light in a particular way at each pixel thereof to provide a modulated input or spatial pattern of N.sup.2 light beams to illuminate the holographic means. The beam modulated by each pixel of the TSLM illuminates N.sup.2 holograms of the holographic means. Each such set of N.sup.2 uniformly illuminated holograms divides the illuminating beam into N.sup.2 interconnecting beams and projects one interconnecting beam to each of the corresponding N.sup.2 pixels defined on the photodetector output device to fully interconnect the input and output devices. The photosensitive screen is disposed in the optical path between either the input TSLM and the holographic means or the holographic means and the photodetector output to set the strengths of the interconnections between the input and output devices.
A modifying device controls the transitivity of the screen. In the preferred embodiments, the modifying device comprises a computer and an optical modifying means. The computer receives detection signals from the photodetector and generates an error signal based upon the detection signals, the input data injected into the network by the TSLM, and signals representative of a desired output pattern. The error signals are applied to the optical modifying means which amplifies the constituent interconnecting beams in the output light pattern which are responsible for generating the error signals. The amplified light is directed back through the network onto the screen, to expose the screen and thereby change the absorption spectrum of selected cells. The network automatically, selectively reduces the transitivity of those cells responsible for transmitting erroneous interconnecting beams in order to modify the interconnection strengths and thereby reduce the error signals as the generated output pattern more nearly resembles the desired output for a given input.
A variable weight optical interconnector in accordance with the present invention comprises a projecting means and an interconnection weighting means remote from the projecting means. The projecting means projects a distribution of interconnecting light beams when illuminated by a spatial light pattern. The interconnection weighting means includes a photosensitive screen provided in optical alignment with the projection means. The screen has variable transitivity to control a parameter of each of the interconnecting light beams and thereby assign an interconnection weight to each such beam.
A highly interconnected optical neural network having capability to learn in accordance with the present invention comprises a spatial modulating means responsive to an input signal for modulating light to provide modulated optical signals, detecting means for providing detection signals in response to light incident thereon, interconnector means for providing a distribution of interconnecting light beams to the detecting means when illuminated by the modulated optical signals to interconnect the modulating means and the detecting means, and means responsive to the detection signals for modifying the interconnection weights assigned by the weighting means. The interconnector means includes projection means which projects interconnecting beams without changing a parameter of the beams relative to each other, and an interconnection weighting means provided in an optical path of the projection means for controlling a parameter of each interconnecting light beam relative to the other beams so as to assign an interconnection weight to each beam and thereby provide weighted, modulated interconnecting beams.
Brief Description of the Drawings
Further aspects and features of the present invention will be even more apparent from the following detailed description and drawings, and the appended claims. In the drawings:
FIG. 1 is a schematic view illustrating a model for an interconnected neuron;
FIG. 2 is a block diagram of a preferred embodiment of the neural network which incorporates the interconnector according to the, present invention;
FIG. 3 is a schematic plan view of a preferred interconnector according to the present invention, which interconnector comprises a photosensitive screen in optical alignment with an N.sup.2 .times.N.sup.2 array of holograms provided in a page-oriented holographic means;
FIG. 4 illustrates a sigmoidal, monotonic input-output response characteristic utilized in the neural network of FIG. 2;
FIG. 5 is a ray diagram useful in illustrating one possible optical interconnection of the input and detection elements in the network of FIG. 2;
FIG. 6 is a ray diagram useful in illustrating modification of the transitivity of the screen in the network of FIG. 2;
FIG. 7 is a block diagram showing an alternative embodiment of an interconnector in accordance with the present invention; and
FIG. 8 is a block diagram of a composite neural network incorporating several neural networks of the type shown in FIG. 2.
Detailed Description of the Invention
In schematic form, FIG. 2 shows a preferred, highly interconnected neural network 10 having learning capability in accordance with the present invention. It is contemplated that preferred network or system 10 can be implemented completely with optical elements. However, as will be seen, network 10 could comprise various combinations of optical and electrical elements while remaining well within the scope of the present invention.
Network 10 is seen to include a laser 12 providing a source of laser light at a particular wave-length. Laser light from laser source 12 is directed onto a spatial light modulator (TSLM) 14. Spatial light modulator 14 is of the polarizing transmissive type, and so when laser light from source 12 is transmitted through TSLM 14, it is modulated to provide a spatial pattern according to input data programmed into the TSLM. Modulated light transmitted through TSLM 14 is expanded by a lens system 16 comprising a double concave lens 18 and a focusing lens 20. Lens system 16 directs the expanded, modulated beams from TSLM 14 through an optical interconnector 22 in accordance with the present invention. Preferred interconnector 22 comprises a photosensitive, transmissive screen 24 and a page-oriented holographic means 26. Holographic means 26 is formed to project N. light beams to beamsplitter 28 when the holographic means is illuminated. It will be seen that the intensity of each beam projected by holographic means 26 depends upon the particular spatial distribution of modulated light transmitted by TSLM 14, which modulated light illuminates the holographic means, and the transmissive condition of photosensitive screen 24. Beamsplitter 28 splits the beams projected by holographic means 26 and directs the split beams onto designated pixels at the photodetecting surface 30 of a detector 32 and onto associated pixels defined on the reflective side 34 of a reflective spatial light modulator (RSLM) 36. Detection signals from the detector 32 are applied to computer 38. Computer 38 also receives signals indicative of a desired output spatial pattern from a desired pattern memory 40 and signals representative of the input signal pattern which are applied simultaneously to TSLM 14. Based upon a comparison of the detection signals and the desired output pattern signals, for a given input signal pattern, computer 38 generates error signals and applies the error signals to RSLM 36. A phase conjugate mirror (PCM) 42 is disposed to receive light reflected from the reflective face 34 of RBLM 36. The PCM 42 amplifies and reverse propagates the received light back to RSLM 36 which then reflects the amplified light to expose transmissive screen 24.
Interconnector 22 provides for complete interconnection between each pixel defined on TSLM 14 and each pixel defined on the photodetection surface 30 of detector 32. Interconnector 22 is depicted in plan view in FIG. 3 as comprising phototransmissive screen 24 in optical alignment with holographic means 26. Holographic means 26 is the element which makes the interconnections in network 10 and therefore will be discussed first. In the preferred embodiment, holographic means 26 is of the page-oriented type which comprises N' individual holograms 44 arranged in an N*.times.N*, spatially localized array. Holograms 44 are affixed in a transparent substrate such as a glass slide. As is well known, each individual hologram 44 can be formed from a thermoplastic material or a photorefractive material. In the preferred system 10, holograms 44 are permanently recorded. Alternatively, if desired, holograms 44 could be erasable. Those of ordinary skill in the art also appreciate that spatially localized holograms 44 could be of the planar type or could be volume holograms. Further still, it will be understood that the interconnections between TSLM 14 and photodetection surface 30 could be made by an array of lenses and/or mirrors However, holographic means 26 is the preferred element to provide for 10.sup.1 2 or more interconnections in network 10.
The TSLM 14 is defined to have N.sup.2 pixels Photodetecting surface 30 and reflecting surface 34 likewise are defined to have N.sup.2 pixels. Each pixel of TSLM 14 modulates a portion of the light from laser 12 and transmits the modulated light therethrough to provide a spatial pattern formed of N.sup.2 modulated light beams. Holographic means 26 and TSLM 14 are optically aligned so that the modulated light beam from each pixel of TSLM 14 illuminates N.sup.2 holograms 44 of the holographic means. Each hologram 44 in turn switches the direction of the modulated light from TSLM 14 to project a modulated interconnecting beam through beamsplitter 26 to a particular pixel on photodetection surface 30. By way of example, when the ith group of N.sup.2 holograms 44 are illuminated by light from the ith pixel of TSLM 14, these N.sup.2 holograms will direct one interconnecting beam, modulated or encoded by the intensity of the light passed by the ith pixel of TSLM 14, to each of the N.sup.2 pixels of photodetection surface 30 (and the reflecting face 34 of RSLM 36). It is thus understood that holographic means 26 divides the N.sup.2 modulated light beams from TSLM 14 which form the input spatial pattern and switches their direction to project N.sup.4 modulated interconnecting beams through beamsplitter 28. All of the N.sup.4 holograms 44 of holographic means 26 have the same transitivity so that the relative intensities of the projected interconnecting beams will not be changed by the holographic means. Indeed, if holographic means 26 were directly exposed to light from laser 12, without the affects of TSLM 14 and transmissive screen 24, each interconnecting beam projected by the holographic means would have the same intensity and therefore the same interconnection weight or strength.
Phototransmissive screen 24 assigns the weight or strength to each interconnection made by interconnector 22. As illustrated in FIG. 3, screen 24 is divided into imaginary partitions or cells 46 delineated by alternating dot-dash lines. There is no physical boundary between the partitions 46. Screen 24 and holographic means 26 are aligned such that there is a one-to one correspondence between each partition 46 and each hologram 44 and therefore each interconnection beam projected by the holographic means. Accordingly, screen 24 is defined to have N.sup.4 such partitions 46 likewise arranged in an N.sup.2 .times.N.sup.2 array. The intensity of each interconnection beam is controlled by the optical transitivity of each partition 46 to assign an interconnection strength to each such beam.
Preferred screen 24 comprises a sheet of photochromogenic material such as a photochromic or photodichromic glass. In the preferred embodiments, screen 24 also could comprise a photodichroic material. It is also contemplated that screen 22 may comprise photographic emulsions. Regardless of which type of material is selected to form screen 24, the screen is fabricated so that when one or more of its cells or partitions 46 are exposed to light having a certain parameter, the screen 22 reacts by modifying its absorption spectrum in the exposed partition or partitions. Where photosensitive screen 24 is constructed from a photochromic material, a photochromic glass such as Corning fast fade No. 8111 glass or a Corning experimental glass known as UXC are preferred. The photodichromic glass darkens where it is exposed to light having an intensity which exceeds a characteristic threshold intensity for the glass. Hereinafter, this characteristic intensity will be referred to as the modification threshold intensity, that is the beam intensity necessary to modify the optical transitivity of the glass. The degree of darkening depends upon the actual intensity of the exposing light and the duration of exposure. If the exposure light intensity fails to exceed the threshold modification intensity, the light can be transmitted through the screen without appreciably affecting the transitivity of the exposed partitions 46 thereof.
It follows that where interconnector 22 comprises a photochromic screen 24 in preferred system 10, laser 12 is provided to have an carrier beam with an intensity somewhat below the threshold intensity of the screen. The TSLM 14 modulates in any of several know ways without amplifying the intensity of the light. The PCM 42 and the RSLM 36 provide the optical means for carrying out modification of the interconnection strengths by amplifying light transmitted through screen 24 to above the modification threshold and directing the amplified light to expose selected partitions of the screen. It is the province of the PCM 42 to amplify the exposure light to the necessary intensity. The RSLM 36 directs the amplified light to the desired partitions 46 to be modified.
The phase conjugating materials from which PCM 42 is provided are now well known in the art. PCM 42 is a nonlinear optical device which reverses the direction of propagation of probe beams incident on it so as to provide return beams which exactly retrace the path of the probe beams back to their points of origin. As explained in U.S. Pat. No. 4,739,496, a diverging light beam emanating from a point source will, upon reflection from a phase conjugating device, result in a converging beam which propagates back to its origin point in space. This property is referred to in the art as wave front reversal or time-reversal reflection.
To suitably amplify the return beams, PCM 42 could be of the degenerate four wave mixing type wherein two pump beams having the same frequency are directed to set up a standing wave in a transparent material whose index of refraction varies with intensity. When the material is exposed by probe beams also having the same frequency, the material emits time-reversed return beams which are amplified by the pump beams. Numerous other phase conjugating devices are readily available 5 and could be used in system 10. Such devices in include three-wave or other four-wave mixing photon scatter devices, Kerr-like phase conjugation devices, stimulated scattering or back scattering devices, nonlinear photon echo devices, surface phase conjugation devices, and various semiconductor or photorefractive devices.
Computer 38 determines how the interconnection strengths set by photosensitive screen 24 are to be modified and controls the reflectivity of RSLM 36 accordingly. Computer 38 could be an electronic computer but it is also contemplated that computer 38 can be constructed from optical components. Computer 38 receives detection signals having analog values from detector 32 and signals representative of a desired pattern from its memory 40. Mathematically, the detection signals received from detector 30 can be thought of as a N.sup.2 .times.1 vector which will be referred to as Y'. The desired pattern signals would be considered as representing a N.sup.2 .times.1 vector T. Computer 38 generates error signals, which will be considered as another N.sup.2 .times.1 vector E, as a function of the vectors Y' and T. The particular function selected for generation of the error signals depends upon the application of preferred system 10. Any of several well known algorithms may be selected.
To generate the error signal vector E, computer 38 compares the detection signals Y' and the desired pattern signals T and calculates the error signals in proportion to the differences between the detection and the desired pattern signals. In detail, detector 32 generates a detection signal in response to the intensity of light at each pixel on its photodetecting surface 30. Detector 32 applies N.sup.2 of these detection signals in parallel to computer 38 as vector Y'. Computer 38 compares each component signal y.sub.1 ' of Y' with a corresponding element t: of T to generate a series of error signals which form the error vector E which is applied to RSLM 36. Each pixel of photodetecting surface 28 has a corresponding pixel on RSLM 36. When the error vector E is applied to RSLM 36, the RSLM responds by altering the reflectivity of the defined pixels of its reflective surface 34. The reflectivity of the ith pixel of surface 34 is changed in proportion to the difference between the element y.sub.i ' of vector Y' and the element t.sub.i of vector I. If computer 38 determines that y.sub.i equals t.sub.i, the error signal representative of e.sub.i, when applied to RSLM 36, will cause the ith pixel of RSLM 36 to be made substantially less reflective and preferably substantially nonreflective. On the other hand, a large difference between a signal y.sub.j and t.sub.j translates to a small reduction or no reduction in the reflectivity of the ith pixel of RSLM 36. Thus, RSLM 36 only weakly partially reflects to PCM 42 each distribution of N* modulated interconnecting beams which provide a detection signal nearly equal to the desired pattern signal when detected at detector 32. The RSLM 36 fully reflects each group of modulated interconnecting beams which cause generation of detection signals that are substantially different from their corresponding desired pattern signals.
The pattern of partially and strongly reflected beams from RSLM 36 are the probe beams for PCM 42. The PCM 42 generates return beams having amplified intensities proportional to the probe beams from RSLM 36. The RSLM 36 in turn reflects the amplified return beams from PCM 42 in the same manner that it directs the modulated interconnecting beams as probe beams onto PCM 42 so that the relative low intensity return beams which correspond to the weakly reflected probe beams are themselves only partially reflected and the relatively high intensity return beams which correspond to the substantially fully reflected probe beams likewise are fully reflected by the RSLM. All of the return beams are directed through holographic means 26 to illuminate photosensitive screen 24. Due to the amplification of the controlled reflectivity of the pixels of reflecting surface 32, the fully or nearly fully reflected return beams have intensities exceeding the modification threshold for screen 24. The highly reflected return beams thus darken their corresponding screen partitions 46. The partitions 46 darkened are those that first pass and encode the interconnecting beams which first become the highly reflected probe beams and thereafter, the high intensity return beams. The interconnection strengths set by the darkened partitions 44 are thus reduced to correspondingly reduce the error signal components e.sub.i with which they are associated. The weaker the reflected return beam directed onto screen 24 by RSLM 36, the less effect the return beam will have on the transitivity of its associated screen partition 46. The weakest reflected return beams will have intensities less than the screen threshold modification intensity and thus leave their associated partitions unchanged.
To generate the N.sup.2 signals defining vector Y', detector 32 performs a detection function and a nonlinear operation. The detection function is elementary and will be considered first. Each pixel of photodetecting surface 30 is exposed to N.sup.2 interconnecting beams projected by holographic means 26. Detector 32 sums the intensities of the N.sup.2 beams incident on each of its N.sup.2 pixels to obtain a summed intensity value at each pixel. With reference to the mathematical representations introduced supra., the N.sup.2 summed intensity values provide the elements y.sub.i of a 1.times.N.sup.2 column vector Y.
Detector means 32 then performs a nonlinear transformation o& each of the N.sup.2 detected intensity values y.sub.i to obtain a new value y.sub.i '. FIG. 4 is a graphical representation of a function G(Y) which describes the transformation carried out by detector 32 for each summed intensity value y.sub.i. The function G(Y) is a continuous, monotone increasing, nonlinear function which maps the vector Y in N.sup.2 space to the new vector Y' in N.sup.2 space to provide a sigmoidal threshold response. Detector 32 can be controlled to set the particular slope or steepness of its sigmoidal response curve G(Y) depending upon different applications of system 10. Preferably detector 32 performs the nonlinear transformation of each of the summed intensity values in parallel to apply the vector Y' in parallel to computer 38.
In describing the operation of preferred system 10, reference will be made to a series of mathematical operations that represent a series of processing steps which the system automatically carries out. According to a mathematical description, system 10 is understood as performing five distinct steps. The first step is a matrix vector multiplication given by
where X is the input vector that represents an input pattern which, for example, system 10 may be made to learn for subsequent identification of such pattern. As described above input vector X may be thought of as a N.sup.2 .times.1 column vector in the form ##EQU1## Input vector X is programmed into system 10 by means of TSLM 14. The TSLM 14 in turn imparts to the light from laser 12, an input spatial pattern representative of the input vector X. As TSLM 14 is defined to have N.sup.2 pixels, each pixel transmits light representing one element x.sub.i of X.
Holographic means 26 and photosensitive screen 24 insert the matrix A into system 10. Corresponding to the N.sup.2 .times.1 vector X, the matrix A is an N.sup.2 .times.N.sup.2 matrix having N elements The magnitude of each element a.sub.i j of A is encoded by photosensitive screen 24. The magnitude of the elements a.sub.i j, represent the strength of each interconnection made between TSLM 14 and detector 32. Light from each pixel of TSLM 14 is switched into N.sup.2 separate beams by holographic means 26. Holographic means 26 thus projects N.sup.2 separate beams corresponding to each pixel of TSLM 14 onto an associated pixel on detecting surface 30 and an associated pixel on reflecting surface 34. When photodetecting surface 30 sums the intensities of the N.sup.2 interconnecting beams incident on each pixel thereof, the matrix multiplication
is completed.
The second step, written as
is carried out by detector 32. Detector 32 nonlinearly transforms the intensity value registered at each detection pixel according to the sigmoid g(y.sub.i)=y.sub.i ' graphically illustrated in FIG. 4. The resulting vector Y' is applied to computer 38 for comparison and calculation of the error signal vector E.
Computer 38 determines the vector E according to a function of Y' and T. This step is written as
The particular function, F, selected for generation of vector E can be any of several well known learning algorithms and could simply be c(Y'-T).sup.1/2 where c is the amplification factor of PCM 42. The vector E is also written as a function of X because each particular desired pattern T is associated with a particular input pattern X.
Computer 38 applies vector E to RSLM 36 to set the reflectivity of each pixel of RSLM 36 accordingly. The weighted interconnecting beams modulated by TSLM 14 are reflected by RSLM 36 as probe beams onto PCM 42. The intensities of the N.sup.4 probe beams are controlled by the reflectivity of each pixel defined on reflective face 34. Thereafter, PCM 42 time reverse reflects and amplifies the probe beams to provide amplified return beams to RSLM 36. Pixels of RSLM 36 controlled to be poorly reflective and pixels adjusted to be highly reflective further adjust the intensities of the return beams and then direct them to illuminate phototransmissive screen 24. This double reflection of the optical signals representing vector Y as probe beams from RSLM 36 onto PCM 42 and back as amplified return beams from the PCM completes an operation described by the vector multiplication
The light beams indicative of the vector Z are amplified by the characteristic gain of PCM 42 to have sufficient intensity to alter the transitivity of selected partitions 46 of photosensitive screen 24 and thereby change the interconnection strengths according to the relation
where c is the amplification of the PCM 42. In the initial state of operation of system 10, there would be no error signal and thus no conjugate beams from PCM 42. Then, a new spatial pattern is transmitted to TSLM 14 to begin a new cycle of learning.
In connection with the operation of the preferred neural network according to the invention, FIGS. 5 and 6 are partial ray diagrams useful in illustrating the optical paths between the elements in the network. The example in FIGS. 5 and 6 of course is greatly simplified where, for purposes illustration, N is considered to equal 2. It is understood by those of ordinary skill in the art that FIGS. 5 and 6 are only one example of the optical paths between the elements shown and that various alternative paths are apparent. As holograms 44 do not affect the modification of the transitivity of screen 24, the holograms are omitted in FIG. 6.
With N considered to equal 2 and with reference to both FIG. 2 and FIG. 5, it is understood that light from each pixel x.sub.i of TSLM 14 illuminates an entire row of partitions 46 of phototransmissive screen 24. The transmissive condition of each partition 46 determines the intensity of the light passed therethrough to its underlying hologram 44. As seen from FIG. 6, the holograms associated with each column of partitions 46 interconnect each pixel x.sub.i of TSLM 14 with its corresponding pixel y.sub.i on photodetector 32 and RSLM 36. The intensity at each pixel y.sub.i of photodetector 32 and RSLM 36 is thus given by
As discussed in the foregoing, photodetector 32 automatically performs the summation to complete the matrix multiplication .DELTA.X=Y and thereafter carries out the nonlinear transformation according to G(Y) to obtain Y'.
When the error vector E is applied to RSLM 36, the reflectivity of each pixel y.sub.i therefore is adjusted accordingly to direct each of the four modulated and weighted interconnecting beams as probe beams onto PCM 42. The PCM 42 in turn generates amplified return beams which retrace the path of the probe beams so that when reflected by RSLM 36, the return beams are directed back to expose the same partition 46 which passed the interconnecting beam providing their respective probe beam. When exposed to the return beams, the partitions 46 darken to reduce the intensity of the return beams and cause the output vector Y' to approach the desired pattern vector T.
Previously, it has been stated that the photosensitive screen in the interconnector of the present invention could comprise a photodichroic material. FIG. 7 shows an alternative embodiment of an interconnector 122 together with means 150 for modifying the weights of the interconnections made by interconnector 122. Interconnector 122 comprises a photodichroic screen 124, and a holographic means 126. Except that screen 124 comprises a photodichroic material, screen 124 can be constructed in essentially the same way as interconnector 22 of FIG. 2. It is contemplated that screen 124 comprises a sheet of sodium fluoride crystals with lithium doping, or a sheet comprising potassium fluoride crystals. Preferably a doped sodium fluoride sheet comprises about 0.175 lithium by weight and measures about 2.5 cm by 2.5 cm on a side. Such a preferred sheet has a thickness of approximately 0.2 mm.
It is contemplated to modify screen 124 by either direct or suppressive extinction writing. These techniques for writing and reading photodichroic materials generally, and lithium doped N.sub.a F materials in particular, have been discussed at length in several articles, notably Casasent and Caimi, "Combined Suppressive And Extinction Writing Using M and M.sub.A Centers and Li-doped NaF", Appl. Phys. Lett. (Vol. 29, No. 10, 15 Nov. 1976); Schneider, "Extinction Technique For Optical Storage Using Anisotropic Color Centers In Alkali Halides" , Applied Physics Letters, (Vol. 25, No. 1, July 1, 1974); Casasent and Caimi, "Photodichroic Crystals for Coherent Optical Data Processing", Optics and Laser Technology, (Apr. 1977); and Casasent and Caimi, "Suppressive Writing in LithiumDoped NaF", (Phys. Stat. Sol. (a)34.461, 1976) which are incorporated by reference herein.
As discussed in detail in the above-listed articles, extinction writing uses the anisotropy and dispersion of "M" or color centers in the screen 124. An "M"center is defined as two nearest neighbor "F" centers along a <011> direction. An "F" center is defined as an electron trapped at a negative-ion vacancy. Depending upon the alignment of their vacancy axes, the M centers are described as existing in a "O" state, i.e. state-0, or a re-oriented state referred to as state-1. M centers have an absorption spectrum substantially comprising two bands, an M band at .lambda..sub.R .perspectiveto.506 nm and an M.sub.F band at .lambda..sub.W in the range of 300 to 400 nm.
In direct writing, the entire screen 124 is exposed to .lambda..sub.W0 light, having a polarization P.sub.O. Exposure to the .lambda..sub.W0 light initially aligns most of the M centers to state-0, that is the .lambda..sub.W0 light bleaches the screen. Thereafter when particular partitions of the screen 124 are exposed to .lambda..sub.W1 light having a different polarization P.sub.1, the M centers of the exposed portions will re-orient to state-1 to carry out writing. The screen then can be nondestructively read by exposing it to read light, .lambda..sub.R in the M band.
For interconnector 122, suppressive extinction writing is preferred because the same wavelength .lambda..sub.R can be used for both writing and non-destructive reading of the screen 124. Interconnector is thus depicted with a source 152 of ultraviolet (UV), .lambda..sub.W light. According to the theory of suppressive writing, the entire screen 124 is continuously exposed to UV light in the M.sub.F band which thus tends to orient the M centers to state-1. However, in partitions of screen 124 which are simultaneously illuminated by .lambda..sub.R light in the M band, the re orienting affects of the UV, .lambda..sub.W light are suppressed. Non-destructive reading is performed only with .lambda..sub.R light. Accordingly, modification of the screen 124 is controlled by the .lambda..sub.R light.
Screen 124 is modified by suppressive extinction writing due to the presence of a polarizor 154 located in front of photodetector 132. According to this technique, spatially modulated .lambda..sub.R light from an input TSLM 114 is linearly polarized, for example vertically polarized. Polarizer 154 then would be a horizontal polarizor so that interconnector 122 is in the optical path between crossed polarizors. If the concentrations of M centers in state-0 and state-1 are equal, the vertical polarization of .lambda..sub.R is unchanged so that all of the light is absorbed by polarizer 154. If M.sub.0 .noteq.M.sub.1, then .lambda..sub.R is transmitted to photodetector 132 at an intensity given by
where I.sub.0 is the incident light intensity, a.sub.0 and a.sub.1 are amplitudes of the incident light polarized along [011]and [011]respectively, d is the screen thickness and n.sub.v and n.sub.h are the refractive indices for vertically and horizontally polarized light respectively. Computer 138 receives magnitude intensity signals for each pixel defined for photodetector 132 following a nonlinear thresholding operation by the photodetector. Based upon a comparison of the threshold output signals from photodetector 132 with the desired input, computer 138 controls TSLM 136 and PCM 142 to generate return beams which change the respective state-1 and state-0 M center concentrations and thereby modify the transitivity of the screen 124 to provide an output pattern similar to the input.
For applications such as recognizing very complex objects or patterns, a composite network comprising several constituent systems according to the present invention can be provided. An example of such a composite network 200 is shown in block-diagram form in FIG. 8. Exemplary composite network 200 has a first stage 202 of four neural networks 210A, 210B, 210C and 210D, and a second stage 204 with one system 210E which receives output from the other four. Each system 210A through 210E can be arranged essentially like optical network 10 discussed in connection with FIG. 2. In operation, each of first stage systems 210A, 210B, 210C, and 210D are made to learn to recognize a particular object or pattern which would be a portion of a larger pattern to be recognized by composite network 200. For instance, first stage network 210A might be made to recognize the wing of a plane, system 210B, the forward portion of the fuselage, system 210C, the tail portion, and system 210D, markings or decals on the body of the plane. When the outputs from the first stage systems 210A through 210D are provided to second stage system 210E, system 210E would recognize the composite object as a particular type of aircraft.
It is to be understood that there can be various changes and modifications to the preferred embodiments of the system of the present invention disclosed herein, which changes and/or modifications may be made by one ordinary skill in the art, but such would still result in a system well within the scope of the invention as set forth in the claims.
Claims
What is claimed is:
1. A highly interconnected optical neural network having capability to learn, said network comprising: spatial modulating means responsive to an input signal for modulating light to provide modulated optical signals; detecting means for providing detection signals in response to light incident thereon; interconnector means for providing a distribution of modulated interconnecting light beams to said detecting means when said interconnector means is illuminated by modulated optical signals from said modulating means so as to interconnect said modulating means and said detecting means according to a predetermined interconnection pattern, said interconnector means including projections means adapted to project a distribution of interconnecting beams along paths without changing a parameter of each of said beams relative to the other interconnecting beams projected thereby, and interconnection weighting means, provided in said paths so as to be illuminated by said interconnecting beams before said parameter of any of said beams has changed, of controlling said parameter of each interconnecting beam relative to the other interconnecting beams to assign an interconnection weight to each said beam and thereby provide weighted, modulated interconnecting beams; and means responsive to said detection signals for changing a condition of said interconnection weighting means to thereby modify said interconnection weights assigned by said weighting means.
2. A network as claimed in claim 1, wherein said interconnection weighting means comprises a photosensitive screen which controls the intensity of each interconnecting beam.
3. A network as claimed in claim 2, wherein said photosensitive screen comprises a photochronic material.
4. A network as claimed in claim 3, wherein said photochromic material is a photochromic glass.
5. A network as claimed in claim 2, wherein said photosensitive screen comprises a photodichroic material
6. A network as claimed in claim 5, further comprising a source of UV light for illuminating said screen.
7. A network as claimed in claim 6, further comprising an analyzing polarizer disposed between said interconnector means and said detecting means.
8. A network as claimed in claim 2, wherein said photodichroic material comprises sodium fluoride doped with about 0.178 percent lithium by weight and said screen has a thickness of about 0.2 mm.
9. A network as claimed in claim 7, wherein said photodichroic material comprises potassium fluoride.
10. A network as claimed in claim 2, wherein said projection means comprises a page-oriented holographic means having an array of spatially-localized holograms associated in one-to-one correspondence with cell partitions defined on said screen.
11. A network as claimed in claim 2, wherein said detecting means comprises photodetecting means for generating signals indicative of the intensity of the weighted, modulated interconnecting beams and means for nonlinearly transforming such generated signals according to a sigmoidal function to provide detection signals.
12. A network as claimed in claim 2, wherein aid photosensitive screen comprises regions having optical transmittivities, and wherein said means for changing a condition of said interconnection weighting means comprises: error signal generation means for generating error signals as a function of (a) said detection signals, (b) signals indicative of a spatial pattern forming a desired output, and (c) modulated optical signals provided by said modulating means and; optical means responsive to said error signals for selecting said regions and varying the transitivity thereof in response to said error signals.
13. A network as claimed in claim 12, wherein said generation means comprises a computer.
14. A network as claimed in claim 13, wherein said means for selecting said regions and varying said transitivity thereof comprises: a spatial light modulator disposed so that weighted, modulated interconnecting beams provided by said interconnector means are incident on said spatial light modulator of said varying means in the same pattern that weighted, modulated interconnecting beams are incident on said detecting means and, a phase conjugate mirror means placed in optical communication with said photosensitive screen by said spatial light modulator of said selecting and varying means.
15. A network as claimed in claim 14, wherein said phase conjugate mirror means provides amplified return beams.
16. A highly interconnected optical processing apparatus for simulating a highly interconnected neural network by carrying out the matrix operation steps of: (1) AX=Y (2) NL(Y)=Y' (3) E=F(Y', T, X) (4) Z=E.sup.2 Y (5) .DELTA.a.sub.i j =ce.sup.2.sub.j X.sub.i a.sub.i j, said apparatus comprising: input means including a spatial light modulator defining pixels for projecting a pattern of modulated light beams representing input data in the form of an input vector X with each modulated beam representing an element X.sub.i of X; a detecting means including a photodetecting means which defines pixels that are associated with said pixels of said spatial light modulator, and a nonlinear transforming means; interconnector means disposed in the optical path between said spatial light modulator and said detecting means for providing a distribution of modulated and weighted interconnecting light beams to interconnect each pixel of said spatial light modulator with each pixel of said detecting means when said interconnector means is illuminated by a pattern of modulated beams from said spatial light modulator, said interconnector means including projection means adapted to transmit modulated light beams from said spatial light modulator as interconnecting beams without relatively changing the intensities thereof, and photosensitive screen means for controlling the relative intensities of interconnecting beams incident thereon from said projection means to encode each such beam to have an interconnection strength according to the element of the matrix A, said photodetector detecting the sum of the intensities of the modulated interconnecting beams incident on each pixel thereof to provide summed intensity values representative of the result of the matrix multiplication (b 1) AX=Y where Y is a vector, and said nonlinear transforming means transforming each of the summed intensity values according to a sigmoidal function to provide detection signals which represent the nonlinear vector transformation (2) NL(Y)=Y' where Y') is a vector; and means responsive to input data representative of the vector X, provided detection signals representative of the vector Y', and data indicative of a transmissive vector T, which vector T represents a desired output pattern for modifying a condition of said photosensitive screen means to thereby change the interconnection weights of the modulated interconnecting beams, said means for modifying a condition of said photosensitive screen including means for generating an error vector signal representative of the result of (3) E=F(Y', T, X) and optical means responsive to the error vector signal and the vector Y for generating modifying signals representative of the result of the matrix multiplication (4) Z=E.sup.2 Y; said photosensitive screen means responding to said modifying signals to modify the intensities of the interconnecting beams according to the result
17. A processing apparatus as claimed in claim 16, wherein said photosensitive screen means comprises a photochromic glass.
18. A processing apparatus as claimed in claim 16, wherein said photosensitive screen means comprises a photodichroic screen.
19. A processing apparatus as claimed in claim 18, further comprising a UV light source adapted to completely expose at least one side of said screen and an analyzing polarizor disposed in an optical path from said interconnector means to said photodetecting means.
20. A processing apparatus as claimed in claim 16, wherein said projection means comprises holographic means in the form of an array of spatially-localized holograms.
21. A processing apparatus as claimed in claim 20, wherein said computer calculates the error vector E=F(Y', T, X) according to the algorithm E=c(Y'-T).sup.1/2 where c is the amplification factor of said phase conjugate mirror.
22. A processing apparatus as claimed in claim 16, wherein said error signal generating means comprises a computer and said responsive optical means comprises a phase conjugate mirror placed in optical communication with said photosensitive screen by a reflective spatial light modulator.
23. A highly interconnected neural network having capability to learn, said network comprising a plurality of first stage neural network devices which are each adapted to provide an output, and a second stage neural network device receiving as inputs, the output of each of said first stage network devices, each of said devices including means for receiving optical signals as input; detecting means for providing detection signals; interconnector means for providing a distribution of modulated interconnecting light beams to said detecting means when illuminated by optical signals transmitted from said input means in order to interconnect said input means and said detecting means, said interconnector means comprising projection means adapted to project a distribution of interconnecting beams along paths without changing a parameter of the beams relative to each other, and interconnection weighting means including a photosensitive screen located remote from said projection means and in the paths of the interconnecting beams projected therefrom, for controlling said parameter of each interconnecting beam relative to the other interconnecting beams in order to assign an interconnection weight to each such beam and thereby provide weighted, modulated interconnecting beams; and means responsive to detection signals from said detecting means for changing a condition of said interconnection weighting means to thereby modify said interconnection weights.
Patent Citations (13)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US3553460 | 1971-01-01 | Preston, Jr. et al. | |
| US4187000 | 1980-02-01 | Constant | |
| US4278318 | 1981-07-01 | Nakayama et al. | |
| US4569033 | 1986-02-01 | Collins et al. | |
| US4603398 | 1986-07-01 | Bocker et al. | |
| US4607344 | 1986-08-01 | Athale et al. | |
| US4620293 | 1986-10-01 | Schlunt et al. | |
| US4660166 | 1987-04-01 | Hopfield | |
| US4696533 | 1987-09-01 | Kingston et al. | |
| US4859012 | 1989-08-01 | Cohn | |
| US4862406 | 1989-08-01 | Fisher | |
| US4877297 | 1989-10-01 | Yeh | |
| US5004309 | 1991-04-01 | Caulfield et al. |
Non-Patent Literature (13)
- Casasent, "Spatial Light Modulators", Proceedings of the IEEE, vol. 65, No. 1, Jan. 1977, pp. 143-157.
- Waibel, "Building Blocks for Speech", Byte, Aug. 1989, pp. 235-242.
- Abu-Mostafa et al., "Optical Neural Computers", Scientific American, Mar. 1987, pp. 88-95.
- Psaltis et al., "Optical Neural Nets Implemented with Volume Holograms", Optical Society of America Topical Meeting on Optical Computing, vol. 11, 1987, pp. 129-132.
- Wagner et al., "Multilayer Optical Learning Networks", SPIE vol. 752 Digital Optical Computing (1987) pp. 86-97.
- Farhat, "Architectures for Optoelectronic Analogs of Self-Organizer Neural Networks", Optics Letters, vol. 12, No. 6, Jun. 1987, pp. 448-450.
- Farhat et al., "Optical Analog of Two-Dimensional Neural Networks and Their Application in Recognition of Radar Targets", Neural Networks for Computing, 1986 AP Conference on Neural Networks, Snowbird, VT, pp. 146-152
- Wagner and Psaltis, "Multilayer Optical Learning Networks", Applied Optics, vol. 26, No. 23, pp. 5061-5076.
- Wagner and Psaltis, "Multilayer Optical Learning Networks", pp. 133-136.
- Casasent and Caimi, "Combined Suppressive and Extinction Writing Using M M.sub.A Centers and Li-Doped NaF", App. Phys. Lett. (vol. 29, 10, Nov. 15, 1976, pp. 660-662.
- Schneider, "Extinction Technique for Optical Storage Using Anisotropic Color Centers in Alkali Halides", Applied Physics Letters, (vol. 25, No. 1, 7-1-74) pp. 77-79.
- Casasent and Caimi, "Photodichroic Crystals for Coherent Optical Data Processing", Optics and Laser Technology, (Apr. 1977) pp. 63-68.
- Casasent and Caimi, "Suppressive Writing in Lithium-Doped NaF", (Phys. Stat. Sol. (a) 34.461, 1976, pp. 461-466.