Inductively Coupled Neural Network

The present invention relates to a neural network and neural coupling of neurons which constitute the neural network.

A proposed prior art type of the computer includes a neural network operating in a mode similar to information processing in the human brain.

This type of device processes mathematically random matter such as pattern recognition and associative memorization, which could not be satisfactorily processed by the sequential processing mode Neumann type computer.

In FIG. 2 is shown an elementary prior art neuron unit of a neural network. The neuron 1 receives n input signals x.sub.1, x.sub.2, . . . x.sub.i, . . . x.sub.n and multiplies each by a synaptic coupling weight w.sub.1, w.sub.2, . . . w.sub.i, . . . w.sub.n. Neuron 1 calculates the sum of the multiplication results. The neuron 1 adds a specific threshold value h to the sum and operates on the added result with a nonlinear function to thereby derives an output signal y. Such operation is mathematically represented by the following relation:

In the operation, if the synaptic coupling weight w.sub.i is positive, it provides an excitatory synaptic coupling, while if the synaptic coupling weight w.sub.i is negative, it provides an inhibitatory synaptic coupling. When simulating a neuron in a mathematical form, the nonlinear function is normally represented by the following sigmoid function:

FIG. 3 is a block diagram of the neuron performance. The neuron is comprised of a multiplication and summation unit 2 for multiplying synaptic coupling weights w.sub.i with the corresponding input signals x.sub.i to obtain the sum of the multiplication results. Nonlinear operating unit 3 adds a specific threshold value to the sum and nonlinearly processes the added result. The value of the synaptic coupling weight w.sub.i is determined according to an adaptive learning function unit (not shown) based on back-propagation method.

FIG. 4 is a block diagram of an example of a prior art neural network. In the Figure, the neural network includes input terminals 4.sub.1, 4.sub.2. . . 4.sub.i, . . . 4.sub.l, primary neurons 5.sub.1, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m, secondary neurons 6.sub.1, 6.sub.2, . . . 6.sub.k, . . . 6.sub.n, and output terminals 7.sub.1, 7.sub.2, . . . 7.sub.k, . . . 7.sub.n. The input terminals 4.sub.1, 4.sub.2, . . . 4.sub.i, . . . 4.sub.l receive each of l number of input signals a.sub.1, a.sub.2, . . . a.sub.i, . . . a.sub.l. These input signals are distributed to and supplied to m neurons 5.sub.1, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m. Each of the neurons r.sub.2, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m carries out the before-mentioned multiplication and summation operations and nonlinear processing. Further, output signals of the primary neurons 5.sub.1, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m are distributed and supplied into respective one of n secondary neurons 6.sub.1, 6.sub.2, . . . 6.sub.k, . . . 6.sub.n. Each of the neurons 6.sub.1, 6.sub.2, . . . 6.sub.k, . . . 6.sub.n also carries out the before-mentioned multiplication and summation operation and the subsequent nonlinear processing, such that respective output signals b.sub.1, b.sub.2 , . . . b.sub.k, . . . b.sub.n are derived from n output terminals 7.sub.1, 7.sub.2, . . . 7.sub.k, . . . 7.sub.n. The values of the synaptic coupling weights of respective neurons 5.sub.1 - 5.sub.m and 6.sub.1 - 6.sub.n are determined by a controlling learning function unit (not shown). The neural network is supplied with a set of input signals a.sub.1 - a.sub.l, the processed results of which are known as a set of output signals b.sub.1 - b.sub.n. A current set of output signals is compared with the known output signals to determine the value of synaptic coupling weights by the learning function unit through the back-propagation method. By repeating such procedure a appropriate number of times, the final synaptic coupling weights are adaptively adjusted to produce optimum output signals when the input signals a.sub.1 - a.sub.l are supplied to the neural network.

As described above, according to adaptive learning by the learning function unit, the neural network can organize itself so as to obtain correct output signals b.sub.1 - b.sub.n from input signals a.sub.1 - a.sub.l. Therefore, the neural network does not need any specific algorithm or program for processing the signals. Moreover, the neural network can process of multiple of data concurrently in parallel so as to provide high speed processing of mathematically random data such as derived during pattern recognition and associative memorization.

The performance of the neural network depends the number of neurons, and performance is more effective as the number of neurons increases . However, when increasing the number of neurons, there is a corresponding increase in the number of couplings between the input terminals and the neurons and between the neurons with each other. Generally, when the number of neurons is multiplied by n-, the number of couplings is increased on the order of n.sup.2 .noteq.times.

The currently available integrated circuit technology is only applicable to a two-dimensional substrate, and the number of multi-layered patterns is limited to a certain degree. Therefore, in order to construct a neural network as shown in FIG. 4 in the form of an integrated circuit, the number of available neurons is considerably restricted, thereby causing a problem. There has been experimentally produced various types of the neural networks. For example, patterns are formed to constitute couplings between input terminals and neurons and among the neurons. The patterns are commonly used in the time-sharing mode for a plurality of input terminals and neurous so as to reduce the actual number of neural couplings. In another conventional neural network, photoemitters, photodetecters, optical fibers and spatial photomodulators are used to provide effective spatial couplings to transmit optical signals between input terminals and neurons and among neurons. Coupling weights can be also optically set in the neural network.

However, in the prior art, perfect parallel processing cannot be performed. In the latter type of the prior art, presently available optical technology is not sufficiently sophisticated to enable device size reduction, and it would be difficult to continuously change coupling weight according to learning results. Either of the prior art structures has drawbacks to impair essential neural network features.

In view of the above noted drawbacks of the prior art, an object of the present invention is to provide neural couplings in the neural network for coupling in parallel the input terminals and the neurous with each other and the neurons by one another with using readily available fabrication technology, without causing device scale-up.

In order to achieve the above noted object, according to a first aspect of the present invention, there is provided a neural coupling method comprising inductively coupling an input terminal and a neuron with each other and inductively coupling a pair of neurons with each other with a magnetic field, or an electromagnetic field.

According to a second aspect, the neural network comprises input terminals, neurons, and a coupling device or circuit for inductively coupling an input terminal and inductively a neuron with each other and coupling a pair of neurons with each other with a magnetic field or electromagnetic field.

According to a third aspect, the coupling device comprises at least a pair of inductance elements magnetically coupled to each other by a magnetic field. According to a fourth aspect, an inductance element comprises a coil. According to a fifth aspect, a inductance element comprises a magneto-resistance element. According to a sixth aspect, the coupling circuit comprises at least a resonance circuit for electromagnetic field coupling. According to a seventh aspect, a neural network comprises a plurality of input terminals, a plurality of primary neurons each having an output, a secondary neuron having an input, a plurality of primary coils magnetically coupled to one another. Each coil is connected to a respective one of the input terminals or each output of the primary neurons. A secondary coil magnetically coupled to the primary coils is connected to the input of the secondary neuron.

According to an eighth aspect, the neural network comprises a plurality of input terminals, a plurality of primary neurons each having an output, and multiple primary coils each connected to each of the input terminals or each output of the primary neurons. Multiple secondary coils are each magnetically coupled to a set of the primary coils. The primary coils are selected from different groups of the primary coils corresponding to respective input terminals or respective outputs of the primary neurons. Multiple secondary, and a multiple of secondary neurons each having an input are connected to each of the secondary coils.

According to a ninth aspect, the neural network includes a plurality of resonance circuits containing the corresponding coils. According to a tenth aspect, the neural network includes a plurality of impedance elements each connected in parallel or series to a corresponding coil to define a synaptic coupling weight. According to an eleventh aspect, each impedance element has a impedance variable to adjust synaptic coupling weight.

With regard to the first aspect of the present invention, since the input terminal and the neuron or the pair of neurons are inductively by a coupled magnetic or electromagnetic field, multiple input terminals and neurons or multiple neurons can be spatially coupled by readily available technology to achieve overall parallel coupling with each other.

With regard to the second aspect, since a coupling device or circuit is utilized to inductively couple between an input terminal and a neuron or between a pair of neurons through a magnetic field, multiple input terminals and neurons and multiple neurons can be spatially coupled with each other by mature technology so as to provide a neural network having parallel couplings.

With regard to the third aspect, at least a pair of magnetically coupled inductance elements are used as a coupling device through a magnetic field so as to facilitate neural network circuit design.

With regard to the fourth aspect, since the inductance element is composed of a coil, couplings can be constructed economically with less spacing between input terminals and neurons and among the neurons.

With regard to the fifth aspect, since a magneto-resistance element is used as the inductance element, the coupling can be effected with a DC magnetic field as well as AC magnetic field.

With regard to the sixth aspect, since at least one resonance circuit is used as the coupling circuit with a magnetic field, efficient spatial couplings can be constructed between input terminals and neurons and among neurons.

With regard to the seventh aspect, because the neural network includes plural input terminals, plural primary neurons each having an output, a secondary neuron having an input, plural primary coils magnetically coupled to each other and connected to each of the input terminals or each output of the primary neurons, and a secondary coil magnetically coupled to the primary coils and connected to the input of the secondary neuron, the parallel couplings can be constructed between the plural input terminals and the secondary neuron and between the plural primary neurons and the secondary neuron.

With regard to the eight aspect, since the neural network includes plural input terminals, plural primary neurons each having an output, multiple primary coils each connected to each of the input terminals or each output of the primary neurons, multiple secondary coils each magnetically coupled to a set of the primary coils each selected from different groups of the primary coils corresponding to respective input terminals or respective outputs of the primary neurons, and multiple secondary neurons each having an input connected to each of the secondary coils; the parallel couplings can be constructed between the plural input terminals or the plural primary neurons and the multiple secondary neurons.

With regard to the ninth aspect, since all or part of the coils are incorporated into a resonance circuit, the couplings can be constructed efficiently in parallel between the plural input terminals or the plural primary neurons and the single or multiple secondary neuron.

With regard to the tenth aspect, since an impedance element is connected in series or parallel to the coil or resonance circuit to define the synaptic coupling weights, the couplings can be ensured between the plural input terminals or the plural primary neurons and the single or multiple secondary neuron to enable the secondary neuron to effect multiplication and summation operations.

With regard to the eleventh aspect, since the synaptic coupling weight can be adjusted by changing impedance of the impedance element, the multiplication and summation operations can be controllably carried out in a single or multiple neurons.

FIG. 1 is a circuit diagram of a first embodiment of the present invention;

FIG. 2 is an illustrative diagram of a prior art neuron of a neural network;

FIG. 3 is a block diagram of a prior art neuron;

FIG. 4 is a diagram of an exemplary prior art neural network;

FIG. 5 is a circuit diagram of a second embodiment of the invention;

FIGS. 6 and 7 are circuit diagrams of neural networks with capacitive coupling;

FIG. 8 is a circuit diagram of an embodiment of the invention with inductive coupling;

FIG. 9 is a circuit diagram of another embodiment of the invention with inductive electromagnetic coupling;

FIG. 10 is an explanatory diagram of an electromagnetically resonant neural network with inductive coupling;

FIG. 11 is an explanatory diagram of the operation of the FIG. 9 circuit; and

FIG. 12 is a graph of the relation between number of noise occurrences in input data and correct answer rate in the recognition.

FIG. 13a and 13b, together, are a circuit diagram of a hybrid version of a neural network in accordance with the present invention; and

FIG. 14 is a schematic diagram of a portion of the circuit elements illustrated in each of FIGS. 13a and 13b.

FIG. 1 is shown a first embodiment of the present invention, in which a pair of nodal neurons 10 and 11 are coupled to each other through inductive AC magnetic field with using an alternating current source 12 and coils 13 and 14. An output current of the alternating current source 12 is controlled by an output signal of the precedent nodal neuron 10. The coils 13 and 14 are connected, respectively, to the alternating current source 12 and to an input of the subsequent nodal neuron 11, and they are arranged to magnetically couple with each other. In such structure, the alternating current source 12 feeds an alternating current to the coil 13 according to an output value of the neuron 10 to generate an inductive AC magnetic field in the coil 13. The AC magnetic field is effective to generate in the coil 14 an induced voltage proportional to the output value of the neuron 10, and the induced voltage is inputted into the neuron 11.

FIG. 5 is shown a second embodiment of the present invention in the form of a neural coupling through inductive DC magnetic field. As shown in the figure, the neural coupling is comprised of a pair of neurons 10 and 11, a coil 13, a direct current source 15, a magnetoresistance element 16, a constant voltage source 17 and a resistor 18. An output current of the direct current source 15 is controlled by an output signal of the precedent neuron 10. The coil 13 is connected to the direct current source. The magnetoresistance element 16 is connected at its one end to the constant voltage source 17, and connected at its other end to ground through the resistor 18 and to an input of the subsequent neuron 11. The coil 13 and the magnetoresistance element 16 are arranged to be magnetically coupled with each other.

In the above described construction, the direct current source 15 feeds a direct current according to an output value of the neuron 10 to the coil 13 to generate therein a DC magnetic field. The DC magnetic field changes the resistance value of the magnetic resistance element 16 proportionally to the output value of the neuron 10. The change of the resistance value causes a corresponding change of an electric current flowing from the constant voltage source 17 through the magnetic resistance element 16 to the resistor 18. A voltage drop developed across the resistor 18 due to the flowing electric current is supplied into the neuron 11 so that the neuron 11 receives a voltage having a magnitude proportional to the output value of the neuron 10.

In FIG. 6 is shown a neural coupling arrangement using an AC electric field. As shown in the Figure, neural coupling between a pair of neurons 10 and 11 is provided by AC voltage source 19 and capacitors 20, 21. The AC voltage source 19 is controlled by an output signal of the neuron 10. The capacitor 20 is connected at its one electrode to the AC voltage source 19, and is connected at its other electrode to an input of the neuron 11 and to one terminal of the capacitor 21. The other terminal of the capacitor 21 is connected to the ground. In such construction, the AC voltage source 19 applies an AC voltage according to the output value of the neuron 10 to one electrode of the capacitor 20 to generate therein an AC electric field. The AC electric field is effective to induce at the other electrode of the capacitor 20 a voltage proportional to the output value of the neuron 10, and the induced voltage is supplied into the neuron 11.

In FIG. 7 is shown a neural coupling arrangement using a DC electric field. In the Figure, the neural coupling between neurons 10 and 11 is provided by capacitors 20, 21 and a DC voltage source 22. The DC voltage source 22 is controlled by an output signal of the neuron 10. One electrode of capacitor 20 is connected to the DC voltage source 22, while another electrode of the capacitor is connected to an input of the neuron 11 and to one terminal of the capacitor 21. The other terminal of the capacitor 21 is connected to the ground. In such construction, the DC voltage source 22 applies a DC voltage according to the output value of the neuron 10 to the one electrode of the capacitor 20 to generate therein a DC electric field. The DC electric field induces at the other electrode of the capacitor 20 a voltage proportional to the output value of the neuron 10. The induced voltage is supplied to the neuron 11.

FIG. 8 is shown another embodiment of the present invention in the form of neural coupling utilizing an AC magnetic field and designed to carry out concurrent multiplication and summation operations. In the figure, neural coupling between neurons 23, 24, 25 and 26 is via, resistors 27, 28 and 29, coils 30, 31, 32 and 33 and a nonlinear operating unit 34. neuron 23 is comprised of the resistors 27, 28 and 29, the coils 30, 31, 32 and 33, the nonlinear operating unit 34 and a learning function unit (not shown). The resistors 27, 28 and 29 constitute synaptic coupling weights together with the coils 30, 31, 32 and 33; the resistance values of resistors 27-29 are determined by the learning function unit. One terminal of the respective resistors 27, 28 and 29 is connected to a corresponding output terminal of the respective precedent neurons 24, 25 and 26, and the other terminal of the respective resistors 27, 28 and 29 is connected to corresponding one of the coils 30, 31 and 32. The coil 33 is connected to the nonlinear operating unit 34. The coils 30, 31, 32 and 33 are arranged to be magnetically coupled in series with each other. Each of neurons 23-26 derives, a separate AC voltage signal.

In the above described construction, the respective each of coils 30, 31 and 32 is responsive to an alternating current having an amplitude depending on the output voltage values of the neurons 24, 25 and 26, the resistance values of the resistors 27, 28 and 29 and reactances of the coils 30, 31, 32 and 33 so as to generate DC magnetic fields. These DC magnetic fields are combined with each other to induce in the coil 33 a voltage proportional to the sum of voltages coils 24, 25 and 26 and proportional to the synaptic coupling weights determined by the resistors 27, 28 and 29 and the coupling coefficients of coils 30, 31, 32 and 33. The voltage induced in coil 33 is supplied to the nonlinear operating unit 34 and is nonlinearly processed. In this embodiment, coils are utilized for the generation and detection of the AC magnetic field; however, other elements can be used. Any impedance element can be used in place of a resistor for constituting synaptic coupling weight. Further, the synaptic compling can be based on induction through DC magnetic field, or AC electric field so as to concurrently effect the multiplication and summation operation in manner similar to the fifth embodiment.

In FIG. 9 is shown a further embodiment of the present invention in the form of neural network through electromagnetic field coupling effective to carry out concurrent multiplication and summation operations. In the Figure, the neural network is comprised of input nodes or terminals 41.sub.1, 41.sub.2, . . . 41.sub.l arranged in one row, synaptic coupling weight control terminals 42.sub.11, 42.sub.12, . . . 42.sub.1l, 42.sub.21, 42.sub.22, . . . 42.sub.2l, . . . 42.sub.m1, 42.sub.m2, . . . 42.sub.ml arranged in m rows and l columns, transistors (FETs) 43.sub.1, 43.sub.2, . . . 43.sub.l arranged in one row, resonance circuits 44.sub.11, 44.sub.12, . . . 44.sub.1l, 44.sub.21, 44.sub.22, . . . 44.sub.2l, . . . 44.sub.ml, 44.sub.m2, . . . 44.sub.ml arranged in an array of m rows and l columns and each being composed of a coil and a capacitor, impedance elements 45.sub.11, 45.sub.12 , . . . 45.sub.1l, 45.sub.21, 45.sub.22, . . . 45.sub.2l, . . . 45.sub.ml, 45.sub.m2, . . . 45.sub.ml arranged in m rows and l columns, constant current circuits 46.sub.1, 46.sub.2, . . . 46.sub.l arranged in one row and each being composed of an operational amplifier, a transistor and a resistor, a direct current source 47, and an alternating current source 48. These elements constitute a first block 40.

A second block 50 is comprised of m parallel rows of linear resonance circuits 51.sub.1, 51.sub.2, . . . 51.sub.m, each circuit being composed of a coil and a capacitor, a column of differential amplifiers 52.sub.1, 52.sub.2, . . . 52.sub.m, a column of comparators 53.sub.1, 53.sub.2 . . . 53.sub.m, and a column of threshold value input terminals 54.sub.1, 54.sub.2, . . . 54.sub.m.

A third block 60 is comprised of a column of intermediate output nodes or terminals 61.sub.1, 61.sub.2, . . . 61.sub.m, m rows and n columns of synaptic coupling weight control terminals 62.sub.11, 62.sub.12, . . . 62.sub.1m, 62.sub.21, 62.sub.22, . . . 62.sub.2m, . . . 62.sub.n1, 62.sub.n2, . . . 62.sub.nm, a column of transistors (FETs) 63.sub.1, 63.sub.2, . . . 63.sub.m, an array of m rows and n columns of resonance circuits 64.sub.11, 64.sub.12, . . . 64.sub.1m, 64.sub.21, 64.sub.22, . . . 64.sub.2m, . . . 64.sub.n1, 64.sub.n2, . . . 64.sub.nm, each being composed of a coil and a capacitor, m rows and n columns of impedance elements 65.sub.11, 65.sub.12, . . . 65.sub.1m, 65.sub.21, 65.sub.22, . . . 65.sub.2m, . . . 65.sub.n1, 65.sub.n2, . . . 65.sub.nm , a column of constant current circuits 66.sub.1, 66.sub.2, . . . 66.sub.m, each being composed of an operational amplifier, a transistor and a resistor, a direct current source 67, and an alternating current source 68.

A fourth block 70 is comprised of n parallel columns of linear resonance circuits 71.sub.1, 71.sub.2, . . . 71.sub.n, each composed of a coil and a capacitor, a row of differential amplifiers 72.sub.1, 72.sub.2, . . . 72.sub.n, a row of comparators 73.sub.1, 73.sub.2, . . . 73.sub.n, a row of threshold value input terminals 74.sub.1, 74.sub.2, . . . 74.sub.n, and a row of output nodes or terminals 75.sub.1, 75.sub.2, . . . 75.sub.n.

The first block 40 and the fourth block 70 are formed integrally on a planar substrate A as shown in the figure, and the second block 50 and the third block 60 are formed integrally on a planar substrate B as shown in the figure. The pair of substrates A and B are superposed with each other to constitute the neural network such that the first block 40 and the second block 50 face each other, and the third block 60 and the fourth block 70 face each other. In such arrangement, electromagnetic coupling is provided between the first block 40 and the second block 50 such that the first row of resonance circuits 44.sub.11, 44.sub.12, . . . 44.sub.1l corresponds to the first row of resonance circuit 51.sub.1, the second row of resonance circuits 44.sub.21, 44.sub.22, . . . 44.sub.2l corresponds to the second row of resonance circuit 51.sub.2, and the m-th row of resonance circuits 44.sub.m1, 44.sub.m2, . . . 44.sub.ml corresponds to the m-th row of resonance circuit 51.sub.m. In similar manner, with regard to the third block 60 and the fourth block 70, electromagnetic coupling is provided between the first column of resonance circuits 64.sub.11, 64.sub.12, . . . 64.sub.1m and the first column of resonance circuit 71.sub.1, between the second column of resonance circuits 64.sub.21, 64.sub.22, . . . 64.sub.2m and the second column of resonance circuit 71.sub.2, and between the n-th column of resonance circuits 64.sub.n1, 64.sub.n2, . . . 64.sub.nm and the n-th column of resonance circuit 71.sub.n.

Next, the description is given for flow or transmission of signals. l is the number of input signals X.sub.1, X.sub.2, . . . X.sub.l, respectively, supplied in parallel to the row of input terminals 41.sub.1, 41.sub.2, . . . 41.sub.l of the first block 40. The input signal distributed by the row of corresponding transistors 43.sub.1, 43.sub.2, . . . 43.sub.l to the corresponding columns of resonance circuits 44.sub.11, 44.sub.21, . . . 44.sub.m1. /44.sub.12, 44.sub.22, . . . 44.sub.m2 / . . . /44.sub.1l, 44.sub.2l, . . . 44.sub.ml. The distributed set of the input signals X.sub.1, X.sub.2, . . . X.sub.l is transmitted as inductive electromagnetic wave from the respective rows of the resonance circuits 44.sub.11, 44.sub.12, . . . 44.sub.1l /44.sub.21, 44.sub.22, . . . 44.sub.2l / . . . /44.sub.m1, 44.sub.m2, . . . 44.sub.ml to the corresponding rows of the resonance circuits 51.sub.1, 51.sub.2, . . . 51.sub.m by the inductive electromagnetic coupling therebetween, and consequently the set of the input signals X.sub.1, X.sub.2, . . . X.sub.l are added to each other in the respective resonance circuit 51.sub.1, 51.sub.2, . . . 51.sub.m.

Each of the resonance circuits 44.sub.11 - 44.sub.ml is connected in parallel to each of the impedance elements 45.sub.11 - 45.sub.ml. Therefore, the set of input signals X.sub.1, X.sub.2, . . . X.sub.l are added to each other through each row of couplings according to the respective synaptic coupling weights determined according to the impedance value of the elements 45.sub.11 - 45.sub.ml and the impedance of the resonance circuits 44.sub.11 - 44.sub.ml and the impedance of the resonance circuits 51.sub.1 - 51.sub.m. The synaptic coupling weights are adaptively adjusted by setting impedances of the impedance elements 45.sub.11 - 45.sub.ml by means of a learning function unit (not shown) connected thereto through the synaptic coupling weight controlling terminals 42.sub.11 - 42.sub.ml.

Output signals from the resonance circuits 51.sub.1, 51.sub.2, . . . 51.sub.m are amplified, respectively, by the column of differential amplifyers 52.sub.1, 52.sub.2, . . . 52.sub.m, and then compared by the column of comparators 53.sub.1, 53.sub.2, . . . 53.sub.m with the respective threshold values T.sub.1, T.sub.2, . . . T.sub.m which are supplied separately by the threshold value input terminals 54.sub.1, 54.sub.2, . . . 54.sub.m so as to produce m number of intermediate output signals Y.sub.1, Y.sub.2, . . . Y.sub.m. The set of intermediate output signals Y.sub.1, Y.sub.2, . . . Y.sub.m are supplied, respectively, to external circuits via the column of intermediate output terminals 61.sub.1, 61.sub.2, . . . 61.sub.m of the third block 60, and are distributed through the column of transistors 63.sub.1, 63.sub.2, . . . 63.sub.m to the respective rows of resonance circuits 64.sub.11, 64.sub. 21, . . . 64.sub.n1 / 64.sub.12, 64.sub.22, . . . 64.sub.n2 / . . . / 64.sub.1m, 64.sub.2m, . . . 64.sub.nm. Then the set of distributed intermediate output signals is transmitted from the respective columns of resonance circuits 64.sub.11, 64.sub.12, . . . 64.sub.1m / 64.sub.21, 64.sub.22, . . . 64.sub.2m / . . . / 64.sub.n1, 64.sub.n2, . . . 64.sub.nm to the corresponding columns of resonating circuits 71.sub.1, 72.sub.2, . . . 71.sub.n of the fourth block 70, which are electromagnetically coupled to the corresponding columns of resonance circuits 64.sub.11, 64.sub.12, . . . 64.sub.1m / 64.sub.21, 64.sub.22, . . . 64.sub.2m / . . . / 64.sub.n1, 64.sub.n2, . . . 64.sub.nm. The sets of transmitted signals are added in each of the resonance circuits 71.sub.1, 71.sub.2, . . . 71.sub.n. Each of the impedance elements 65.sub.11 - 65.sub.nm is connected in parallel to each of the corresponding resonance circuits 64.sub.11 - 64.sub.nm so that the set of intermediate output signals are added to each other with the synaptic coupling weights which are determined by impedances of the impedance elements 65.sub.11 - 65.sub.nm, the resonance circuits 64.sub.11 - 64.sub.nm and the resonance circuits 71.sub.1 - 71.sub.n. The synaptic coupling weights can be adaptively adjusted by setting impedance values of the impedance elements 65.sub.11 - 65.sub.nm through the corresponding synaptic coupling weight control terminals 62.sub.11 - 62.sub.nm by the learning function unit (not shown).

Output signals of the columns of the resonance circuits 71.sub.1, 71.sub.2, . . . 71.sub.n are amplified by the respective differential amplifiers 72.sub.1, 72.sub.2, . . . 72.sub.n, and further compared by the respective comparators 73.sub.1, 73.sub.2, . . . 73.sub.n with respective threshold values S.sub.1, S.sub.2, . . . S.sub.n which are supplied separately from threshold value input terminals 74.sub.1, 73.sub.2, . . . 73.sub.n to thereby finally develop output signals at the n output terminals 75.sub.1, 75.sub.2, . . . 75.sub.n.

Next, the general description is given for the operation of the electromagnetically coupled resonance circuits with reference to FIG. 10. The electromagnetic coupling is comprised of coils 80, 81, capacitors 82, 83, a resistor 84 and an alternating current source 85. A primary resonance circuit is comprised of coil 80, capacitor 82, resistor 84 and alternating current source 85, all connected in parallel to the primary resonance circuit. A secondary resonance circuit comprises coil 81 and the capacitor 83. Here, L.sub.1 and L.sub.2 respectively denote the inductances of the coils 80 and 81, C.sub.1 and C.sub.2 respectively denote the capacitances of capacitors 82 and 83, respectively, R denotes resistance of the resistor 84, and I.sub.s =I.sub.o sin .omega.t denotes an output current of the alternating current source 85. The primary resonance circuit is held in the resonant state to satisfy the following relation:

On the other hand, an induced voltage is generated in the secondary resonance circuit due to mutual induction between the coils 80 and 81. The voltage V.sub.0 developed across the terminals of the capacitor 83 is represented by the following relation:

where M denotes mutual inductance between the coils 80 and 81 and is represented by M.sup.2 =K.sup.2 L.sub.1 L.sub.2 where K indicates coupling coefficient.

According to the relation (4), the voltage V.sub.0 induced in the secondary resonance circuit is proportional to the product of the current I.sub.s in the primary resonance circuit and the resistance R. Therefore, by changing the value of resitance R, the voltage V.sub.0 can be varied. Changes of the value of the resistance R change the resonance circuit selectively.

Next, the description is given for the operation of the FIG. 9 neural network with reference to FIG. 11 which examplifies one circuit section of the first block 40. The circuit section is comprised of an input terminal 41.sub.j, a synaptic coupling weight control terminal 42.sub.ij, a transistor 43.sub.j (FET), a resonance circuit 44.sub.ij, an impedance element 45.sub.ij, a constant current circuit 46.sub.j composed of an operational amplifyer, a transistor and resistors, a direct current source 47, and an alternating current source 48 (where j=1, 2, . . . l, i=1, 2, . . . m). The input terminal 41.sub.j receives an input signal voltage X.sub.j which may have either of high and low levels. When the input voltage X.sub.j exceeds a threshold value of the transistor 43.sub.j, a current I.sub.j is flown throughout the j-th column of resonance circuits 44.sub.ij (i=1, . . . m). The current I.sub.L flowing in the coil of the resonance circuit 44.sub.ij is represented by the following relation:

where L.sub.1 denotes inductance of the coil of the resonance circuit L.sub.1 denotes inductance of the coil of the corresponding resonance circuit 51.sub.i (not shown) in FIG. 11, but shown in FIG. 9 of the second block 50, which corresponds to the resonance circuit 44.sub.ij, M denotes mutual inductance therebetween, C.sub.1 and C.sub.2 denote capacitances of the capacitors in the resonance circuits 44.sub.ij and 51.sub.i, respectively, and R.sub.ij denotes impedance of the impedance element 45.sub.ij. The resonance circuit 44.sub.ij of the first block 40 transmits an electromagnetic wave proportional to the current I.sub.L flowing through the coil. As understood from relation (5), the current I.sub.L is proportional to R.sub.ij I.sub.j. As decribed before, the i-th row of the resonance circuits 44.sub.ij (j=1, 2, . . . l) of the first block 40 are electromagnetically coupled to the i-th lenear resonance circuit 51.sub.i of the second block 50 so that the i-th resonance circuit 51.sub.i of the second block 50 induces a voltage proportional to sum of the currents flowing through the respective coils of the resonance circuits 44.sub.ij (j=1, 2, ... l) of the first block 40.

In the second block 50, the induced voltage is amplified by the i-th differential amplifier 52.sub.i, and thereafter is detected by the i-th comparator 53.sub.i. Therefore, the output signal voltage Y.sub.i of the i-th comparator 53.sub.i is represented by the following relation:

where .alpha. denotes a proportionality constant. According to the relation (6), the output signal voltage Y.sub.i of the second block 50 is obtained by a multiplication and summation operation in the form of matrix/vector products of impedance R.sub.ij and current I.sub.j.

Similar operation is effected between the respective resonance circuits of the third block 60 and the corresponding resonance circuit of the fourth block 70. The output signal voltage Z.sub.k(k=1, 2, . . . n) is likewise obtained by a multiplication and summation operation in the form of matrix/vector products of the impedance Q.sub.ki of the impedance element 65.sub.ki and the current I.sub.i.

The impedances R.sub.ij and Q.sub.ki are controlled, respectively, through the synaptic coupling weight control terminals 42.sub.ij and 62.sub.ki by the learning function unit. In the preferred illustrated embodiment, the impedances R.sub.ij and Q.sub.ki are purely resistive so that the selectivity is changed in the resonance circuits 44.sub.ij and 64.sub.ki by the learning operation. Accordingly, the phase of the transmitted electromagnetic wave does not change, and therefore it is not necessary to compensate for any phase difference.

Further, the currents I.sub.j and I.sub.i are controlled, respectively, by the input voltage X.sub.j to transistor 43.sub.j and the intermediate input voltage Y.sub.i to transistor 63.sub.i, hence the currents I.sub.j and I.sub.i are, respectively, functions of the input voltages X.sub.j and Y.sub.i, i.e., I.sub.j =f(X.sub.j) and I.sub.i =f(Y.sub.i). These functions are determined by the characteristics of the transistors 43.sub.j and 63.sub.i. In the later-described simulation, the functions are approximated as being of the sigmoid type. In the foregoing description, the voltages X.sub.j, Y.sub.i and Z.sub.k are actually values before processing by a nonlinear function such as a sigmoid function. To provide positive and negative synaptic coupling weights in a practical device, the resonance circuits 44.sub.ij and 64.sub.ki are combined with other resonance circuits (not shown in FIGS. 9 and 11) having reversely wound coils and corresponding impedance elements and another synaptic coupling weight control terminal.

Each of impedance elements 45.sub.ij and 65.sub.ki may be composed, for example, of a MOS-FET. Such, a MOS-FET has its source and drain terminals connected in parallel to the individual resonance circuit, and its gate terminal connected to the synaptic coupling weight control terminal. The synaptic coupling weight control terminal is supplied with a gate voltage controlled by the learning function unit so that the channel resistance between the source and drain terminals is adaptively set to determine optimum synaptic coupling weight.

Alternatively, the synaptic coupling weight may be adjusted by using a voltage-controlled variable capacitor of a type generally utilized in an electronic synchronization circuit. Namely, the resonance circuits 44.sub.ij and 64.sub.ki can be comprised of a variable capacitor, and the capacitance value thereof can be controlled by a voltage derived inputted from the learning function unit. Concurrently, the current flowing through the resonance circuits 44.sub.ij and 64.sub.ki is varied to thereby change the magnitude of voltage induced in the corresponding secondary resonance circuits 51.sub.i and 71.sub.k.

The constant current circuits 46.sub.j and 66.sub.i respectively drive the j-th column of the resonance circuits of the first block 40 and i-th row of the resonance circuits of the third block 60; since circuits 46.sub.j and 66.sub.1 are constant current sources that the currents I.sub.j and I.sub.i are not affected by changes of the above-mentioned impedances.

Lastly, simulation is undertaken to evaluate the performance of the FIG. 9 neural network. Namely, the first block 40 has 81 input terminals, the second block 50 has 20 output terminals which correspond to those of the intermediate output terminals of the third block 60, and the fourth block 70 has 26 final output terminals. The input voltage X.sub.j is set to .+-.5V, the inductance of the coils of the resonance circuits is set to 10 nH in the first block 40 and the third unit 60, the capacitance of the capacitors of these resonant circuits is set to 2.533 pF, the frequency of the power source is set to 1 GH.sub.z, so the resonance circuits are in the resonant state. A limiter is utilized with the impedance elements such that the impedance R.sub.ij has value in the range of 0-1K .OMEGA. for positive and negative input values. Consequently, if the adjusted value of impedance R.sub.ij according to the learning mode exceeds 1K .OMEGA., the impedance R.sub.ij is actually limited to 1K .OMEGA.. The amplitude change of the currents I.sub.j, I.sub.i fed from the constant current circuits is set in the range of 0-10 mA. The inductance of coils of the secondary resonance circuits is set to 20 nH in the second block 50 and the fourth block 70, and capacitance of capacitors of the secondary resonance circuits is set to 2.533 pF. Lastly, the coupling coefficient K between the coils of the primary resonance circuits in the first and third blocks 40, 60 and the corresponding coils of the secondary resonance circuits in the second and fourth blocks 50, 70 is set to 0.1. Learning is carried out according to back-propagation learning rule. Under the above-mentioned condition, 277 learning trials are undertaken until saturation to establish stable recognition of the 26 capital letters of the English alphabet. Further, in FIG. 12 is shown the relation between number of noise occurrences in the input data and correct recognition answer rate. In the graph, the solid curve 90 indicates the results when the number of input data is set to 8.times.8=64, and the broken curve 91 indicates the results when the number of input data is set to 40.times.40=1600.

In order to practice the invention, the neural network can be constructed in the form of an integrated circuit which contains coils and capacitors. In such case, spiral coils can be formed of gold plating film on the integrated circuit substrate, and capacitors having capacitance of 1-20 pF can be formed of dielectric material film such as silicon nitride Si.sub.3 N.sub.4 on the integrated circuit of the planar type. Further, an integrated circuit utilizing FETs of gallium arsenide can be used in the high frequency range of more than 40 GHz. By increasing the operating frequency, inductances and capacitances in the resonance circuits can be reduced in size to thereby enable production of a practical neural network by currently available fabrication technology. In the present invention, analog signals are transmitted through magnetic, electric or electromagnetic field coupling by using coils and capacitors. Variations may be caused in circuit parameters such as inductance of coils and capacitance of capacitors during the fabrication processing. However, such circuit parameter variations can be effectively canceled during the learning procedure, because the synaptic coupling weights are adaptively determined during the learning operation to obtain optimum results in the neural network to thereby absorb the above-noted variations.

Reference is now made to FIGS. 13a, 13b and 14, schematic diagrams of a hybrid apparatus for performing the same functions as performed by the apparatus illustrated in FIG. 9. Elements in FIGS. 13a and 13b corresponding to elements in FIG. 9 are indicated in FIGS. 13a and 13b by digits having the same units and tens value as the reference numerals of FIG. 9; however in FIGS. 13a and 13b these corresponding elements have a value of 1 in the hundreds digit. The apparatus illustrated in FIG. 13a corresponds with the elements on the left side of FIG. 9, i.e., the elements in FIG. 9 responsive to signals X.sub.1, X.sub.2. . . X.sub.1 and the remaining apparatus for deriving signals that are compared with signals T.sub.1, T.sub.2. . . T.sub.m. The apparatus illustrated in FIG. 13b corresponds to the structure illustrated on the right side of FIG. 9 and thus includes the comparators responsive to signals T.sub.1, T.sub.2 . . . T.sub.m and the remaining apparatus for deriving output signals Z.sub.1, Z.sub.2 . . . Z.sub.n.

The apparatus illustrated in FIG. 13a includes a two dimensional x-y matrix of elongated coils 151.sub.1, 151.sub.2 . . . 151.sub.m, which are spatially parallel to each other and extend in the x direction. The matrix of FIG. 13a also includes elongated coils 252.sub.1, 252.sub.2 . . . 252.sub.l. Each of coils 252.sub.1, 252.sub.2 . . . 252.sub.l extends in the y direction, i.e., at right angles to coils 151.sub.1, 151.sub.2 . . . 151.sub.m. At each intersection of coils 151 and 252 there is a cylindrical magnetic core 253. For example, at the intersection of coils 151.sub.1 and 252.sub.1 is located core 253.sub.11 ; at the intersection of coils 151.sub.m and 252.sub.l, core 253.sub.ml is located; and at the intersection of coils 151.sub.h and 252.sub.i, core 253.sub.hi is located, where h is selectively each of 1, 2 . . . m and i is selectively each of 1, 2 . . . l. The magnetic core 253 at each intersection magnetically couples the coils at the particular intersection to each other. Each of cores 253 and the coils associated therewith correspond with the resonant coupling circuits 44 on the left side of FIG. 9.

A preferred configuration of coils 151 and 252 and one of cores 253 is illustrated in FIG. 14. Core 253 is positioned so its axis extends between coils 151 and 252 which are located in close proximity to opposite end faces of the core. Helical winding 255, wound about the side wall of core 253, is shunted by fixed capacitor 257 and variable capacitor 259, both of which are connected in parallel with the source drain path of field effect transistor 261, to form a circuit resonant to the frequency of AC source 148. The resistance of the source drain path of field effect transistor 261 is controlled by the magnitude of the weight setting voltage at terminal 142, selectively applied through switch 265 to the gate of transistor 261, shunted by capacitor 267. Hence, while switch 265 is closed, variations in the value of the weight setting voltage at terminal 142 control the resistance of the resonant circuit including coil 255 and capacitors 257 and 259.

AC source 148 drives each of coils 252.sub.1, 252.sub.2 . . . 252.sub.l through amplifiers 146.sub.1, 146.sub.2 . . . 146.sub.l, respectively. The magnitude of the AC current flowing in coils 252.sub.1, 252.sub.2 . . . 252.sub.l is respectively controlled by the source drain resistance of field effect transistors 143.sub.1, 143.sub.2 . . . 143.sub.l. The source drain resistance of each of field effect transistors 143.sub.1, 143.sub.2 . . . 143.sub.l is in turn controlled by the magnitude of the DC signals X.sub.1, X.sub.2 . . . X.sub.l. The magnitude of the AC voltage thereby induced in a particular coil 151.sub.h in response to the current flowing in a particular coil 252.sub.i is thereby proportional to the AC current magnitude flowing in the particular coil 252.sub.i coupled to the coil 151.sub.h and the magnitude of the weight setting voltage at terminal 142.sub.hi applied to the field effect transistor at a particular intersection, as well as the magnitude of the voltage X.sub.i applied to transistor 143.sub.i connected to the particular coil 252.sub.i. The total voltage induced in a particular coil 151.sub.i is thereby proportional to the sum of the voltages induced in the coil as a result of coupling from coils 252.sub.1, 252.sub.2 . . . 252.sub.l. Hence, the voltage at output terminals of each of coils 151.sub.1, 151.sub.2 . . . 151.sub.m is analogous to the voltage derived from each of coils 51.sub.1, 51.sub.2 . . . 51.sub.m. The voltages at the output terminals of coils 151.sub.1, 151.sub.2 . . . 151.sub.m are respectively applied to input terminals of amplifiers 152.sub.1, 152.sub.2 . . . 152.sub.m.

The output voltages of amplifiers 152.sub.1, 152.sub.2 . . . 152.sub.m are respectively applied to input terminals 272.sub.1, 272.sub.2, 272.sub.m of each of comparators 153.sub.1, 153.sub.2 . . . 153.sub.m, FIG. 13b. Comparators 153.sub.1, 152.sub.2 . . . 153.sub.m compare the signal supplied to them by amplifiers 152.sub.1, 152.sub.2 . . . 152.sub.m with signals having values T.sub.1, T.sub.2 . . . T.sub.m, respectively. Each of comparators 153.sub.1, 153.sub.2 . . . 153.sub.m derives an supplied to it. The outputs of comparators 153.sub.1, 153.sub.2 . . . 153.sub.m are respectively applied to the gates of field effect transistors 163.sub.1, 163.sub.2 . . . 163.sub.m. The source drain resistances of field effect transistors 163.sub.1, 163.sub.2 . . . 163.sub.m are thereby respectively controlled by the magnitude of the output voltages of comparators 153.sub.1, 153.sub.2 . . . 153.sub.m. The source drain paths of transistors 163.sub.1, 1632 . . . 163m are energized by DC voltage source 167.

Transistors 163.sub.1, 163.sub.2 . . . 163.sub.m are connected to an x-y matrix of mutually orthogonal elongated coils 171.sub.1, 171.sub.2 . . . 171.sub.n and 272.sub.1, 272.sub.2 . . . 272.sub.m. Each of coils 171.sub.1, 171.sub.2 . . . 171.sub.n is elongated so it extends in the y direction while each of coils 272.sub.1, 272.sub.2 . . . 272.sub.m is elongated to extend in the x direction so coils 171.sub.1, 171.sub.2 . . . 171.sub.n are all parallel to each other and at right angles to coils 272.sub.1, 272.sub.2 . . . 272.sub.m, which are parallel to each other. At each intersection of coils 171.sub.1, 171.sub.2 . . . 171.sub.n and 272.sub.1, 272.sub.2 . . . 272.sub.m is a cylindrical magnetic core 273 so that at the intersection of coils 171.sub.1 and 272.sub.1, core 273.sub.11 is located, at the intersection of coils 171.sub.h and 272.sub.i, core 273.sub.hi is located and at the intersection of coils 171.sub.n and 272.sub.m is core 273.sub.nm, where h is selectively each of 1, 2 . . . n and i is selectively each of 1, 2 . . . m.

Core 273 at each intersection of coils 171 and 272 is part of a resonant circuit, constructed identically to the resonant circuit of FIG. 14 to include a fixed capacitor, a variable capacitor and a variable, voltage controlled resistance. The voltage controlled resistor is the source drain path of a field effect transistor having a gate electrode responsive to a weight setting voltage corresponding to the voltage at terminal 62. The resonant circuit at each intersection of coils 171 and 272 is resonant to the frequency of AC source 168 which drives coils 272.sub.1, 272.sub.2 . . . 272.sub.m via the outputs of AC amplifiers 148.sub.1, 148.sub.2 . . . 148.sub.m, respectively. The inputs of amplifiers 148.sub.1, 148.sub.2 . . . 148.sub.m are driven in parallel by AC source 168. The AC currents flowing in coils 272.sub.1, 272.sub.2 . . . 272.sub.m are controlled, inter alia, by the magnitude of the source drain resistances of transistors 163.sub.1, 163.sub.2 . . . 163.sub.m, respectively.

The voltage induced in winding 171.sub.1 by core 273.sub.11 is proportional to the current flowing in winding 272.sub.1 and the coupling coefficient of core 273.sub.11 between coils 272.sub.1 and 171.sub.1. The core coupling coefficient is determined by the value of the voltage controlled resistor in parallel with the coil corresponding with coil 255, FIG. 14. A similar relationship exists between the voltage induced in coils 171.sub.1, 171.sub.2 . . . 171.sub.n at each intersection of these coils with coils 272.sub.1, 272.sub.2 . . . 272.sub.m. Hence, the voltage induced in coil 171.sub.1 is equal to the sum of the voltages induced in the coil by coupling via cores 273.sub.11, 273.sub.12 . . . 273.sub.1m. In a similar manner, the sum of the voltages induced in each of coils 171.sub.1, 171.sub.2 . . . 171.sub.n is equal to the sum of the individual voltages induced in the particular coil at each intersection thereof with coils 272.sub.1, 272.sub.2 . . . 272.sub.m.

Coils 171.sub.1, 171.sub.2 . . . 171.sub.n have output terminals respectively connected to input terminals of amplifiers 172.sub.1, 172.sub.2 . . . 172.sub.n. Amplifiers 172.sub.1, 172.sub.2 . . . 172.sub.n derive output signals respectively applied to one input terminal of each of comparators 173.sub.1, 173.sub.2 . . . 173.sub.n, also having input terminals responsive to AC voltages having values represented by S.sub.1, S.sub.2 . . . S.sub.n. Comparators 173.sub.1, 173.sub.2 . . . 173.sub.n thereby derive output signals Z.sub.1, Z.sub.2 . . . Z.sub.n representing the difference between the AC voltages applied to the input terminals thereof. All AC voltages applied to the circuits of FIGS. 13a, 13b and 14 have the same frequency, equal to the resonant frequency of the magnetic core at each intersection in the matrices of FIGS. 13a and 13b.

As described above, according to the first aspect of the present invention, since the input terminal and the neuron or the pair of neurons are inductively coupled by a magnetic or electromagnetic field multiple input terminals and neurons or multiple neurons themselves can be spatially coupled by the mature technology to achieve overall parallel coupling with each other.

According to the second aspect, since a an inductive coupling device or circuit is utilized to couple between an input terminal and a neuron or between a pair of neurons through the magnetic, multiple input terminals and neurons and multiple neurons themselves can be spatially coupled with each other by the mature technology so as to provide a neural network having parallel couplings.

According to the third aspect, at least a pair of magnetically coupled inductance elements are used as a coupling device through magnetic field so as to facilitate circuit design of the neural network, and to realize a practical neural network.

According to the fourth aspect, since the inductance element is composed of a coil, coupling can be provided economically with less spacing between input terminals and neurons and among the neurons.

According to the fifth aspect, since a magnetic resistance element is used as the inductance element, coupling can be effected through DC magnetic fields as well as AC magnetic fields, thereby increasing design freedom.

According to a sixth aspect, since at least one resonance circuit is used as a coupling circuit through magnetic field, efficient spatial couplings can be constructed between input terminals and neurons and among neurons.

According to the seventh aspect, the neural network is comprised of a plurality of input terminals, a plurality of primary neurons each having an output, a secondary neuron having an input, a plurality of primary coils magnetically coupled to each other and each connected to each of the input terminals or each output of the primary neurons, and a secondary coil magnetically coupled to the primary coils and connected to the input of the secondary neuron; hence the parallel couplings can be constructed between the plural input terminals and the secondary neuron and between the plural primary neurons and the secondary neuron, thereby reducing overall neural network size.

According to the eighth aspect, the neural network is comprised of a plurality of input terminals, a plurality of primary neurons each having an output, multiple primary coils each being connected to each of the input terminals or each output of the primary neurons, multiple secondary coils each magnetically coupled to a set of the primary coils each selected from different groups of the primary coils corresponding to respective input terminals or respective outputs of the primary neurons, and a multiple of secondary neurons each having an input connected to each of the secondary coils; hence the parallel couplings can be constructed between the plural input terminals or the plural primary neurons and the multiple secondary neurons, thereby enabling further reduction of the overall size of the neural network circuit size.

According to the ninth aspect, since all or part of the coils are incorporated into a resonance circuit, the couplings can be constructed efficiently in parallel between the plural input terminals or the plural primary neurons and the single or multiple secondary neuron.

According to the tenth aspect, since an impedance element is connected in series with or parallel to the coil or resonance circuit so as to define the synaptic coupling weights, the couplings can be ensured between the plural input terminals or the plural primary neurons and the single or multiple secondary neuron to enable the secondary neuron to effect multiplication and summation operations. Consequently, there is realized a neural network having a smaller size and a greater number of neurons.

According to the eleventh aspect, since the synaptic coupling weight can be adjusted by changing the impedance of the impedance element, the multiplication and summation operations can be controllably carried out in single or multiple neurons. Consequently, there can be realized an efficient neural network having a smaller size and a greater number of neurons.