Simple Distance Neuron

The present invention relates to an artificial neuron using a simple distance calculation between the input signal vector and the synapse weight elements for providing an output signal.

A biological neuron is a single nerve cell responsive to stimuli through weighted inputs known as synapses. The weighted stimuli are summed and processed through a particular nonlinearity (sigmoid) associated with the neuron. The output of the neuron may be coupled to the synapses of other neurons forming an interconnection known as a neural network which possesses many desirable properties including the ability to learn and recognize information patterns in a parallel manner.

One conventional technique for processing the input signal vector through an artificial neuron involves forming a dot product combination between the elements of the input signal vector and the weight elements associated with the synapses. The dot product computation typically utilizes a plurality of multipliers coupled one each for receiving an element of the input signal vector and an element of the weight vector. The output signals of the plurality of multipliers are summed and processed through the sigmoid for providing the output signal of the neuron. A more detailed explanation of the dot product operation of conventional neurons is described in the detailed description of the prior art section.

One of the shortcomings of the aforedescribed neuron is the requirement for multiplier circuits which perform the dot product between the input signal vector and the weight vector. In analog form, multiplier circuits tend to be limited in accuracy and sensitive to process variation during manufacturing and temperature variation during operation. On the other hand, four quadrant digital multipliers are often large and complex taking significant portions of the available integrated circuit area, especially for neurons processing large input signal vectors since each element of the input signal vector is typically matched with a dedicated multiplier. Hence, it is desirable to remove multiplier circuits from neural networks as much as possible.

The nonlinear sigmoid function is provided for limiting the magnitude of the output signal should the sum of the products of the input signal vector and the weight elements become excessively large. While it is possible to limit the magnitude of the output signal by normalizing the input signal vector (dividing each element by a predetermined constant), a sigmoid function positioned at the output of the neuron has proven to be a simpler solution. Yet, it is still desirable to eliminate the sigmoid function to simplify the design and reduce the physical size of the neuron thereby allowing more neurons per unit area on the integrated circuit.

Hence, there is a need for an improved artificial neuron for processing an input signal vector through the synapse weight elements without the use of multiplier circuits for performing the dot product combination thereof and further without the sigmoid function for limiting the magnitude of the output signal of the neuron.

Briefly stated, the invention comprises an artificial neuron including a first circuit for subtracting a weight element from an input signal and providing a difference signal, and a second circuit for processing the difference signal through a weighting function of the first circuit for providing an output signal having a predetermined polarity, while a third circuit accumulates the output signal of the second circuit.

In another aspect the present invention is a neural network comprising a digital processing element including a first digital input bus coupled for receiving a digital input signal and a digital address bus coupled for receiving a digital address signal. A first circuit stores a plurality of digital weight elements and provides one of the plurality of digital weight elements at an output port according to the digital address signal. A second circuit is coupled for receiving the digital input signal and one of the plurality of digital weight elements for providing a difference signal as the difference of the digital input signal and one of the plurality of digital weight elements. A third circuit receives the difference signal and applies a weighting function to the difference signal for providing an output signal having a predetermined polarity, and a fourth circuit receives the output signal of the third circuit and accumulating the value thereof.

One of the advantages of the present invention is the combination of the input signal vector and the synapse weight elements with a simple distance calculation for providing the output signal of the neuron without the use of multiplier circuits and nonlinear sigmoid functions. A difference signal is developed by subtracting a weight element from an input signal. The difference signal is processed through a weighting function having a predetermined polarity and accumulated for providing the output signal of the neuron.

FIG. 1 is an illustration of a conventional neuron;

FIG. 2 is a simplified block diagram illustrating the function of the present invention;

FIG. 3 is a graph of the weighting functions useful in the explanation of the present invention;

FIG. 4 is a simplified block diagram illustrating a digital embodiment of the present invention;

FIG. 5 is a simplified block diagram illustrating an alternate digital embodiment of the present invention;

FIG. 6 is a simplified block diagram illustrating an analog embodiment of the present invention;

FIG. 7 is a schematic diagram illustrating further detail of an absolute value comparator; and

FIG. 8 is a graph use in the explanation of the absolute value comparator.

In FIG. 1 there is shown neuron 10 as a conventional element of a modern neural network. An input signal vector comprising elements I.sub.1, I.sub.2 and I.sub.3 is applied at inputs 12, 14 and 16 at the first inputs of four quadrant multipliers 18, 20 and 22, respectively. The second inputs of multipliers 18, 20 and 22 are coupled for receiving weight elements W.sub.1, W.sub.2 and W.sub.3, while the output signals of multipliers 18, 20 and 22 are summed through summing junction 24 and processed through sigmoid function 26 (.sigma.()) for providing output signal S.sub.28 of neuron 10 at output 28. The sigmoid function limits the magnitude of output signal S.sub.28 with increasing signal levels from summing junction 24. The operation of neuron 10 may be described mathematically as a dot product combination of the input signal vector I.sub.1 -I.sub.3 and the weight elements W.sub.1 -W.sub.3 as follows:

From Equation (1), output signal S.sub.28 is the dot product of the input signal vector I and the weight vector W processed through the sigmoid function. The same relationship is expanded in Equation (2) with the elements of the input signal vector I.sub.1 -I.sub.3 and the weight vector W.sub.1 -W.sub.3. A theorem from vector calculus shows that Equation (3) holds in that the output signal S.sub.28 is the product of the magnitude of the input signal vector I and the magnitude of the weight vector W times the cosine of the angle between the vectors processed through the sigmoid function. With the magnitude of vectors I and W normalized, the output signal S.sub.28 is a function of the angle .beta. between the vectors I and W and thus is a measure of their similarity.

Multiplier circuits 18, 20 and 22 are required for the dot product combination, i.e., I.sub.1 .times.W.sub.1 +I.sub.2 .times.W.sub.2 +I.sub.3 .times.W.sub.3. It is desirable for improving accuracy and reducing complexity and physical area to eliminate multiplier circuits 18-22 from the operation of neuron 10, especially for large input signal vectors. Furthermore, the dot product operation of neuron 10 requires either a normalization of the input signal vector I.sub.1 -I.sub.3 (not shown), or as is more common, a limitation on the output signal of summing junction 24 as provided by nonlinear sigmoid function 26 which clips the magnitude of the output signal should one or more elements of the input signal vector times the associated weight become excessively large. Neural networks often contain hundreds or even thousands of neurons for performing a useful function. Thus, by eliminating multipliers 18-22 and sigmoid function 26 and thereby reducing the complexity and area of the individual neuron, more neurons may be allocated per unit area. Accordingly, it is desirable to provid a neuron operating without multipliers like 18-22 and a nonlinear sigmoid function like 26.

Referring to FIG. 2, there is shown a functional block diagram of the present invention as neuron 30. An input signal vector I.sub.1, I.sub.2 and I.sub.3 is applied through inputs 32, 34 and 36 to the first inputs of subtracter circuits 38, 40 and 42. The second inputs of subtracter circuits 38, 40 and 42 are coupled for receiving weight elements W.sub.1, W.sub.2 and W.sub.3, respectively. The difference between elements I.sub.1, I.sub.2 and I.sub.3 of the input signal vector and the corresponding weight elements W.sub.1, W.sub.2 and W.sub.3 are processed through weighting functions 44, 46 and 48, respectively, and summed through summing junction 50 for providing an output signal V.sub.52 at output 52. The difference signal .DELTA..sub.1 provided at the output of subtracter circuit 38 is equal to I.sub.1 -W.sub.1, while the difference signal .DELTA..sub.2 at the output of subtracter circuit 40 is I.sub.2 -W.sub.2, and the difference signal .DELTA..sub.3 of subtracter circuit 42 is I.sub.3 -W.sub.3. Thus, output signal V.sub.52 may be mathematically described as:

where:

F.sub.44 () is weighting function 44

F.sub.46 () is weighting function 46

F.sub.48 () is weighting function 48

Weighting function 44 is graphically illustrated in FIG. 3 as four separate embodiments. Weighting functions 46 and 48 typically follow the same pattern as weighting function 44. In one embodiment, function F.sub.1 causes the square of the Euclidean distance to be computed by squaring the difference signal .DELTA..sub.1 provided at the output of subtracter circuit 38 (F.sub.1 =.DELTA..sup.2). Function F.sub.2 is simply the absolute value of the difference signal .DELTA..sub.1 (F.sub.2 =.vertline..DELTA..vertline.), and function F.sub.3 is the magnitude of the difference signal divided by an integer K (F.sub.3 =.vertline..DELTA..vertline./K), where K is typically a binary value in powers of two. Function F.sub.4 is the maximum of zero and the magnitude of the difference signal minus a constant C (F.sub.4 =Max(.vertline..DELTA..vertline.-C,0)), wherein the minimum value of the function is zero when the magnitude of the difference signal is less than C and increases in a linear positive direction as the difference signal exceeds the value of C as shown in FIG. 3.

Thus, the operation of weighting functions 44-48 provides a means of weighting the difference signals for statistical processing. Function F.sub.1 weights the difference signal by a squaring operation whereby larger difference signals produce greater output signals in a nonlinear fashion while smaller difference signals have much less of an impact. Functions F.sub.2 and F.sub.4 provide similar linear weighting for the difference signal, although function F.sub.4 requires a minimum value C of the difference signal before producing an output signal. When the input signal and the weight vector are very close, i.e. a small difference signal, the output signal V.sub.52 of neuron 30 should be small. Function F.sub.3 includes the advantages of minor contributions from low value difference signals while providing larger output signals of the weighting function in a stair-step manner as the difference signal increases.

The output signals of weighting functions 44, 46 and 48 are summed through summing junction 50 for providing the output signal V.sub.52 of neuron 30 at output 52. The operation of neuron 30 determines how close the weight vector W.sub.1 -W.sub.3 comes to matching the input signal vector I.sub.1 -I.sub.3. Thus, instead of trying to calculate the angle .beta. between the input signal vector I and the weight vector W as is done for dot product combinations, the present invention simply calculates a distance between the end points of the input signal vector I and the weight vector W. The nonlinear sigmoid function is not needed since the magnitude of the output signal is inherently limited by the subtraction process. A highly correlated input vector and weight vector produces a small value whereas a dot product type neuron produces a large value, thus, as the input vector and weight vector become more highly correlated, the output signal of neuron 30 tends toward smaller values while the dot product neuron of the prior art tends toward larger values. This change in polarity must be accounted for in network design with neuron 30 as is understood.

Referring to FIG. 4, there is shown a digital embodiment of the present invention as neuron 60 including weight memory 62 coupled for receiving a 3-bit address word A.sub.i applied at address bus 64 and providing an 8-bit digital weight value W.sub.i at the first input of subtracter circuit 66 in response to the address word A.sub.i, where i ranges from 1 to 8. Weight memory 62 includes a predetermined number of 8-bit memory locations, W.sub.1 -W.sub.8, and may be implemented as RAM, ROM, EPROM, EEPROM or any other suitable digital memory device. Weights W.sub.i are precalculated possibly from an external neural network computer simulation and written into weight memory 62 in a conventional manner through input bus 68. It is known that weight memory 62 can be divided into several logical areas allowing multiple sets of weights W.sub.i to be stored concurrently and accessed as needed with different input signal vectors. This allows physical neurons (excluding weight memory 62) to operate on several possibly distinct neural applications in a time multiplexed fashion thereby increasing the overall operational speed by avoiding excessive swapping of weight W.sub.i sets for combination with different input signal vectors.

Continuing with FIG. 4, the second input of subtracter 66 is coupled for receiving a digital input signal I.sub.i applied at input bus 70, whereby an 8-bit difference signal .DELTA..sub.i =I.sub.i - W.sub.i is formed at the output of subtracter circuit 66 and applied as an address into lookup table 72 for performing any one of the weighting functions shown in FIG. 3. For example, squaring function F.sub.1 is achieved by storing the algebraic square of the difference signal .DELTA..sub.i in the memory locations of lookup table 72 according to the address value .DELTA..sub.i. The difference signal .DELTA..sub.i may be treated as a fractional binary, i.e., "0010.0011", wherein values greater than "0001.000" become larger through the squaring function and values smaller than "0001.0000" become smaller when squared. If .DELTA..sub.i ="0010.0000", then address "00100000" of lookup table 72 would contain the value "00000100.00000000". Alternately, If .DELTA..sub.i ="0000.0011" , then address "00000011" of lookup table 72 would contain the value "00000000.00001001". The size of the RAM is dependent upon the desired precision as is understood. For example, it is possible to discard lesser significant bits at the extremes of the binary point at the cost of precision. The technique of using a lookup table for processing a signal through an algebraic function, such as functions F.sub.1 -F.sub.4 of FIG. 3, is well known in the art. Lookup table 72 may be implemented as RAM, ROM, EPROM, EEPROM or any other suitable digital memory device and sized as need for the application. The output signal of lookup table 72 is accumulated in accumulator 74 for providing the output signal of neuron 60 at output 76.

The digital embodiment of FIG. 4 performs a Euclidian distance calculation (function F.sub.1) on the digital input signal series I.sub.1 -I.sub.8 and weights W.sub.1 -W.sub.8 yielding a 16-bit value V.sub.76 in accumulator 74 as follows:

For one computation series according to equation (5), accumulator 74 is zeroed after which a first 8-bit digital input signal, I.sub.1, is applied at the second input of subtracter 66 via input bus 70, while an address word, A.sub.1, is applied at the address input of weight memory 62 by way of address bus 64. The value of address word A.sub.1 retrieves the 8-bit weight W.sub.1 from weight memory 62 and applies the same to the first input of subtracter 66. The difference of digital input signal I.sub.1 and weight W.sub.1 (.DELTA..sub.1 =I.sub.1 -W.sub.1) becomes an address into lookup table 72 for performing a squaring operation thereon. The square of the difference signal (.DELTA..sub.1).sup.2 is stored in accumulator 74. A second digital input signal, I.sub.2, is then applied at the second input of subtracter 66 while a second address word, A.sub.2, retrieves weight W.sub.2 and applies the 8-bit value thereof to the first input of subtracter 66. The difference signal .DELTA..sub.2 from the digital input signal I.sub.2 and weight W.sub.2 is also squared in lookup table 72 and summed with the prior contents of accumulator 74. The process continues for third, fourth, fifth, sixth, seventh and eighth combination of digital input signals I.sub.3 -I.sub.8 and weights W.sub.3 -W.sub.8 according to equation (5) leaving the 16-bit value V.sub.76 in accumulator 74 as the output signal of neuron 60. Again, the weighting function stored in lookup table 72 may comprise any one of the functions F.sub.1 -F.sub.4 disclosed in FIG. 3 and others. Furthermore, the invention supports practically any size input signal vector I and any number of weight elements W in weight memory 62.

An alternate digital embodiment for performing the weighting function F.sub.2 is disclosed in FIG. 5 as neuron 80 including weight memory 82 coupled for receiving the 3-bit address word A.sub.i applied at address bus 84 and providing an 8-bit digital weight value W.sub.i at the first input of subtracter 86 in response to the address word A.sub.i. Weight memory 82 includes a predetermined number of 8-bit memory locations, W.sub.1 -W.sub.8, and may be implemented as RAM, ROM, EPROM, EEPROM or any other suitable digital memory device. Weights W.sub.i are written into weight memory 82 in a conventional manner through input bus 88. The second input of subtracter 86 is coupled for receiving a digital input signal I.sub.i applied at input bus 90, whereby the difference signal .DELTA..sub.i =I.sub.i -W.sub.i is formed at the output of subtracter circuit 86 and applied at the first input of 16-to-8 multiplexer 92. The 8-bit difference signal of subtracter 86 is also complemented through inverter 94 and applied to the second input of multiplexer 92, while the control input of multiplexer 92 is coupled for receiving the sign bit of the difference signal of subtracter 86. The output signal of multiplexer 92 is accumulated with the sign bit of the difference signal in accumulator 96 for providing the output signal of neuron 80 at output 98.

The digital embodiment of FIG. 5 performs a twos complement absolute value combination of the digital input signal series I.sub.1 -I.sub.8 with weights W.sub.1 -W.sub.8 yielding a 16-bit value V.sub.98 in accumulator 98 as follows:

For one computation series according to equation (6), accumulator 96 is zeroed after which a first 8-bit digital input signal, I.sub.1, is applied at the second input of subtracter 86 via input bus 90, while an address word, A.sub.1, is applied at the address input of weight memory 82 by way of address bus 84. The value of address word A.sub.1 retrieves the 8-bit weight W.sub.1 from weight memory 82 and applies the same to the first input of subtracter 86. The difference signal .DELTA..sub.1 and its complement .DELTA..sub.1 are applied at the first and second inputs of multiplexer 92, respectively, while the sign bit of the difference signal .DELTA..sub.1 is applied to the control input of the same. If the sign bit is logic one indicating that the difference signal is negative, the complemented difference signal .DELTA..sub.1 is passed through multiplexer 92 yielding a positive value at the output thereof by inverting the negative signal. If the sign bit is logic zero, the difference signal .DELTA..sub.1 is selected. Hence, multiplexer 92 performs an absolute value on the difference signal in accordance with function F.sub.2 of FIG. 3. The output signal of multiplexer 92 is summed in accumulator 96 with the sign bit of the difference signal for providing a carry-in to the least significant bit of the accumulation.

A second digital input signal, I.sub.2, is applied at the second input of subtracter 86 as a second address word, A.sub.2, retrieves weight W.sub.2 and applies the 8-bit value thereof to the first input of subtracter 86. The sign bit of the difference signal .DELTA..sub.2 =I.sub.2 -W.sub.2 again selects either the difference signal .DELTA..sub.2 or its complement .DELTA..sub.2 for providing an absolute value at the output of multiplexer 92 for accumulating with the sign bit and the prior contents of accumulator 96. The sign bit provides a carry-in to the least significant bit of the accumulation. The process continues for third, fourth, fifth, sixth, seventh and eighth combination of digital input signals I.sub.3 -I.sub.8 and weights W.sub.3 -W.sub.8 according to equation (6) leaving the 16-bit value V.sub.98 as the output signal of neuron 80 in accumulator 96.

Turning to FIG. 6, there is shown an analog embodiment of the present invention as neuron 100 including comparators 102, 104 and 106 for providing the absolute value of the difference between respective elements of the input signal vector and the weight signals and summing the results in a parallel manner. Thus, the analog embodiment of FIG. 6 is analogous to the digital embodiment of FIG. 5. Accordingly, the first input of absolute value comparator 102 is coupled for receiving input signal V.sub.IN1 applied at input 108 and the second input of absolute value comparator 102 is responsive to weight signal V.sub.W1 applied at input 110 and stored across capacitor 112. Another technique of supplying the weight signal to absolute value comparator 102 may involve storing the weight value in a digital EEPROM and converting the digital output signal thereof via a digital-to-analog converter (not shown) to an appropriate analog signal. Thus, it is understood that the source of the analog weight signal may take a variety of forms. The output signal of absolute value comparator 102 is current I.sub.102 which is approximately proportional to the absolute value of the difference between input signal V.sub.IN1 and weight signal V.sub.W1. The proportionality is determined by the bias signal V.sub.BIAS applied through input 114 to absolute value comparators 102-106. Likewise, the first input of absolute value comparator 104 is coupled for receiving input signal V.sub.IN2 applied at input 116 while the second input of the same is coupled for receiving weight signal V.sub.W2 applied at input 118 and stored across capacitor 120. The output signal of absolute value comparator 104 is current I.sub.104 approximately proportional to the absolute value of the difference between input signal V.sub.IN2 and weight signal V.sub.W2. The first input of absolute value comparator 106 is coupled for receiving input signal V.sub.IN3 applied at input 122 and the second input is coupled for receiving weight signal V.sub.W3 applied at input 124 and stored across capacitor 126. The output signal of absolute value comparator 106 is current I.sub.106 approximately proportional to the absolute value of the difference between input signal V.sub.IN3 and weight signal V.sub.W3. The output signal of neuron 100 is approximately equal to the sum of currents I.sub.102 +I.sub.104 +I.sub.106 and may be described as follows: ##EQU1## where: tanh () is the hyperbolic tangent function

K is a lumped constant related to Boltzmann's constant with units of volts.sup.-1

More generally, the output signal of neuron 100 is presented in Equation (8) where the integer n is the number of elements of the input signal vector. ##EQU2##

A more detailed description of absolute value comparator 102 is shown in FIG. 7 where components having a similar function are assigned the same reference numbers used in FIG. 6. Absolute value comparators 104 and 106 follow a similar construction as shown for comparator 102. The input signal V.sub.IN1 is applied at the gate of transistor 130 while the weight signal V.sub.W1 is applied at the gate of transistor 132. The drains of transistors 132 and 130 are respectively coupled to the input and output of a current mirror formed of transistors 134 and 136. The gates of transistors 134 and 136 are coupled together to the drain of transistor 132, while the sources of the same are coupled to power supply conductor 138 typically operating at a positive potential such as V.sub.DD. The sources of transistors 130 and 132 are coupled together for sinking a tail current I.sub.BIAS through transistor 140 in response to the bias voltage V.sub.BIAS applied at the gate of the latter. The source of transistor 140 is coupled to power supply conductor 142 typically operating at ground potential. Thus, transistors 130-140 form a differential amplifier, wherein the input signal V.sub.IN1 is applied at the inverting input and the weight signal V.sub.W1 is applied at the non-inverting input thereof.

Continuing with FIG. 7, the input signal V.sub.IN1 is also applied at the gate of transistor 146 and the weight signal V.sub.W1 is applied at the gate of transistor 148. The drains of transistors 146 and 148 are respectively coupled to the input and output of a current mirror formed of transistors 150 and 152. The gates of transistors 150 and 152 are coupled together to the drain of transistor 146, while the sources of the same are coupled to power supply conductor 138. The sources of transistors 146 and 148 are coupled together for sinking a tail current I.sub.BIAS through transistor 154 in response to the bias voltage V.sub.BIAS applied at its gate. The source of transistor 154 is coupled to power supply conductor 142. Transistors 146-154 also form a differential amplifier although the input signal V.sub.IN1 is now applied at the non-inverting input while the weight signal V.sub.W1 is applied at its inverting input.

The output signal of the differential amplifier formed of transistors 130-140 is provided at the drain of transistor 130 which is coupled to the input of a current mirror formed of transistors 158 and 160. The gates of transistors 158 and 160 are coupled together to the drain of transistor 130, while the sources are coupled to power supply conductor 138. The output current I.sub.160 of current mirror circuit 158-160 flows through the output of comparator 102 at output 162. Likewise, the output signal of the differential amplifier formed of transistors 146-154 is provided at the drain of transistor 148 which is coupled to the input of a current mirror formed of transistors 164 and 166. The gates of transistors 164 and 166 are coupled together to the drain of transistor 148, while the sources are coupled to power supply conductor 138. The output current I.sub.166 of current mirror circuit 164-166 also flows through the output of comparator 102 at output 162 such that current I.sub.102 is equal the sum of current I.sub.160 and I.sub.166.

The operation of the differential amplifier circuits like 130-140 and 146-154 is believed to be well known in the art. Briefly, the output signal of differential amplifier circuit 130-140 becomes more positive as the weight signal V.sub.W1 exceeds the input signal V.sub.IN1. Conversely, the output signal of differential amplifier circuit 130-140 becomes more negative as the input signal V.sub.IN1 exceeds the weight signal V.sub.W1. The opposite is true for the differential amplifier circuit 146-154, wherein the output signal becomes more negative as the weight signal V.sub.W1 exceeds the input signal V.sub.IN1 and more positive as the input signal V.sub.IN1 exceeds the weight signal V.sub.W1. The output signals of the differential amplifier circuits are converted to currents through current mirror circuits 158-160 and 164-166 for providing current I.sub.102 flowing through the output of absolute value comparator 102.

The relationship is graphically illustrated in FIG. 8, wherein the current I.sub.102 has a positive value and negative slope when the difference V.sub.IN1 -V.sub.W1 is negative and a positive value and positive slope when the difference V.sub.IN1 -V.sub.W1 is positive. When the weight signal V.sub.W1 is more positive than the input signal V.sub.IN1 (negative difference), transistors 132 and 134 sink the majority of the tail current I.sub.BIAS flowing through transistor 140 and reflect the same through transistor 136 thereby increasing the potential at the drain of transistor 130 and disabling current mirror circuit 158-160 such that no current I.sub.160 flows. At the same time, the more positive weight signal V.sub.W1 causes transistor 148 to conduct the majority of the tail current I.sub.BIAS flowing through transistor 154 such that less current flows into the input of current mirror circuit 150-152. The potential at the drain of transistor 148 falls, enabling current mirror circuit 164-166 for providing positive current I.sub.166 as current I.sub.102. Conversely, when the input signal V.sub.IN1 is more positive than the weight signal V.sub.W1 (positive difference), transistors 146 and 150 sink the majority of the tail current I.sub.BIAS and reflect the same through transistor 152 thereby increasing the potential at the drain of transistor 148 and disabling current mirror circuit 134-136 such that current I.sub.166 is zero. The less positive weight signal V.sub.W1 causes transistor 130 to conduct the majority of the tail current I.sub.BIAS flowing through transistor 140 whereby less current flows into the input of current mirror circuit 134-136. The potential as the drain of transistor 130 falls enabling current mirror circuit 158-160 which provides positive current I.sub.160 as current I.sub.102.

As illustrated in FIG. 8, the output signal of comparator 102 approximates an absolute value function in the linear region of the curve such as function F.sub.2 in FIG. 3. Furthermore, it should be appreciated that the simple distance neuron of the present invention may be implemented in either digital or analog form as examples of both embodiments have been demonstrated.

Hence, what has been provided is a novel neuron using a simple distance calculation between the input signal vector and the synapse weight elements for providing an output signal. A difference signal is developed by subtracting a weight signal from an input signal for providing a difference signal which is processed through weighting function having a predetermined polarity and accumulated for providing the output signal of the neuron.