Analysis Arrangement Based on a Model of Human Neural Responses

The invention relates to signal processing and more particularly to processing arrangements for forming signals representative of sensory information based on a model of human neural responses.

Many different types of processing arrangements have been devised to analyze sensory information. With respect to sensory signals derived from sounds such as speech, some processing systems extract specific features such as pitch, formants, or linear predictive parameters to detect, recognize, enhance or synthesize the speech or sounds. Other systems are adapted to form frequency spectra directly from the speech wave. It is generally agreed that the human heating apparatus does not process speech waves in these or similar ways and that human perception of speech for recognition or other purposes is superior to such automatic processing systems.

Little is known about the processing principles in the brain stem, auditory nuclei and the auditory cortex. It is well recognized, however, that sound waves entering the ear cause hair cells in the cochlea to vibrate, and that the sound waves are represented at the cochlear nucleus solely by the auditory nerve firing patterns caused by the hair cells in the cochlea. Such knowledge has been utilized as described for example in U.S. Pat. No. 4,532,930 issued to Peter A. Crosby et al., on Aug. 6, 1985 to provide auditory prosthesis for profoundly deaf persons. It is further known that human understanding of speech in the presence of noise is very good in comparison to automated recognition arrangements whose performance deteriorates rapidly as the noise level increases. Consequently, it has been suggested in the article "Recognition system processes speech the way the ear does" by J. R. Lineback appearing in Electronics, vol. 57, No. 3, Feb. 9, 1984, pp. 45-46 and elsewhere, that speech analysis may be modeled on the auditory nerve firing patterns of the human hearing apparatus.

U.S. Pat. No. 4,536,844 issued to Richard F. Lyon, Aug. 20, 1985, discloses a method and apparatus for simulating aural response information which are based on a model of the human hearing system and the inner ear and wherein the aural response is expressed as signal processing operations that map acoustic signals into neural representations. Accordingly, the human ear is simulated by a high order transfer function modeled as a cascade/parallel filter bank network of simple linear, time invariant filter sections with signal transduction and compression based on half-wave rectification with a nonlinearly coupled variable time constant automatic gain control network. These processing arrangements, however, do not correspond to the nerve firing patterns characteristic of aural response.

U.S. Pat. No. 4,075,423 issued to M. J. Martin et al. on Feb. 21, 1978 disclosed sound analyzing apparatus for extracting basic formant waveforms present in a speech signal, and examining the format waveforms to identify the frequency components thereof using a histogram of the frequency patterns of detected waveform peaks developed over successive sampling periods in a digital processor. The Martin et al arrangement, however, is limited to forming a particular set of acoustic features, i.e., formants but does not address the problem of utilizing the information available in the time differences of level crossings to characterize the acoustic wave more fully than the generation of the few formants there disclosed. In particular, the Martin et al arrangement treats each of the frequency sub-band components of the acoustic wave completely separately. Others have employed techniques somewhat similar to the techniques of the Martin et al patent and have also limited their analysis to formant extraction. See the article by Russell J. Niederjohn et al, "A Zero-Crossing Consistency Method for Format Tracking of Voiced Speech in High Noise Levels", IEEE Transactions on Acoustics, Speech and Signal Processing, vol. ASSP-33, No. 2, Apr. 1985, the article by M. Elghonemy et al, "An Iterative Method for Formant Extraction Using Zero-Crossing Interval Histograms" Melecon '85, vol. II, Digital Signal Processing, A. Luque et al (eds.) Elsevier Science Publishers B. V. (North-Holland) 1985, and the article of one of us, O. Ghitza, "A Measure of In-Synchrony Regions in the Auditory Nerve Firing Patterns as a Basis for Speech Vocoding", International Conference, Acoustics, Speech and Signal Processing, '85, Tampa, Fla., Mar. 26-29, 1985. In the latter article the analysis is advanced, with respect to the different frequency subband components of the acoustic wave, by a nonlinear combination thereof which picks "dominant frequencies" when present in at least 6 adjacent bands and suppresses other distributional information regarding the crossing time differences. We now believe that process causes the loss of valuable information regarding the input bandlimited signal, and that an analysis (a multiplicative nonlinear process) as employed in the article by the other of us, J. B. Allen, "Cochlear Modeling", IEEE ASSP Magazine, January, 1985 has disadvantages in characterizing the input bandlimited signal. It is an object of the invention to provide improved spectral representation of the neural response to sensory patterns that simulates the operation of biological organs and to adapt the technique to processing of bandlimited signals generally.

The foregoing object is achieved by performing a timing synchrony analysis on a sensory pattern in which the spectrum of the sensory pattern is divided into spectral portions and the spectral distribution of neural response to the sensory pattern waveform is obtained using multilevel neural response thresholds. Nerve firing patterns are detected and the spectral distribution of the counts of nerve firings of the individual spectral portions are combined to form a spectral representation corresponding to the operation of the sensory organ. For sound patterns, multilevel sound intensity thresholds are established and crossings of the plurality of sound intensity thresholds by the spectral portion waveforms are counted to produce a neural response histogram. The spectral portion histograms are combined to produce an auditory spectral representation of the input sound pattern.

The invention is directed to a sensory type pattern analysis arrangement in which a plurality of neural response intensity levels is defined. The frequency spectrum of a received sensory type pattern is divided into a plurality of spectral portions by filters each having a prescribed spectral response. The output of each filter is partitioned into successive time segments. Responsive to the output of each filter in the present time segment, a set of signals is generated which represent a histogram of the inverse time intervals between crossings of each of the neural response intensity levels by the filter output as a function of frequency for the present time segment. The inverse interval histogram signals from the filters for the present time segment are combined to produce a signal corresponding to the spectral distribution of the neural responses to the time segment waveform of the sensory pattern. Autocorrelation signals for the time segment formed from the neural response spectral distribution signals permit accurate speech recognition in high noise environments.

FIG. 1 depicts a general block diagram of an arrangement illustrative of the invention which produces spectral representations based on auditory neural patterns responsive to sounds;

FIGS. 2, 3, 4 and 13 show flow charts illustrating the operation of the arrangement of FIG. 1;

FIGS. 5 and 6 depict signal processing circuits useful in the arrangement of FIG. 1;

FIG. 7 show waveforms illustrating the operation of the partial interval histogram processors of FIG. 1;

FIG. 8 show waveforms illustrating the spectral representations obtained from the arrangement of FIG. 1;

FIG. 9 shows waveforms illustrating the spectral portion filtering in the arrangement of FIG. 1;

FIG. 10 shows curves illustrating time segment arrangements in the circuit of FIG. 1;

FIG. 11 illustrates diagrammatically the operation of one of the partial interval histogram processors;

FIG. 12 illustrates diagrammatically the operation of a plurality of partial interval histogram and ensemble histogram processors of the circuit of FIG. 1.

FIG. 1 depicts a general block diagram of an arrangement adapted to analyze sensory information by partitioning an input signal into a plurality of spectral portions, detecting occurrences of particular events in each spectral portion i.e., crossings of sensory thresholds, and combining event information i.e., counts of intervals between sensory threshold crossings for evaluation. While FIG. 1 is described in terms of a speech analyzer, it should be understood that it may be used for the spectral analysis of visual or other sensor like signals. The circuit of FIG. 1 produces a frequency domain representation of an input sound measured from firing patterns generated by a simulated nerve fiber array and simulates the temporal characteristics of the information in the auditory nerve fiber firing patterns by transforming the frequency domain representation into autocorrelation signals for use in speech processing. As a result, the information obtained therefrom corresponds to that derived from the human hearing mechanism rather than that obtained by a direct analysis of a signal from an electroacoustic transducer. Priorly known human hearing simulation arrangements are based on a single auditory nerve threshold level and produce only limited auditory feature information. The simulation circuit according to the invention utilizes a plurality of auditory nerve threshold levels to provide much better resolution of the auditory response.

The model of human hearing used for the circuit of FIG. 1 comprises a section representing the peripheral auditory system up to the auditory nerve level. This section simulates the mechanical motion at ever point along the basilar membrane as the output of a narrow band-pass filter with frequency response produced by the mechanical tuning characteristics at that place as described in the article "Cochlear Modeling" by J. B. Allen appearing in the IEEE ASSP Magazine, January 1985, page 3. The shearing motion between the basilar membrane and the sectorial membrane is sensed by the cilia of the inner hair cell and transduced, in a highly nonlinear manner, to the primary nerve fibers attached to the cell. Each of these fibers is characterized by its threshold level and its spontaneous rate as disclosed in the article "Auditory-Nerve Response from Cats Raised in a Low-Noise Chamber" by M. C. Liberman appearing in Journal of the Acoustical Society of America, vol. 63, 1978, pp. 442-455. The mapping of places along the basilar membrane to frequency is approximately logarithmic, and the distribution of the inner hair cells along the membrane is uniform.

The filtering section may be represented by a plurality of filters each having a prescribed response corresponding to the cochlea. A set of 85 such cochlear filters equally spaced on a log-frequency scale from 0 Hz to 3200 Hz may be used. It is to be understood, however, that other filter characteristics may be used depending on the intended use of the analyzer. The nerve fiber firing mechanism is simulated, according to the invention, by a multilevel crossing detector at the output of each cochlear filter. In contrast to other arrangements which assume a single nerve fiber at each point in the basilar membrane, the arrangement according to the invention is in accordance with a multifiber model in which each fiber fires at a different sound intensity threshold. We have found that the multilevel arrangement corresponds more closely to the physiology of hearing and provides improved spectral representation in the presence of noise. The level crossings measured at threshold levels corresponding to predetermined sound intensities are uniformly distributed in a log scale over the dynamic range of the signal. While positive going threshold levels are used in embodiment described herein and positive going crossings of the threshold levels are measured, it is to be understood that other threshold and crossing arrangements may be used. The ensemble of the multilevel crossing intervals corresponding to the firing activity at the auditory nerve fiber-array. The interval between each successive pair of same direction, e.g., positive going, crossings of each predetermined sound intensity level is determined and a count of the inverse of these interspike intervals of the multilevel detectors for each spectral portion is stored as a function frequency. The resulting histogram of the ensemble of inverse interspike intervals forms a spectral pattern that is representative of the spectral distribution of the auditory neural response to the input sound. Advantageously, the ensemble histogram pattern is relatively insensitive to noise compared to priorly known Fast Fourier Transform derived spectra. The auditory neural response is the firing pattern of the ensemble of primary fibers in the auditory nerve.

FIG. 1, sound waves such as speech are converted into an electrical signal s(t) by a transducer 101 which may be a microphone. Signal s(t) is sampled at a prescribed rate, e.g., 40 Ksamples/sec., and the successive samples are converted in digital representations thereof in signal converter 103. The digitally coded signal is applied to filter processor circuit 105. The filter processor which may comprise a processor arrangement incorporating for example the type MC68020 microprocessor or the type TMS 32020 digital signal processor is operative to partition the digitally coded sequence corresponding to signal s(t) into a plurality of prescribed spectral potion signals s.sub.1, s.sub.2, . . . s.sub.i, . . . s.sub.I by means of spectral filtering well known in the art. Each spectral portion may have the prescribed characteristic of a cochlear filter as aforementioned. Alternatively, each spectral portion may have a Hamming window type or other type characteristic well known in the art. Waveforms 905-1 through 905-I of FIG. 9 show the spectral characteristics of the passbands of such a set of cochlear filter characteristics, and waveforms 910-1 through 910-I illustrate the spectral response of a set of overlapping Hamming window type filters.

The spectral portions defined in filter processor 105 generally have a dominant frequency range that is relatively narrow. As a result, the spectral portion signal in the time domain comprises a sinewave type signal having relatively slowly changing peaks. The spectral portion signals may also be generated by applying the output of transducer 101 to a plurality of analog filters each having a prescribed spectral response. The spectral portion from each filter is then applied to a digital converter circuit operative to same the filter output at a prescribed rate and to transform the sampled filter output into a sequence of digital codes. The spectral portion digital codes from the converter circuits then corresponds to prescribed spectral portion signals s.sub.1, s.sub.2, . . . s.sub.i, . . . s.sub.I.

The time domain digital signal sequence for prescribed spectral portion s.sub.1 is applied to partial interval histogram processor 110-1. Similarly, prescribed spectral portions s.sub.2, . . . s.sub.i, . . . s.sub.I are supplied to partial interval histogram processors 110-2 through 110-I, respectively. Each partial interval histogram processor is operative to detect the time intervals between crossings of the sound intensity levels by the spectral portion waveform as illustrated in FIG. 7 and to store the counts of the inverse time intervals as a function of frequency. Referring to FIG. 7, waveform 720 represents a time segment of the output in analog form from signal converter 107-1. A prescribed time segment, e.g., 40 milliseconds, is selected for all partial interval histogram processors although as will be explained the time segment may be further limited to a particular number of detected time intervals, e.g., 20. Waveforms 701-1 through 701-7 are a succession of positive threshold levels scaled logarithmically as indicated in FIG. 7.

Processor 110-1 is adapted to detect same direction, i.e., positive going crossings, of the same sound intensity level by the spectral portion waveform within the prescribed time segment TS and to generate signals each representing the inverse of the time interval of each successive pair of positive going sound intensity level crossings. The analysis time segment TS starts at the present time t.sub.0 and extends into the past (right to left) until time t.sub.f. Waveform 720 is a typical analog representation of the input spectral portion waveform to partial interval histogram processor 110-1. Waveform 720, while positive going, crosses level 701-1 at time t.sub.1, t.sub.11, t.sub.21, and t.sub.31 going right to left. These positive going crossings are detected and a signal corresponding to the inverse interval between each pair of successive crossings is obtained. With respect to level 701-1 in FIG. 7, indications of the inverse intervals 1/(t.sub.11 -t.sub.1), 1/(t.sub.21 -t.sub.11) and 1/(t.sub.31 -t.sub.21) are recorded in a histogram store having bins or storage cells arranged according to inverse interval frequency. In similar fashion, inverse intervals 1/(t.sub.12 -t.sub.2), 1/(t.sub.22 -t.sub.12), and 1/(t.sub.32 -t.sub.22) are formed for level 701-2, inverse intervals 1/(t.sub.13 -t.sub.3), 1/(t.sub.23 -t.sub.13), and 1/(t.sub.33 -t.sub.23) for level 701-3, and inverse intervals 1/(t.sub.14 -t.sub.4), 1/(t.sub.24 -t.sub.14), and 1/(t.sub.34 -t.sub.24) for level 704-4. With respect to level 701-5, only inverse interval 1/(t.sub.35 -t.sub.5) is generated. No inverse time intervals are obtained for level 701-6 since there is only one crossing of this level at time t.sub.6.

Counts of the inverse intervals are stored in the histogram bins which are memory locations arranged according to a frequency scale. The first bin may correspond to a frequency range between 0 and 32 Hz. The nex bin then corresponds to the frequency range .DELTA. of 32 Hz-64 Hz. Other bins are arranged in like manner to cover the frequency spectrum of interest e.g., 0-3200 Hz. Assume for purposes of illustration that inverse intervals 1/(t.sub.11 -t.sub.1), 1/(t.sub.21 -t.sub.11) 1/(t.sub.31 -t.sub.21) 1/(t.sub.12 -t.sub.2), 1/(t.sub.22 -t.sub.12), 1/(t.sub.32 -t.sub.22) 1/(t.sub.13 -t.sub.3), 1/(t.sub.23 -t.sub.13), and 1/(t.sub.33 -t.sub.23) are all in the frequency range of a single bin. According to the invention, that bin will store the number of inverse time intervals within its range, i.e., 9, obtained in the time segment TS being analyzed. Inverse intervals 1/(t.sub.14 -t.sub.4), 1/(t.sub.24 -t.sub.14), and 1/(t.sub.34 -t.sub.24) for level 704-4 may fall within the range of an adjacent bin so that the count in the adjacent bin for time segment TS would be 3. The inverse time interval 1/(t.sub.35 -t.sub.5) of course falls within a completely different frequency range and a count of 1 would be stored in the bin corresponding to that frequency range.

The bin counts are representative to the synchrony in the neural firing pattern of the cochlea. The use of a plurality of logarithmically related sound intensity levels accounts for the intensity of the input signal in a particular frequency range. Thus, a signal of a particular frequency having high intensity peaks results in a much larger count in the bin covering that frequency than a low intensity signal of the same frequency. The counts are independent of the spectral portion source in which they occurred. Priorly known histogram analysis arrangements utilize a single crossing count or peak counts so that variations in intensity are not readily detectable. In accordance with the invention, multiple level histograms of the type described herein readily indicate the intensity levels of the nerve firing spectral distribution and cancel noise effects in the individual intensity level histograms.

As is well known in the art, the use of a predetermined time segment for signal analysis tends to average the data obtained over the time segment. While a time segment of 40 milliseconds is appropriate for the analysis of low frequency spectral portions, it may not be appropriate for signal components in the high frequency spectral portions. A different time segment may be used for each spectral portion so that an appropriate time scale may be obtained for each spectral range. In the partial interval histogram circuit of FIG. 1, the time segment is made appropriate for each spectral range by using overlapping segments of TS duration. For example, the time segment duration for the analysis may be nominally 40 milliseconds while each analysis occurs every 5 milliseconds. The nominal TS segment is changed so that there is a maximum number of counts permitted in each bin of the histogram store. Consequently, a high count for a bin in effect shortens the time segment TS for that bin. Higher counts are expected for the higher frequency components of the input signal where the signal makes more level crossings within a given time. The time segment for such higher components is relatively short compared to the time segments for lower frequency components. Thus, the time resolution for the higher frequency components is made finer than for lower frequency components.

FIG. 10 illustrates the variable time interval arrangement. Line 1001 represents the time axis and time segement t.sub.O -t.sub.f is marked as a sampling time period at which the analysis is performed. Line 1005 represents the frequency axis along which are a low frequency limit, e.g., 200 Hz and a high frequency limit, e.g., 3200 Hz. An analysis time segment TS, e.g., 40 milliseconds, shown by line 1010-1 is used at the low frequency limit at which a maximum inverse interval count of 20 cannot be expected. Similarly, a 40 millisecond analysis interval is used at somewhat higher frequencies as indicated by line 1010-2 and 1010-3. Line 1010-4, however, is at a frequency where a count of 20 results in a shorter interval than TS=40 milliseconds. In the highest frequency ranges, the count of 20 occurs within a much shorter analysis window as indicated by lines 1010-)(I-2) and 1010-I. The resulting analysis window is indicated by curve 1015 which is of 40 milliseconds duration at low frequencies and decreases at higher frequencies. Thus, a long analysis window TS takes into account the effects of low frequency components while the shorter window obtained by limiting the count of inverse time intervals permits accurate analysis of high frequency changes.

As an illustration of the partial interval histogram operation, consider an input signal of the form

applied to a cochlear type filter of FIG. 9 having a center frequency

For a given intensity A, the output signal of the cochlear filter will provide only some sound intensity level-crossings. For a given level, the time interval between two successive up going level crossings is generally 1/f.sub.0 and the inverse of this time interval is f.sub.0. Since a histogram of the inverse of the intervals is generated, this interval between a pair of positive going crossings contributes one count to the f.sub.0 bin of the histogram. For the illustrative input signal of frequency f.sub.0, all the intervals are identical. This results in a histogram which is zero everywhere, except for the bin corresponding to f.sub.0. As the amplitude A of the input signal increases, there are crossings of higher value sound intensity levels, whereby this cochlear filter contributes more counts to the f.sub.0 bin of the partial interval histogram processor. For sound intensity crossing levels equally distributed on a log amplitude scale, the partial interval histogram is related to the dB scale.

The filters whose characteristics are shown in FIGS. 9 are overlapping so that more than one partial ensemble histogram processor contributes to the f.sub.0 bin. In fact, all the cochlear filters which produce

will contribute to the f.sub.0 bin of the EIH, provided that A.vertline.H.sub.i (f.sub.0).vertline. exceeds any of the level crossing thresholds. Consequently, there are several spectral portion sources contributing counts to the f.sub.0 bin in a nonlinear manner. The resulting inverse interval histogram obtained by combining the outputs of the partial interval histograms, e.g., by summation, corresponds to the extent of the neural response of the cochlea.

FIG. 11 illustrates diagrammatically the operation of one of the partial interval histogram processors responsive to a sinewave input

within the passband of its associated filter. Box 1101 illustrates the level detector arrangement of a partial interval histogram processor (PIH) such as 110-1 in FIG. 1 and shows logarithmically related sound intensity threshold levels 1103-1 through 1103-7 which are incorporated in the PIH processor. The outputs of the level detector arrangement illustrated in box 1101 are applied to partial interval histogram level stores corresponding to the amplitude vs. frequency plots 1105-1 through 1105-7. The positive portions of the waveform applied to the partial interval histogram processor that occur during analysis time segment TS are shown in box 1101 and the detected intensity level points where the positive going waveform crosses levels 1103-1 through 1103-4 are indicated therein. As a result of the detected positive going crossings, an inverse interval count of 4 for level 1103-1 is stored in a memory location bin corresponding to f.sub.0 line 1110-1 in plot 1105-1. In similar manner, inverse level counts of 4 are stored as shown in plots 1105-2, 1105-3 and 1105-4 as lines 1110-2, 1110-3 and 1110-4, respectively. Corresponding bins having the same frequency range of the level stores indicated in plots 1105-1 through 1105-7 are summed to form the partial interval histogram indicated in plot 1125. Since, a count of 4 is stored in each of the bins containing f.sub.0 in plots 1105-1 through 1105-4, the inverse interval count in the bin for f.sub.0 of plot 1125 is 16.

FIG. 12 illustrates diagrammatically the operation of the plurality of partial histogram processors responsive to a sinewave signal

Line 1205 of FIG. 12 represents the amplitude A of this sinewave at frequency f.sub.0 on a log frequency scale and the spectral characteristics of a set of 5 overlapping filters 1201-1 through 1201-5 are indicated on the same log frequency scale. Each filter exhibits a prescribed shaped spectral portion. While triangle shape spectral portions are shown, it is to be understood that the actual spectral portions correspond to the cochlear filters of FIG. 9. It is apparent that signal s(t) falls within the passbands of filter characteristics 1201-2, 1201-3, and 1201-4 but outside the passbands of filter characteristics 1201-1 and 1201-5. Boxes 1210-1 through 1210-5 diagrammatically represent the operation of the set of partial interval histogram level detection arrangements associated with filters 1201-1 through 1201-5, respectively. The horizontal lines within each of boxes 1210-1 through 1210-5 correspond to the aforementioned logarithmically related positive amplitude sound intensity crossing levels for a predetermined time segment TS. The time segment TS for the partitioned input signal s(t) results in signal outputs from spectral filter processor 105 to partial interval histogram boxes 1210-2, 1210-3, and 1210-4, but no signal outputs to partial interval histogram boxes 1210-1 or 1210-5 as indicated.

The positive portions of the sinewave applied to boxes 1210-2, 1210-3 and 1210-4 shown as waveforms 1212-2, 1212-3 and 1212-4 result in an inverse interval count of 24 at frequency f.sub.0 shown at line 1215-2 on a log frequency scale, an inverse interval count of 16 at frequency f.sub.0 shown at line 1215-3 and an inverse interval count of 8 at frequency f shown at line 1215-4. These counts are summed in summer 1220 and the resultant count for the bin is indicated at line 1225 at frequency f.sub.0. In general, signal s(t) is a complex speech waveform having many components so that the partial interval histogram counts and the resulting combination in the ensemble interval histogram (EIH) represents the spectrum of the speech waveform as derived from the synchrony of neural firings.

FIGS. 2 and 3 show a flow chart that illustrates the general method of operation of the circuit of FIG. 1, and the general sequence of operations of control 130 used to coordinate the signal processors in FIG. 1 is set forth in Fortran language form in Appendix A hereto. Referring to FIGS. 1, 2 and 3, step 200 is initially entered wherein a plurality of logarithmically relates sound intensity threshold levels L.sub.1, L.sub.2, . . . , L.sub.j, . . . , L.sub.J are set in each partial interval histogram processor 110-1 through 110-I to values such as 4.sup.j, j=1, . . . , J. The sound intensity threshold levels for each spectral portion may be the same or may differ from one another. Where the threshold levels are different, they may be set randomly with respect to one another so as to better simulate the behavior of the acoustic nerve cell arrangement. Input sound signal s(t) from transducer 101 is digitized in signal converter 103 and partitioned into I spectral portions s.sub.1, s.sub.2, . . . , s.sub.i, . . . , S.sub.I in spectral processor 105 (step 201) in a manner well known in the art. A set of stored instructions for performing the spectral filter operations of signal converter 103 and processor 105 is shown in Fortran language form in Appendix B hereto.

Time segment index I.sub.TS is reset to one in step 203 and the sequence of digital codes x.sub.1, x.sub.2, . . . , x.sub.n, . . . , x.sub.N for the spectral portion waveform, e.g., s.sub.i, of the current time segment TS illustrated in FIG. 7 is formed in processor 105 (step 205). The digital code sequence for the present time segment TS spectral portion s.sub.1 is applied to partial interval histogram processor 110-1 from processor 105 in step 205. Similarly, the time segment digital code sequences for spectral portions s.sub.2 through s.sub.I are applied to partial histogram processors 110-2 through 110-I, respectively. Time segment TS may, for example, bet set to 40 milliseconds. The codes may be received by the partial interval histogram processor as generated, stored therein and segmented into groups of N for processing in the current and succeeding time segments TS.

Sound intensity threshold index j is reset to zero (step 207) preparatory to formation of partial interval histograms as aforementioned with respect to FIGS. 11 and 12. The partial interval histograms for the different spectral portion waveforms s.sub.1, s.sub.2, . . . , s.sub.i, . . . , s.sub.I are produced concurrently in processors 110-1 through 110-I. The inverse interval histogram processing for spectral portion s.sub.i l in processor 110-i is shown in the loop including steps 218, 220 and 225. The inverse interval histogram processing for the other spectral portions is performed concurrently so that a set of PIH.sub.ij (k) partial interval histogram signals are produced where is the spectral portion index, j is the sound intensity level index and k is the histogram frequency bin index. Threshold level index j is incremented in step 218. The partial interval histogram signal set PIH.sub.ij (k) for the current level j and spectral portion s.sub.i is generated as per step 220 by determining the count of the time intervals between positive going crossings of threshold level j by the spectral portion waveform of the current time segment and storing the counts in storage location bins k which span the frequency range of interest, e.g., the speech spectral range. The result is a frequency distribution of the inverse time interval counts for the current time segment of spectral portion i and level j. After the partial interval histogram signals for level j are formed, threshold index incrementing step 218 is reentered via decision step 225 until the final level J has been processed.

The formation of the partial interval histogram for level j of step 220 is shown in greater detail in the flow chart of FIGS. 4 and 13 with reference to the processor arrangement of FIG. 5. FIG. 5 depicts the arrangement that may be used as the partial interval histogram processor of FIG. 1. The circuit of FIG. 5 processes the partial interval histogram for one of the spectral portions, e.g., s.sub.i and comprises input interface 501, signal processor 505, partial interval histogram program instruction store 520, data signal store 525, output interface 510, and bus 530. Program instruction store 520 is a read only memory storing the instructions for implementing the partial interval histogram processing according to the flow charts of FIGS. 2 and 3. The instructions of store 520 are set forth in Fortran language in Appendix C hereto. Input interface 501 receives the sequence of digital codes x.sub.1, x.sub.2, . . . , x.sub.N for the corresponding spectral portion e.g., s.sub.i from spectral filter processor 105. Signal processor 505 is adapted to perform the partial interval histogram processing operations under control of the instructions from store 520 as is well known in the art. Data signal store 525 includes k=1, 2, . . . K memory locations arranged to store the inverse interval counts for the histogram of each level j and the counts for the histogram of the combined levels j=1, 2, . . . , J. Each memory location bin k receives the count of inverse intervals corresponding to a particular frequency range .DELTA. in bin k as will be described. Output interface 510 is operative to transfer the PIH.sub.i (k) signals representing the partial histogram of inverse interval counts for all levels j of the present time segment of spectral portion i to ensemble histogram processor 115 in FIG. 5.

Referring to FIG. 4, the digital codes x.sub.1, x.sub.2, . . . , x.sub.n, corresponding to the spectral portion signal s.sub.i are received by input interface 501 of FIG. 5 and are transferred to data signal store 525 under control of instructions from instruction store 520 (step 401). Each sequence of N digital codes corresponds to a predefined maximum analysis time segment for which a histogram is to be formed. The filtered sample signals x.sub.n are stored (step 401). Sample index n is initially set to N in step 405 since the histogram analysis is performed on the sequence of past N samples in descending order x.sub.N, x.sub.N-1, . . . , x.sub.n, . . . , x.sub.1 and the time segment determining count index m set to zero (step 410) preparatory to the histogram formation. As aforementioned with respect to FIG. 10, the analysis time segment is preset, e.g. 40 milliseconds, but may be shortened to correspond to a predetermined count of inverse time intervals, e.g. M=20 so that a finer time resolution may be obtained. Consequently, the count index m is used to determine the duration of the time segment so that the analysis time segment for higher frequency spectral portions is shortened. The partial interval histogram count signals PIH.sub.ij (k) for all frequency bins k=1, 2, are reset to zero (step 415) and a temporary sample storage location S1 is set to value of digital code x.sub.N (step 420) preparatory to the level detection operations in the loop from step 425 of FIG. 4 to step 1378 of FIG. 13.

Detection of a positive upgoing crossing of sound intensity threshold level j is implemented according to steps 425, 430, and 435 in which sample index n is decremented in step 425. Signal S1 is made equal to the previous sample, e.g., x.sub.n+1 and signal S2 is set to the current sample, e.g., x.sub.n in step 430. If signal S2 corresponding to current sample x.sub.n is greater than or equal to the threshold level L.sub.i and signal S1 corresponding to the immediately preceding sample x.sub.n+1 is less than threshold level L.sub.j (step 435), the threshold has been crossed in the upward or positive going direction and step 440 is entered. Otherwise, step 425 is reentered so that the pair of samples x.sub.n and x.sub.n-1 may be processed.

In the event that the conditions of decision step 435 have been satisfied for current sample x.sub.n and the preceding sample x.sub.n+1, a signal representative of the time at which the upcross of threshold level L.sub.j has occurred

is produced by linear interpolation (step 440). Decision step 445 is then entered to determine if tcross is the time of the first positive going level j crossing in the current time segment. This is done by checking signal tmem which represents the time of the preceding crossing. If signal tmem is zero, there have been no prior crossings in the current time segment and signal tcross produced in step 445 is the first upcross. tmem is then set equal to tcross (step 450), and step 425 is reentered to detect the next upcross of level j. Otherwise a signal representing the time interval between the previous and the current upcrossings of level j

is generated in step 1355 of FIG. 13 and the inverse interval count in the kth frequency bin of the PIH.sub.ij (k) histogram in data signal store 525 is incremented.

The frequency bin incrementing responsive to the inverse interval count signals performed in step 1360 wherein the count signal is placed in the bin k corresponding to the inverse of the time interval signal (1/.tau.) modulo .DELTA.. .DELTA. is equal to the range of frequencies in one bin. Each frequency bin indexed by k corresponds to a predetermined frequency range k.DELTA. to (k+1(.DELTA. where .DELTA. is, for example, 32 Hz. The k=1 bin may, for example, correspond to the frequency range between 32 Hz and 64 Hz while the highest frequency bin K=100 corresponds to the frequency range between 3200 Hz and 3232 Hz. Step 1365 is then entered.

In step 1365, the most recent tmem signal is made equal to the most recent tcross signal obtained in step 440. The time segment determining count index m for level j and filter i is then incremented (step 1370) and the incremented time segment determining count index m is compared to a prescribed maximum M, e.g., 20 (step 1375). As aforementioned, the histogram analysis time segment TS ends after the time period of N samples or may be terminated earlier when the maximum inverse interval count M is reached. If m is less than M in step 1375, the sample index n is tested against zero in step 1378. As long as m is less than M and n is greater than zero, step 425 of FIG. 4 is reentered to generate the next inverse interval signal for level j. Otherwise, all input samples of the time segment have been processed and the partial interval histogram signals PIH.sub.ij (k) for frequency bins k=1, 2, . . . , K of level j of spectral portion i are stored (step 1380). Control is then passed to step 225 of FIG. 2 in which threshold level index j is compared to the last index J. As long as index j is less than J, step 218 is reentered to process the next level to form the partial interval histogram signals PIH.sub.ij (k) for the set of frequency bins k=1, 2, . . . , K of the next level j.

Upon formation of partial interval histogram signal set PIH.sub.iJ (k) for the last level J, the partial interval histogram signals for the levels j=1, 2, . . . , J are combined by summing the level partial histogram signals to form the ith filter partial histogram signal set ##EQU1## as per step 330 of FIG. 3. The partial interval histogram signal set PIH.sub.i (k) for spectral portion s.sub.i is then stored in data signal store 525 of FIG. 5. All of the partial interval histogram processors 110-1 through 110-I of FIG. 1 operate concurrently as described with respect to processor 110-i. It is readily seen from FIG. 2 and 3 that the steps described with respect to processor 110-i for spectral portion s.sub.i are the same for all partial interval histogram processors. The partial interval histogram processing steps for such other spectral portions is indicated in FIG. 3 by the arrows entering step 335.

Ensemble histogram processor 115 of FIG. 1 shown in greater detail in FIG. 6 is operative to combine the signal sets PIH.sub.1 (k), PIH.sub.2 (k), . . . , PIH.sub.i (k), . . . , PIH.sub.I (k) for frequency bins k=1, 2, . . . , K obtained from the spectral portion partial interval histogram processors 110-1 through 110-I to form an ensemble interval histogram signal set EIH(k) by combining the filter interval histogram signals according to ##EQU2## as indicated in step 335 of FIG. 3. Each EIH(k) signal for the present time segment TS corresponds to the neural response for the frequency range of bin k so that the set of EIH(k) signals represents a spectral distribution of the neural response to the input sound. The processor of FIG. 6 comprises input interface 601, signal processor 605, output interface 610, ensemble histogram formation instruction store 620, data signal store 625 and bus 630. The ensemble histogram formation instruction store is a read only memory containing a set of instruction codes adapted to implement the operations of step 335 of FIG. 3. The instructions stored in store 620 are set forth in Fortran language form in Appendix D hereto. Input interface 601 receives the partial interval histogram signal sets PIH.sub.1 (k), PIH.sub.2 (k), . . . , PIH.sub.I (k), . . . , PIH.sub.I (k) from processors 110-1 through 110-I and transfers them via signal processor 605 and bus 630 to data signal store 625. When all of the partial interval histogram signal sets for the present time segment are stored in the data signal store, signal processor 605 is operative to sum the corresponding frequency bin counts partial interval histogram signal sets in accordance with equation 9 to form the ensemble interval histogram signal set EIH(k) of step 335 of FIG. 3.

The ensemble histogram signal set EIH(k) represents the frequency distribution of inverse interval counts over the spectrum covered by spectral portions obtained form spectral filter processor 105 of FIG. 1. Consequently, the EIH(k) signal set corresponds to a spectrum directly related to the nerve firing pattern in the auditory nerve and the resulting spectral distribution is representative of the response of the aural sensing mechanism rather than a frequency distribution of the amplitudes of a sound pattern segment obtained by direct Fourier analysis.

Advantageously, the use of multiple sound intensity threshold levels in the inverse interval counts and the combining of the partial interval histogram signals provides a direct measure of the intensity of the individual frequency components of the time segment neural response spectral distribution and results in a high degree of noise immunity over conventional Fourier analysis arrangements. The noise immunity is illustrated in the waveforms of FIG. 8. Referring to FIG. 8, waveform 801 is the Fourier power spectrum for the speech pattern /e/ in a noise-free environment and waveform 821 is the Ensemble Interval Histogram for the same sound obtained using the circuit of FIG. 1. Since waveform 821 represents a neural response spectral distribution rather than a Fourier type analysis, it is completely different that waveform 801. Waveform 805 represents the Fourier power spectrum for the sound /e/ obtained in a noisy environment while waveform 825 is the Ensemble Interval Histogram for the same sound in the same noisy environment. While there are marked differences between the power spectrums of waveforms 801 and 805 attributable to noise, there are only minor differences between Ensemble Interval waveforms 821 and 825. Further in this regard, the LPC fit waveforms 807 and 810 for the noise-free and noisy power spectra of waveforms 801 and 805 show significant disparities but the LPC fit for the Ensemble Interval Histogram waveforms 821 and 825 indicate very minor differences. The LP fit arrangements and waveforms are discussed on page 431 of the volume Digital Processing of Speech Signals, by L. R. Rabiner and Schafer, Prentice Hall 1978.

The ensemble interval histogram arrangement according to the invention may be utilized in many sound processing applications. One example of its use, i.e. forming autocorrelation signals for speech recognition arrangements, is illustrated in the circuit of FIG. 1. The ensemble interval histogram signal set EIH(k) for the current time segment is transferred to inverse FFT and autocorrelation signal processor 120 wherein an inverse Fourier transform of the 2 to the power of the EIH(k) signal set is generated as per step 340 of FIG. 3 and autocorrelation signals are produced in accordance with

The FFT.sup.-1 processing arrangements described in chapter 8.2 of Programs for Digital Signal Processors published by the IEEE Press, 1974, may be used to convert the spectral distribution signals from EIH processor 115 to an equivalent autocorrelation domain signal in processor 120. The autocorrelation signals obtained from processor 120 are applied to utilization device 125 which may comprise an automatic speech recognizer well known in the art utilizing such autocorrelation signals. Each time segment in FIG. 1 is set to a time frame of the speech recognizer and the autocorrelation obtained from processor 120 correspond to the spectral distribution signals of the auditory model neural response for the time frame with appropriate intensity weighting. Appendix E hereto sets forth in Fortran language form the instructions for operation of processor 120.

The invention has been illustrated and described with reference to a particular embodiment thereof. It is to be understood, however, that various changes and modifications may be made by those skilled in the art without departing form the spirit and scope of the invention. ##SPC1##