System and a method for detecting the direction of arrival of a sound signal

[0001] The present invention relates to the field of sound direction detection.

[0002] The need to detect the exact direction of arrival (DOA) of a sound is highly important in a variety of fields. Detecting the direction of the sound emitted by a shooting firearm, for instance, might be crucial both for military and civic situations. Due to the changing nature of armed conflicts and the increase in attacks by guerillas, armed civil conflicts, and of humanitarian action given in unstable regions, receiving accurate and reliable data about the position of a shooter may allow damage reduction by enabling the appropriate responses. U.S. Pat. No. 5,586,086 provides a means for locating a firearm on the basis of acoustic detection. The disclosed method and system detect the location of a firearm firing projectiles by using an acoustic detection antenna that includes at least three microphones spaced apart from one another to detect and record signals and time offsets of the signals which are representative of the muzzle noise of the firearm and/or the sound wave emitted by the mach cone generated by a projectile that has supersonic muzzle velocity, and processing the signals and their time offsets in such a manner as to determine at least the direction in which the firearm is located.

[0003] U.S. Pat. No. 6,185,153 is a system for detecting gunshots, which includes an input device including a microphone for converting acoustic noises into signals and amplifiers for amplifying the input signals, a threshold detector for receiving the amplified signals and comparing the signals with a predetermined threshold value and for producing an output signal when the threshold value is exceeded. A pulse width detector is connected to the threshold detector for producing an output signal only if the width of the threshold detector output signal is within a predetermined range of values. A pulse count detector is also connected to the threshold detector for producing an output signal when the level of the threshold output signal is above a peak threshold level or the number of threshold level output signals that exceed a threshold level are less than a preset limit. An output device indicates that a gunshot has occurred only when signals are received from the pulse width detector and the pulse count detector during a sampling period.

[0004] Another field in which sound direction detection is important is in the field of security systems which include surveillance cameras. The limited angle of view of security cameras often dictates making use of several cameras in order to provide a full visual coverage of a given area. This solution increases the complexity and cost of such systems. One traditional means for overcoming this problem includes regularly rotating the direction of the camera, but this solution might cause the camera to be directed in the wrong direction in critical moments. The integration of a sound direction detection system into such systems helps overcome this limitation and to make them more efficient and reliable.

[0005] Security systems that do implement such methods integrate other methods of directional detection as described in U.S. Pat. No. 4,857,912. This system utilizes multiple sensors for detecting intrusion into an area. Each of the sensors operates on a different principle to detect intrusion. Sound, vibration, infra-red, motion detection, microwave and light level sensors are used, for example, which obviously makes it a highly complex and costly solution.

[0006] There is therefore a need for a simple and highly reliable system and method for detecting the direction of an emitted sound which can be adjusted to suit the different needs of different systems. The solution should also be able to supersonic as well as sonic sounds and be compact and lightweight enough to be carried by a person.

[0007] The present invention discloses an apparatus for automatically the detecting direction of sound events. The apparatus comprised of at least four sound detectors positioned in the vicinity of the device; signal pattern recognition means for identifying the real event's signal; processing and analyzing means for detecting direction of arrival of sound events in accordance with signal pattern identification. The sounds detectors are positioned at a maximum spread within the physical boundaries of the device.

[0008] The detection of sound direction is based on first signal event detection technique. The sound level is determined according to the sampled signal's frequency and energy level. The detection of the first signal event is based on a pattern identification criterion by estimating the envelope of the sounds signal energy. The processing and analyzing means include analyzing the signals' differential arrival time as sampled by the different microphones. The estimation of time of arrival differences between the sound detectors may be computed according to frequency properties.

[0009] Also within the scope of the present invention is a video camera and means for turning the video camera towards the detected direction.

[0010] The present invention also discloses a method for automatically detecting the direction of noise events. The method is comprised of the following steps: sampling sound signals from at least four different sources in space; applying pattern recognition techniques for identifying the signal's first event; processing and analyzing all signals for detecting direction of arrival of sound events based on pattern recognition results.

[0011] The detection of direction of arrival is based on first signal event detection technique. The first signal event detection is based on a floating threshold criterion by estimating the sounds signal energy envelope. The estimation of time of arrival differences between the sound detectors may be computed according to frequency properties. The disclosed method may also include the step of turning a surveillance camera in the direction of the detected event.

[0014] The present invention is a highly reliable sound direction detection system. It is an optimal solution for detecting the direction of an abrupt sound source in wide variety of spaces which are not necessarily sterile. When an event which produces a loud noise occurs, such as a shout or a gun shot, it can be detected by the said invention, even in a noisy and crowded area.

[0015] The estimation of the DOA of a known signal is a central problem in many fields including radar, sonar, and communication. The problem is to find the best estimation of the source of a transmitted signal. In classical DOA estimations, one calculates the multiple reflections having different DOAs of the reflected signal. This is a multi dimensional complex algorithm that requires heavy resources depending on the number of reflections and microphones.

[0016] The method described in the present invention eliminates the need for complex calculation abilities by identifying the sound emanating from the actual sound source event, the “real” signal, then calculating the direction using the classical method for one source. This significantly reduces the complexity of the calculation, while maintaining a high direction resolution.

[0017] The main problem in determining the direction of an incoming sound signal derives from the fact that in most cases the signal is composed of multiple reflections each coming from a different direction and with a different time delay. This is especially true for sound signals in urban spaces. A sound which is caused by an event happening in one location bounces off all walls and surfaces in the vicinity and is received by the sensors as a multiplicity of sounds coming from different directions with different time delays. Some of these sound signal reflections may overlap in time. In such cases the conventional techniques of determining the direction of a sound source become ineffective and/or demand extensive processing resources, mainly because of the crucial need to correctly estimate the exact number of sources. New estimation techniques that are designed to easily handle overlapping reflections are therefore required. These new techniques also have to perform efficiently and overcome continuous static background noises.

[0018] The preferred embodiment of this invention makes use of such a technique. The device is comprised of four microphones, a digital signal processing (DSP) and a command processing board. The microphones are set at a spread of at lease 5 cm from each other. This spread creates the differential time of arrival of the sound to the different microphones, according to the relative location of the sound source and the device, which is necessary for the calculation of the direction. The processing board executes the direction of arrival (DOA) calculation algorithm, translates its results into directional output. This output may come in the form of a graphical and textual display, informing the user about the direction of the sound source, or in the form of commands, changing the point of view (POV) of a camera to the appropriate direction.

[0019] In order to determine with a high level of certainty the DOA of a sound signal the system continuously samples and processes a plurality of digitized inputs samples received from the physical layer (PHY) module. This data stream is a sequence of samples of a plurality of analog signals taken each at a sample frequency. These samples, which are records of the same event taken by different sources, are processed in order to determine the beginning of an event. For example, a floating threshold criterion, which is also noise resistant, may be used to estimate the envelope of the energy signals in order to determine the beginning of the event using a pattern recognition algorithm. The graph of such algorithm is displayed in FIG. 1.

[0020] The pattern recognition algorithm allows the system to recognize the signal of an actual event and distinguish it from echoes and reflections. The first event is not easily recognized since in different circumstances reflection signals might be detected by the microphones before the signal of the first event reaches it. The purpose of pattern recognition algorithm is therefore to identify the first event according to several parameters whereas the time parameter is only one of them. Other parameters may include the signal's energy and amplitude levels and its frequency. The pattern of the recorded signals is analyzed according to prevalent signal patterns and then the real event can be identified. Once the beginning point of an event is known, the DOA calculation of a single source can be performed using standard methods by comparing the time delay which can be calculated with a simple maximum likelihood estimator of that event in the four microphones and using a simple trigonometric calculation for finding the direction.

[0021] FIG. 1 illustrates an exemplary case of 4 samples, x_j=(a_j, m_j), j=1,2,3,4 marked with O. a_j stands for the position on the time axis (X) and m_j stands for the value of the sample (Y).

[0022] Line 1 in the graph is the sampled signal x(t). Line 3 is the calculated envelope of the energy of the sampled signal. Line 2 represents the threshold signal and is calculated based on the energy envelope. The noise threshold determines the beginning of the event.

[0023] FIG. 2 is a flowchart of the sequence of operations. First, sound signals are sampled by the PHY module. For each sample the beginning of the event is calculated, the direction is determined by comparing data from the four microphones and the nature of the event is analyzed. The system then determines whether it is a significant event according to predefined parameters such as the level of energy and frequency of the signal. If the event is significant, its data is transferred to the appropriate output format. In the case of devices which inform the user about the location of the sound source, graphical and textual data are displayed on a user interface display. When the device controls the POV of a camera the motors which control the camera are given a command to turn it in the appropriate direction. In addition, the output may be used for any other type of devices or a combination thereof.

[0024] The preferred embodiment is based on an algorithm which uses four microphones (marked in the following description as P₁, P₂, P₃, P₄). The input samples form microphone P_i are marked as x₁ⁱ,x₂ⁱ,x₃ⁱ, . . . x_nⁱ. The time of arrival of each sample to each of the microphones P₁, P₂, P₃, P₄ is calculated. Let us assume that the input sample rate is 1/T. Let xⁱ(t) be a continuous function such that, xⁱ(m·T)=x_mⁱ (for all 1≦m≦n). xⁱ(t) is the input signal from microphone P_i.

[0025] The algorithm is comprised of two parts: computation of the a coarse time of arrival t_i (first event) of the received signal for each microphone; A algorithm for estimating the exact difference in the time of arrival (dⁱ₁,dⁱ₂,dⁱ₃) for each of the microphones (P_i) in comparison to the other microphones (P_j, j≠i), i is chosen to give a maximum signal to noise ratio (SNR), according to some criterion; translation of the time differences dⁱ₁,dⁱ₂,dⁱ₃ (i is chosen with the maximal SNR value) to an accordant position in space.

[0026] In order to calculate the time of arrival t_i of the transmitted signal for microphone P_i the energetic structure of the transmitted signal is assumed to be a continuous function ƒ, which is piecewise linear function, combining 6 points:
x_j=(a_j,m_j),j=0 . . . 5. :

[0027] $f\left(t\right)=\left\{\begin{array}{cc}{m}_0& t\le a_0\\ t{\mathrm{•\alpha }}_i+{\beta }_i& a_i\le t\le a_{i+1}\left(0\le i<5\right)\\ m_5& t>a_5\end{array}\right\}$

[0028] Where α_i, β_i are unknown, calculated under the constrain that
ƒ is continuous and ƒ(a_i)=m_i.

[0029] This implies, for example that m₀=a₁□α₁+β₁·α_i, β_i can be completely determined from the above constrain, as a function of a_j,m_j (j=0, . . . , 5).

[0030] Let P_i(t) be the energy envelope of the input signal from microphone P_i. Our Algorithm minimizes the following expression
e(a_{0 . . . ,} a₅, m_{0 . . .} m₅)=∫{P_i(t)−ƒ(t)}²dt

[0031] Standard least square calculation is used to find the appropriate a_{0, . . . ,} a₅, m_{0 . . . ,} m₅ that minimize the last expression.

[0032] Now in order to find t_i, one can apply threshold criteria such as m₁/m₀>Thd₀ and m₂/m₀>Thd₁ where Thd₀, Thd₁ are some threshold numbers. Then we define t_i=a₁. We can also find an interval L=a₂−a₀ where we can assume that no other real event lies between a₀ and a₂. The last step is finding the exact time delay between for this event. This is a standard problem for a single source DAO calculation, for example a Maximum Likelihood solution can be found in the classical literature. The finale DOA can be found by translating (using simple trigonometry) the time delay into a DOA.

[0033] This calculation is based on the assumption that no reflections occur during the interval L. To insure this assumption the device needs to be installed so that no reflectors (such as a wall) are positioned in a distance greater than 0.5 meter from it.

[0034] The uniqueness of this algorithm lays in the fact that the signals are calculated according to their time attribute whereas prior art usually performs the calculation according to their frequency. This manner of computation is much more accurate, especially when a small number of samples are involved and the frequency axis becomes problematic and unreliable.