Method and device for generating information relating to the relative position of a set of at least three acoustic transducers

[0001] The invention refers to a method and a device for generating information relating to the relative position of a set of at least three transducers designed for treating acoustic signals,

[0002] In recent years, many different inventions have been proposed to detect and compute the rotation and position of a listener's head wearing headphones. One class of inventions proposes a transmitter which is mounted on the headphones. It is used for emitting a signal only, from which the head position and orientation can be extracted. The transmitted signal is either based on infrared, ultrasonic, or magnetic-field signals. A corresponding receiver will feed the received signal to a signal-processing unit where the rotation and/or position of the listener's head is calculated. Such device is disclosed in WO 92/07346.

[0003] Another class of inventions propose a gyrator or an angular velocity device, which is mounted on the headphone. In this case, the rotation of the head can be acquired just from the angular velocity device without any other receiver device. Such devices are used for example in digital cameras where hand movements are compensated. However, the drift of angular velocity devices can be significant. Therefore, this method is preferred for measuring relative motions only. Such device is disclosed in EP-1 176 848 A2.

[0004] A third class of inventions is a combination of the latter two methods. Therefore, an angular velocity device is used for measuring the head movements, whereas an additional ultrasonic or infrared transmitter and receiver are used to calibrate the drift of the angular velocity device.

[0005] Different methods also exist for tracking the listener's position in a room with loudspeakers. The most common method is to reproduce an audible signal using the loudspeakers and measure the delay using one or several microphones. The position of the listener can be computed from the known position of the loudspeakers and the measured delay. Until now, all methods and devices in that field measure the listener's position once before sound reproduction starts. After that, a fixed sweet spot exists where a listener has the best sound experience. Changing the sweet spot requires to stop the sound reproduction and to perform a new measurement.

[0006] JP 06 082242 discloses a device for calculating the position and orientation of a headphone in space using ultrasonic signals. In addition to the transducers used for the acoustical signal, three more transducers are fixed on the headphone for treating ultrasonic signals used for defining the position and orientation in space. Further three transducers are positioned in space for receiving ultrasonic signals. In this case a electrical or optical reference signal is transmitted from the headphone to the receivers of the ultrasonic signals and the delays of each ultrasonic signal is calculated and its emitter is determined. The reference signals are separated by using different frequency bandwidths. One drawback of this apparatus is to be seen in the fact that special transducers for determining the position and orientation are necessary in addition to the transducers used for acoustical signals. Also an electrical or optical reference signal has to be provided. This arrangement and the division of the ultrasonic signals into several frequency bands leads to a very sophisticated method for the treatment of the signals in order to determine the position and orientation of the headphone.

[0007] JP 01 276900 discloses a device with loudspeakers dispersed in a space and whereas the position of a listener should be determined. Therefore, the loudspeakers emit ultrasonic signals which are received by a receiver located on the listeners position. Starting from the different running times needed by said ultrasonic signals to reach the receivers, the position of the receiver or the listener can be determined. As no separate ultrasonic transducers are provided, the step of determining the position must be performed ahead of emitting the acoustical signal, that is the effective operation of the loudspeakers. The loudspeakers are in a fixed position and the position is therefore determined once only in advance to the operation of the loudspeakers. The position of the listeners head can not be measured continuously.

[0008] Both devices take advantage of correlating incoming ultrasonic signals to determine the position. That means that said devices use the signals having the most important amplitudes, but, such signals are often those who are reflected by other objects. For this reason, the position determined by such devices is heavily influenced in a negative manner by external conditions.

[0009] The major objective of this invention as claimed is to provide a method and apparatus for continuously measuring and calculating the relative position of at least three audio transducers and especially the rotation and the position of a listener's head wearing headphones within a space without any additional transmitter device attached to the listener's head, headphone or body during the session of listening to sound emitted.

[0010] The major objective of this invention is achieved by providing a first electric audio signal to a first transducer for transducing the first electric audio signal to a first acoustic signal, providing a second electric audio signal to a second transducer for transducing the second electric audio signal to a second acoustic signal, adding a first ultrasonic signal to the said first electric audio signal and a second ultrasonic signal to said second electric audio signal before such audio signals enter the said respective first and second transducer, then, emitting first and second combined acoustical and ultrasonic signals emitted from the said first and second transducers, sensing the said first and second combined acoustical and ultrasonic signals in a third transducer and converting them into a electrical signal, filtering out the ultrasonic signal from the sensed combined first and second signals, computing a first and a second delay signal from the said sensed and filtered first and second ultrasonic signals and computing from said first and second delay signal a signal representing the position of at least one transducer with respect to the at least two other transducers of the said set of three transducers.

[0011] Additionally a set of at least two transducers of the set may be in a fixed positional relationship. Then, according to the disclosed method, the position or orientation of at least two transducers having a fixed relative positional relationship with respect to the third transducer of the set can be calculated.

[0012] Then, a first set of at least two transducers for emitting acoustical signals which are separated by a fixed relative distance and a second set of at least two more transducers for receiving acoustical signals separated by a fixed relative distance may be arranged. Computing a signal representing the position and orientation of the said first or second set to the other first or second set is performed in a two-dimensional plane according to the invention.

[0013] Finally, a first set of at least three transducers for emitting or receiving acoustical signals which are separated by a fixed relative distance and a second set of at least two more transducers for emitting or receiving acoustical signals separated by a fixed relative distance may be arranged. Computing a signal representing the position and orientation of the said first or second set to the other first or second set is performed in three-dimensional space.

[0014] The device for performing the method according to the invention may comprise a first transducer, a second transducer and a third transducer. A digital system is connected to said at least first, second and third transducer and has an output audio path for feeding audio signals to at least one of said first, second and third transducers and an input audio path, for receiving audio signals from at least one of said first, second and third transducer. Then, a source of audio signals and a ultrasonic signal generator is comprised in said digital system. Both are connected to the said output audio path in said digital system. A position information generator is connected to the said input audio path. The device may further comprise a signal filter and analyzer connected to the position information generator in said input path, and the said first and second transducers may be the left and right side transducers of a headphone and said third transducer may be a stationary microphone. In an alternative embodiment, said first and second transducers may be stationary loudspeakers and said third transducer may be a movable microphone.

[0015] According to the invention, e.g. in the case of headphones, the built in transducers of standard headphones are used instead of added devices. More precisely, the transducers of the headphones are used for both, sound reproduction and ultrasonic emittance. The same is true for the loudspeakers: The transducers of the loudspeakers are use for emitting sound signals as well as for emitting ultrasonic signals. Therefore, any headphones without additional cabling can be applied. Additionally the position and orientation of the human head may be measured without interrupting the emittance of sound.

[0016] The present method and device refers to two types of applications namely head tracking and listener tracking, both during sound reproduction and both without an additional sender device.

[0017] The first type covers the field of all applications, where head tracking in combination with headphones is required or useful. Head tracking is the process of continuously keeping track of movements, that is the rotation and position changes of a human head wearing headphones. More particularly the invention relates to multimedia or game applications where a personal computer, a game console or any other device can use the three-dimensional position and rotation information of the users head.

[0018] Moreover, applications with headphones for telephone or radio transmission are covered, such as conference calls or airplane cockpits, where audio sources are “placed” in a virtual listening environment according to the head's rotation and position.

[0019] In a slightly different arrangement, the same invention can be used for tracking a listener in a listening room with a multi-loudspeaker sound installation. Such an installation can be a DVD reproduction system with five loudspeakers and one subwoofer or any other number of loudspeakers placed at arbitrary positions in the room. By tracking the listener in the room it is possible to adjust the sweet spot to the actual position of the listener. The sweet spot is the area in the room where the reproduction of sound from all loudspeakers is equalised and the best listening experience exists. Unlike other methods the present method and device allows to dynamically adjust the measurement of the listener's position during sound reproduction.

[0025] As can be seen from FIG. 1, a headphone set 7 comprising a transducer or headphone 7a and 7b for the right and left ear is connected to a digital system 13 with audio capabilities by lines 5 or a wireless connection. Such digital system 13 can be or at least comprises a personal computer, a game station, a set-top box or any similar device known per se, which produces an audio signal for a listener 6. This digital system 13 therefore comprises a source 1 for emitting non-modified audio signals, a ultrasonic signal generator 2, which produces non-audible signals above audible frequencies, typically above 20 kHz and a digital to analog converter 4 (DAC 4). All three elements are connected via lines 15 and adders 3a and 3b. The source 1 for a non-modified audio signal may be a CD-player, a wave-file or any other source within or outside the system 13. This source 1 emits the audio signal that the listener 6 wants to listen to. Additionally, one or more transducer 8 such as microphones or ultrasonic receivers are positioned in front of the listener 6, and are connected by a transmission line 9 to the digital system 13. Again this transmission line 9 can be a cable or a wireless connection. An analogue to digital converter 10 (ADC 10) is connected via lines 16 and 17 to a signal filter and analyzer 11 and further to a position information block 12. The source 1 with lines 15, the converter 4 and lines 5 define an output audio path 19, whereas transmission line 9, the converter 10, the filter and analyzer 11 with lines 16 and 17 as well as the position information block 12 are elements of an input audio path 20.

[0026] In operation, this system 13 produces an audio signal which is emitted by the source 1 and transmitted through lines 15 and after conversion in the converter 4 through lines 5 to the first and second transducers 7a, 7b in the headphone 7 attached to a listeners head 6. Two distinct ultrasonic signals, one for each transducer 7a and 7b are emitted by the ultrasonic signal generator 2. The ultrasonic signal generator 2 produces at least two non-audible signals above 20 kHz which may use neighbouring frequency bands.

[0027] Another possibility is to use different time sequences, one sequence for the right transducer 7a and one sequence for the left transducer 7b in the headphone 7. Both sequences in the same frequency band above 20 kHz for both ultrasonic signals are emitted during a short alternating time span.

[0028] In both cases the ultrasonic signals are added to the audio signal in the lines 15 by means of the adders 3a, 3b. The ultrasonic signals are also converted in the digital to analog converter 4 and are sent together with the audio signal to the headphones 7a, 7b. The digital to analog converter 4 may be a common audio-converter having a sampling rate of 48 kHz. As now converters having a sampling rate of 96 kHz are also available, a ultrasonic signal having a frequency much higher than 24 kHz may be used. This ultrasonic signal may be converted back with the corresponding ADC 10, which operates at a sampling rate of 96 kHz.

[0029] The said ultrasonic signals are continuously emitted by the transducers 7a and 7b. The person wearing the headphone 7 will not hear the ultrasonic signals emitted by the transducers. A third transducer 8, such as a microphone or a receiver for ultrasonic waves positioned e.g. in front of the listener 6, will receive the ultrasonic signals.

[0030] Depending on the characteristics of the headphones 7, the emitted ultrasonic signal can be weak, because the attenuation of a signal in air as well as on absorbing materials above 20 kHz is rather strong. Normally, the signals can have any type of signal form. However, in order to still get a good signal quality and receive the ultrasonic signal at a distance of several meters, a band-limited white noise with pseudo-random phase is best fitted in this case. Suitable signals, such as maximum-length sequences are known to have an auto-correlation function with a clearly defined single maximum. Other possible signals include Golay sequences and a periodic-phase sequences that can improve detection accuracy in noisy environments. At least two signal adders 3a and 3b in the output path 19 are used where the output signal is the sum of the signals from the source 1 and the ultrasonic signal generator 2. A digital to analogue converter DAC 4 has at least two digital signal inputs connected to adders 3a and 3b and two analogue signal outputs. A transmission line 5, such as an electric cable or a wireless connection is used to transmit the signals from the system 13 to the headphones 7. A person's head 6 is wearing standard, usual in the trade headphones 7. The ADC 10 converts at least one analogue electrical signal as received by the transducer 8 into at least one digital signal. This received signal comprises the ultrasonic signals from both transducers 7a and 7b. The output signal of the ADC 10 will have a shape as shown in FIG. 2. In FIG. 2 signals are plotted over a horizontal axis having values of frequencies and adjacent to a vertical axis having values of the amplitude of the signal indicated thereon. As can be seen from FIG. 2 a frequency band 23 for a non-modified audio signal extends up to a frequency limit of approximately 20 kHz. Two distinct frequency bands 24 and 25 extend above the said frequency limit, whereas one frequency band 24 may be directed to the right transducer 7a and the frequency band 25 to the left transducer 7b. The signal filter and analyser 11 connected to the ADC 10 processes the signal received in a high-pass filter and filters out the frequency band below 20 kHz from the waveform of FIG. 2. The remaining signal only consists of possibly weak ultrasonic signals 24, 25 from the right transducer 7a and the left transducer 7b of the headphone 7. A second filter in said analyser 11 is provided to separate the two reconstructed ultrasonic signals into two individual signals one containing the ultrasonic signal 24 from the right transducer 7a of the headphone 7, and the other containing the signal 25 from the left transducer 7b. In the case, where the two ultrasonic signals are in adjacent frequency bands for either signal, the said second filter is a bandpass filter and it is applied for each frequency band. Using appropriate cross-correlation with the known input signal may improve the resulting signal to noise ratio. Such filtering techniques using cross-correlation are well known in the art and therefore not described in more detail here.

[0031] In the case where the two ultrasonic signals are temporal sequences with one sequence for the right transducer 7a of the headphones 7, and another sequence for the left transducer 7b, the said second filter is a decoder of the said sequences. As a result, for each receiver or microphone 8 the ultrasonic signal from the right and the left transducer of the headphone is separated and reconstructed in the filter and analyser 11 and both signals are then propagated to the position information block 12.

[0032] As such devices are most often operated in closed rooms, most probably several signals will be received by the receiver 8. Such ultrasonic signals are emitted by one specific transducer 7a, 7b. Due to reflexion on a wall or an object, a first signal issued by the same transducer 7a, 7b may directly reach the receiver 8 and a second signal may reach the receiver 8 indirectly after being reflected. Additionally the transducers 7a, 7b in the headphone may have strong directional effects. Therefore, the strongest signal reaching the receiver 8 may well be a reflected signal. Filter and analyzer 11 is therefore designed in such a manner as to detect and separate the signal reaching the receiver first instead of the signal having the biggest amplitude.

[0033] The position information block 12 will also receive the original ultrasonic signal as emitted by line 18 from the ultrasonic generator 2 together with the reconstructed ultrasonic signal received from filter 11 over line 17. He can now compare the original ultrasonic signal with the reconstructed ultrasonic signal by means of a digital signal cross-correlation. Two delays, for the left and the right signal, respectively, result individually for each receiver or microphone 8 depending on the distance between the respective first and second transducer 7a, 7b and the third transducer or receiver 8. If one receiver or microphone 8 is used, the rotation or orientation of the listener's head 6 can be computed through their relative distance to the receiver 8. As will be explained using FIG. 3, the difference of the delays, as explained above, delivers the information about the difference of the distances 37, 38 of the right transducer 7a to the microphone 8 and of the left transducer 7b to the microphone 8. The rotation of the head 6, expressed in the angle α, 39, can be computed through known triangulation procedure knowing the real or average diameter of a person's head. A more exact calibration based on the individual head size is easily feasible through personal measurements. The average of the two distances 37 and 38 gives the information about the absolute distance from the centre of the head 6 to the receiver 8.

[0034] In case of two receivers 8a and 8b, shown in FIG. 4, the rotation and a two-dimensional exact position of the head 6 with respect to the receivers 8a, 8b can be calculated with more accuracy. As above, it can be computed through triangulation with the additional information of the two distances 41 and 42 to the second receiver 8b and the distance 40 between the two receivers 8a and 8b. In this and in the above mentioned case the transducers are located in one plane, which is identical to the plane of the said figure or which is parallel to said plane.

[0035] In case of three and more receivers, the rotation and the exact position in three dimensions is also computable. Again, it can be computed through triangulation with the additional information of the distances between all receivers.

[0036] After the described computation performed by the position information block 12, the information about the rotation and the position is then available in digital form as a signal in line 14 (FIG. 1). It can be used by an application running on the digital system. In most cases, the audio-visual content provided to the user through displays, loudspeakers, headphones, etc. is modified depending on the head's rotation and position.

[0037] A second embodiment of the invention is shown in FIG. 5. In FIG. 5, the same digital system 13 with audio capabilities as known from FIG. 1 can be used for tracking a listener in a room with two or more loudspeakers 51a, 51b installed. In this case, the two loudspeakers 51, 51b replace the transducers 7a, 7b of the headphones known from FIG. 1. As described in the previous section, at least two ultrasonic signals from the ultrasonic signal generator 2 are added to the non-modified audio signal from source 1 in the adders 3a and 3b. The digital to analogue converter DAC 4 converts the signals to the apalogue domain. A transmission line 5, either cable based or wireless, is used to transmit the signals to the two or more loudspeakers 51a and 51b. The same receivers 8 as described in the previous section can be used to detect the audio and ultrasonic signals. A transmission line 9, again cable based or wireless, propagates the signal to an analogue to digital converter ADC 10. The signal filter and analyser 11 filters out the audio signal and extracts the ultrasonic signals. In this case, the position of the loudspeakers is known and information about the said position is entered into position information block 12 by known means such as digital information fed from a position measurement system or a keyboard here not shown. The information block 12 calculates out of the said extracted ultrasonic signals the position of the receiver 8 which is unknown. This can be done through triangulation given that the positions of the loudspeakers 51a and 51b as well as the distance between the loudspeakers 51a and 51b are known.

[0038] During sound reproduction, the listener can wear or hold the receiver 8 in any suitable manner and can move in the room surrounding the loudspeakers. The listener cannot hear the ultrasonic signals emitted by the loudspeakers. Nevertheless the receiver 8 can detect them. Therefore, the position information block 12 can keep track of the listener's actual position. Also the position of loudspeakers may be determined depending on the known position of the listener in the same way.

[0039] As shown in FIG. 6, the listener may also wear or hold two receivers 8a and 8b. In this case, not only the position of the listener but also his rotational position or orientation in space can be calculated. In this case, the receivers 8a, 8b have a fixed distance 65 from each other and are mounted on the same device or holder. The following distances have to be calculated from the measurements of the corresponding delays. The distance 61 from the first loudspeaker 51a to the first receiver 8a, the distance 62 from the first loudspeaker 51a to the second receiver 8b, the distance 63 from the second loudspeaker 51 b to the first receiver 8a and the distance 64 from the second loudspeaker 51 b to the second receiver 8b. Through triangulation the position of both receivers 8a and 8b can be calculated. In a further embodiment, the non-modified audio signals from source 1 and the ultrasonic signals from generator 2 can also be added in the analogue domain after the DAC 4. In this case, a separate DAC is used to convert the ultrasonic signals from generator 2 to the analogue domain.