Method of positioning sound image with distance adjustment

[0002] 1. Field of the Invention

[0003] The present invention generally relates to a method of positioning a sound image and a sound image positioning apparatus for use in a three-dimensional sound system or else. More particularly, the present invention relates to a method of simulating acoustic transfer characteristics from a virtual sound source in a virtual sound field.

[0004] 2. Description of Related Art

[0005] In a three-dimensional virtual reality system for example, a sound image positioning apparatus is conventionally used as a means for enhancing presence of virtual reality experience. In such a system, a cubic sound field is generated by creating direction perspective and distance perspective in auditory sensation by producing audio signals from a monaural sound source through a plurality of channels having time difference, amplitude difference, and frequency characteristic difference based on binaural technique. To be more specific, an input audio signal is attenuated in a particular frequency component by a notch filter, for example, to create elevation. The input audio signal is also converted by a delay circuit into left channel and right channel signals having a time difference, and is further given acoustic transfer characteristic from a virtual sound source by a FIR (Finite Impulse Response) filter. A parameter of the FIR filter is given from an HRTF database storing head-related transfer functions (HRTF) measured with using a dummy head in advance.

[0006] In the above-mentioned conventional sound image positioning apparatus, it is impracticable to store the HRTFs corresponding to all virtual sound source points included in a sound field. Normally, only the transfer characteristics at points radially away from a listener by a certain distance, for example one meter, are measured and stored. Therefore, if a virtual sound source is located one meter away from a listener as shown in FIG. 5, proper sound image positioning can be provided. However, if the virtual sound source is located from the listener at a distance more or less one meter, problem occurs that sound images sensed by the right and left ears of the listener do not match each other, losing good positioning. Especially, it is known that the human ear has an angular resolution of ±3° for acoustic direction. Therefore, if a virtual sound source passes across the listener, an error higher than ±3° may occur, thereby causing a sense of incongruity.

[0007] It is therefore an object of the present invention to provide a sound image positioning method and a sound image positioning apparatus for positioning a sound image to a correct point even if there is a difference between a reference distance used for measuring a head-related transfer function stored beforehand and a setting distance of a virtual sound source.

[0008] The inventive method pans a sound image of a virtual sound source to a listener in a virtual sound field by filtering audio signals of left and right channels through left and right filters which simulate acoustic transfer characteristics of the virtual sound field. The inventive method comprises the steps of provisionally memorizing acoustic transfer characteristics of the virtual sound field which are distributed radially around a center point of the listener, designating a source point at which the virtual sound source is to be located within the virtual sound field in terms of a geometric distance and a geometric direction relative to the center point, computing a leftward acoustic direction from the source point to a left ear of the listener according to the geometric distance, the geometric direction and an offset of the left ear from the center point, computing a rightward acoustic direction from the source point to a right ear of the listener according to the geometric distance, the geometric direction and an offset of the right ear from the center point, determining an effective acoustic transfer characteristic based on the memorized acoustic transfer characteristics according to the leftward acoustic direction so as to enable the left filter to simulate said effective transfer characteristic, determining another effective acoustic transfer characteristic based on the memorized acoustic transfer characteristics according to the rightward acoustic direction so as to enable the right filter to simulate said another effective transfer characteristic, and filtering an audio signal of the left channel through the left filter and filtering another audio signal of the right channel through the right filter to thereby direct the sound image of the virtual sound source located at the source point to the listener positioned at the center point.

[0009] Further, the inventive apparatus is constructed for directing a sound image of a virtual sound source at a designated source point to a listener in a virtual sound field. In the inventive apparatus, a database provisionally memorizes acoustic transfer characteristics of the virtual sound field in correspondence to reference source points distributed radially around a center point of the listener. Left and right filters respectively filter audio signals of left and right channels according to the acoustic transfer characteristics loaded from the database. A processor computes a leftward acoustic direction from the designated source point to a left ear of the listener, and computes a rightward acoustic direction from the designated source point to a right ear of the listener. A controller specifies a leftward reference source point coincident with the leftward acoustic direction to load an effective acoustic transfer characteristic corresponding to the leftward reference source point from the database into the left filter, and specifies a rightward reference source point coincident with the rightward acoustic direction to load another effective acoustic transfer characteristic corresponding to the rightward reference source point from the database into the right filter. A feeder feeds an audio signal of the left channel to the left filter and feeds another audio signal of the right channel to the right filter to thereby direct the sound image of the virtual sound source located at the source point to the listener positioned at the center point.

[0010] In a different view, the inventive apparatus is arranged for directing a sound image of a virtual sound source to a listener in a virtual sound field. In the apparatus, a database provisionally memorizes a pair of leftward and rightward acoustic transfer characteristics of the virtual sound field in correspondence to each of sample points distributed radially around a center point of the listener at a fixed radius, the leftward acoustic transfer characteristic simulating a path from each sample point to a left ear of the listener and the rightward acoustic transfer characteristic simulating another path from each sample point to a right ear of the listener. Left and right filters respectively filter audio signals of left and right channels according to the left and right acoustic transfer characteristics loaded from the database. An input designates a source point at which the virtual sound source is to be located within the virtual sound field in a distance which may be different from the fixed radius relative to the center point. A processor computes a leftward acoustic direction from the source point to the left ear of the listener, and computes a rightward acoustic direction from the source point to the right ear of the listener. A controller specifies a leftward sample point substantially coincident with the leftward acoustic direction to load the leftward transfer characteristic corresponding to the leftward sample point from the database into the left filter, and specifies a rightward sample point substantially coincident with the rightward acoustic direction to load the rightward acoustic transfer characteristic corresponding to the rightward sample point from the database into the right filter. A feeder feeds an audio signal of the left channel to the left filter and feeds another audio signal of the right channel to the right filter to thereby direct the sound image of the virtual sound source located at the source point to the listener positioned at the center point.

[0011] According to the present invention, as shown in FIG. 3, based on the geometric distance r and geometric direction θ to the virtual sound source point Ps and the offset 2hbetween both ears of the listener, the acoustic directions R and L from the virtual sound source point Ps to the right and left ears of the listener are calculated separately for the right and left channels. Acoustic transfer characteristics of the right and left filters are determined by these acoustic transfer directions R and L. To be more specific, sample points PR and PL coincident with the acoustic directions R and L are identified on a circumference having the fixed radius r0 that is the reference distance. If the setting distance r to the virtual sound source point Ps differs from the reference distance r0 at which the acoustic transfer characteristics are measured, the acoustic transfer characteristics corresponding to the sample points PR and PL are used as the effective acoustic transfer characteristics of both channels. Approximating the effective or true characteristics from the object distance r by these acoustic transfer characteristics provides high fidelity of the sound image positioning. The above-mentioned acoustic transfer characteristics are stored in the transfer characteristic database beforehand at a fine angular pitch, for example, in unit of 1° on the circumference away from the listener by the fixed reference distance r0. Alternatively, the data may be stored at a coarse angular pitch, for example, in unit of 90° in forward, backward, rightward and leftward directions. In the latter case, the effective acoustic transfer characteristic corresponding to the acoustic direction concerned may be obtained by vector compoint or composition according to the calculated acoustic directions R and L. Consequently, according to the present invention, an acoustic transfer characteristic of higher fidelity can be obtained with generally the same data volume as that used in the prior art technology. Conversely, a smaller data volume than that conventionally used may be enough for achieving generally the same acoustic transfer characteristic as that of the prior art technology.

[0012] These and other objects of the invention will be seen by reference to the description, taken in connection with the accompanying drawing, in which:

[0013] FIG. 1 is a block diagram illustrating constitution of a sound image positioning apparatus practiced as one preferred embodiment of the present invention;

[0014] FIG. 2 is a geometric diagram illustrating a point of a virtual sound source and a point of a listener in a virtual sound field;

[0015] FIG. 3 is a geometric diagram for describing acoustic directions of right and left channels in the preferred embodiment shown in FIG.

[0016] FIG. 4 is a block diagram illustrating constitution of a FIR filter associated with another preferred embodiment of the present invention; and

[0017] FIG. 5 is a geometric diagram for describing a problem of the prior art.

[0018] This invention will be described in further detail by way of example with reference to the accompanying drawings. Now, referring to FIG. 1, there is shown a block diagram illustrating a sound image positioning apparatus, panning apparatus or localizing apparatus practiced by a personal computer or the like as one preferred embodiment of the present invention. An input monaural audio signal SI is supplied to a notch filter 1 from a feeder 12 or monaural sound source. The notch filter 1 attenuates a particular frequency component Nt of the audio signal SI based on human auditory characteristic to impart elevational positioning to the input audio signal SI. An output of the notch filter 1 is delayed by a delay circuit 2 to produce two-channel stereophonic audio signals imparted with a sound transfer time lag T from a virtual sound source point to both ears. These stereo signals are supplied to FIR filters 3 and 4, respectively. The FIR filters 3 and 4 impart acoustic transfer characteristics to the audio signals of each channel based on parameters fR(θR) and fL(θL) read from an HRTF database 5. Outputs of the FIR filters 3 and 4 are adjusted in right and left amplitude balance by amplifiers 6 and 7, respectively. Outputs of the amplifiers 6 and 7 are treated by a cross talk canceler (XTC) 8 to eliminate a cross talk that would enter from right and left speakers (not shown) into both ears. The outputs eliminated of cross talk are supplied to the speakers as two-channel audio signals S0R and SOL. A central processing unit (CPU) 9 accepts positional information r, θ, and φ of a virtual sound source designated by an input 10. Based on these positional data, a processor in the CPU 9 calculates control parameters for various blocks concerned, and a controller in the CPU 9 supplies the calculated parameters to the various blocks. A disk drive 13 is connected to the CPU 9. The disk drive 13 receives a machine readable medium 14 such as a floppy disk or CD-ROM.

[0019] As shown in FIG. 2, let an intermediate point between both ears of a listener 11 be a center point Po of a three-dimensional coordinates system. Rightward, forward, and upward directions of the listener 11 are set to X axis, Y axis, and Z axis of the absolute coordinate system, respectively. Then, a source point Ps of the virtual sound source is given in terms of a geometric distance r from the center point Po to the virtual sound source, an azimuth angle θ in the horizontal direction of the virtual sound source Ps as viewed from the front side (Y-axis direction) of the listener 11, and an elevation angle φ in the vertical direction as viewed from the angle θ relative to the front side of the listener 11.

[0020] It is known that, in the human ear, as the angle φ in the elevation of the sound source increases, the dead band frequency shifts to higher ranges. The CPU 9 determines the attenuation frequency component Nt by the elevation φ to control the frequency response of the notch filter 1. Also, the CPU 9 obtains the transfer time lag T of the right and left channels based on the difference in distance from the virtual sound source point Ps to both ears.

[0021] The CPU 9 also calculates acoustic transfer angles θR and θL from the virtual sound source to both ears of the listener 11 based on the azimuth θ. To be more specific, as shown in FIG. 3, the HRTF database 5 stores acoustic transfer characteristics from sample points or reference points distributed along a circumference having radius r0 toward the center point Po. The acoustic transfer characteristics are measured beforehand with using a dummy head. The rightward acoustic transfer angle θR of a sound generated from the virtual sound source point Ps to enter into the right ear of the listener positioned at offset distance +h in the X-axis direction is represented by equation (1) below.

[0022] θR=cos⁻¹(e/r0)  (1)

[0023] Let straight line R that passes the right ear (x=+h) and the virtual sound source point Ps be:

[0024] R: y=dx−dh.

[0025] Since coordinates (xs, ys) of the point Ps are

[0026] xs=r sinθ

[0027] ys=r cosθ,

[0028] d is represented by equation (2) below:

[0029] d=ys/(xs−h)=cosθ/(sinθ−h/r)  (2)

[0030] Further, e equivalent to Y coordinate at intersection PR between the circumference having radius r0 and the straight line R is represented by equation (3) below:

[0031] (e/d+h)²+e²=r0²

[0032] e={−b±(b²−4ac)}/2a  (3)

[0033] where,

[0034] a=d²+1

[0035] b=2dh

[0036] c=d²(h²r0²)

[0037] Therefore, the acoustic transfer angle θR of the right channel can be calculated by obtaining e by substituting equation (2) into equation (3) and by substituting obtained e into equation (1). The leftward acoustic transfer angle θL of the left channel can be obtained in similar manner. Consequently, the HRTF database 5 is referenced based on the calculated transfer angles θR and θL of the right and left channels obtained by the CPU 9. To be more specific, the HRTF database 5 stores a pair of filter parameters fR(θ) and fL(θ) at each sample point to represent the acoustic transfer characteristics or functions up to the right and left ears. Each sample point is determined by the fixed distance r0 and the calculated acoustic transfer angle θ. For the rightward transfer angle θR, fR(θR) of the pair of filter parameters fR(θR) and fL(θL) is selected. For the leftward transfer angle θL, fL(θL) of the pair of filter parameters fR(θR) and fL(θL) is selected. The FIR filters 3 and 4 may be operated by these obtained filter parameters fR(θR) and fL(θL), respectively, which represent effective acoustic transfer characteristics from the virtual sound source to the listener 11.

[0038] If the sampling point pitch of the HRTF database is coarse and therefore the effective acoustic transfer characteristic data corresponding to the obtained right and left transfer angles θR and θL does not exist in the HRTF database, vector compoint or composition may be performed on a pair of sample points oppositely adjacent to the calculated transfer angle so as to obtain the effective acoustic transfer characteristic data for each of the transfer angles θR and θL by interpolation.

[0039] It should be noted that the present invention is not limited to a system that has filter parameters in the database as mentioned above. For example, as shown in FIG. 4, the present invention is also applicable to a system in which the head-related transfer functions corresponding to the forward, backward, rightward, and leftward directions of the listener are given as fixed parameters of FIR filters 21 and 22. Directivity of the sound is imparted by performing amplitude control through amplifiers 23 and 24 according to the obtained transfer angles θR and θL on audio signals SR and SL supplied from the FIR filters 21 and 22, and then by adding amplified results through adders 25 and 26. It should be noted that the FIR filter 21 is divided into sections FRF, FRB, FRL, and FRR corresponding to the forward, backward, leftward and rightward directions. The amplifier 23 is also divided into sections VRF, VRB, VRL and VRR corresponding to the forward, backward, leftward and rightward directions. This holds true with the FIR filter 22 and the amplifier 24 of the left channel.

[0040] For summary, referring back again to FIG. 1, the inventive apparatus is constructed for positioning a sound image of a virtual sound source relative to a listener in a virtual sound field. In the apparatus, the database 5 provisionally memorizes acoustic transfer characteristics (fL, fR) of the virtual sound field in correspondence to sample points (θL, θR) distributed radially around a center point of the listener at a fixed radius. The left and right filters 3 and 4 respectively filter audio signals of left and right channels according to the acoustic transfer characteristics (fL, fR) loaded from the database 5. The input 10 designates a source point at which the virtual sound source is to be located within the virtual sound field in terms of a geometric distance r which may be different from the fixed radius and a geometric direction θ relative to the center point. The processor in the CPU 9 computes a leftward acoustic direction from the source point to a left ear of the listener according to the geometric distance r, the geometric direction θ and an offset of the left ear from the center point, and computes a rightward acoustic direction from the source point to a right ear of the listener according to the geometric distance r, the geometric direction θ and an offset of the right ear from the center point. The controller in the CPU 9 specifies a leftward sample point (PL) coincident with the leftward acoustic direction to load an effective acoustic transfer characteristic (fL) corresponding to the leftward sample point (PL) from the database 5 into the left filter 3, and specifies a rightward sample point (PR) coincident with the rightward acoustic direction to load another effective acoustic transfer characteristic (fR) corresponding to the rightward sample point (PR) from the database 5 into the right filter 4. The feeder 12 feeds an audio signal of the left channel to the left filter 3 and feeds another audio signal of the right channel to the right filter 4 to thereby direct the sound image of the virtual sound source located at the source point to the listener positioned at the center point.

[0041] The controller may specify a pair of leftward sample points which lie oppositely relative to the leftward acoustic direction such that said effective acoustic transfer characteristic is determined by interpolating the acoustic transfer characteristics corresponding to the pair of the leftward sample points, and may specify another pair of rightward sample points which lie oppositely relative to the rightward acoustic direction such that said another effective acoustic transfer characteristic is determined by interpolating the acoustic transfer characteristics corresponding to the pair of the rightward sample points.

[0042] The processor may compute an azimuth component of the leftward acoustic direction from the source point to the left ear of the listener in a three-dimensional space of the virtual sound field such that the leftward sample point is selected substantially coincident with the azimuth component of the leftward acoustic direction, and may compute an azimuth component of the rightward acoustic direction from the source point to the right ear of the listener such that the rightward sample point is selected substantially coincident with the azimuth component of the rightward acoustic direction, thereby directing the sound image of the virtual sound source to the listener in an azimuth direction of the three-dimensional space. Further, the controller may compute an elevation component of an acoustic direction from the source point to the listener according to the geometric distance r and the geometric direction φ, and the notch filter 1 filters the audio signal SI according to the elevation component of the acoustic direction so as to direct the sound image of the virtual sound source to the listener in an elevation direction of the three-dimensional space.

[0043] The machine readable medium 14 is for use in the inventive apparatus having the CPU 9 and positioning a sound image of a virtual sound source relative to a listener in a virtual sound field by filtering audio signals of left and right channels through left and right filters 3 and 4 which simulate acoustic transfer characteristics of the virtual sound field. As shown in FIG. 3, the medium contains program instructions executable by the CPU for causing the apparatus to perform the steps of provisionally memorizing acoustic transfer characteristics of the virtual sound field allotted to sample points (PL, PR) distributed radially around a center point Po of the listener at a fixed distance r0, designating a source point Ps at which the virtual sound source is to be located within the virtual sound field in terms of a geometric distance r and a geometric direction θ relative to the center point Po, computing a leftward acoustic direction (L) from the source point Ps to a left ear of the listener according to the geometric distance r, the geometric direction θ and an offset −h of the left ear from the center point Po, computing a rightward acoustic direction (R) from the source point Ps to a right ear of the listener according to the geometric distance r, the geometric direction θ and an offset +h of the right ear from the center point Po, selecting a leftward sample point PL substantially coincident with the leftward acoustic direction L to determine an effective acoustic transfer characteristic based on the acoustic transfer characteristic allotted to the leftward sample point PL so as to enable the left filter to simulate said effective transfer characteristic, selecting a rightward sample point PR substantially coincident with the rightward acoustic direction R to determine another effective acoustic transfer characteristic based on the acoustic transfer characteristic allotted to the rightward sample point PR so as to enable the right filter to simulate said another effective transfer characteristic, and filtering an audio signal of the left channel through the left filter and filtering another audio signal of the right channel through the right filter to thereby pan the sound image of the virtual sound source located at the source point Ps to the listener positioned at the center point Po.

[0044] Preferably, the step of selecting a leftward sample point comprises selecting a pair of leftward sample points which lie oppositely relative to the leftward acoustic direction such that said effective acoustic transfer characteristic is determined by interpolating the acoustic transfer characteristics allotted to the pair of the leftward sample points, and the step of selecting a rightward sample point comprises selecting a pair of rightward sample points which lie oppositely relative to the rightward acoustic direction such that said another effective acoustic transfer characteristic is determined by interpolating the acoustic transfer characteristics allotted to the pair of the rightward sample points.

[0045] Preferably, as shown in FIG. 3, the step of computing a leftward acoustic direction comprises computing an azimuth component (θL) of the leftward acoustic direction from the source point Ps to the left ear of the listener in a three-dimensional space of the virtual sound field such that the leftward sample point is selected substantially coincident with the azimuth component of the leftward acoustic direction, and the step of computing a rightward acoustic direction comprises computing an azimuth component (θR) of the rightward acoustic direction from the source point Ps to the right ear of the listener such that the rightward sample point is selected substantially coincident with the azimuth component of the rightward acoustic direction, thereby directing the sound image of the virtual sound source to the listener in an azimuth direction (θ) of the three-dimensional space. The inventive steps further comprise computing an elevation component (φ) of an acoustic direction from the source point Ps to the listener according to the geometric distance r and the geometric direction, and filtering the audio signal according to the elevation component (φ) of the acoustic direction so as to direct the sound image of the virtual sound source to the listener in an elevation direction of the three-dimensional space.

[0046] As described and according to the present invention, based on the distance and direction from a listener to a virtual sound source point and the distance between both ears of the listener, the acoustic directions from the virtual sound source point to both ears of the listener are independently calculated for the right and left channels. The acoustic transfer characteristics of the filters are determined by the obtained acoustic directions of the right and left channels. Consequently, even if the setting point from the listener to the virtual sound source point differs from the reference distance used for measurement of the acoustic transfer characteristics, the effective acoustic transfer characteristics corresponding to a specified virtual sound source point can be obtained, thereby providing good fidelity of the virtual sound field.

[0047] While the preferred embodiments of the present invention have been described using specific terms, such description is for illustrative purposes only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the appended claims.