Filter bank approach to adaptive filtering method using independent component analysis

[0001] The present invention relates to a filter bank approach to adaptive filtering method using independent component analysis. More particularly, the invention relates to a method of improving the performance of adaptive filtering method by applying independent component analysis that is capable of reflecting the secondary or even higher order statistical characteristics to adaptive filtering algorithm using the filter bank approach.

[0002] Independent component analysis is capable of separating default signals by receiving input signals of mixed sound sources from unknown channels and restoring mutually independent sound source signals.

[0003] Let us assume that N number of probabilistically independent sound sources s(t)=[s₁(t), s ₂(t), . . . , s_N(t)]^T are mixed through arbitrary channels and the sound sources are measured by N number of sensors.

[0004] In this instance, if the channels could be modelled by a simple addition of weighting factors, then the measured signals x(t)=[x₁(t), x₂(t), . . . , x_N(t)]^T by the sensors could be represented as follows.
x(t)=A·s(t)  [Mathematical Equation 1]

[0005] where, A is an unknown invertible matrix which is known as mixed matrix.

Hence, the problem here is to obtain a reverse matrix of the mixed matrix A in order to restore the sound source signals only using the sensor signals that can be measured.

[0006] However, from the perspective of the sound source signal separation, the process of obtaining a restored signal whose original sequence of the sound source signal is reversed or a restored signal whose magnitude differs significantly from the sound source signal, would not cause a serious problem since this process does not affect the waveform itself.

[0007] Consequently, a restored signal u(t) whose magnitude and sequence are different from the sound source signal is obtained from the following mathematical equation by deducing a separation matrix W which permits the above process.
u(t)=W·x(t)  [Mathematical Equation 2]

[0008] In this instance, in order to deduce the separation matrix W, an assumption which makes the sound sources to be mutual independent is established. This means that a signal originating from one sound source dose not interfere with another signal which is originating from another sound source. This is a reasonable assumption to be made in the real world and this statistical independence is inclusive of all the statistical characteristics for any order.

[0009] Also, since the statistical independence is not related to magnitude and sequence, the separation matrix W in Mathematical Equation 2 can be obtained. The separation matrix could be trained by using the following training method which maximizes the statistical independence among the sound source signals.

[0010] $\begin{array}{cc}\Delta W\propto {\left[W^T\right]}^{-1}-\phi \left(u\right)x^T,\phi \left(u_i\left(t\right)\right)=-\frac{\frac{\partial P\left(u_i\left(t\right)\right)}{\partial u_i\left(t\right)}}{P\left(u_i\left(t\right)\right)}.& \left[\mathrm{Mathematical}\mathrm{Equation}3\right]\end{array}$

[0011] where, P(u_i(t)) represents a value which approximates the probability density function of the sound source signal u_i(t) which is to be deduced.

[0012] A mixed signal, which is a simple addition of weighting factors, is hard to find in the real world, instead the sensor receives a mixed signal that is a convolution type mixture between time delays due to the transmission and surrounding environment. The convolution type mixed signal could be represented as follows.

[0013] $\begin{array}{cc}{x}_i\left(t\right)=\sum _{j=1}^N\sum _{k=0}^{K-1}{a}_{\mathrm{ij}}\left(k\right)s_j\left(t-k\right)& \left[\mathrm{Mathematical}\mathrm{Equation}4\right]\end{array}$

[0014] where, x_it) represents measured signals, s_j(t) represents N number of sound sources and a _ij(k) represents mixed filter coefficients for the length K.

[0015] In order to separate the sound source signal, a feedforward filter structure in FIG. 1 or a feedback filter structure in FIG. 2 could be utilized.

[0016] In the feedforward filter structure, a restored signal u_i(t) could be expressed as follows.

[0017] $\begin{array}{cc}{u}_i\left(t\right)=\sum _{j=1}^N\sum _{k=0}^K{w}_{\mathrm{ij}}\left(k\right)x_j\left(t-k\right)& \left[\mathrm{Mathematical}\mathrm{Equation}5\right]\end{array}$

[0018] where, w_ij(k) is filter coefficients for restoring the original sound source signal.

[0019] In this structure, the corresponding training methods for the coefficients w_ii(0) with no time delay between input and output and the coefficients w_ii(k), k≠0 with a time delay between the input and output could be expressed as follows.

[0020] $\begin{array}{cc}\Delta W\left(0\right)\propto {\left[{W\left(0\right)}^T\right]}^{-1}-\phi \left(u\left(t\right)\right){x\left(t\right)}^T,\Delta w_{\mathrm{ij}}\left(k\right)\propto -\phi \left(u_j\left(t\right)\right)x_j\left(t-k\right),\phi \left(u_i\left(t\right)\right)=\frac{\frac{\partial P\left(u_i\left(t\right)\right)}{\partial u_i\left(t\right)}}{P\left(u_i\left(t\right)\right)}& \left[\mathrm{Mathematical}\mathrm{Equation}6\right]\end{array}$

[0021] where, w(0) is a matrix consisting of the coefficients with no time delay w_ij(0) and each of u(t) and x(t) represents vectors consisted of u_i(t) and x_i(t) respectively.

[0022] The coefficients w_ij(0) with no time delay control the size of data in order to maximize the amount of information which is transferred through a nonlinear function. The coefficients w_ij(k) with a time delay remove the correlation between each of the outputs φ(u_i(t)) of the nonlinear functions and input signals x_j(t).

[0023] The restored signals u_i(t) for the feedback filter structure could be expressed as follows.

[Mathematical Equation 7]

[0024] $\begin{array}{cc}\begin{array}{c}{u}_i\left(t\right)=\sum _{k=0}^K{w}_{\mathrm{ii}}\left(k\right)x_i\left(t-k\right)+\\ \sum _{j=1}^N\sum _{j\ne ik=1}^K{w}_{\mathrm{ij}}\left(k\right)u_j\left(t-k\right)\end{array}& \left[\mathrm{Mathematical}\mathrm{Equation}7\right]\end{array}$

[0025] In this structure, there exists three different cases of filter coefficients. More specifically, they are the coefficients with no time delay between the input and output w_ii(0), the coefficients with a time delay w_ii(k), k≠0 and the differential feedback filter coefficients w_ij(k),i≠j. The training methods for each of the cases are as follows.

[0026] $\begin{array}{cc}\Delta w_{\mathrm{ii}}\left(0\right)\propto 1/w_{\mathrm{ii}}\left(0\right)-\phi \left(u_i\left(t\right)\right)x_i\left(t\right),\Delta w_{\mathrm{ii}}\left(k\right)\propto -\phi \left(u_i\left(t\right)\right)x_i\left(t-k\right),\Delta w_{\mathrm{ij}}\left(k\right)\propto -\phi \left(u_i\left(t\right)\right)u_j\left(t-k\right),\phi \left(u_i\left(t\right)\right)=\frac{\frac{\partial P\left(u_i\left(t\right)\right)}{\partial u_i\left(t\right)}}{P\left(u_i\left(t\right)\right)}& \left[\mathrm{Mathematical}\mathrm{Equation}8\right]\end{array}$

[0027] The coefficients w_ii(0) control the size of data in order to maximize the amount of information which is transferred through a nonlinear function. The other coefficients w_ii(k),k≠0 with a time delay produce output signals corresponding to whitening of each of the input signals with respect to time. The coefficients w_ij(k),i≠j remove the correlation between each of the outputs φ(u_i(t)) of the nonlinear functions and input signals x_j(t).

[0028] FIG. 1 and FIG. 2 represent a feedforward and feedback filter structures in which each structure consists of two input and two output terminals, however, the total number of input and output terminals for each structure could be increased in order to construct a filter structure with an arbitrary number of input and output terminals. For the filter structure with an arbitrary number of input and output terminals, the same mathematical equations as represented in mathematical equations 4 through 8 which describe a convolution type mixed signal, restored signal and filter coefficient training method could equally be applied.

[0029] The restored sound source signal for the convolution type mixed signal and original sound source signal appears as whitened with respect to time. This whitening problem could be resolved by leaving the filter with the coefficients w_ii(0) between the inputs and outputs with the same index and fixing the value of the coefficients w_ii(k), k≠0 with a time delay to zero.

[0030] In case when pure noises can be observed, the independent component analysis could be utilized as an adaptive noise filter where the noise component of the mixed signal is removed.

[0031] FIG. 3 shows a diagram which illustrates the structure of a conventional adaptive noise filter.

[0032] As shown in FIG. 3, a signal source S(10) is transferred to a sensor via a channel, and a mixed signal with noise s+n₀ which is generated in the sensor by addition of a noise source n₀ (20) forms a primary input (30) for the noise filter.

[0033] A secondary sensor receives a noise n₁ via a channel which is different to that of the first sensor, it forms a reference input (40) with respect to the noise filter.

[0034] The noise n₁ is transformed into an output Z which is nearest to n₀ through an adaptive filter (50) and a system output u=s+n₀−Z (70) of the noise filter is formed by deducting the output Z from the primary input s+n₀.

[0035] The purpose of the conventional adaptive noise filter is to obtain an output u=s+n₀−Z which is nearest to the signal S from the perspective of least square. In order to achieve this object, the filter is adapted with the Least Mean Square algorithm (LMS) to minimize the total outputs of the noise filter. More specifically, the outputs of the adaptive noise filter act as an error signal for the duration of the adaptive process.

[0036] This coefficient training of the filter follows the LMS algorithm (Widrow-Hoff) and could be expressed as follows.

[0037] $\begin{array}{cc}\Delta w\left(k\right)\propto u\left(t\right)n_1\left(t-k\right)& \left[\mathrm{Mathematical}\mathrm{Equation}9\right]\end{array}$

[0038] where, w(k) represents a k^th filter coefficient and t is a sampling number.

[0039] The conventional noise filter as shown above utilizes the LMS method as the adaptive algorithm which only uses the secondary statistical characteristic between the noise component which is to be removed and noise sources.

[0040] In order to obtain a better noise filtering performance than the LMS adaptive algorithm, the independent component analysis could be utilized.

[0041] As shown in FIG. 3 which shows a conventional adaptive noise filter, when a signal which forms the primary input, a mixed signal of noise s+n₀, and a noise n₁ in the secondary sensor are being received, it could be assumed that the noise n₁ is independent from the signal S whereas it is related to n₀ in unknown ways.

[0042] This is a reasonable assumption to be made since the noise and signal do not affect their respective sound sources. Using the independent component analysis, the noise dependent components of the first input could be removed by using the second input as following.

[0043] $\begin{array}{cc}\Delta w\left(k\right)\propto \phi \left(u\left(t\right)\right)n_1\left(t-k\right),\phi \left(u\left(t\right)\right)=\frac{\frac{\partial P\left(u\left(t\right)\right)}{\partial u\left(t\right)}}{P\left(u\left(t\right)\right)}& \left[\mathrm{Mathematical}\mathrm{Equation}10\right]\end{array}$

[0044] From the output of the adaptive noise filter which is obtained from the filter coefficient training, the required signal components which are independent from the noise components are obtained. The adaptive noise filter as in the case of default signal separation, the total number of input and output terminals for each structure could be increased in order to construct a filter structure with an arbitrary number of input and output terminals. For the filter structure with an arbitrary number of input and output terminals, the same mathematical equations as represented in mathematical equations 4 through 8, which describe the filter coefficient training method, could equally be applied.

[0045] However, in order to implement the adaptive filter algorithm using independent component analysis to the real world problem, a large number of filter training coefficients are required and also a large amount of calculation is required when a training is undertaken. This results in a very slow learning speed and the deterioration in the quality of result signals.

[0046] The object of the present invention is to provide a filter bank approach to adaptive filtering method using independent component analysis in order to improve the performance of the conventional adaptive filtering method using independent component analysis.

[0047] The adaptive filtering method using independent component analysis according to the present invention provides a method of reducing the large amount of calculation required for filter training, improving the learning speed and the quality of result signals by utilizing a filter bank approach

• [0054] 10: Signal Source
• [0055] 20: Noise Source
• [0056] 30: Primary Input
• [0057] 40: Reference Input
• [0058] 50: Adaptive Filter
• [0059] 60: Adder
• [0060] 70: System Output
  • [0061] w₁₁w₁₂w₂₁w₂₂: Adaptive Filter

[0062] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0063] FIG. 4 shows a diagram which illustrates a multi-sampling system using the filter bank approach according to the present invention. An input signal x(n) is separated into K number of signals through K number of analysis filters H_k(Z) and takes in a sample signal at every M_k≦k time from each signal x_k(n) in order to form a signal v_k(n). In this instance, the signal v_k(n) could be represented in a Z domain.

[0064] $\begin{array}{cc}\begin{array}{c}{V}_k\left(z\right)=\frac{1}{M_k}\sum _{m=0}^{M_k-1}{X}_k\left(z{W}_{M_k}^m\right)\\ =\frac{1}{M_k}\sum _{m=0}^{M-1}{H}_k\left(z{W}_{M_k}\right)X\left(z{W}_{M_k}\right)\end{array}& \left[\mathrm{Mathematical}\mathrm{Equation}11\right]\end{array}$

[0065] where W_Mi=e^−j2zmk. From this signal, u_k(n) is formed by inserting M_k−1 number of samples whose value are zero between every two consecutive samples and this could be represented in a Z domain.

[0066] $\begin{array}{cc}\begin{array}{c}{U}_k\left(z\right)=V_k\left(z^{M_l}\right)\\ =\frac{1}{M_k}\sum _{m=0}^{M-1}{H}_k\left(z{W}_{M_l}^m\right)X\left(z{W}_{M_l}^m\right)\end{array}& \left[\mathrm{Mathematical}\mathrm{Equation}12\right]\end{array}$

[0067] A signal which is nearest to the original signal {circumflex over (x)}(n) is restored from u_k(n) through a synthesis filter F_k(Z).

[0068] $\begin{array}{cc}\hat{X}\left(z\right)=\sum _{k=0}^{K-1}{F}_k\left(z\right)U_k\left(z\right)=\frac{1}{M_k}\sum _{m=0}^{M-1}X\left({\mathrm{zW}}_{M_k}^m\right)\sum _{k=0}^{K-1}{H}_k\left({\mathrm{zW}}_{M_k}^m\right)F_k\left(z\right)& \left[\mathrm{Mathematical}\mathrm{Equation}13\right]\end{array}$

[0069] A polyphase representation is another way of analyzing a multi-sampling system and this representation enables an implementation which reduces the amount of calculation. Also, an implementation based on Fast Fourier Transform could also be considered.

[0070] FIG. 5 shows a diagram which illustrates the structure of default signal separator using the filter bank approach according to the present invention. Each input signal passes through an analysis filter bank and K number of signals are constructed by taking one sample at M_k time for an output signal H_k(z). The separated signals could be obtained by applying the default signal separation method using independent component analysis to this signal and the original separation signal could be restored by inserting M_k−1 number of samples whose value are zero between every two consecutive samples and passing them through synthesis filter banks.

[0071] In this instance, as mentioned previously, the default signal separator using independent component analysis produces output signals whose magnitude and sequence are reversed. Hence, the output signals whose magnitude and sequence are reversed are produced after receiving input signals from the analysis filter bank. In order to recover the original sound source signal, the processes of compensating the magnitude of the signals and collecting the output signals from an identical source among the reversed signals are required. In general, the process of magnitude compensation for a feedback filter structure is carried out by fixing the filter coefficients as ^w_ij(0)=1 which have no time delay between the input and output with the same index or fixing specific values to the specific time delay filter coefficients. Also, the process of a magnitude compensation for the feedforward filter structure is carried out by fixing specific values to the specific time delay coefficients or normalizing the filter coefficients in order to make the energy ratio constant between the input and output which are transferred through a network.

[0072] The process of collecting the output signals from an identical source among the signals whose magnitude and sequence are reversed is carried out, by comparing the a number of characteristics including a statistical characteristic of cumulant or the correlation of the output signals which corresponds to other analysis filters or by comparing the characteristics of the feedback or feedforward filter structures.

[0073] FIG. 5 represent a structure which consists of two input and two output terminals respectively, however, the total number of input and output terminals for each structure could be increased in order to construct a filter structure with an arbitrary number of input and output terminals. For the filter structure with the arbitrary number of input and output terminals, the same methods could be applied.

[0074] The filter bank approach could be applied to adaptive noise filtering method for removing pure noises.

[0075] FIG. 6 shows a diagram which illustrates an adaptive noise filter using the filter bank approach according to the present invention.

[0076] Each of the primary and reference inputs pass through an analysis filter bank and after taking one sample at M_k time for an output signal H_k(t), a signal without noises are obtained by applying the adaptive noise filtering method using independent component analysis. The noise filter signal could be restored by passing through a synthesis filter bank after inserting M_k−1 number of samples whose values are zero to the above signal.

[0077] The adaptive noise filter as in the case of the default signal separator, the total number of input and output terminals for each structure could be increased in order to construct a filter structure with an arbitrary number of input and output terminals. For the filter structure with an arbitrary number of input and output terminals, the same methods could be applied.

[0078] The independent component analysis could be applied to various types of adaptive filter algorithms other than default signal separator or adaptive noise filter.

[0079] In this case, the filter bank approach, which uses the same method of passing each input signal through an analysis filter bank and applying an adaptive noise filter using independent component analysis after taking one sample at M_k time for an output signal of Kth analysis filter bank II_k(z) and inserting M_k−1 number of samples whose values are zero to the above signal and again passing through the analysis filter bank, could be adopted.

[0080] The invention has been described with reference to the preferred embodiments. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the invention be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof. Finally, as explained so far, according to the filter bank approach to adaptive filtering method using independent component analysis according to the present invention, an improved performance is achieved in terms of reducing the amount of calculation for filter training, improving the learning speed and the quality of result signals when compared to the other methods which do not use the filter bank approach.