An Affine Combination Of Two Lms Adaptive Filters

Asilomar, Tuesday, November 3, 2009

An Affine Combination of Two LMS Adaptive Filters

Statistical Analysis of an Error Power Ratio Scheme

Neil Bershad(1), Jose C. M. Bermudez(2) and Jean-Yves Tourneret(3)

(1) University of California, Irvine, USA

[email protected]

(2) Federal University of Santa Catarina, Florianopolis, Brazil

[email protected]

(3) University of Toulouse, ENSEEIHT-IRIT-TeSA, Toulouse, France

[email protected]

Asilomar Conf. on Signals, Systems, and Computers – p. 1/21


Property of most adaptive algorithms

Large step size µ

Fast convergence

Large steady-state weight misadjustment

Small step size µ

Slow convergence

Small steady-state weight misadjustment

☞ Possible solution: Variable µ algorithms

T. Aboulnasr and K. Mayyas, “A robust variable step-size LMS type algorithm: Analysisand simulations," IEEE Trans. Signal Process., vol. 45, pp. 631-639, March 1997.

H. C. Shin, A. H. Sayed and W. J. Song, “Variable step-size NLMS and affine projectionalgorithms,” IEEE Trans. Signal Process. Lett., vol. 11, pp. 132-135, Feb. 2004.



Affine combination of two adaptive filters

New approach recently proposedUse two adaptive filters with different step-sizesadapting on the same dataConvex combination of the adaptive filter outputsJ. Arenas-Garcia, A. R. Figueiras-Vidal, and A. H. Sayed, “Mean-square

performance of a convex combination of two adaptive filters,” IEEE Trans. Signal

Process., vol. 54, pp. 1078-1090, March 2006.

Affine combinationRecent study for affine combination of LMS filtersN. J. Bershad, J. C. M. Bermudez, and J-Y Tourneret, “An Affine Combination of

Two LMS Adaptive Filters - Transient Mean-Square Analysis,” IEEE Trans. Signal

Process., vol. 56, pp. 1853-1864, May 2008.



+

+

+

+

+

−

−

−

U (n)

W 1(n)

W 2(n)

y1(n)

y2(n)

e1(n)

e2(n)

y(n)

d(n)

λ(n)

1 − λ(n)

Adaptive combining of two transversal adaptive filters.Convex: λ(n) ∈ (0, 1) Affine: λ(n) ∈ R



Affine Combination Schemes

Two schemes for updating λ(n) proposed in 2008

Stochastic gradient approx. of opt. sequence λo(n).Analyzed inR. Candido, M. T. M. Silva and V. Nascimento, “Affine combinations of adaptive

filters,” (Asilomar 2008).

Power error ratioVery good performance but still not analyzed.



This paper

Analysis of the power ratio combination scheme

Mean behavior of λ(n)

Mean square deviation of weight vector

Monte Carlo simulations to verify the theoretical models

Statistical Assumptions

Input signal u(n) is white

Additive noise is white and uncorr. with u(n)

Unknown system is stationary



The Affine Combiner – Brief Review

LMS adaptation rule

W i(n + 1) = W i(n) + µiei(n)U(n), i = 1, 2 (1)

ei(n) = d(n) − WTi (n)U(n), (2)

d(n) = e0(n) + WTo U (n), (3)

Combination of filter outputs

y(n) = λ(n)y1(n) + [1 − λ(n)]y2(n), (4)

e(n) = d(n) − y(n), (5)

where yi(n) = WTi U(n) and λ(n) ∈ R.



Optimal Combining Rule

Optimal Combiner

λo(n) =

[

W o − W 2(n)]T [

W 1(n) − W 2(n)]

[

W 1(n) − W 2(n)]T [

W 1(n) − W 2(n)]

Steady-State Behavior

limn→∞

E[λo(n)]

≃ limn→∞

E[

WT2 (n)W 2(n)

]

− E[

WT2 (n)W 1(n)

]

E

{

[

W 1(n) − W 2(n)]T [

W 1(n) − W 2(n)]

} .



Optimal Combining Rule (cont.)

Expected Values

limn→∞

E[W T2 (n)W 1(n)]

= WTo W o +

µ1µ2Nσ2o

(µ1 + µ2) − µ1µ2(N + 2)σ2u

and

limn→∞

E[W Ti (n)W i(n)] = W

To W o+

µiNσ2o

2 − µi(N + 2)σ2u

, i = 1, 2.

After Simplifications

limn→∞

E[λo(n)] ≃ δ

2(δ − 1), δ = (µ2/µ1) < 1.



Error Power Ratio Based Scheme

λ(n) = 1 − κ erf(

e21(n)

e22(n)

)

where

e21(n) =

1

K

n∑

m=n−K+1

e21(m)

e22(n) =

1

K

n∑

m=n−K+1

e22(m)

and

erf(x) =2√π

∫ x

0

e−t2 dt.



The Value ofκ

Objective

limn→∞

E[λ(n)] ≃ limn→∞

E[λo(n)].

First order approximation

E[λ(n)] ≃ 1 − κ erf{

E[e21(n)]

E[e22(n)]

}

with

E[e2i (n)] = σ2

o +σ2

u

K

n∑

m=n−K+1

MSDi(m), i = 1, 2.

MSDi(n) = E{[W o − W i(n)]T [W o − W i(n)]}



The Value ofκ (cont.)

Taking the limn→∞

limn→∞

E[λ(n)] ≃ 1 − κ erf[

σ2o + σ2

u MSD1(∞)

σ2o + σ2

u MSD2(∞)

]

For limn→∞ E[λ(n)] ≃ limn→∞ E[λo(n)]

κ =

[

1 − δ

2(δ − 1)

]{

erf[

σ2o + σ2

u MSD1(∞)

σ2o + σ2

u MSD2(∞)

]}−1



Mean Behavior ofλ(n)

Define

ξ =e21(n)

e22(n)

, η = E (ξ) and σ2ξ = E(ξ2) − η2

Second order approximation

E[g(ξ)] ≃ g(η) +σ2

ξ

2g′′(η)

Mean Behavior

E[λ(n)] ≃ 1 − κ

[

erf(η) −2ησ2

ξ√π

e−η2

]

Expressions required for η and σ2ξ .



Mean Behavior ofλ(n) (cont.)

Approximation for η

Writing

e2i (n) = E[e2

i (n)] + εi = mi + εi, i = 1, 2

the mean η is approximated as

η = E

[

m1 + ε1

m2 + ε2

]

≃ m1

m2

=E [e2

1(n)]

E [e22(n)]



Mean Behavior ofλ(n) (cont.)Approximation for σ2

ξ

σ2ξ = E

{

(

e21(n)

e22(n)

)2}

− η2 ≃E

{

[e21(n)]

2}

E{

[e22(n)]

2} −

{

E [e21(n)]

E [e22(n)]

}2

Thus,

σ2ξ ≃ m2

2E(ε21) − m2

1E(ε22)

[m22 + E(ε2

2)]m22

with

E(ε2i ) =

2

K2

n∑

m=n−K+1

[

σ2o + σ2

u MSDi(m)]2

, i = 1, 2.



Mean-Square Deviation (MSD)

Error signal

e(n) = eo(n) +{

λ(n)[W o − W 1(n)]

+ [1 − λ(n)][W o − W 2(n)]}T

U (n)

Squaring and averaging

MSDc(n) = E[e2(n)] − σ2o ≃ σ2

u

{

E[λ2(n)]MSD1(n)

+ {1 − 2E[λ(n)] + E[λ2(n)]}MSD2(n)

+ 2{E[λ(n)] − E[λ2(n)]}MSD21(n)}

Expression for E[λ2(n)] is necessary.



Mean-Square Deviation (MSD) (cont.)

From the expression of λ(n)

E[λ2(n)] = 1 − 2κE[erf(ξ)] + κ2E[erf2(ξ)]

Expression of E[erf(ξ)]

E[erf(ξ)] ≃ erf(η) −2ησ2

ξ√π

e−η2

Second order approximation of E[erf2(ξ)]

E[erf2(ξ)] ≃ erf2(η) +2σ2

ξ√π

[

2√π

e−η2 − 2 η erf(η)

]

e−η2

The model for MSDc(n) is complete.



Simulation ResultsResponses to be identified: W o = [wo1

, . . . , woN]T

wok=

sin[2πfo(k − ∆)]

2πfo(k − ∆)

cos[2πrfo(k − ∆)]

1 − 4rfo(k − ∆), k = 1, . . . , N

In all simulations: N = 32, K = 100, σ2u = 1, 50 MC runs.

Example 1 Example 2

∆ = 10, r = 0.2, α = 1.2 ∆ = 5, r = 0, α = 3.8

0 5 10 15 20 25 30 35−0.2

0

0.2

0.4

0.6

0.8

1

1.2

sample k

Wo

0 5 10 15 20 25 30 35−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

sample k

Wo



Simulation Results – Example 1

σ2o = 10−4, δ =

µ2

µ1

= 0.1

0 500 1000 1500 2000 2500 3000 3500 4000−0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

iteration n

λ(n

)

0 500 1000 1500 2000 2500 3000 3500 4000−60

−50

−40

−30

−20

−10

0

iteration n

MS

Dc(n

)Optimal combination λo(n)

λ(n) obtained from the error power ratioTheoretical model for λ(n)



Simulation Results – Example 2

σ2o = 10−3, δ =

µ2

µ1

= 0.3

0 500 1000 1500 2000 2500 3000 3500 4000−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

iteration n

λ(n

)

0 500 1000 1500 2000 2500 3000 3500 4000−40

−35

−30

−25

−20

−15

−10

−5

0

5

iteration n

MS

Dc(n

)Optimal combination λo(n)

λ(n) obtained from the error power ratioTheoretical model for λ(n)



Conclusions

Affine combination two LMS adaptive filters studied.

Analysis of an error power ratio scheme

Tuning parameter κ determined for optimalsteady-state performance

Analytical model for E[λ(n)]

Analytical model for MSDc(n)

Monte Carlo Simulations show that

Error power ratio scheme is close to optimum

Analytical models are very accurate


An Affine Combination Of Two Lms Adaptive Filters

Documents

n asilomar

msdi n

schemee2 n

n rasilomar

y1 n w

w t u n

wheren1 e2 n

e1 n e e