ESE 531: Digital Signal Processing - seas.upenn.edu

ESE 531: Digital Signal Processing

Week 14Lecture 25: April 18, 2021

Adaptive Filters

Penn ESE 531 Spring 2021 – Khanna

Adaptive Filters

Penn ESE 531 Spring 2020 - Khanna 2

Adaptive Filters

! An adaptive filter is an adjustable filter that processes in time" It adapts…

3

AdaptiveFilter

Update Coefficients

x[n] y[n]d[n]

e[n]=d[n]-y[n]

+_

Penn ESE 531 Spring 2020 - Khanna

Adaptive Filter Applications

! System Identification

4Penn ESE 531 Spring 2020 - Khanna

Adaptive Filter Applications

! Identification of inverse system

5Penn ESE 531 Spring 2020 - Khanna

Adaptive Filter Applications

! Adaptive Interference Cancellation

6Penn ESE 531 Spring 2020 - Khanna

Adaptive Filter Applications

! Adaptive Prediction

7Penn ESE 531 Spring 2020 - Khanna

Stochastic Gradient Approach

! Most commonly used type of Adaptive Filters! Define cost function as mean-squared error

" Difference between filter output and desired response

! Based on the method of steepest descent" Move towards the minimum on the error surface to get to minimum" Requires the gradient of the error surface to be known

8Penn ESE 531 Spring 2020 - Khanna

Stochastic Gradient Approach

! Most commonly used type of Adaptive Filters! Define cost function as mean-squared error

" Difference between filter output and desired response

! Based on the method of steepest descent" Move towards the minimum on the error surface to get to minimum" Requires the gradient of the error surface to be known

9Penn ESE 531 Spring 2020 - Khanna

Least-Mean-Square (LMS) Algorithm

! The LMS Algorithm consists of two basic processes" Filtering process

" Calculate the output of FIR filter by convolving input and taps" Calculate estimation error by comparing the output to desired signal

" Adaptation process" Adjust tap weights based on the estimation error

10Penn ESE 531 Spring 2020 - Khanna

Adaptive FIR Filter: LMS

11Penn ESE 531 Spring 2020 - Khanna

Adaptive FIR Filter: LMS

12Penn ESE 531 Spring 2020 - Khanna

Adaptive FIR Filter: LMS

13Penn ESE 531 Spring 2020 - Khanna

aTb = a1 a2 a3 a4⎡⎣⎢

⎤⎦⎥

b1b2b3b4

⎡

⎣

⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥

= a1b1 + a2b2 + a3b3 + a4b4

∂ aTb( )∂a

=∂ aTb( )∂a1

∂ aTb( )∂a2

∂ aTb( )∂a3

∂ aTb( )∂a4

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

T

=

b1b2b3b4

⎡

⎣

⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥

= b

Adaptive FIR Filter: LMS

14Penn ESE 531 Spring 2020 - Khanna

Adaptive FIR Filter: LMS

15Penn ESE 531 Spring 2020 - Khanna

Adaptive Filter Applications

! Adaptive Interference Cancellation

16Penn ESE 531 Spring 2020 - Khanna

Adaptive Interference Cancellation

17Penn ESE 531 Spring 2020 - Khanna

Adaptive Interference Cancellation

18Penn ESE 531 Spring 2020 - Khanna

s[n]= s[n]+w[n]− w[n]= s[n]+w[n]− hn

T !wn

Adaptive Interference Cancellation

19Penn ESE 531 Spring 2020 - Khanna

s[n]= s[n]+w[n]− w[n]= s[n]+w[n]− hn

T !wnMinimizing (ŝ[n])2 removes noise w[n]

Adaptive Interference Cancellation

20Penn ESE 531 Spring 2020 - Khanna

s[n]= s[n]+w[n]− w[n]= s[n]+w[n]− hn

T !wnMinimizing (ŝ[n])2 removes noise w[n]

s[n]( )2= s[n]+w[n]− hn

T !wn( )2

∂s2[n]∂hn

= 2 s[n]+w[n]− hnT !wn( )(− !wn )

= 2s[n](− !wn ) = −2s[n] !wn

Adaptive Interference Cancellation

21Penn ESE 531 Spring 2020 - Khanna

Stability of LMS

! The LMS algorithm is convergent in the mean square if and only if the step-size parameter satisfy

! Here lmax is the largest eigenvalue of the correlation matrix of the input data

! More practical test for stability is

! Larger values for step size" Increases adaptation rate (faster adaptation)" Increases residual mean-squared error

22

max

20l

<µ<

power signal input20 <µ<

Big Ideas

23Penn ESE 531 Spring 2020 – KhannaAdapted from M. Lustig, EECS Berkeley

! Adaptive Filters" Use LMS algorithm to update filter coefficients" Applications like system ID, channel equalization, and

signal prediction

Admin

! Project 2" Out now" Due 4/30

! Final Exam – 5/5" Administered in Canvas

" 2hr timed exam in 12hr window

" Open notes" Covers lec 1-24*

" Doesn’t include lecture 13

24Penn ESE 531 Spring 2021 – KhannaAdapted from M. Lustig, EECS Berkeley