Page 1
Professor A G
Constantinides© 1
AGC
DSP
AGC
DSP
Adaptive Signal Processing Problem: Equalise through a FIR filter the
distorting effect of a communication channel that may be changing with time.
If the channel were fixed then a possible solution could be based on the Wiener filter approach
We need to know in such case the correlation matrix of the transmitted signal and the cross correlation vector between the input and desired response.
When the the filter is operating in an unknown environment these required quantities need to be found from the accumulated data.
Page 2
Professor A G
Constantinides© 2
AGC
DSP
AGC
DSP
Adaptive Signal Processing The problem is particularly acute when
not only the environment is changing but also the data involved are non-stationary
In such cases we need temporally to follow the behaviour of the signals, and adapt the correlation parameters as the environment is changing.
This would essentially produce a temporally adaptive filter.
Page 3
Professor A G
Constantinides© 3
AGC
DSP
AGC
DSP
Adaptive Signal Processing
A possible framework is:
][nd][ˆ nd]}[{ nx
][new:Filter
Adaptive
Algorithm
Page 4
Professor A G
Constantinides© 4
AGC
DSP
AGC
DSP
Adaptive Signal Processing Applications are many
Digital Communications Channel Equalisation Adaptive noise cancellation Adaptive echo cancellation System identification Smart antenna systems Blind system equalisation And many, many others
Page 5
Professor A G
Constantinides© 5
AGC
DSP
AGC
DSP
Applications
Page 6
Professor A G
Constantinides© 6
AGC
DSP
AGC
DSP
Adaptive Signal Processing
Echo Cancellers in Local Loops
-
+
-
+Rx1 Rx2
Tx1 Rx2
Echo canceller Echo canceller
Adaptive Algorithm Adaptive Algorithm
Hybrid Hybrid
Local Loop
Page 7
Professor A G
Constantinides© 7
AGC
DSP
AGC
DSP
Adaptive Signal Processing
Adaptive Noise Canceller
Noise
Signal +Noise
-
+
FIR filter
Adaptive Algorithm
PRIMARY SIGNAL
REFERENCE SIGNAL
Page 8
Professor A G
Constantinides© 8
AGC
DSP
AGC
DSP
Adaptive Signal Processing
System Identification
Unknown System
Signal
-
+
FIR filter
Adaptive Algorithm
Page 9
Professor A G
Constantinides© 9
AGC
DSP
AGC
DSP
Adaptive Signal Processing
System Equalisation
Unknown System
Signal
-
+
FIR filter
Adaptive Algorithm
Delay
Page 10
Professor A G
Constantinides© 10
AGC
DSP
AGC
DSP
Adaptive Signal Processing
Adaptive Predictors
Signal
-
+
FIR filter
Adaptive Algorithm
Delay
Page 11
Professor A G
Constantinides© 11
AGC
DSP
AGC
DSP
Adaptive Signal Processing
Adaptive Arrays
Linear Combiner
Interference
Page 12
Professor A G
Constantinides© 12
AGC
DSP
AGC
DSP
Adaptive Signal Processing Basic principles: 1) Form an objective function
(performance criterion) 2) Find gradient of objective function
with respect to FIR filter weights 3) There are several different
approaches that can be used at this point
3) Form a differential/difference equation from the gradient.
Page 13
Professor A G
Constantinides© 13
AGC
DSP
AGC
DSP
Adaptive Signal Processing Let the desired signal be The input signal The output Now form the vectors
So that
][nd][nx
][ny
Tmnxnxnxn ]1[.]1[][][ x
Tmhhh ]1[.]1[]0[ h
hx Tnny ][][
Page 14
Professor A G
Constantinides© 14
AGC
DSP
AGC
DSP
Adaptive Signal Processing
The form the objective function
where
}][][{)( 2nyndEJ w
Rhhphhpw TTTdJ 2)(
}][][{ TnnE xxR
]}[][{ ndnE xp
Page 15
Professor A G
Constantinides© 15
AGC
DSP
AGC
DSP
Adaptive Signal Processing
We wish to minimise this function at the instant n
Using Steepest Descent we write
But
][])[(
21
][]1[nnJ
nnhh
hh
Rhphh
22)(
J
Page 16
Professor A G
Constantinides© 16
AGC
DSP
AGC
DSP
Adaptive Signal Processing So that the “weights update equation”
Since the objective function is quadratic this expression will converge in m steps
The equation is not practical If we knew and a priori we could
find the required solution (Wiener) as
])[(][]1[ nnn Rhphh
pR
pRh 1opt
Page 17
Professor A G
Constantinides© 17
AGC
DSP
AGC
DSP
Adaptive Signal Processing However these matrices are not known Approximate expressions are obtained
by ignoring the expectations in the earlier complete forms
This is very crude. However, because the update equation accumulates such quantities, progressive we expect the crude form to improve
Tnnn ][][][ˆ xxR ][][][ˆ ndnn xp
Page 18
Professor A G
Constantinides© 18
AGC
DSP
AGC
DSP
The LMS Algorithm Thus we have
Where the error is
And hence can write
This is sometimes called the stochastic gradient descent
])[][][]([][]1[ nnndnnn T hxxhh
])[][(])[][][(][ nyndnnndne T hx
][][][]1[ nennn xhh
Page 19
Professor A G
Constantinides© 19
AGC
DSP
AGC
DSP
Convergence
The parameter is the step size, and it should be selected carefully
If too small it takes too long to converge, if too large it can lead to instability
Write the autocorrelation matrix in the eigen factorisation form
ΛQQR T
Page 20
Professor A G
Constantinides© 20
AGC
DSP
AGC
DSP
Convergence Where is orthogonal and is
diagonal containing the eigenvalues The error in the weights with respect
to their optimal values is given by (using the Wiener solution for
We obtain
Q Λ
])[(][]1[ nnn optoptopt RhRhhhhh
p
][][]1[ nnn hhh Reee
Page 21
Professor A G
Constantinides© 21
AGC
DSP
AGC
DSP
Convergence Or equivalently
I.e.
Thus we have
Form a new variable
][)1(]1[ nn hh eΛQQe T
][)(
][)1(]1[
n
nn
h
hh
eΛQQQQ
eΛQQQQeT
T
][)1(]1[ nn hh QeΛQe
][][ nn hQev
Page 22
Professor A G
Constantinides© 22
AGC
DSP
AGC
DSP
Convergence So that
Thus each element of this new variable is dependent on the previous value of it via a scaling constant
The equation will therefore have an exponential form in the time domain, and the largest coefficient in the right hand side will dominate
][)1(]1[ nn vΛv
Page 23
Professor A G
Constantinides© 23
AGC
DSP
AGC
DSP
Convergence
We require that
Or
In practice we take a much smaller value than this
11 max
max
20
Page 24
Professor A G
Constantinides© 24
AGC
DSP
AGC
DSP
Estimates Then it can be seen that as the
weight update equation yields And on taking expectations of both
sides of it we have
Or
n
]}[{]}1[{ nEnE hh
])}[][][]([{]}[{]}1[{ nnndnEnEnE T hxxhh
])}[][][][][{(0 nnnndnE T hxxx
Page 25
Professor A G
Constantinides© 25
AGC
DSP
AGC
DSP
Limiting forms
This indicates that the solution ultimately tends to the Wiener form
I.e. the estimate is unbiased
Page 26
Professor A G
Constantinides© 26
AGC
DSP
AGC
DSP
Misadjustment The excess mean square error in the
objective function due to gradient noise Assume uncorrelatedness set
Where is the variance of desired response and is zero when uncorrelated.
Then misadjustment is defined as
optT
dJ hp 2min
2d
opth
minmin /))(( JJJJ LMSXS
Page 27
Professor A G
Constantinides© 27
AGC
DSP
AGC
DSP
Misadjustment
It can be shown that the misadjustment is given by
m
i i
iXS JJ
1min 1
/
Page 28
Professor A G
Constantinides© 28
AGC
DSP
AGC
DSP
Normalised LMS To make the step size respond to
the signal needs
In this case
And misadjustment is proportional to the step size.
][][][1
2][]1[ 2 nen
nnn x
xhh
10
Page 29
Professor A G
Constantinides© 29
AGC
DSP
AGC
DSP
Transform based LMS
][nd][ˆ nd]}[{ nx
][new:Filter
Adaptive
Algorithm
Transform
Inverse Transform
Page 30
Professor A G
Constantinides© 30
AGC
DSP
AGC
DSP
Least Squares Adaptive with
We have the Least Squares solution
However, this is computationally very intensive to implement.
Alternative forms make use of recursive estimates of the matrices involved.
n
i
Tiin1
][][][ xxR
n
indnn
1][][][ xp
][][][ 1 nnn pRh
Page 31
Professor A G
Constantinides© 31
AGC
DSP
AGC
DSP
Recursive Least Squares Firstly we note that
We now use the Inversion Lemma (or the Sherman-Morrison formula)
Let
][][]1[][ ndnnn xpp
Tnnnn ][][]1[][ xxRR
Page 32
Professor A G
Constantinides© 32
AGC
DSP
AGC
DSP
Recursive Least Squares (RLS)
Let
Then
The quantity is known as the Kalman gain
][]1[][1][]1[
][ 1
1
nnnnn
n T xRxxR
k
1][][ nn RP
]1[][][]1[][ nnnnn T PxkRP
][nk
Page 33
Professor A G
Constantinides© 33
AGC
DSP
AGC
DSP
Recursive Least Squares Now use in the computation
of the filter weights
From the earlier expression for updates we have
And hence
][][][ nnn xPk
])[][]1[]([][][][ ndnnnnnn xpPpPh ][nP
]1[]1[][][]1[]1[]1[][ nnnnnnnn T pPxkpPpP
])1[][][]([]1[][ nnndnnn T hxkhh
Page 34
Professor A G
Constantinides© 34
AGC
DSP
AGC
DSP
Kalman Filters Kalman filter is a sequential estimation
problem normally derived from The Bayes approach The Innovations approach
Essentially they lead to the same equations as RLS, but underlying assumptions are different
Page 35
Professor A G
Constantinides© 35
AGC
DSP
AGC
DSP
Kalman Filters The problem is normally stated as:
Given a sequence of noisy observations to estimate the sequence of state vectors of a linear system driven by noise.
Standard formulation][][]1[ nnn wAxx ][][][][ nnnn νxCy
Page 36
Professor A G
Constantinides© 36
AGC
DSP
AGC
DSP
Kalman Filters Kalman filters may be seen as RLS with
the following correspondenceSate space
RLS Sate-Update matrix Sate-noise variance Observation matrix Observations State estimate
][nA
][nx
Tn][x][nC][ny
][nh
}][][{][ TnnEn wwQ I0
][nd
Page 37
Professor A G
Constantinides© 37
AGC
DSP
AGC
DSP
Cholesky Factorisation In situations where storage and to some
extend computational demand is at a premium one can use the Cholesky factorisation tecchnique for a positive definite matrix
Express , where is lower triangular
There are many techniques for determining the factorisation
TLLR L