05/07/09 Ec44440.SpFY09/MPF - Section VI 1 VI. Linear Models [p. 3] Definition of AR, MA & ARMA models [p. 5] Relationship between AR, MA, & ARMA models [p. 10] How to find ARMA model parameters: Yule-Walker equation solutions [p. 29] AR modeling - [p. 32] Autocorrelation method - [p. 41] covariance method - [p. 46] SVD-based autocorrelation/covariance method - [p. 55] Applications to signal frequency estimation techniques - [p. 59] Burg method - [p. 62] Filter order selection - [p. 65] Model validation - [p.66] Appendices
72
Embed
faculty.nps.edufaculty.nps.edu/fargues/teaching/ec4440/SpringFY09/ec4440-VI-SpFY... · 05/07/09 Ec44440.SpFY09/MPF - Section VI 2 References: [Therrien], [Manolakis], [Hayes]: M.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
05/07/09 Ec44440.SpFY09/MPF - Section VI 1
VI. Linear Models
[p. 3] Definition of AR, MA & ARMA models[p. 5] Relationship between AR, MA, & ARMA models[p. 10] How to find ARMA model parameters: Yule-Walker equation solutions[p. 29] AR modeling
- [p. 32] Autocorrelation method- [p. 41] covariance method- [p. 46] SVD-based autocorrelation/covariance method- [p. 55] Applications to signal frequency estimation techniques - [p. 59] Burg method- [p. 62] Filter order selection- [p. 65] Model validation- [p.66] Appendices
05/07/09 Ec44440.SpFY09/MPF - Section VI 2
References:[Therrien], [Manolakis], [Hayes]: M. Hayes, Statistical Digital Signal Processing and Modeling, Wiley & Sons, 1996.[Haykin]: S. Haykin, Modern Filters, MacMillan, 1989.[Kumaresan82a]: R. Kumaresan & D. Tufts, “Estimating the parameters of exponentially damped sinusoids and pole-zero modeling in noise,” IEEE Trans. on ASSP, Vol. ASSP-30, no. 6, Dec. 1982, pp. 833 - 840.[Kumaresan82b]:D. Tufts & R. Kumaresan, “Estimation of frequencies of multiple sinusoids: Making linear prediction perform like maximum likelihood,” IEEE Proceedings, Vol. 70, Issue 9, Sept. 1982 pp. 975 – 989. [MathWorks]: www.mathworks.com
05/07/09 Ec44440.SpFY09/MPF - Section VI 3
• To model a stationary RPuse -- correlation function
-- power spectral density -- parametric model
• Why are parametric models important?
(1) Useful in spectral estimation
-- if we have some knowledge about the process generating the RP, we can use it to select a model which is an approximation of the process
-- next, we can get an estimated PSD by using the model
(2) Useful for time series extrapolation and and interpolation.
(3) Useful for bandwidth compression. (speech processing applications)
05/07/09 Ec44440.SpFY09/MPF - Section VI 4
( )( ) ( )( ) ( ) ( )
( ) ( )( )
( )
* 1
* 1
2
2
n w
jj j
x wj
B z B zS z S z
A z A z
B eS e S e
A e
ωω ω
ω
−
−=
=
• Types of models considered
( )zAb
w(n) x(n)
( )( )zAzB
w(n) x(n)
( )zBw(n) ( ) ( )inwbnxP
ii −= ∑
0=
(1) AR
(2) Moving average (MA)
(3) AR-MA (ARMA)
(IIR)
(FIR)
white noise
noise white:)(nwez j
+= ω ARMA representation:
- more flexible (has both poles and zeroes
- more difficult to study
05/07/09 Ec44440.SpFY09/MPF - Section VI 5
• How to relate AR, MA, ARMA modelsWold decomposition theorem [1954]: Any wss RP can be decomposed into:
-- a random component
-- a “deterministic” random component (perfectly predictable based on the infinite past)
( ) ( ) ( )nxnxnx dr +=
( ) ( )inxanx di
id −−= ∑∞
1=
*where
{ai } are chosen so that one can perfectly match xd (n)
05/07/09 Ec44440.SpFY09/MPF - Section VI 6
• How to relate AR, MA, ARMA models
• Consequences:
• if PSD is continuous; any AR or ARMA process can be represented by a MA model of infinite order.
• Kolmogorov’s Theorem [1941]: an ARMA or MA process can be represented by an AR model of infinite order.
• Practical Results: if we choose the wrong model, we can still get a good approximation by choosing a high enough order!!
05/07/09 Ec44440.SpFY09/MPF - Section VI 7
( )( )
( ) 1−
1−
−∞
1
+1+1
=≡+1
1=
∑ azbzzH
zkczH ARMA
kAR ?
( ) 1−
1−−
∞
0= +1+1
≡∑ bzazzkc k
k
( ) 1−
1−
+1+1
=azbzzHEx: ARMA (1, 1)
numerator order denominator order
• How to represent ARMA (1, 1) with AR (∞)
⇓
compute inverse z-transform to get c(k)
05/07/09 Ec44440.SpFY09/MPF - Section VI 8
( )⎩⎨⎧
≥=
=1 0
kk
kc
1
1Recall ,1
( )n
z aaz
a u n
− >−↓
( ) 1−
1−−
∞
0= +1+1
≡∑ bzazzkc k
k
05/07/09 Ec44440.SpFY09/MPF - Section VI 9
To approximate ARMA (1, 1) with AR(P), then we need P so that:
c(P) =
needed order-P depends on _________
Trick: Choose P so that:
05/07/09 Ec44440.SpFY09/MPF - Section VI 10
• How to find the model parameters: via solving “Yule-Walker” (Y-W) equations for (AR)
models
• via modified Y-W equations for (ARMA) models
w(n) x(n)ARMA
ai : AR parameters bi : MA parameters
00 0
( )( )( )
( ) , 1, ( )P P
i ii i
i i
B zH zA z
A z a z a B z b z− −
= =
=
= = =∑ ∑
05/07/09 Ec44440.SpFY09/MPF - Section VI 11
( ) ( ) ( )
( ) ( )[ ] ( ) ( )[ ]
( ) ( )ikRbikRa
knxEknxE
inwbinxanx
wx
Q
iix
P
ii
Q
ii
P
ii
−=−
−=−
−=−+
∑∑
∑∑
0=0=
0=1=
• How to find the model parameters:
ai : AR parameters bi : MA parameters
00 0
( )( )( )
( ) , 1, ( )P P
i ii i
i i
B zH zA z
A z a z a B z b z− −
= =
=
= = =∑ ∑
05/07/09 Ec44440.SpFY09/MPF - Section VI 12
( ) ( ) ( )
( ) ( )[ ] ( ) ( )[ ]
( ) ( )ikRbikRa
knxEknxE
inwbinxanx
wx
Q
iix
P
ii
Q
ii
P
ii
−=−
−=−
−=−+
∑∑
∑∑
0=0=
0=1=
( ) ( )( )
1 ncewith varia
noise whitewhen w(n) __________________________
*
==
−=
xw
xwwx
RRR
( )0 0
QP
i xi i
aR k i= =
− =∑ ∑Y-W equations
Use:
05/07/09 Ec44440.SpFY09/MPF - Section VI 13
( ) ( )*
0___________ˆ
__________
Q
ii
c k bh i k+∞
= −∞
= − =
=
∑ ∑
If h(n) is causal
• Solution of modified Yule-Walker equations:
( )⎩⎨⎧ ≤≤
=ow
Qibib i
00
( ) ( )
( ) ( )
*
00 for
Q
ii k
k
c k bh i k k Q
k Qc k
b k Q
=
= − ≤
>=
= >
∑
with
05/07/09 Ec44440.SpFY09/MPF - Section VI 14
( ) ( )kcikRa x
P
ii =−∑
0=
⇒
( )( )
0 _____________________ 0
_____________________1 _____________________ ____
_____________________ ____
k c
k Q c Qk Q
k Q P
= → =
= → =
= + → == + → =
matrix form
05/07/09 Ec44440.SpFY09/MPF - Section VI 15
⇒
[ ] [ ][ ]
[ ]
⎥⎥⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢
⎣
⎡
0
0
10
1
_______
_______
_______
______________
____
____
________
____
____
____
____0
1 Qc
cc
a
a
R
p
x
P+Q unknown
Modified Y-W equations
05/07/09 Ec44440.SpFY09/MPF - Section VI 16
( )( )
( )
( ) ( ) =++
==
==
≠=
−+∞→+∞→ 11
0
1limlim0
______0________0
0____
zab
zHh
cc
kkc
zz
[ ] [ ] [ ][ ] [ ] [ ]
[ ] [ ] [ ] ⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
+−−−
0
0_____1
01
10110
1
pxxx
xxx
xxx
a
a
RPRPR
PRRRPRRR
Solve for {ai , b0 } (Levinson algorithm)
Case I: AR model Q = 0
05/07/09 Ec44440.SpFY09/MPF - Section VI 17
( )( )
( )
( )
( )⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡ 0
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡10
Qc
c
QR
RR
x
x
x
Case II: MA model P = 0
(1) identify c(k)(2) identify b(k) 2 methods available
option (a) to identify b(k):
Recall MA model is a FIR filter ⇒ h(n) = b(n)
( ) ( ) ( ) ( ) ( )kibibkbkbkc −=−= ∑ ***
And extract b(k) from c(k)
….. Which may be hard….
05/07/09 Ec44440.SpFY09/MPF - Section VI 18
( )( )( )
( )( )( )
( ) ( )
( ) ( )( ) ( )
22
0* *
1 0 2 1*
2 0
0 00 01 1 1 12 2 2 2
i xx
x x
x x
c b RR cR c c bb b b RR c c b b R
⎧= =⎪⎡ ⎤ ⎡ ⎤ ⎪⎪⎢ ⎥ ⎢ ⎥= ⇒ = + =⎨⎢ ⎥ ⎢ ⎥
⎪⎢ ⎥ ⎢ ⎥ = =⎣ ⎦ ⎣ ⎦ ⎪⎪⎩
∑
and solve for b0 , b1 , b2 (may not be easy)
Ex: Q = 2, P = 0
05/07/09 Ec44440.SpFY09/MPF - Section VI 19
Option (b) to identify b(k): use spectral factorization
Use the fact that
• Compute Sx (z) = Z [Rx (k)]
• Extract B(z); so that zeros are inside u.c. (may be hard)
( ) ( ) ( ) ( )1=
1= zSzBzBzS wx**
05/07/09 Ec44440.SpFY09/MPF - Section VI 20
[ ] [ ] [ ][ ] [ ] [ ]
[ ] [ ] [ ]⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−1−
−101−1−0
=
PQRQRQR
PRRRPRRR
R
xxx
xxx
xxx
B
Case III: ARMA model
Rewrite Y-W equations as:0
B
A
R ca
R⎡ ⎤ ⎡ ⎤
=⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦
[ ]
[ ]
[ ] [ ]
[ ] ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ 1+−
+
1+=
QR
PQRQR
PQR
QRR
x
xx
x
x
A
[ ] [ ] [ ]0 , 1 ,...,T
c c c c Q⎡ ⎤= ⎣ ⎦
where RB is the (Q+1) ×
(P+1) matrix
RA is the P ×
(P+1) matrix
c is the vector
05/07/09 Ec44440.SpFY09/MPF - Section VI 21
Case III: ARMA model
• Solve for a with RA a = 0
• Solve for c with RB a = c
• How to find b from c non-linear job!
Use spectral factorization (as for MA model)
0B
A
R ca
R⎡ ⎤ ⎡ ⎤
=⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦
05/07/09 Ec44440.SpFY09/MPF - Section VI 22
( ) ( )zSzS wx .
=
( )zA1( )zB
( )nx′( )nw ( )nx
residual process
Decompose ARMA into MA + AR portions
( ) ( )( ) ( )
( ) ( )
________*
_____*
_____*
x w
x x
x x
S z S z
S z S z
S z S z
′
′
′
=
=
→ =
Note:
05/07/09 Ec44440.SpFY09/MPF - Section VI 23
Procedure: 1) find a from Y-W equations
2) recall ( ) ( ) ( ) ⎟⎠⎞
⎜⎝⎛ 1=′ *
*
zAzSzAzS xx
( ) ( ) ( ) ( )kRkaaikRkc xi
P
ix *=−= ∑
0=
( ) Qkkc ≤0≠
( )( ) _____________
_____________ ==
′ zSzC
x
Note:
( )for , 0, ,ka k a k P= =
05/07/09 Ec44440.SpFY09/MPF - Section VI 24
( ) ( ) ⎟⎠⎞
⎜⎝⎛ 1
=′ **
zBzBzS x
• Find the spectral factorization of ( )zSx′
pick B(z): minimum phase
(to insure causality and invertibility)
• Note: Extension to the Levinson algorithm available to solve for (ai , bi ) for an ARMA (p, q) system is
possible (see appendix)
05/07/09 Ec44440.SpFY09/MPF - Section VI 25
( ) 1−1
1−10
+1+
=zazbbzHEx: to be fit to Rx (k)
Rx (k)
k
3 21
4321
05/07/09 Ec44440.SpFY09/MPF - Section VI 26
05/07/09 Ec44440.SpFY09/MPF - Section VI 27
05/07/09 Ec44440.SpFY09/MPF - Section VI 28
05/07/09 Ec44440.SpFY09/MPF - Section VI 29
• AR ModelingAssume Rx (k) not known but estimated
Autocorrelation method Covariance method
SVD-based covariance /correlation method
BurgNotes:
(1)
• Performances are the same for large data records
• Strong differences for short data records.
(2) Easy to implement ⇒ “fast” algorithms implemented
(3) well suited for data with PSD behavior large at certain frequencies, i.e., with “peaky behavior”. Well suited for speech data.
I
IIIII
05/07/09 Ec44440.SpFY09/MPF - Section VI 30
AR Methods lead to characterizing the data with a pole-only model, (can be done simply)
A(z)x(n) e(n)
White if model is accurateStochastic data
1/A(z)
Analysis (Modeling) Phase
e(n) x(n)
Synthesis Phase
Need stable filter only if x(n) is to be regenerated
If A(z) is good then: 2( ) should be "close" to |1/A( ) |j j
• No data windowing• Minimizesforward & backward prediction errors
Advantages • Always gives a stable model• Fast Levinson implementations
• Better resolution for short data length than correlation
• Cleans up noise impacts• Easier to estimate frequency locations
• Good resolution for short data length • Always gives a stable model
Drawbacks • Poor performances for short data length• Frequency bias
• May produce unstable model• May have false peaks• Frequency bias
• Require good estimation of the matrix rank• Computationally expensive
• May suffer from spectral line splitting in high SNRs• Frequency bias• Lower accuracy for high-order models, or long data length
Overall AR Schemes Comparisons
05/07/09 Ec44440.SpFY09/MPF - Section VI 32
Problem? What happens when is estimated from short data records?
Poor resolution!
( ) ( )
( ) ( ) ⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
0
0=
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡ 1
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
0
−0 2
1
εσ
pxx
xx
a
a
RPR
PRR
ˆˆ
ˆˆ
Autocorrelation Method
1) Replace true Rx (k) with estimated in Weiner Hopf eqs and solve using Levinson algorithm.
Solve:
( )kR xˆ
( )kRxˆ
A(z) is insured to be minimum phase
H(z) = 1/A(z) insured to be stable
05/07/09 Ec44440.SpFY09/MPF - Section VI 33
2) Using the data directly. ⎥⎦
⎤⎢⎣
⎡
0=
2εσ
aR• Recall a is obtained by solving
• Correlation matrix can be estimated as: given we have {x(0), …, x(n−1)}
XHX =
x(0), x(1), …, x(p), …, x(n−1), 0, …
x(0), …, x(p−1), … x(n−1)
x(0), …, x(n−p−1) x(n−1)
x(0)
x(1) x(0)
x(p) x(0)
x(n−1) x(n−p−1)
x(n−p)
x(n−1)
*
Note: implicitly assume data is zero outside {x(0), …, x(n-1)}
Data windowing
05/07/09 Ec44440.SpFY09/MPF - Section VI 34
XHX (1, 1) =
XHX (2, 2) =
XHX (2, 1) =
XHX =
x(0), x(1), …, x(p), …, x(n−1), 0, …
x(0), …, x(p−1), … x(n−1)
x(0), …, x(n−p−1) x(n−1)
x(0)
x(1) x(0)
x(p) x(0)
x(n−1) x(n−p−1)
x(n−p)
x(n−1)
*
05/07/09 Ec44440.SpFY09/MPF - Section VI 35
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡=optaXX H
• Error variance can be found as:
=2εσ
05/07/09 Ec44440.SpFY09/MPF - Section VI 36
⎥⎦
⎤⎢⎣
⎡′1
=opt
opt aa
⎥⎦
⎤⎢⎣
⎡0
=S
aXXHopt
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡= 10 XXX
• Goal: Compute
by solving
(1)
• To solve (1): we need to split the matrix equation to
• get first
• compute S after (and )2εσ
opta′
05/07/09 Ec44440.SpFY09/MPF - Section VI 37
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡= 10
1
0 XXXX
XXH
HH
then:
05/07/09 Ec44440.SpFY09/MPF - Section VI 38
• Goal: need to solve:
( ) 0111 −=′ XXaXX HHopt
0 0 0 1
01 1 1
H H
H
H H
X X X XX X
X X X X
⎡ ⎤⎢ ⎥
= ⎢ ⎥⎢ ⎥⎣ ⎦
⎥⎦
⎤⎢⎣
⎡0
=S
aXX Hopt ⎥
⎦
⎤⎢⎣
⎡′1
=opt
opt aawhere
05/07/09 Ec44440.SpFY09/MPF - Section VI 39
• Different procedures to estimate the correlation matrix exist, resulting in different X1 and X0
Define:
X =x(0) x(1)
x(p) x(0)
x(n−1) x(n−p−1) x(n−p)
x(n−1)
B4
B3
B1 B2
05/07/09 Ec44440.SpFY09/MPF - Section VI 40
• Correlation method (B1 range of equations)
X =x(0) x(1)
x(p) x(0)
x(n−1) x(n−p−1) x(n−p)
x(n−1)
B4
B3
B1B2
• Resulting XHX is Toeplitz
• Data window is “assumed” implicitly to solve this case
-- reduces resolution of AR spectrum
-- always stable
05/07/09 Ec44440.SpFY09/MPF - Section VI 41
• Covariance method (B2 range of equations)
B4
X =x(0) x(1)
x(p) x(0)
x(n−1) x(n−p−1) x(n−p)
x(n−1)
B3
B1B2
• Resulting XHX not Toeplitz (but fast solver 0(P2) exist)
• A(z) not minimum phase (may be unstable, happens rarely)
• Spectrum may have false peaks
05/07/09 Ec44440.SpFY09/MPF - Section VI 42
• Pre-windowed method (B3 range of equations)
• Post-windowed method (B4 range of equations)
B4
X =
x(0) x(1)
x(p) x(0)
x(n−1) x(n−p−1) x(n−p)
x(n−1)
B3
B1B2
05/07/09 Ec44440.SpFY09/MPF - Section VI 43
(1) using generic matrix approach
(2) using SVD-based approach
• 2 possible methods to solve system of equations:
(1) Generic approach
( ) 0111 −=′ XXaXX HHopt
(fast solvers exist in certain cases)
(2) SVD-based approach
generalized inverse
( ) 01
1−
11
0+1
−=′ XXXXaXX
HHopt
05/07/09 Ec44440.SpFY09/MPF - Section VI 44
Example: Compare FFT and Correlation models, using generic approach
Speech signal1024 fft length14th order model correlation (Y-W) method
[Mathworks]
05/07/09 Ec44440.SpFY09/MPF - Section VI 45
Example: Compare FFT and Correlation models, using generic approach
To generate AR parameters from data x, for filter of order 7[d1,p1] = aryule(x,7);
To visualize how good the model is: overlay the periodogram to 1/A(z) s = spectrum.periodogram; Hpsd = psd(s,x); plot(Hpsd); hold on; hp = plot(w1/pi,20*log10(2*abs(H1)/(2*pi)),'r');
% Scale to make one-sided P SD set(hp,'LineWidth',2); xlabel('Normalized frequency (\times \pi rad/sample)') ylabel('One-sided PSD (dB/rad/sample)') legend('PSD estimate of x','PSD of model output')
[www.mathworks.com]
05/07/09 Ec44440.SpFY09/MPF - Section VI 46
∑=1HVUX
• Generalized inverse can be computed with the SVD
when is singular or ill-conditioned.( )11 XX H
• SVD computes:
PK× KK×PP×
PK×
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
00
000
=
1
∑pσ
σ
U V unitary UH U = U UH = I VH V = I
P columns
K rows
• SVD-based Autocorrelation/Covariance Method
Assume K>P
05/07/09 Ec44440.SpFY09/MPF - Section VI 47
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
00
000
=
1
∑pσ
σ
Assume K > P
• if rank (X1 ) = r < P ⇒ only r σi are nonzero
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
00
0=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
00
0=
11
∑S
rσσ
P columns
K rows
• SVD-based Autocorrelation/Covariance Method
05/07/09 Ec44440.SpFY09/MPF - Section VI 48
[ ]⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎦
⎤⎢⎣
⎡000
==
2
11
211 ∑H
H
H
V
VS
UUVUX
• Properties of the SVD.
HVSUX 1111 =⇒
Comment: Only the portions of U and V which are associated to the non-zero singular values are taken into account.
Contains the non-zero singular values
05/07/09 Ec44440.SpFY09/MPF - Section VI 49
(1) Using the SVD for the generalized inverse: +1X
Comment: Only the portions of U and V which are associated with the non-zero values of X1 are taken into account.
05/07/09 Ec44440.SpFY09/MPF - Section VI 53
0opt 1
1
01 1
1 0
0 1
H
r
a X X
V U Xσ
σ
+′ = −
⎡ ⎤⎢ ⎥= − ⎢ ⎥⎢ ⎥⎣ ⎦
• Generalized inverse can be used to stabilize the solution when data is noisy
Recall:
What if we have?σi
i
05/07/09 Ec44440.SpFY09/MPF - Section VI 54
⎪⎩
⎪⎨⎧
==
∑++
+
HUVXdXh
• Assume K < P -- less equations than unknowns -- no unique solution
pick the one with minimum-norm
-- leads to same solution
Practical problems using the SVD
correct determination of the rank r of the SVD
( )( )
( )
( )
( )
( )( )
( )
( )
( ) ( )Nkpk
kp
kMDL
kpp
kp
kAIC
Nkp
P
kii
kpi
P
kk
Nkp
P
kii
kpi
P
kk
log221
1log
221
log
1
1
110
1
1
110
−+
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
Π−=
−+
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
Π−=
−
+=
−
+=
−
+=
−
+=
∑
∑
λ
λ
λ
λ
05/07/09 Ec44440.SpFY09/MPF - Section VI 55
( ) ( )( )( )
exp 0.1 2 0.52
exp 0.2 2 0.42
y n j n
j n
π
π
⎡ ⎤= − +⎣ ⎦+ − +⎡ ⎤⎣ ⎦
Figure 1. Zeroes of B(z); 40 independent trials. Intersecting radial lines and arcs show the true locations of the two zeros. If the intersecting lines are inside the unit circle they correspond to exp(s1 ) and exp(s2 ) and if they are outside they correspond to exp(-s1
*) and exp(-s2* )
(a) Prony method, L = M = 2.
(b) Covariance method, L = 4.
(c) Covariance method, L = 8.
(d) Autocorrelation method, L = 4.
(e) Autocorrelation method, L = 8.
(f) Backward covariance method, L = 8. [Kumaresan82a]
b)
c) d)
e)f)
a)
05/07/09 Ec44440.SpFY09/MPF - Section VI 56
Figure 1 (continued).
(g) Backward covariance method using SVD, L = 8.
(h) Backward covariance method using SVD, L = 18.
(i) Covariance method using SVD, L = 8. [Kumaresan82a]
05/07/09 Ec44440.SpFY09/MPF - Section VI 57
( ) ( )[ ]
ex p 2 0 .5
ex p 2 0 .5 2
y n j n
j n
π
π
= ⎡ ⎤⎣ ⎦+
Figure 4. (a)The zeros of the prediction-error filter for 50 independent trials. True frequency locations are shown by arrows. Two unit amplitude-sinusoid as f1 = 0.52 Hz and f2 = 0.5 Hz. SNR −10 dB, L = 4, 25 data samples are used.
(b) L = 8.
(c) L = 12.
(d) L = 16.
(e) L = 18.
(f) L = 20.
[Kumaresan82b]
05/07/09 Ec44440.SpFY09/MPF - Section VI 58
( ) ( )[ ]
ex p 2 0 .5
ex p 2 0 .5 2
y n j n
j n
π
π
= ⎡ ⎤⎣ ⎦+
Figure 4. (a)The zeros of the prediction- error filter for 50 independent trials. True frequency locations are shown by arrows. Two unit amplitude-sinusoiad as f1 = 0.52 Hz and f2 = 0.5 Hz. SNR −10 dB, L = 4, 25 data samples are used.
(a) L=4.
(b) L = 8.
(c) L = 12.
(d) L = 16.
(e) L = 18.
(f) L = 20. (g) L = 22.
[Kumaresan82b]
• SVD-based techniques
05/07/09 Ec44440.SpFY09/MPF - Section VI 59
• Burg’s AlgorithmMethod
• Estimates the reflection coefficients which minimize the sum of forward and backward prediction errors.
• Guaranteed to generate a stable filter. Solved from reflection coefficients. Does not need correlation estimates
• AR parameters obtained from reflection coefficients with Levinson algorithm. Fats implementation exists
• Problems may occur for high SNRs or long data length(see equation details in appendix)