6/22/2009 1 1 Lecture 09: Spectrum estimation – parametric methods Instructor: Dr. Gleb V. Tcheslavski Contact: [email protected]Office Hours: Room 2030 Class web site: Summer I 2009 ELEN 5301 Adv. DSP and Modeling http://www.ee.lamar.edu/ gleb/adsp/Index.htm Material comes from Hayes; Image was downloaded from http://www.bubble ology .com/seeps/SeepBubbleMusic.html 2 Introduction One of the limitations of the non-parametric spectral analysis methods is that they do not incorporate information that may be available about the process. For , , an autoregressive model on the speech waveform. Therefore, for the quasi- stationary time intervals, the spectrum of the process would be ( ) 2 0 1 1 j x p jkkkb P e a e ω ω − = = + ∑ while the eriodo ram fo r inst an ce would return an estimate co nsis ten t with a (9.2.1) Summer I 2009 ELEN 5301 Adv. DSP and Modeling moving average process: ( ) ( ) ( ) 1 2 1 ˆ ˆ Nj j jk per N x k NP e X e r k e ω ω ω − − =− + = = ∑ Therefore, incorporating a process modelinto the spectrum estimation algorithm could lead to a more accurate and higher resolution estimate. (9.2.2)
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
With a parametric approach, the first step is to select an appropriate model for theprocess. This selection may be based on a priori knowledge about the way the
model suits well. The commonly used models are autoregressive (AR), moving
average (MA), and autoregressive moving average (ARMA).
The next step (once the model is selected) is to estimate the model parameters
from the given data.
The final step is to estimate the power spectrum by using the estimated parameters.
Although it is possible to
si nificantl im rove the( ) ( )5 sin 0.45 5 sin 0.55
n n x n n wπ π = + +
Summer I 2009ELEN 5301 Adv. DSP and Modeling
resolution of the spectral estimate
with a parametric method, it is
important to realize that, unlessthe model that is used is
appropriate for the analyzedprocess, inaccurate or misleading
estimates may be obtained.
Blackman-Tukey
AR MA processsinusoids
4
Signal modeling
Assume that the time-domain signal x n that consists of N data samples x 0 ,…,x N-1 is
transmitted across a communication channel. We state that it is possible to. ,
would be more efficient to transmit or store these parameters instead of signalvalues. The signal would be then reconstructed from the parameters.
We view the signal x n as the response of a linear time-invariant filter to an input v n .
Therefore, our goal is to find the filter H (z ) and the input v n that make the output “asclose as possible” to x n .
We will use the rational model for
the filter in the form
Summer I 2009ELEN 5301 Adv. DSP and Modeling
( )( )
( )0
1
1
qk
k q k
pk p
k
k
b z B z
H z A z
a z
−
=
−
=
= =+
∑
∑Therefore, the signal model will include the filtercoefficients a k and b k and a description of an
Here the autocorrelation estimates (the hats are omitted) are computed as:
1*1 N k − −
0
; , ,..., x n k n
n
r x x p N
+=
= =
Solving (9.7.2) for the coefficients a k and setting
( ) ( )2 *
0
1
0 p
p x k x
k
b r a r k ε
=
= = + ∑produces an estimate for power spectrum sometimes referred to as the Yule-Walker
(9.8.1)
(9.8.2)
Summer I 2009ELEN 5301 Adv. DSP and Modeling
method, which is equivalent to the maximum entropy method. The only difference isthat the Y-W assumes that x n is an AR process while the ME assumes that x n is
Gaussian.
Since the autocorrelation matrix Rx is Toeplitz, the Levinson-Durbin recursion may
be used to solve these equations. Furthermore, if the autocorrelation matrix is
positive definite, the roots of A(z ) will be inside the unit circle and the model will bestable.
However, since the autocorrelation method applies a rectangular window to the
data, the data will be extrapolated with zeros. As a consequence, the resolution of…
The autocorrelation method is generally not used for short data records.
There is an artifact that may be observed with the autocorrelation method calledspectral line splitting. As a result, a spectral peak may appear as two separate
distinct peaks. This usually happens when x n is overmodeled, i.e., when p is too
large. Consider, for example an AR(2) process described by the following differenceequation:
= − 9.9.1
Summer I 2009ELEN 5301 Adv. DSP and Modeling
2.n n n−
where w n is unit variance white noise. The true spectrum has a single peak at ω =
π /2.
The data record of length N = 64 was generated according to (9.9.1) and two
estimates were found using model orders p = 4 and p = 12…
. .
10
Autoregressive SE: theAutocorrelation method
p = 4
p = 12
Line splitting for p = 12.
We notice that the autocorrelation estimate in (9.8.1) is biased. A variation of the
autocorrelation method is to use the unbiased estimate:
Summer I 2009ELEN 5301 Adv. DSP and Modeling
( )1
*
0
1; 0,1,...,
N k
x n k n
n
r k x x k p N k
− −
+=
= =−
∑However, in this case, the autocorrelation matrix is not guaranteed to be positive
definite and the variance of the spectrum estimate tends to become large when thematrix is ill-conditioned. Therefore, biased autocorrelation estimates are generally
− −== ∑Unlike the linear equations in the autocorrelation method, equations in (9.11.1) arenot Toeplitz.
(9.11.2)
12
Autoregressive SE: theCovariance method
The advantage of the covariance method over the autocorrelation method is that no
windowing of the data is required in the formation of the autocorrelation estimates
r x , . ere ore, or s or a a recor s, e covar ance me o genera y pro uces
higher resolution spectrum estimates than the autocorrelation method.
When the data length is large compared to the model order, N >> p , the effect of thedata window becomes small and the difference between the two approaches
becomes negligible.
3. Modified Covariance method:
Summer I 2009ELEN 5301 Adv. DSP and Modeling
The modified covariance method is similar to the covariance method in that no
window is applied to the data. However, instead of finding an AR model that
minimizes the sum of the squares of the forward prediction error, the modifiedcovariance method minimizes the sum of the squares of the forward and backward
The AR parameters in the modified covariance method are found by solving a set of
linear equations of the form given in (9.11.1) with
( )1
* *, N
x n l n k n p l n p k
n p
r k l x x x x−
− − − + − +=
⎡ ⎤= +⎣ ⎦∑ (9.13.1)
We notice that the autocorrelation matrix is not Toeplitz.
Unlike the autoregressive method, the modified covariance method appears to givestatistically stable spectral estimates with high resolution. Furthermore, in the
Summer I 2009ELEN 5301 Adv. DSP and Modeling
,method is characterized by a shifting of the peaks from their true locations due to
additive noise, this shifting is less pronounced than with other AR estimators.
Also, the peak location tends to be less sensitive to phase.Finally, unlike the previous methods, the modified covariance method is not subject
for line splitting.
14
Autoregressive SE: the Burgalgorithm
4. Burg method:
As with the modified covariance method, the Burg algorithm finds a set of AR
parameters that minimizes the sum of the squares of the forward and backwardprediction errors. However, in order to assure that the model is stable, this
minimization is performed sequentially with respect to the reflection coefficients.
Although less accurate than the modified covariance method, the AR estimates aremore accurate than those obtained with the autocorrelation method (since no
window is applied to the data).
Summer I 2009ELEN 5301 Adv. DSP and Modeling
,splitting and the peak locations are highly dependent upon the phases of the
It is not easy to provide a set of examples illustrating properties of the ARestimators. We consider the AR(4) process generated according to the difference
1 2 3 42.7377 3.7476 2.6293 0.9224n n n n n n x x x x x w− − − −= − + − + (9.15.1)
where w n is unit variance white Gaussian noise.
The filter generating x n has two pairs of complex poles at
0.20.98 j z e π ±=
Summer I 2009ELEN 5301 Adv. DSP and Modeling
0.30.98 j z e π ±=
Using data records of length N = 128, an ensemble of 50 spectrum estimates werecomputed by the methods we discussed…
. .
16
Autoregressive SE: Examples
1. Autocorrelation method – does not resolve peaks, biased, higher variance
AR model order p selection is an important problem.
If the order that is used is too small, the resulting spectral estimate will be smoothed
an w ave poor reso u on. n e o er an , e mo e or er s oo arge, espectral estimate may contain spurious peaks and lead to spectral line splitting (in
addition to a higher computational load).
In selecting order, one approach would be to increase the order until the modelingerror is minimized. The difficulty with this is that the error is a monotonically
nonincreasing function of the model order p . This problem may be mitigated byincorporating a penalty function that increases with the model order. Several criteria
proposed include a penalty term that increases linearly with p in form
Summer I 2009ELEN 5301 Adv. DSP and Modeling
( ) ( )log pC p N f N pε = + (9.20.1)
where ε p is the modeling error, N is the data record length, and f (N ) is a constant
that may depend on N . The idea is to select a value of p that minimizes C (p ).
Although (9.24.3) is equivalent to the Blackman-Tukey estimate using a rectangularwindow, there is a subtle difference in the assumptions behind these two estimators.
, . . n ,
autocorrelation sequence is zero for |k | > q . Thus, if an unbiased estimate of theautocorrelation sequence is used for |k | ≤ q , then
( ){ } ( )ˆ j j
MA x E P e P eω ω =
Therefore, the power spectrum estimate is unbiased.
(9.25.1)
Summer I 2009ELEN 5301 Adv. DSP and Modeling
e ac man- u ey me o , on e o er an , ma es no assump ons a ou x n
and may be applied to any type of process. Therefore, due to the windowing the
autocorrelation sequence, unless x n is an MA process, the Blackman-Tukey estimate
will be biased.
26
Moving Average SE: Durbin’smethod
2. Estimate the MA parameters b k from x n and then substitute these estimates as
2
( )0
ˆˆq
j jk
MA k
k
P e b eω ω −
=
= ∑
For example, MA parameters may be estimated by the Durbin’s method:
1) Find a high-order all-pole (AR) model Ap (z ) for the MA process;
2) Consider the coefficients a k to be a new “data set” and find the coefficients of a
q th
order MA model as a q th
order AR process for the sequence a k .
(9.26.1)
Summer I 2009ELEN 5301 Adv. DSP and Modeling
Therefore, let x n be an MA process of order q with
We consider a 4th-order MA process generated according to the difference equation:
1 2 3 4. . . .
n n n n n n x w w w w w− − − −= − + − +
Where w n is a unit variance white Gaussian process. The filter generating x n has the
following 4 complex zeros:0.2
0.5
0.98
0.98
j
j
z e
z e
π
π
±
±
=
=
(9.29.1)
(9.29.2)
Summer I 2009ELEN 5301 Adv. DSP and Modeling
s ng e a a eng = , an ensem e o spec rum es ma es were
computed using the Blackman-Tukey method with rectangular window extendingfrom -4 to 4. Then, MA spectra were estimated via the Durbin’d method with q = 4
Since h n is assumed to be causal, then c k = 0 for k > q and Yule-Walker equationsfor k > q are a function only of the coefficients a k :
( ) ( )1
0; p
x l x
l
r k a r k l k q=
+ − = >∑which, in matrix form for k = q +1, q +2, …, q +p , is
( ) ( ) ( )
( ) ( ) ( )
( )
( )1
2
1 1 1
1 2 2
x x x x
x x x x
ar q r q r q p r q
ar q r q r q p r q
− − + +⎡ ⎤ ⎡ ⎤⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥+ − + +⎢ ⎥ ⎢ ⎥⎢ ⎥ = −
(9.35.1)
(9.35.2)
Summer I 2009ELEN 5301 Adv. DSP and Modeling
( ) ( ) ( ) ( )1 2 p x x x xar q p r q p r q r q p
⎢ ⎥ ⎢ ⎥⎢ ⎥+ − + − +⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦⎣ ⎦ ⎣ ⎦
(9.35.2) is a set of p linear equations in the p unknowns a k . These equations arecalled the Modified Yule-Walker equations and allow finding the coefficients a k in the
filter H (z ) from the autocorrelation sequence r x (k ) for k = q, q+ 1,…, q+p . This is theModified Yule-Walker method. Usually, autocorrelation estimates are used.
36
Autoregressive Moving AverageSE: MYWE
Once the AR coefficients a k are estimated, the MA parameters b k can be found
using one of several approaches…
x n s an p,q process w power spec rum
( )( ) ( )( ) ( )
* *
* *
1
1
q q
x
p p
B z B zP z
A z A z=
Therefore, filtering x n with an LTI filter having a system function Ap (z ) produces an
MA (q ) process y n with a power spectrum
(9.36.1)
Summer I 2009ELEN 5301 Adv. DSP and Modeling
( ) ( ) ( )* *1 y q q
P z B z B z=
Therefore, the MA parameters b k may be estimated from y n , for instance, by theDurbin’s method.