Modeling with ARMA Processes: I • goal: determine which ARMA(p, q ) process is best model for observed time series x 1 ,...,x n • tasks at hand include - determining p and q (order selection) - estimating process mean (easily done!), coefficients φ j & ✓ j (not so easy) and white noise variance σ 2 (relatively easy) - subjecting selected model to goodness-of-fit tests • note: will assume that, if need be, series x 1 ,...,x n has been adjusted so that it can be regarded as realization of zero mean stationary process (usual procedure: take sample mean ¯ x 0 = P n t=1 x 0 t of original series x 0 1 ,...,x 0 n and set x t = x 0 t - ¯ x 0 ) BD–137, CC–149, SS–121 XIII–1
142
Embed
Modeling with ARMA Processes: I - University of Washingtonfaculty.washington.edu/dbp/s519/PDFs/2013/13-overheads.pdf · Modeling with ARMA Processes: I • goal: determine which ARMA
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Modeling with ARMA Processes: I
• goal: determine which ARMA(p, q) process is best model forobserved time series x1, . . . , xn
• tasks at hand include
� determining p and q (order selection)
� estimating process mean (easily done!), coe�cients �j & ✓j(not so easy) and white noise variance �2 (relatively easy)
� subjecting selected model to goodness-of-fit tests
• note: will assume that, if need be, series x1, . . . , xn has beenadjusted so that it can be regarded as realization of zero meanstationary process (usual procedure: take sample mean x0 =Pn
t=1 x0t of original series x01, . . . , x0n and set xt = x0t � x0)
BD–137, CC–149, SS–121 XIII–1
Modeling with ARMA Processes: II
• with p & q assumed initially to be known, will advocate Gaussianmaximum likelihood (ML) estimators for �j, ✓j & �2
• requires use of nonlinear optimization procedure, for which needgood initial estimates of coe�cients �j & ✓j
• can base initial estimates on easier-to-compute estimators
� Yule–Walker (Y–W) estimator (good for AR(p) case)
� Burg estimator (for AR(p) also)
� innovations algorithm (handles MA(q) and ARMA(p, q))
� Hannan–Rissanen (adapts Y–W to handle ARMA(p, q))
• will now described these estimators, along with some prelimi-nary discussion about order selection
BD–138, CC–149, SS–121 XIII–2
Yule–Walker Estimation: I
• assume causal AR(p) model �(B)Xt = Zt, i.e.,
Xt �pX
j=1
�jXt�j = Zt, (⇤)
with {Zt} ⇠WN(0,�2)
• can develop set of p linear equations linking
� = [�1, . . .�p]0
to ACVF values �(0), �(1), . . . , �(p� 1)
• Y–W estimators gotten by substituting usual estimator �(h)for �(h) in p equations (so-called ‘moment matching’)
• one additional equation needed to estimate �2 via this scheme
• to (mis)quote Yogi Berra: ‘This is like deja vu all over again!’
BD–139, CC–149, SS–121 XIII–5
Yule–Walker Estimation: IV
• exactly same matrix equation arose when trying to find coe�-cients of best linear predictor of Xn+1 given Xn, . . . , X1 for ageneral stationary process {Xt} (i.e., not necessarily AR(p))
• given time series x1, . . . , xn, form usual estimate of ACVF:
�(h) =1
n
n�|h|Xt=1
xtxt+|h|
• with �p & �p formed by replacing �(h)’s in �p & �p by �(h)’s,Y–W estimator � of AR(p) coe�cients � given by
� = ��1p �p,
where inverse ��1p exists as long as time series isn’t ‘boring’
BD–139, 140, CC–149, SS–121 XIII–6
Yule–Walker Estimation: V
• can solve equation using Levinson–Durbin recursions
• note: provides Y–W estimates not only for order p, but also forall lower orders 1, . . . , p� 1
• once � has been computed, can return to h = 0 case of (⇤⇤),namely,
�2 = �(0)�pX
j=1
�j�(j), to get estimator �2 = �(0)�pX
j=1
�j�(j)
(similar equation arose for getting MSE of best linear predictor)
• fitted model has theoretical ACVF that is identical to esti-mates �(h) at lags h = 0, 1, . . . , p, but in general is di↵erentat higher lags
• as an example, let’s revisit the sunspot time series:
� fit AR models of orders p = 1, . . . , 8 & then 29 using Y–W
� compare sample ACVF to the theoretial ACVFs correspond-ing to 9 fitted AR models
BD–141, CC–149, SS–121 XIII–8
Sunspots (1749–1963)
1750 1800 1850 1900 1950
050
100
150
year
x t
BD–99 I–7
Sample PACF for Sunspots
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
PAC
F
BD–99 XII–13
Sample and Fitted AR(1) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–9
Sample and Fitted AR(2) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–10
Sample and Fitted AR(3) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–11
Sample and Fitted AR(4) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–12
Sample and Fitted AR(5) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–13
Sample and Fitted AR(6) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–14
Sample and Fitted AR(7) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–15
Sample and Fitted AR(8) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–16
Sample and Fitted AR(29) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–17
Yule–Walker Estimation: VII
• distribution of Y–W estimators � is approximately multivariatenormal with mean � & covariance �2��1
p /n for large n
• large sample distribution of ML estimators is the same
• don’t even need to worry about inverting �p: can show that
�2��1p = A0A�B0B = AA0 �BB0,
where A and B are p⇥ p lower triangular matrices whose firstcolumns are, respectively,2
6641��1
...��p�1
3775 and
2664
�p�p�1
...�1
3775 ;
A has the same element along any given diagonal; and B has asimilar structure (sometimes referred to as a Toeplitz structure)
BD–141, CC–161, SS–122 XIII–18
Confidence Intervals and Regions for �
• can use large sample distribution to get approximate confidenceintervals for individual �j’s or confidence region for vector �
• approximate 95% confidence interval for �j given by24�j � 1.96
v1/2j,jpn
, �j + 1.96v
1/2j,jpn
35 ,
where vj,j is jth diagonal element of �2��1p
• letting �20.95(p) denote 95% quantile of chi-squared distribution
with p degrees of freedom, approximate 95% confidence regionfor � is the set of all �’s such that
(�� �)0�p(�� �) �20.95(p)
�2
n
BD–142, 143 XIII–19
Yule–Walker Estimation and Order Selection: I
• when Y–W is used to estimate coe�cients for AR(h) model
Xt � �1Xt�1 � · · ·� �hXt�h = Zt,
estimate �h is same as �h,h (hth member of sample PACF)
• as noted before, large sample theory suggests that �h,h is ap-proximately N (0, 1/n) for h > p (the true AR model order)
• given estimates of �h,h out to some maximum order, say H,Brockwell & Davis suggest setting p to be smallest m such that|�h,h| < 1.96/
pn for m < h H
• obvious danger: sampling variability might result in p being settoo high
� with H = 40 in sunspot example, would select p = 29, whichmight not be a reasonable choice
BD–96, 141, CC–115, SS–122 XIII–20
Yule–Walker Estimation and Order Selection: II
• another approach is to select order that minimizes AICC statis-tic (biased-corrected version of Akaike’s information criterion):
AICC = �2 ln (L(�, S(�)/n)) +2(p + 1)n
n� p� 2,
where L is Gaussian likelihood function, and S(�) is definedbelow
• given zero-mean Gaussian AR(p) time series Xn with covari-ance matrix �n (implicitly dependent on � & �2), can write
L(�n) = (2⇡)�n/2(det �n)�1/2 exp�� 1
2X0n��1
n Xn�
and hence
�2 ln (L(�n)) = n ln (2⇡) + ln (det �n) + X 0n��1n Xn
BD–141, 142, 158, CC–130, SS–53, 153 XIII–21
Yule–Walker Estimation and Order Selection: III
• when considering ML estimation later on, will argue that
�2 ln (L(�n)) = n ln (2⇡) + ln (det �n) + X 0n��1n Xn
can be rewritten in AR(p) case as
�2 ln L(�,�2) = n ln(2⇡�2) +p�1Xj=0
ln(rj) +nX
j=1
(Xj � bXj)2
�2rj�1,
where rj ⌘ vj/�2 (note: rj = 1 for j � p)
• dependence on � is through vn’s and coe�cients determiningbXj (can get these from � using reverse L–D recursions)
• can remove �2 by replacing it with S(�)/n, where
S(�) =nX
j=1
(Xj � bXj)2
rj�1
BD–141, 142, 158, CC–130, SS–53, 153 XIII–22
Yule–Walker Estimation and Order Selection: IV
• with removal of �2, AICC statistic becomes
AICC = Cn + n ln
0@ nX
j=1
(Xj � bXj)2
rj�1
1A +
p�1Xj=0
ln(rj) +2(p + 1)n
n� p� 2,
whereCn ⌘ n + n ln(2⇡/n)
• note: will discuss other order selection statistics (BIC etc.) later
• let’s see what order the AICC picks out for sunspot series
BD–141, 142, 158, CC–130, SS–53, 153 XIII–23
AICC for Sunspots
0 10 20 30 40
1850
1900
1950
model order
AIC
C
XIII–24
Sample and Fitted AR(9) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–25
Example – Recruitment Time Series: I
• monthly measure of number of new fish entering Pacific Ocean(453 months covering 1950–87; Shumway & Sto↵er got it fromRoy Mendelssohn, NOAA/PFEL, who got it from Pierre KleiberNOAA/NMFS, who generated measures using a model . . . )
SS–7 XIII–26
Recruitment Time Series (1950–1987)
0 100 200 300 400
020
4060
8010
0
t (months starting with Jan 1950)
x t
SS–8 XIII–27
Sample ACF for Recruitment Series
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
ACF
SS–109 XIII–28
Sample PACF for Recruitment Series
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
PAC
F
SS–109 XIII–29
Sample & Fitted AR(2) ACFs for Recruitment Series
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
ACF
XIII–30
Example – Recruitment Time Series: II
• for AR(2) model, Y–W estimates are
� =
"�1
�2
#.=
1.3316�0.4445
�and �2 .
= 94.171
(note: R function ar gives �2 .= 94.799 . . . hmmm)
• using large sample approximation that � is multivariate normalwith mean � and covariance �2��1
2 , can get 95% confidenceintervals (CIs) and regions based upon
�2��12 = �2
�(0) �(1)�(1) �(0)
��1
=
v1,1 v1,2v2,1 v2,2
�.=
0.8024 �0.7396�0.7396 0.8024
�
• usingh�j � 1.96 v
1/2j,j /p
n, �j + 1.96 v1/2j,j /p
ni
yields 95% CIs
[1.2491, 1.4141] for �1 and [�0.5270,�0.3621] for �2
XIII–31
AICC for Recruitment Series
0 10 20 30 403320
3340
3360
3380
3400
3420
model order
AIC
C
XIII–32
Sample & Fitted Y–W AR(13) ACFs
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
ACF
XIII–33
Burg’s Algorithm: I
• Y–W estimator � of � is based on L–D recursions with �(h)replaced by �(h)
• given �k�1 & vk�1, recursion gives us �k & vk via 3 steps
1. get kth order partial autocorrelation:
�k,k =�(k)�
Pk�1j=1 �k�1,j�(k � j)
vk�1
2. get remaining �k,j’s:264
�k,1...
�k,k�1
375 =
264
�k�1,1...
�k�1,k�1
375� �k,k
264�k�1,k�1
...�k�1,1
375
3. get kth order MSE: vk = vk�1(1� �2k,k)
BD–147, 148 XIII–34
Burg’s Algorithm: II
• when k = p, get Y–W estimators � = �p and �2 = vp
• start procedure by setting
�1 = [�1,1] =
�(1)
�(0)
�and v1 = �(0)(1� �2
1,1)
• sample ACVF comes into play in forming �1 and in 1st step ofL–D recursions, but not in 2nd and 3rd steps
• sample ACVF just used to get PACF estimates �1,1, . . . , �p,p
• kth component of PACF is a correlation coe�cient:
�k,k = corr {Xk � bXk,X0 � bX0|k�1}• Burg’s algorithm is based on estimating �k,k in keeping with
the above rather than via sample ACVF
BD–147, 148 XIII–35
Burg’s Algorithm: III
• let �k�1 = [�k�1,1, . . . , �k�1,1]0 be Burg estimator of coe�-
cients for AR(k � 1) process based on X1, . . . , Xn
• can show that, for any estimator �k,k with �k,1, . . . , �k,k�1generated by step 2 of L–D, have, for k + 1 t n
�!Ut(k) =
�!Ut(k � 1)� �k,k
�Ut�k(k � 1)
�Ut�k(k) =
�Ut�k(k � 1)� �k,k
�!Ut(k � 1)
BD–147, 148 XIII–36
Burg’s Algorithm: IV
• Burg’s idea: choose �k,k that minimizes
SSk(�k,k) ⌘nX
t=k+1
�!U 2
t (k) + �U 2
t�k(k)
• yields Burg’s estimator
�k,k ⌘Pn
t=k+1�!Ut(k � 1)
�Ut�k(k � 1)
12Pn
t=k+1�!U 2
t (k � 1) + �U 2
t�k(k � 1)
• compare above to following expression:
�k,k = corr {Xk � bXk,X0 � bX0|k�1}
=cov {Xk � bXk,X0 � bX0|k�1}�
var {Xk � bXk} var {X0 � bX0|k�1}�1/2
BD–147, 148 XIII–37
Burg’s Algorithm: V
• initialize with�!Ut(0) ⌘ Xt and
�Ut�1(0) ⌘ Xt�1
• guaranteed to have |�k,k| 1
• if |�p,p| 6= 1, Burg estimators � = �p of coe�cients � alwayscorrespond to stationary & causal AR(p) process (same is truefor Y–W, except that |�p| = 1 can’t happen)
• large sample distribution for Burg same as for Y–W and ML,but Monte Carlo studies show Burg outperforming Y–W
• fitted model has theoretical ACVF that need not be identicalto sample ACVF, as another visit to sunspot time series shows
BD–147, 148 XIII–38
Sample and Fitted AR(5) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–39
Sample and Fitted AR(10) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–40
Sample and Fitted AR(15) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–41
Sample and Fitted AR(20) ACVFs for Sunspots
0 10 20 30 40
−500
050
010
0015
00
h (lag)
ACVF
XIII–42
Example – Recruitment Time Series: III
• reconsider recruitment time series, this time using Burg’s algo-rithm
• can use Burg to get estimate of PACF that is an alternative tosample PACF (latter is based on Y–W)
XIII–43
Sample PACF for Recruitment Series
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
PAC
F
SS–109 XIII–29
Burg Estimate of PACF for Recruitment Series
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
PAC
F
XIII–44
Sample & Fitted AR(2) ACFs for Recruitment Series
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
ACF
XIII–45
Example – Recruitment Time Series: IV
• for AR(2) model, Burg estimates are
� =
�1�2
�.=
1.3515�0.4620
�and �2 .
= 89.337,
as compared to Y–W estimates:
� =
"�1
�2
#.=
1.3316�0.4445
�and �2 .
= 94.171
• can determine Burg estimator �(h) of ACVF by feeding � intoone of the methods for computing theoretical ARMA ACVFs
• yields
�2��12 = �2
�(0) �(1)�(1) �(0)
��1
=
v1,1 v1,2v2,1 v2,2
�.=
0.7866 �0.7271�0.7271 0.7866
�
& 95% CIs [1.2698, 1.4332] for �1 & [�0.5436,�0.3803] for �2
XIII–46
95% Confidence Regions for � (Y–W and Burg)
1.20 1.25 1.30 1.35 1.40 1.45 1.50
−0.60
−0.50
−0.40
−0.30
q1
q 2
XIII–47
95% Confidence Regions and Causality Region for �
−2 −1 0 1 2
−1.0
−0.5
0.0
0.5
1.0
q1
q 2
XIII–48
AICC for Recruitment Series
0 10 20 30 403320
3340
3360
3380
3400
3420
model order
AIC
C
XIII–49
Sample & Fitted Burg AR(13) ACFs
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
ACF
XIII–50
Sample & Fitted Y–W AR(13) ACFs
0 10 20 30 40
−1.0
−0.5
0.0
0.5
1.0
h (lag)
ACF
XIII–33
Moment Matching and MA(q) Processes: I
• Y–W & Burg give preliminary estimates of � & �2 for AR(p)
• Y–W estimator based on moment matching
• relates �1, . . . , �p and �2 to ACVF values �(0), . . . �(p) viap + 1 linear equations
• knowing 1 . . . , p+q, can solve for �i’s & ✓j’s, as follows
• equation (⇤) for j = 1, . . . , q gives
1 = ✓1 + �1, . . . , q = ✓q +
min {p,q}Xi=1
�i q�i (•)
• (⇤) for j = q + 1, . . . , q + p does not involve ✓j’s directly:
q+1 =
min {p,q+1}Xi=1
�i q+1�i, . . . , q+p =pX
i=1
�i q+p�i (†)
• use (†) to solve for �i’s, after which (•) gives
✓j = j �min {p,j}X
i=1
�i j�i, j = 1, . . . , q
BD–154, 155 XIII–77
Innovations Algorithm for Mixed ARMA Models: III
• IA takes ACVF and gives ✓m,j’s, where ✓m,j ! j as m!1• to get estimates of �i’s and ✓j’s,
1. use IA with sample ACVF to get estimates of ✓m,j (with mchosen large enough to ensure convergence)
2. set j equal to estimate of ✓m,j
3. use j’s with p + q equations to get �i’s and ✓j’s
• B&D note that
� resulting �i need not correspond to a causal process
� order selection using sample ACVF and PACF dicey withmixed models because no clear patterns to distinguish be-tween, e.g., ARMA(2,1) and ARMA(1,2)
� order selection can still be done using AICC
BD–154, 155 XIII–78
Innovations Algorithm for Mixed ARMA Models: IV
• can base estimate of �2 on normalized one-step-ahead MSEs:
�2 =1
n
nXt=1
(Xt � bXt)2
rt�1, where rt�1 =
E{(Xt � bXt)2}�2 ,
and bXt is predictor of Xt based upon Xt�1, . . . , X1
• can get �2 by using �(0) (sample variance for time series)along with estimates �i & ✓j to calculate ACVF for theoreticalARMA(p, q) process and feeding this ACVF into L–D recur-sions – desired estimate �2 is nth order MSE vn
• alternatively, once ARMA(p,q) ACVF has been determined,apply IA to it with m set large enough so that vm is stable,and then use �2 = vm
BD–154, 155 XIII–79
Example – Atomic Clock Series: I
• as an example, let’s use IA to fit an ARMA(1,1) model
Xt � �Xt�1 = Zt + ✓Zt�1, {Zt} ⇠WN(0,�2),
to atomic clock series
• the p + q = 2 relevant equations are
1 = ✓ + � & 2 = � 1, yielding � = 2
1& ✓ = 1 � �
• basing our estimates of 1 and 2 on
✓15,1.= �0.5879 and ✓15,2
.= �0.1316
(see overhead XIII–71) yields
�.= 0.2238 and ✓
.= �0.8117
(corresponds to a causal and invertible ARMA(1,1) model)
• get �2 = 20.860 (compared to �2 .= 20.782 for MA(2) model)
XIII–80
Estimation of �2 via v100 from Innovations Algorithm
0 20 40 60 80 100
2224
2628
m
v m
XIII–81
Sample & ARMA(1,1) ACF for Atomic Clock
0 10 20 30 40
−0.4
−0.2
0.0
0.2
0.4
h (lag)
ACF
XIII–82
Sample & ARMA(1,1) PACF for Atomic Clock
5 10 15 20
−0.4
−0.2
0.0
0.2
0.4
h (lag)
PAC
F
XIII–83
AICC for Atomic Clock Series
0 10 20 30 40
6020
6040
6060
6080
number of parameters in model
AIC
C
XIII–84
Higher-Order Yule–Walker Method: I
• alternative to IA algorithm for handling mixed ARMA modelsis based on structure of ACVF {�(h)} for such models
• as noted before (overhead IX–20), ARMA(p, q) ACVF satisfies
�(k)� �1�(k � 1)� · · ·� �p�(k � p) = 0
for all k � q + 1
• does not involve MA coe�cients
• can use so-called higher-order Y–W equations to get �i’s:
• with �i’s known, can filter time series X1, . . . , Xn and getoutput Yp+1, . . . , Yn with MA(q) structure:
Yt ⌘ Xt � �1Xt�1 � · · ·� �pXt�p
= Zt + ✓1Zq�1 + · · · + ✓qZt�q
• higher-order Y–W method with IA thus consists of
� substituting �(h)’s into higher-order Y–W equations andsolving to get estimates �i
� using �i’s to filter time series to get output, say Y 0t� forming sample ACVF for Y 0t ’s and using these as input to
IA to estimate MA coe�cients ✓j
BD–145 XIII–86
Example – Atomic Clock Series: II
• as an example, let’s use scheme to fit an ARMA(1,1) model toatomic clock series
• relevant higher-order Y–W equation is ��(1) = �(2), yielding
� =�(2)
�(1).= 0.1823, as compared to �
.= 0.2238 using IA
• estimate corresponds to a causal process, but might not happenfor other time series (no reason why �(1) ⇡ 0 can’t occur)
• forming sample ACVF for Y 0t = Xt � �Xt�1, t = 2, . . . , n,and feeding it into IA yields ✓n,j’s and vn’s shown on nextoverheads
XIII–87
Convergence of ✓n,j’s for Y 0t ’s
0 10 20 30 40
−0.8
−0.6
−0.4
−0.2
0.0
n
e nj
j = 1 j = 2 j = 3 j = 4
XIII–88
Convergence of vn’s for Y 0t ’s
0 10 20 30 40
2022
2426
2830
32
n
v n
XIII–89
Example – Atomic Clock Series: III
• using ✓15,1 to estimate ✓ in ARMA(1,1) model, get ✓.= �0.7748
compared to ✓.= �0.8117 using IA by itself (overhead XIII–80)
• using v15 to estimate �2 yields �2 .= 20.631 as compared to
IA-based �2 .= 20.860
• sampling theory for ✓n,j’s suggests that those for j = 2, 3 and 4are not significantly di↵erent from zero; i.e., ARMA(1,q) modelwith q > 1 not indicated
• AICC for fitted ARMA(1,1) model is 6022.7, so model is lesslikely than IA-based model with AICC of 6016.2
• next overheads
� show AICC compared to ones for IA-based MA(q) models
� compare theoretical and sample ACVFs and PACFs
XIII–90
AICC for Higher-Order Y–W Method
0 10 20 30 40
6020
6040
6060
6080
number of parameters in model
AIC
C
XIII–91
Sample & ARMA(1,1) ACF for Atomic Clock
0 10 20 30 40
−0.4
−0.2
0.0
0.2
0.4
h (lag)
ACF
XIII–92
Sample & ARMA(1,1) PACF for Atomic Clock
5 10 15 20
−0.4
−0.2
0.0
0.2
0.4
h (lag)
PAC
F
XIII–93
Least Squares Estimators: I
• as prelude to Hannan–Rissanen algorithm, consider least squares(LS) estimators for AR(p) coe�cients
• H–R estimator is � = � + �†, but B&D stick with just �
• three comments
1. can handle both pure MA(q) & mixed ARMA(p, q) models
2. usual formulation of H–R calls for use of Y–W, but Burg isa better choice (in particular, Zt’s are computed as part ofBurg’s algorithm)
3. as in IA, choice of m > max {p, q} requires some care
BD–155, 156, 157 XIII–102
Example – Atomic Clock Series: IV
• as an example, let’s use H–R to fit an MA(4) model to atomicclock series to compare with IA results
• next two overheads look at
1. dependence of estimates ✓1, . . . , ✓4 on order m for approx-imating AR process; m = 5 to 40; m = 15 looks like goodchoice; dotted lines indicate ✓j = ✓15,j from IA
2. same, but now for refinement ✓j; will use m = 15 again