Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Class 1: Stationary Time Series Analysis Macroeconometrics - Fall 2013 Jacek Suda, Banque de France September 20, 2013 Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
109
Embed
Class 1: Stationary Time Series Analysis - Jacek Suda
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation
Class 1: Stationary Time Series Analysis
Macroeconometrics - Fall 2013
Jacek Suda, Banque de France
September 20, 2013
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Website Description Requirements Readings Outline
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Website Description Requirements Readings Outline
Description
The purpose of this course is to familiarize students with current tech-niques used in macroeconomic time series models with applications inmacroeconomics, international finance, and finance; with the ultimateaim of providing necessary tools to conduct original research in the area.
The focus is on implementation of the models presented in the course.
Topics include ARMA models, VARs and impulse response functions;unit roots, and structural breaks; spurious regressions; cointegration andVECM; ARCH models of volatility, and trend/cycle decomposition meth-ods, including Kalman filtering. We will mostly work with the classicalframework in the time domain.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Website Description Requirements Readings Outline
Requirements
Two homework assignments with the weight of 40% of final grade.
The class ends either in-class 2 hour long exam.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Website Description Requirements Readings Outline
Readings
The book that covers most of the material isTime Series Analysis by James D. Hamilton, Princeton University Press,1994.
Other texts areState-Space Models with Regime Switching by Chiang-Jin Kim andCharles R. Nelson, MIT Press, 1999.New Introduction to Multiple Time Series Analysis by Helmut Lütkepohl,Springer-Verlag, 2005.Introduction to Bayesian Econometrics by Edward Greenberg, CambridgeUniversity Press, 2007.Structural Macroeconometrics by David N. DeJong with Chetan Dave,Princeton University Press, 2007.
Series of lectures by James H. Stock and Mark W. Watson"What’s New in Econometrics - Time Series", NBER Summer Institute2008
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Website Description Requirements Readings Outline
Outline
1 Stationary Time Series AnalysOverview of ARMA modelsState-Space RepresentationKalman Filter
2 Structural AnalysisGranger Causality, VAR, IRFs, Estimation, Variance decomposition
Reduced-form VAR modelsStructural VAR models
3 Unit Roots and Structural BreaksUnit root testsStructural break testsCointegrationVEC models
4 NonlinearityARCH modelsGARCH models
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Website Description Requirements Readings Outline
Today
Outline:
1 Covariance-Stationary Processes
2 Wold Decomposition Theorem
3 ARMA Models
1 Auto-Correlation Function (ACF)
2 Partial Auto-Correlation Function (PACF)
3 Model Selection
4 Estimation
5 Forecasting
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Stochastic Process
Stochastic process: a collection of random variables
Observed series {y1, y2, . . . , yT} – realizations of a stochastic process.
We want a model for {Yt}−∞∞ to explain observed realizations {yt}T1 .
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Covariance-Stationary
Definition
{Yt} is covariance-stationary (weak stationary) if(i) E[Yt] = µ ∀t
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Covariance-Stationary
Definition
{Yt} is covariance-stationary (weak stationary) if(i) E[Yt] = µ ∀t
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Covariance-Stationary
Definition
{Yt} is covariance-stationary (weak stationary) if(i) E[Yt] = µ ∀t
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Covariance-Stationary
Definition
{Yt} is covariance-stationary (weak stationary) if(i) E[Yt] = µ ∀t
It is weak stationarity because it only relates to the first two moments.Higher moments can be time-variant.
Examples1 {Yt} with E(Yt) = 0, Var(Yt) = σ2, and E(YtYτ )⇒ white noise (WN).
2 Yt ∼ iid(0, σ2)⇒ independent white noise.3 Yt ∼ iidN(0, σ2)⇒ Gaussian white noise.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Strict (Strong) Stationary
Definition
{Yt} is (strictly/strongly) stationary if for any values of j1, j2, . . . , jn the jointdistribution of (Yt,Yt+j1 ,Yt+j2 , . . . ,Yt+jn) depends only on the intervalsseparating the dates (j1, j2, . . . , jn) and not on date itself (t).
If a process is strictly stationary with a finite second moment it is alsocovariance-stationary.Normality⇒ strong stationarity: whole distribution depends on the firsttwo moments.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
t - time dummy
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
deterministic part
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
stochastic component
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
E[Yt] = β · t depends on tBut Xt = Yt − β · t is covariance stationary.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
E[Yt] = β · t depends on tBut Xt = Yt − β · t is covariance stationary.
2 Yt = Yt−1 + εt, εt ∼ WN,Y0 constantRandom walk
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Stochastic Process Covariance-Stationary Strict (Strong) Stationary Nonstationary Processes
Nonstationary Processes
Examples:
1 Yt = β · t + εt, εt ∼ WN
E[Yt] = β · t depends on tBut Xt = Yt − β · t is covariance stationary.
can be rewritten as a 1st order systemYt − µYt−1 − µYt−2 − µ
...Yt−p+1 − µ
=
φ1 φ2 . . . φp−1 φp
1 0 . . . 0 00 1 . . . 0 0...
.... . .
...0 0 .. 1 0
Yt−1 − µYt−2 − µYt−3 − µ
...Yt−p − µ
+
vt
00...0
In the state-space, companion form notation
βt = F · βt−1 + εt
and we are back in 1st order system.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations
AR(p): Stability
Consider a state space form
βt+j = Fj+1βt−1 + Fjεt + . . .+ Fεt+j−1 + εt+j.
AR(p) is stable and stationary if
limj→∞
Fj = 0
,i.e. when eigenvalues of F are inside unit circle (have modulus < 1).Shocks die out.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations
Eigenvalues
Consider equationFx = λx.
x is eigenvector and λ is a corresponding eigenvalue.To compute the eigenvalue, write it as
(F − λI)x = 0.
If x is a non-zero vector then
F − λI is singular ⇒ det(F − λI) = 0.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations
Example
Consider the AR(2)
Yt − µ = φ1(Yt−1 − µ) + φ2(Yt−2 − µ) + vt
Then, in a matrix notation
βt = Fβt−1 + εt,
where
βt =
[Yt − µ
Yt−1 − µ
], F =
[φ1 φ21 0
]Eigenvalues λ of the matrix F solves
⇒ det(φ1 − λ φ2
1 −λ
)= λ2 − λφ1 − φ2 = 0,
λi =φ1 ±
√φ2
1 + 4φ2
2If |λi|<1 for i = 1, 2, the AR(2) is stable .
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations
AR(p): Stability
For pth-order SDE, solve
λp − φ1λp−1 − . . .− φp−1λ− φp = 0
In general, the solution involve complex and real roots.The AR(p) system is stable if all eigenvalues are inside the unit circle.Note: complex eigenvalues imply periodic behavior.
2 4 6 8 10
-0.4
-0.3
-0.2
-0.1
0.1
0.2
AR2, f1= + 0.4 f2= - 0.5
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations
i.e. AR(∞) with φ(L) = 1 + θ − θ2 + θ3 − θ4 + . . ..|θ| < 1 is not a stability requirement, MA system is always stable. Itallows invertibility and AR(∞) representation.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF
Partial Autocorrelation Function (PACF)
Definition
kth-order partial autocorrelation is regression coefficient (for the population)φkk in kth-order autoregression
Yt = c + φk1Yt−1 + φk2Yt−2 + . . .+ φkkYt−k + εt
It measures how important is the last lag in the process.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Box-Jenkins Approach
Matching model with actual dataTransform data to “appear” covariance stationaryWe may have data that is not covariance stationary, e.g. GDP.
Box-Jenkins approach: maybe GDP is not covariance-stationary but some transformation of it is and
we can get forecast of it from transformed series.
take logs (natural)differencesdetrend
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
GDP
8000,0
10000,0
12000,0
14000,0
16000,0
GDP
GDP
0,0
2000,0
4000,0
6000,0
1947-01-01
1948-08-01
1950-03-01
1951-10-01
1953-05-01
1954-11-29
1956-06-29
1958-01-29
1959-08-29
1961-03-29
1962-10-29
1964-05-29
1965-11-29
1967-06-29
1969-01-29
1970-08-29
1972-03-29
1973-10-29
1975-05-29
1976-11-29
1978-06-29
1980-01-29
1981-08-29
1983-03-29
1984-10-29
1986-05-29
1987-11-29
1989-06-29
1991-01-29
1992-08-29
1994-03-29
1995-10-29
1997-05-29
1998-11-29
2000-06-29
2002-01-29
2003-08-29
2005-03-29
2006-10-29
2008-05-29
2009-11-29
We can’t use ARMA model as the graph shows it’s not covariance stationaryseries.Also, not linear.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Log of GDP
Take natural logarithm.
6
8
10
12
LN(GDP)
LN(GDP)
0
2
4
1947-01-01
1948-08-01
1950-03-01
1951-10-01
1953-05-01
1954-11-29
1956-06-29
1958-01-29
1959-08-29
1961-03-29
1962-10-29
1964-05-29
1965-11-29
1967-06-29
1969-01-29
1970-08-29
1972-03-29
1973-10-29
1975-05-29
1976-11-29
1978-06-29
1980-01-29
1981-08-29
1983-03-29
1984-10-29
1986-05-29
1987-11-29
1989-06-29
1991-01-29
1992-08-29
1994-03-29
1995-10-29
1997-05-29
1998-11-29
2000-06-29
2002-01-29
2003-08-29
2005-03-29
2006-10-29
2008-05-29
2009-11-29
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Difference in log of GDP = Growth rate
If we take first difference of the log series, we get growth rate of level series.
0,02
0,03
0,04
0,05
0,06
0,07
Diff ln(GDP)
Diff ln(GDP)
-0,03
-0,02
-0,01
0
0,01
19
47
-01
-01
19
48
-09
-01
19
50
-05
-01
19
52
-01
-01
19
53
-09
-01
19
55
-05
-01
19
57
-01
-01
19
58
-09
-01
19
60
-05
-01
19
62
-01
-01
19
63
-09
-01
19
65
-05
-01
19
67
-01
-01
19
68
-09
-01
19
70
-05
-01
19
72
-01
-01
19
73
-09
-01
19
75
-05
-01
19
77
-01
-01
19
78
-09
-01
19
80
-05
-01
19
82
-01
-01
19
83
-09
-01
19
85
-05
-01
19
87
-01
-01
19
88
-09
-01
19
90
-05
-01
19
92
-01
-01
19
93
-09
-01
19
95
-05
-01
19
97
-01
-01
19
98
-09
-01
20
00
-05
-01
20
02
-01
-01
20
03
-09
-01
20
05
-05
-01
20
07
-01
-01
20
08
-09
-01
20
10
-05
-01
It looks much more like AR or MA process.Just looking at graph may not be enough.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
50 100 150 200 250time
�2
�1
1
2
3
4
y: AR�1�
AR�1�, �� � 0.1
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
50 100 150 200 250time
�3
�2
�1
1
2
3
4
y: AR�1�
AR�1�, �� � 0.4
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
50 100 150 200 250time
�2
2
4
y: AR�1�
AR�1�, �� � 0.7
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
50 100 150 200 250time
5
10
15
20
25
y: AR�1�
AR�1�, �� � 1
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
50 100 150 200 250time
�3
�2
�1
1
2
3
y: AR�1�
AR�1�, ��.��0.2�
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(2)
50 100 150 200 250time
�2
�1
1
2
3
4
y: AR�2�
AR�2�, �1�.0.3 �2�.0.3
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
MA(1)
50 100 150 200 250time
�2
�1
1
2
3
4
y: MA�1�
MA�1�, ��.0.3
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
MA(1)
50 100 150 200 250time
�2
2
4
y: MA�1�
MA�1�, ��.0.7
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
MA(2)
50 100 150 200 250time
�2
2
4
y: MA�2�
MA�2�, �1�.0.7 �2�.0.4
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
It’s not that bad, though:
10 20 30 40 50time
�2
�1
1
2
3
y: AR�1�
AR�1�, ��.0.5
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
AR(1)
It’s not that bad, though:
10 20 30 40 50time
�2
2
4
y: AR�1�
AR�1�, ��.0.9
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Box-Jenkins Approach
Matching model with actual dataTransform data to “appear” covariance stationary
take logs (natural)differencesdetrend
Examine the sample ACF and PACFWe know that there is a 1-1 mapping between ACF and series
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
ACF: AR(1)
2 4 6 8 10
0.2
0.4
0.6
0.8
1.0
ACF AR�1�, �� � 0.6
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
ACF: AR(2)
2 4 6 8 10
0.5
0.6
0.7
0.8
ACF AR�2�, �1� � 0.6 �2� � 0.3
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
ACF: MA(1)
2 4 6 8 10
0.2
0.4
0.6
0.8
1.0
ACF MA�1�, �� � 0.7
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
ACF: MA(2)
2 4 6 8 10
0.2
0.4
0.6
0.8
1.0
ACF MA�2�, �� � 0.7 �2� � 0.4
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Sample ACF
y =1T
T∑t=1
yt,
γj =1T
T∑t=j+1
(yt − y)(yt−j − y), sample auto covariance estimate
ρj =γj
γ0sample auto correlation
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Sample PACF
Use OLS
yt = c + φ1kyt−1 + . . .+ φkkyt−k + εt, k = 1, 2, . . .
If yt ∼ iid(µ, σ2), ρj = 0, ∀j 6= 0 .(no correlation across observation)
ˆvar(ρj) ≈1T
and ˆvar(φkk) ≈1T
This values are used when reporting bounds in ACF and PACF:5%-95%: ±1.96 ˆvar(ρj) or ±1.96 ˆvar(φkk).
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Model selection
Assume we can reject iid, i.e. PACF and ACF show some significantlags. How to determine which model is correct?
Process ACF PACFAR(p) Exponential or oscillatory decay φkk = 0, k > pMA(q) ρk = 0, k > q Exponential or oscillatory decayARMA(p,q) decay begins at lag q decay begins at lag p
But we have many models we can’t really discriminate between just bylooking.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Box-Jenkins Approach
Matching model with actual dataTransform data to “appear” covariance stationary
take logs (natural)differencesdetrend
Examine the sample ACF and PACFEstimate ARMA modelsPerform diagnostic analysis to confirm that model is consistent withdata
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Box-Ljung Statistics
Use modified Box-Ljung Statistics (Q-Stat) to test joint statisticalsignificance of ρ1, . . . , ρk:
Q∗(k) = T(T + 2)
k∑i=1
1T − i
ρ2i .
Under H0 : Yt ∼ iid(µ, σ2), Q∗(k)A∼ χ2(k).
For significance level α, the critical region for rejection of thehypothesis of randomness (i.e. reject ρ1 = . . . = ρk = 0) is
Q∗(k) > χ21−α(k).
For smaller sample, empirical distribution has fatter tail and one needsto move the critical point to the right .
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Akaike Information Criterion (AIC)
Akaike Information Criterion (AIC):
AIC(p, q) = ln(σ2) +1T
2(p + q),
where the first term is responsible for the fit of the model and the second oneis a penalty for number of parameters.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Schwarz (Bayesian) Information Criterion
Schwarz (Bayesian) Information Criterion (BIC):
BIC(p, q) = ln(σ2) +1T
ln(T)2(p + q),
where now penalty is related to sample size.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Selection: AIC vs BIC
Calculate selection criterium for a set of estimated models and try tofind the one that minimize the AIC or BIC.Result:If p, q considered are larger than “true” orders
(i) BIC picks “true” ARMA(p,q) model as T →∞.(ii) AIC picks overparameterized model as T →∞.
For smaller sample it might be a a problem with choice BIC:Trade-off between efficiency and consistency:
Better to overparameterize than to have incorrect result.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation Box-Jenkins Approach GDP Sample ACF Sample PACF Box-Ljung Statistics Akaike Information Criterion (AIC) Schwarz (Bayesian) Information Criterion: Selection Residual Diagnostic
Residual Diagnostic
Use sample ACF and PACF of sample residuals.Compute Box-Ljung Statistics for sample of residuals:
Q∗(k)A∼ χ2(k − (p + q)),
where (p+q) is a degree of freedom adjustmentUse LM test (it has higher power)
T · R2 ∼ χ2(k),
with R2 computed from the regression
ε2t = c + α1εt−1 + . . .+ αkεt−k + vt.
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation OLS
Estimation
OLS
MLE
AR(1)
MA(1)
Macroeconometrics - Fall 2013 Class 1: Stationary Time Series Analysis
Syllabus Stationarity Wold ARMA AR MA Model Selection Estimation OLS
OLS
For AR(p), OLS is equivalent to Conditional MLEModel: