OLS Regression Auto-correlated Models Regression with Autocorrelated Errors Regression With Autocorrelated Errors EDU 7309 Project Xiaowen Hu & Wenkai Bao Southern Methodist University Apr. 7th, 2010 Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
39
Embed
Regression With Autocorrelated Errorsfaculty.smu.edu/kyler/courses/7309/hu_bao_autocorrelation.pdf · OLS Regression Auto-correlated Models Regression with Autocorrelated Errors Linear
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Regression With Autocorrelated ErrorsEDU 7309 Project
Xiaowen Hu & Wenkai Bao
Southern Methodist University
Apr. 7th, 2010
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Linear Regression Model
Settings and Assumptions
Linear Regression Model
yi = β0 + β1xi1 + . . . + βkxik + εi ,
where yi , xi1, . . . , xik are observations of k + 1 variables, andεi
iid∼ N(0, σ2).
E(εi) = 0 for i = 1, . . . , nVar(εi) = σ2 for i = 1, . . . , ncov(εi , εj) = E(εiεj) = 0 for i 6= j
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Linear Regression Model
Least Square Regression
β0, . . . , βk unknownResidual ei = yi − β̂0 − β̂1xi1 − . . .− β̂kxik
β̂0, . . . , β̂k minimizes∑n
i=1 e2i
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Autocorrelated Errors
Relaxing The Assumptions
What if cov(εi , εj) 6= 0?More specific, autocorrelation among errorsThis may occur when
Missing true explanatory variablesMisspecification of models (linear vs. quadratic)Pure correlated errors (true autocorrelation)
β̂0, . . . , β̂k are still unbiasedthat is, expectations of β̂’s are β’s.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Autocorrelated Errors
Relaxing The Assumptions
However...Variance of errors may be underestimatedVariance of β̂’s may be underestimatedConfidence intervals may not be applicableSpurious regressione.g. two uncorrelated variables may appear related.
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Autocorrelated Errors
Remedial Options
Add predictorsHigher order predictorsTransform variablesCochrane-Orcutt (1949)
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
ARMA Models
Basic Definitions
Time series processA collection of random variables Xt ’s. i.e. {Xt}, where t istime index.Autocovariance
Residual standard error: 6.385 on 503 degrees of freedom Multiple R‐squared: 0.5954, Adjusted R‐squared: 0.5922 F‐statistic: 185 on 4 and 503 DF, p‐value: < 2.2e‐16
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors
Illustrative Example
Obtain OLS Residuals
80 90 100 110 120
0.0
0.5
1.0
1.5
2.0
Fitted values
Sta
nd
ard
ize
d r
esid
ua
ls
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●
●
●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●●
●
Scale−Location152
154257
●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●
●
●●●
●
●
●
● ●●● ●
●
●
●
●
●● ●●
●
●
●●
●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
● ●
●●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
● ●
●●●
●●
●●●
●
●
●
● ●
●●
●
●
●
●●
●
●
●●
●
●
●●●
●●
●
●
●
● ●
●
●
●
●●
●
●
● ●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●● ●
●●
●
●
●
●
●
●●
●
●
●●
●
●●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●●
●
●●
●
●
●
●
●
●
●● ●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
● ●
●●
●
●
●●●●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
● ●
●
●
●
●● ●
●●●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−2
02
4
Theoretical Quantiles
Sta
nd
ard
ize
d r
esid
ua
ls
Normal Q−Q
152
154257
0 5 10 15 20 25
0.0
0.2
0.4
0.6
0.8
1.0
Lag
AC
F
ACF of residuals
0 5 10 15 20 25
−0
.10
.00
.10
.20
.3
Lag
Pa
rtia
l A
CF
PACF of residuals
Xiaowen Hu & Wenkai Bao Regression With Autocorrelated Errors
OLS Regression Auto-correlated Models Regression with Autocorrelated Errors