Introduction Prediction Mean Square Error Prediction Intervals Empirically Based P.I.s Summary Interval Forecasting Based on Chapter 7 of the Time Series Forecasting by Chatfield Econometric Forecasting, January 2008 Pekalski, Swierczyna, Zalewski interval forecasting
44
Embed
Interval Forecasting - Zentraler Informatikdienst ...homepage.univie.ac.at/robert.kunst/pres07_prog_pekswizal.pdf · Title: Interval Forecasting Author: Based on Chapter 7 of the
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Interval Forecasting
Based on Chapter 7 of the Time Series Forecasting byChatfield
Most forecasters realize the importance of providinginterval forecasts and point forecasts in order to:
asses future uncertainty,enable different strategies to be planned for the range ofpossible outcomes indicated by the interval forecast,compare forecasts from different methods more thoroughly.
Reasons for not using interval forecasts:rather neglected in statistical literature,no generally accepted method for calculating predictionintervals (with some exception),theoretical prediction intervals are difficult or impossible toevaluate for many econometric models containing manyequations or which depend on non-linear relationship.
By density forecast we mean finding the entire probabilitydistribution of a future value of interest.Linear models with normally distributed innovations:
the density forecast is typically normal with mean equal tothe point forecast and variance equal to that used incomputing a predicion interval.
Linear model without normally distributed innovations:seems to be more prevalent and used e.g. in forecastingvolatility,different percentiles or quantiles of the conditionalprobability distribution of future values are estimated.
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorPotential Pitfall
Assesing forecast uncertainty, remember:
the var of the forecast 6= the var of the forecast error.
Given data up to time N and a particular method or model:the forecast x̂N(h) is not a random variable, it has varianceof zero,XN+h and eN(h) are random variables, condtioned by theobserved data.
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorBias of PMSE
Even assuming that the true model is known a priori, there willstill be biases in the usual estimate obtained by substitutingsample estimates of the model parameters and the residualvariance into the true-model PMSE formula.
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorConditional and Unconditional Errors
Looking once more at equation
eN(1) = (α− α̂)xN + εN+1.
Consider:conditional on xN forecast error: xN is fixed α̂ - biasedestimator of α, then the expected value of eN(1) need notbe zero.unconditional forecast error: If, however, we average overall possible values of xN , as well as over εN , then it can beshown that te expectation will indeed be zero giving anunbiased forecast.
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorUnconditional PMSE
Unconditional PMSE can be usefull to assess the ’success’ of aforecasting method on average. This apporach if used tocompute P.I.s, it effectively assumes that te observatoins usedto estimate the model parameters are independent of thoseused to construct the forecasts. This assumption can bejustified asymptotically. Box assesed that the correction termswould generally be of order 1
Properties and assumptions of the formula for P.I.:symmetric about x̂N(h),assumes the forecast is unbiased with PMSEE [eN(h)2] = Var [eN(h)],forecast errors are assumed to be normally distributed.
Note: some authors state that the zα/2 should be replaced bythe precentage point of a t-distribution, with appropriate numberof degrees of freedom (worth making for less than 20 obs).
is generally used for P.I.s. preferably after checking theassumptions (e.g. forecast errors are approximately normallydistributed) are at least reasonably satisfied. For any givenforecasting method the main problem will then lie withevaluating Var [eN(h)].
P.I.s derived from a fitted probability modelFormulas for PMSE
PMSE can be derived for:ARMA models (also seasonal and integrated),structural state-space,various regression models (typically allow for parameteruncertainty and are conditional in the sense that theydepend on the particular values of the explanatoryvarialbes from where a prediction is being made).
Cannot be derived:some complicated simultaneous equation,non-linear.
P.I.s when assumed that method is optimalWhen it is reasonable ?
When it is reasonable?Observed one-step-ahead forecast errors show no obviousautocorrelation.No other obvious features of the data (e.g. trend) whichneed to be modeled.
Forecasting methods not based on a probability model
Assume that the method is optimal in the sense that theone-step ahead errors are uncorrelated.Easy to check by looking at the correlogram of theone-step-ahead errors:
if there is correlation we have more structure in the datawhich should improve the forecast.
When to use?1st method2nd methodSimulation and Resampling
Empirically based P.I.s1st method
Apply forecasting method to all the past data.Find within-sample ’forecasts’ at 1, 2, 3, . . . steps ahead(from all available time origins).Find the variance of these errors (at each lead time overthe periods of fit).Assume normality.
When to use?1st method2nd methodSimulation and Resampling
Empirically based P.I.s1st method
Result:approximate empirical 100(1− α)% P.I. for XN+h is given by
x̂N(h) + /− zα/2√
Var [eN(h)].
Problems:if N small - assume t-distribution,long series is needed to get reliable values for se,h,smooth values to make them increase monotonically with hvalues of
√Var [eN(h)] based on in-sample residuals not
on out-of-sample forecast errors,results comparable to theoretical formulae (if available).
When to use?1st method2nd methodSimulation and Resampling
Simulation and resampling methodsSimulation (Monte Carlo approach)
Assumption:Probability time-series model is known (and identifiedcorrectly)Generate random innovationsGenerate possible past and future valuesRepeat many timesFind the interval within which the required percentage offuture values lie
When to use?1st method2nd methodSimulation and Resampling
Simulation and resampling methodsResampling (bootstrapping)
Sample from the empirical not theoretical distribution
⇒ distribution-free approach
The idea (the same as for simulation):use the knowledge about the primary structure of themodelgenerate a sequence of possible future valuesfind a P.I. containing the appropriate percentage of futurevalues
When to use?1st method2nd methodSimulation and Resampling
BootstrappingProperties
Properties of bootstrap:Bootstrap P.I.s are useful non-parametric alternative to theusual Box-Jenkins intervals.It is difficult to resample correlated data.
When to use?1st method2nd methodSimulation and Resampling
Uncertainty in Forecasts
Sources of uncertainty in forecasts from econometric models:the model innovations,having estimates of model parameters rather than truevalues,having forecasts of exogenous variables rather than truevalues,misspecification of the model.
Summarized main findings and recommendations:Formulate a model, that provides a reasonable apporx forthe process generating a given series, derive PMSE, anduse the formula.Distinction between a forecasting method and a forecastingmodel should be borne in mind. The former may, or maynot, depend (explicitly or implicitly) on the latter.Use not model but method based approach (e.g. theHolt-Winters method).
No theoretical formulae, or there are doubts about modelassumptions, use the empirically based approach.The reason why out-of-sample forecasting ability is worsethan within-sample fit is that the wrong model may havebeen identified or may change through time.The formula x̂N(h) + /− zα/2
√Var [eN(h)] assumes:
model has been correctly identified,innovations are normally distributed,the future will be like the past.
Rather than compute P.I.s based on a single ’best’ model,use Bayesian model averaging, or not model-basedapproach.