Increasing uncertainties? A post-mortem on the Federal Planning Bureau’s medium-term projections

Federal Planning Bureau Economic analyses and forecasts

http://www.plan.be

Increasing uncertainties?Increasing uncertainties? A post-mortem on the Federal Planning Bureau’s A post-mortem on the Federal Planning Bureau’s

medium-term projectionsmedium-term projections

Francis Bossier

Igor Lebrun

39th CMTEA - Romania, Iaşi, September 25-27th, 2008

Federal Planning BureauEconomic analyses and forecasts

How to address uncertainty in forecasting?How to address uncertainty in forecasting?

Scenario analysis (‘variants’):Scenario analysis (‘variants’):

To illustrate effects of potential risks surrounding baseline To illustrate effects of potential risks surrounding baseline (results are often presented as deviations from baseline)(results are often presented as deviations from baseline)

Confidence intervals: Confidence intervals:

Traditionally symmetric (ECB projections for the euro Traditionally symmetric (ECB projections for the euro area) but can also be asymmetric to reveal unbalanced area) but can also be asymmetric to reveal unbalanced risks (‘fan chart’ in IMF WEO)risks (‘fan chart’ in IMF WEO)


Why is it useful to evaluate past forecast Why is it useful to evaluate past forecast performances ?performances ?

To assess whether forecasts meet basic quality To assess whether forecasts meet basic quality standards: standards:

• Unbiasedness and absence of serial correlation (weak Unbiasedness and absence of serial correlation (weak efficiency)efficiency)

• Information set fully exploited (informational efficiency)Information set fully exploited (informational efficiency)

• Size of error declines when forecasting horizon shortens Size of error declines when forecasting horizon shortens

• Directional accuracyDirectional accuracy

• Prediction of turning pointsPrediction of turning points


Why is it useful to evaluate past forecast Why is it useful to evaluate past forecast performances? (2)performances? (2)

To identify possible methodological weaknesses that To identify possible methodological weaknesses that should be improvedshould be improved

To give users a broad idea of the precision of the To give users a broad idea of the precision of the forecastforecast

International organizations tend to asses their International organizations tend to asses their forecasts on a relatively regular basis:forecasts on a relatively regular basis:

• IMF: Working Paper by Allan Timmerman (2006) IMF: Working Paper by Allan Timmerman (2006)

• OECD: Working Paper by Lukas Vogel (2007) OECD: Working Paper by Lukas Vogel (2007)

• EC: Economic paper by Melander et. al. (2007) EC: Economic paper by Melander et. al. (2007)


The Belgian Federal Planning Bureau:The Belgian Federal Planning Bureau: Short-term macro-economic forecasts Short-term macro-economic forecasts

Economic budget:Economic budget:

• Released twice a year: September for the year t+1 and February Released twice a year: September for the year t+1 and February for the year tfor the year t

• Produced with quarterly macro-econometric model but adjusted for Produced with quarterly macro-econometric model but adjusted for recent business cycle informationrecent business cycle information

2004: track record for GDP growth and inflation (WP 13-04)2004: track record for GDP growth and inflation (WP 13-04)

2006: update of post-mortem analysis (WP 4-06)2006: update of post-mortem analysis (WP 4-06)

Project: evaluation of forecasts for larger set of variablesProject: evaluation of forecasts for larger set of variables


The Belgian federal Planning Bureau: The Belgian federal Planning Bureau: Medium-term macro-economic projections Medium-term macro-economic projections

Economic outlook:Economic outlook:

• Released in Spring with an update in AutumnReleased in Spring with an update in Autumn

• Very detailed macroeconomic projectionVery detailed macroeconomic projection

• Forecast for current year based on a economic budgetForecast for current year based on a economic budget

• Simulation with six-year horizon using the HERMES modelSimulation with six-year horizon using the HERMES model

• Unchanged policy scenario with regard to fiscal and social policiesUnchanged policy scenario with regard to fiscal and social policies

2006: evaluation of GDP growth and budgetary projections 2006: evaluation of GDP growth and budgetary projections (in European Economy, Economic Papers No. 275)(in European Economy, Economic Papers No. 275)

2007: evaluation of past projection errors for a large set of 2007: evaluation of past projection errors for a large set of variables (WP 8-07)variables (WP 8-07)


The methodology used to evaluate the The methodology used to evaluate the medium-term projectionsmedium-term projections

Focus on GDP trend growth and trend inflationFocus on GDP trend growth and trend inflation

Methodology proposed by Jonung and Larch (EP, 2006):Methodology proposed by Jonung and Larch (EP, 2006):

• For each historical data vintage trend growth is obtained by HP-For each historical data vintage trend growth is obtained by HP-filtering the series (historical data + projections)filtering the series (historical data + projections)

• The value of year t+1 of the filtered series is taken as the one-The value of year t+1 of the filtered series is taken as the one-year-ahead forecast of trend growthyear-ahead forecast of trend growth

• The outcome is computed by applying recursively the HP-filter The outcome is computed by applying recursively the HP-filter on a rolling sample based on the latest data vintageon a rolling sample based on the latest data vintage

Sample starts with 1986 outlook (covering 1987-1990) and Sample starts with 1986 outlook (covering 1987-1990) and ends with 2006 issue (covering 2007-2011) ends with 2006 issue (covering 2007-2011)

Outcome given by the 2008 issue (covering 2009-2013)Outcome given by the 2008 issue (covering 2009-2013)


GDP trend growth: one-year-ahead forecastGDP trend growth: one-year-ahead forecast


GDP trend growth: one-year-ahead forecast vs. GDP trend growth: one-year-ahead forecast vs. actual estimatesactual estimates


GDP trend growth: Forecasting errorGDP trend growth: Forecasting error


GDP trend growth: Forecasting error and output gapGDP trend growth: Forecasting error and output gap


GDP trend growth and potential export markets: GDP trend growth and potential export markets: Forecasting errorsForecasting errors


Output gap: one-year-ahead vs. actual estimatesOutput gap: one-year-ahead vs. actual estimates


GDP trend growth: Contributions to forecasting errorsGDP trend growth: Contributions to forecasting errors


Descriptive statistics for projection errors Descriptive statistics for projection errors ((1987-2007)1987-2007)

Mean errorMean error No bias No bias No corrNo corr

GDPGDP -0.10 -0.10 0.37 0.37 0.00 0.00

ProductivityProductivity -0.30 -0.30 0.00 0.00 0.00 0.00

EmploymentEmployment 0.20 0.20 0.03 0.03 0.00 0.00

Positive (negative) mean error = underestimation (overestimation)Positive (negative) mean error = underestimation (overestimation)

No bias = probability for zero mean errorNo bias = probability for zero mean error

No corr = probability for uncorrelated errorsNo corr = probability for uncorrelated errors


Productivity growth: actual and trendProductivity growth: actual and trend


Productivity growth: one-year-ahead forecast vs. Productivity growth: one-year-ahead forecast vs. actual estimatesactual estimates


Trend inflation: one-year-ahead forecastTrend inflation: one-year-ahead forecast


Trend inflation: one-year-ahead forecast vs. actual estimatesTrend inflation: one-year-ahead forecast vs. actual estimates


Trend inflation: Forecasting errorsTrend inflation: Forecasting errors


Price trend growth: Forecasting errorsPrice trend growth: Forecasting errors


ConclusionsConclusions

Difficulty to disentangle trend from cycle at the end of Difficulty to disentangle trend from cycle at the end of the samplethe sample

Risk of running behind the facts because trend growth Risk of running behind the facts because trend growth only “observable” with a huge delayonly “observable” with a huge delay

Increased uncertainty? No clear indication that Increased uncertainty? No clear indication that projection errors have increased recently but only projection errors have increased recently but only future will tell future will tell

Despite size and inertia of projection errors we believe Despite size and inertia of projection errors we believe our medium-term outlook remains an important tool our medium-term outlook remains an important tool for policy-makers, social partners and public in general for policy-makers, social partners and public in general in terms of diagnosis for the Belgian economyin terms of diagnosis for the Belgian economy

Increasing uncertainties? A post-mortem on the Federal Planning Bureau’s medium-term projections

Documents

forecast gdp trend growth

evaluation of gdp growth

output gap gdp trend

year horizon

trend inflationmethodology

current year

year tproduced

trend growththe outcome