© Crown copyright Met Office Long-range forecasting Emily Wallace Nov 2012.

© Crown copyright Met Office

Long-range forecasting

Emily Wallace Nov 2012


Content

• How is long-range forecasting possible?

• predictability vs chaos, drivers of predictability, what is predictable?

• Dynamical seasonal prediction systems

• Initialisation, coupled modelling, assessing uncertainty

• Hindcasts

• Bias correction, model climatology, skill assessment

• Products

• Standard products, bespoke products


How is it possible?


Predictability and chaos

‘chaos’

dry wetThe distribution is analogous to the climatology of a meteorological variable (here, rainfall).

The ball drops can be seen as values corresponding to individual years.

• The precise bin in which a ball falls cannot be predicted (‘chaos’).

• If many drops are made, the ‘distribution’ of balls in the bins can be described.

Predictability and chaos

wetdry

‘chaos’

large-scale influences


Individual ball drops are analogous to individual forecasts, all with similar starting points.

The prediction consists in quantifying the difference between the two distributions (climatology and forecast).

Example of large-scale influence:ocean temperatures

• The precise bin in which a ball falls still cannot be predicted (‘chaos’).

• The tilt of the table changes the shape of the distribution (‘predictability’).

• If many drops are made, the new distribution of balls in the bins can be described.

© Crown copyright 2011

time

pred

icta

bilit

y

ICs

SSTs, surface, etc

external forcings

days months years

Sources of predictability: Initial conditionsBoundary conditions (SST, soil moisture, etc); External forcing (emissions, etc)

Seasonal: probabilistic forecast


Example of forcing: sea surface temperature anomalies

The forcing pattern is large scale and slow-varying in time.The impact is also large scale.


Teleconnections: typical El Niño impacts


Teleconnections: typical La Niña impacts

• ‘climate’ (seasonal averages), not ‘weather’ (conditions on specific days)

• large-area averages, not localised events

• range of outcomes, with probabilities attached to them (risk)

What is predictable at long range?

How are long-range predictions done?

• statistical methods – using empirical relationships derived from historical records

• dynamical methods – using dynamical (climate) models

Statistical models – using empirical relationships derived from historical records

Statistical models are…

• Cheap – equations are far less complex than dynamical models

But…

• Require a long, good quality, observational dataset to train the model on

• Will produce poor predictions if the assumptions change


Seasonal forecasting with dynamical models

A seasonal forecasting system requires:

• definition of starting point (initial conditions; data assimilation)

• model of the climate system

• description of uncertainties (ensembles)

Dynamical methods: seasonal forecasting systems


For seasonal forecasting, assimilation of ocean state is important

Tropical Atmosphere Ocean array (TAO)

ARGO floats

SST

Subsurface ocean

Climate simulation Vs. seasonal prediction


Climate model

Observed state of ocean/atmosinitialisation time

Synchronised to real world

Forecast from initialisation time

Climate model

Climate simulationArbitrary or climatology start






Coupled and uncoupled seasonal forecast systems http://www.wmolc.org “Data ->system configuration”

• Coupled (1-tier) systems

• Model: Includes interactive 3D ocean model

• Initial conditions: atmosphere + 3D ocean

• GPCs: Exeter, ECMWF, Washington, Toulouse, Melbourne, Montreal, Tokyo, Beijing

• Forecast range: typically 6 months +

• Uncoupled (2-tier) systems

• Model: atmosphere only + prescribed SST. Atmosphere ‘forced’ with predicted (or persisted) SST anomalies. No 2-way atmosphere/ocean interaction

• Initial conditions: atmosphere (usually) + SST

• GPCs: Moscow, Seoul, CPTEC, Pretoria

• Forecast range: typically 3-5 months


http://www.wmolc.org/








Uncertainty type 1: initial condition uncertainty


Climate model

Data assimilation: ocean/atmos

• Run model forecasts from many slightly different initial conditions• Forecasts may evolve differently • Collectively, the ensemble estimates the range of uncertainty stemming from sensitivity to initial conditions

forecastEnsemble prediction

Addressing initial condition uncertainties


Uncertainty in initial atmospheric

state Uncertainty in future atmospheric

state

Ensemble forecast explores part of the future uncertainty (from initial

condition) uncertainty

Uncertainty type 2: uncertainty in model formulation• When climate models are developed choices must be

made on schemes to represent physical processes

• e.g. Convection scheme, radiation scheme...

• Forecasts from the same basic model and same initial state may give different forecasts when different physics schemes are used.

• Choice of physics scheme is often centre dependent

• Thus more uncertainty!

• Model formulation uncertainties are addressed by:

• Stochastically perturbing model variables (and/or tuneable physics parameters) as the model runs

• Combining ensembles from different modelling centres. Typically each centre will have made different choices in model formulation. Thus multi-model: e.g. LC-LRFMME, EUROSIP, APCC


Addressing model formulation uncertainties


Uncertainty in initial atmospheric

state Uncertainty in future atmospheric

state

Ensemble forecast from model 1 explores part of the future uncertainty (from initial

condition) uncertainty

Ensemble forecast from model 2 (i.e. perturbed physics), run from same set of initial states, typically explores additional future uncertainties (from model

formulation uncertainty)

Including representation of model formulation uncertainties gives better sampling of the true uncertainty.

Example ensemble predictionsMet Office GloSea4 system

• Initial condition uncertainty (lagged analysis)

• 21 different initial ocean/atmos states used (daily lag)

• Model formulation uncertainty

• Stochastic (kinetic energy) perturbations to model wind field as the model runs


22nd Feb‘11

23rd

Mar’1121st Feb‘11

Forecast

2 perturbed runs from daily start dates ->14 runs to (7 months) each week, after 3 weeks we have a 42 member ensemble

GloSea4 ensemble prediction of Nino3.4 SST anomaly from March 2010


Ensemble ‘postage’ stamps


Ensemble mean: reinforces commonalities, masks uncertainties


Qualities of the ensemble mean

• Considered the ‘most likely’ single (deterministic) prediction.

• Usually lies near centre of the ensemble distribution

• Picks out the dominant signal:

• Commonalities across the members ‘reinforce’

• Differences across members tend to cancel

Important ‘but’…..

• Quantitative information on uncertainty is removed by the averaging process

Seasonal prediction systems


Climate model

Initialisation:Current state of ocean/atmos

forecast

Ensemble generation


Hindcasts: Correcting model climatology and assessing model quality

Hindcasts: for bias correction and skill assessments

• To adjust for biases in the seasonal forecast we generate a set of retrospective forecasts (hindcasts) that describe the ‘climatology’ of the model

• Model climatologies are defined over all retrospective years and all members

• For GloSea4: 14 hindcast years, 12 members = 168 realisations of each season.

• Note: most systems have more ensemble members in the real-time forecast than in the hindcast set.

• Hindcasts are also the basis of assessing forecast performance



Model bias

• In this example:

• 14 years (1989-2002)

• 8 members per year

Calibrated forecast

obs

Hindcastmean

Forecastmembers

At long range, predict anomalies

black line: observed climatologyPale red lines: hindcast membersblue line: model climatology

MAM Temperatures in SE Asia

March-April-May

Temperature Anomalies relative to 1961-1990

Black Line:

CRUTEM3

Red Lines:

8 GCM simulations• CNRM-CM5

• CanEMS2

• GISS-ES-R

• HadGEM2-ES

• NorESM1-M

• bcc-csm1-1

• inmcm4

• ipsl-csma

The model simulations were extended to 2010 following RCP8.5

Courtesy of Nikos Christidis


• PROBLEM:

• What does “climate” mean under climate change?

Calculating anomalies: the importance of the reference period


Ensemble-mean forecast for the average temperature anomaly over MAM 2011

1996-2009

1981-2010

1971-2000

Reference period


Generating probabilistic forecasts

Probabilistic forecasts and bias correction e.g. precipitation forecast


wet

dry

Observed climatology,

Lower tercile (obs)

Upper tercile (obs)

Model climatology, e.g. wet bias

Upper tercile (model)

Lower tercile (model)

Ensemble member

Member is counted as a prediction of the average (obs) tercile category

Generating probability forecasts from the ensemble

• An estimate of the forecast probability of an event is the proportion of the ensemble members that predict the event


Category No. Members that predict category

Fraction of total ensemble members

Forecast probability

above 5 5/9 55%

average 3 3/9 33%

below 1 1/9 11%

Seasonal prediction systems


Climate model

Initialisation:Current state of ocean/atmos

forecast

Ensemble generation

Retrospective forecasts

(hindcasts)Skill

assessment(verification)

Forecast bias correction


Hindcasts: Assessing skill

Skill of seasonal forecasting systems

• Skill is assessed on the hindcast (covering a number of past years)

• Can (and should) be done in several ways:

• Statistical assessment of skill

• Process based assessment

What is in Malaysia for me?What’s in Malaysia for me?


Statistical skill of forecast products, estimated from hindcasts:http://www.metoffice.gov.uk/research/climate/seasonal-to-decadal/gpc-outlooks/glob-seas-prob-skillROC curves Reliability

diagram


• Forecasts are generated monthly using data from GloSea4 and ECMWF

• Skill (linear correlation) of 6-month forecasts from March to September is detailed below

Skill (linear correlation)

Mar Apr May Jun Jul Aug Sep

TS 0.26 0.49 0.59 0.33 0.55 0.50 0.42

ACE 0.14 0.25 0.74 0.61 0.56 0.46 0.17

Perfect forecasts would have a skill of 1.0

Deterministic skill assessmentSkill of tropical storm seasonal forecast 1987–2009

Skill of seasonal forecasting systems

• Skill is assessed on the hindcast (covering a number of past years)

• Can (and should) be done in several ways:

• Statistical assessment of skill

• Process based assessment


Time

Lat

itud

e

Colours: 5-day average rainfall in mm/day, 10°W-10°E

Red line: Timing of monsoon onset , early July

Time

Lat

itud

e

Mean observed rainfall (TRMM1998-2010) GloSea4 mean rainfall (1996-2009), 25 April start dates

•Good agreement between observed and GloSea4 temporal evolution of monsoon and onset timing

•Some skill in predicting late/early onset (ROC score ~0.6)

Seasonal forecasts with GloSea4 of timing of monsoon onset over Sahel


ENSO teleconnections: precipitation

JJA

DJF

Forecast (E-L) Observed (E-L)

Skilful reproductions in the tropics – even for rainfall

Red = El Nino is drier Blue = El Nino is wetter


Examples of simple forecast products:http://www.metoffice.gov.uk/research/climate/seasonal-to-decadal/gpc-outlooks


‘Raw’ products


Examples of bespoke forecast products and information

GloSea4 Forecast probabilities for 2011 Short Rains (Sep-Nov)

Early onset: Late onset:

Courtesy of Michael Vellinga

Observations for 2011

Plots courtesy of Lizzie Good Courtesy of Michael Vellinga

Forecast products

Deterministic forecasts

• Provides a best estimate and forecast range (±1 stdev interval) for:

• Numbers of named storms

• ACE index

• During the following 6 months

Probabilistic forecasts

• Probability distributions

• Exceedance of thresholds (to aid assessment of risk)

• Help to quantify and communicate the inherent uncertainties in the forecast.

Public forecast

Tailored products


Summary

How is long-range forecasting possible?

• Large scale forcing that evolve slowly can make climate predictable

Dynamical seasonal prediction systems

• Must include: initialisation, a climate model, and a way to assess uncertainty

Hindcasts

• Due to model biases hindcasts are needed for correction of forecasts, They are also used to assess forecast quality, and can lead to model improvements

Products

• We are developing new and exciting bespoke products

Questions and answers

© Crown copyright Met Office Long-range forecasting Emily Wallace Nov 2012.

Documents