Joe Tribbia NCAR IPAM lecture 15 February 2007

Uncertainty QuantificationUsing Ensemble Methods:

Predictability of Extremes and Coherent Vortices

Joe TribbiaNCAR

IPAM lecture 15 February 2007

Outline

• General problem of uncertainty prediction

• Reliability prediction as practiced operationally

• Specific problem of extreme events

• Stochastic physics

• Path prediction and shadowing

Uncertainty prediction

Prior to 1990 all numerical weather forecasts deterministic (n.b. Pitcher and Epstein,1974)

• Post 1990 Modus Operandi: Numerically forecast weather and its uncertainty (0-10 day) time range

• Gigantic numerical model, dynamical system: degrees of freedom

• Uncertainty prediction obtained from ensemble of <100 forecasts with representative initial condition uncertainty

87 1010

The probabilistic approach to NWP: ensemble prediction

A complete description of the weather prediction problem can be stated in terms of the time evolution of an appropriate probability density function (pdf).

Ensemble prediction based on a finite number of deterministic integration appears to be the only feasible method to predict the PDF beyond the range of linear growth.

fc

0

fcj

reality

pdf(0)

pdf(t)

Temperature Temperature

Forecast time

Sampling strategies for small samples in high dimensional

systems

Bred vectors and Singular vectors

Basic state jet

Singular vector (upper)Bred vector (lower)

Singular vectors are the fastest growing structures into the futureBred vectors are the fastest growing structures in the past.

Operational centers battled over which was superior.NB: Inconsistencies in initial error will disappear with Ens KF

Predictability is flow dependent: spaghetti plots

The degree of mixing of Z500 isolines is an index of low/high perturbation growth.

The atmosphere exhibits a chaotic behavior: an example

A dynamical system shows a chaotic behavior if most orbits that pass close to each other at some point do not remain close to it as time progresses.

This is illustrated by the forecasts of the storm that hit northern Europe on 4 December 1999.

4 Dec 1999, 00UTC : verifying analysis (top-left) and t+132h ensemble forecasts of mean-sea-level pressure started from slightly different initial conditions (i.e. from initially very close points).

Forward looking SVs (possibly)better for extrema

Quantifying known unknowns:model error

Ensemble prediction demonstrated that IC error was important but the imperfection of models needed to be accounted for in any UQ for weather prediction

Rank histogram shows the verification of 72hr temperature predictions with ECMWFensemble. A perfect system would have aflat histogram. U shape indicates the system is underpredicting uncertainty.

Rationale for stochastic terms

MOTIVATION:

• Traditional dimensional reduction/closure-account for unresolved scales

• Weather uncertainty prediction-should take into account all sources of uncertainty in particular model error

• May induce extremes

Growth of model error (T&B)

T&B examined the growth of errors due to the impact of unresolvedscales by comparing integrations with identical ICs and differing horizontal resolutions from T170 to T42.

Stochasticity: sub-grid distributionconvection parameterization

Each ensemble member evolution is given by the time integration

of the perturbed model equations starting from the perturbed initial conditions

The model tendency perturbation is defined at each grid point by

where rj(x) is a set of random numbers.

‘Stochastic physics’ and the ECMWF EPS

T

t

jjjjj dttePtePteATe0

)],(),(),([)(

)0()0()0( 0 jj eee

),,(),(),,( pPrpP jjj

Figure 6. May-June-July 2002 average RMS error of the ensemble-mean (solid lines) and ensemble standard deviation (dotted lines) of the EC-EPS (green lines), the MSC-EPS (red lines) and the NCEP-EPS (black lines). Values refer to the 500 hPa geopotential height over the northern

hemisphere latitudinal band 20º-80ºN.

Buizza et al. (2004)

Spread and forecast skill

Not enough spread

BAD NEWS FOR EXTREMES

• Even with stochastic forcing, predicted (conditional) distribution deficient in wings

• SVs need unrepresentative amplitude to represent total initial uncertainty

• Stochastic forcing can alleviate under-dispersion but masks model rectifiable(?) model variability deficiencies

Gratuitous Hurricane picture:(easier problem?)

ECMWF uses targeted SVs with stochastic physics for TCs

TR-SVs’ target areas: impact of the Sep ’04 change

Results based on 44 cases (from 3 Aug to 15 Sep 2004) indicate that the implemented changes in the computation of the tropical areas has a positive impact on the reliability diagram of strike probability.

Reliability diagram for strike probabilities

Old CY28R2 EPSNew CY28R3 EPS

Ensemble prediction of tracks

Simplistic TC track model

• Barotropic model with point vortex

• Metaphor/model of tropical cyclone track

• Ref:Kasahara1963,

Morikawa1960,

Zabusky and McWilliams1982

contcont

cont

s

scont

q

tsK

tsq

qqq

qJtq

)(

))((

))((

0),(

22

0

rr

r

Point vortex stream function

Model simulationPoint vortex in hyperbolic flow

Weak point vortex advected inflow; would be sensitive tovariation in x(0).

Interaction makes the track lessSensitive.

Chris Velden (U.Wisc/CIMSS)

Reality: multi-scale interaction and weather

Water Vapor

Channel

Note the smaller scale structure in tropics

Ensemble of tracksTrack distributionvarying x(0),y(0)and s(0)

Variational shadowing

• Shadowing trajectories needed to separate model errors from observational errors

• Objective measure of trajectory accuracy• Four dimensional variational minimization of

cost J(x)

))()(())()(())0(( iobsit

iobsi

i ttttJ xxWxxx

Use ensemble to minimize cost function J :1-d slices

J is strongly dependent on x(0); weakly dependent on y(0) and s(0)

J as function of ensemble index and 2-d x-y surface

J(x(0),y(0))

x

y

J_min=0.4436

Bayesian Data Assimilation

Posteriordistributionproportionalto product

EDA: towards a probabilistic analysis & forecast system?

EDA ensemble-mean

EDA perturbed members

High-resolution forecast

Low resolution forecast

• Ensemble assimilation predicts covariance

• Variational smoother gets optimal trajectory

Conclusions

• Ensemble techniques offer method of uncertainty/predictability prediction

• Can be tailored for extrema, but extremes must exist in the ensemble (i.e. seeds in the conditional distribution)

• Stochastic terms needed to inflate ensemble variance• Shadowing can be used to ensure that verifying analysis

is part of model repertoire and calibrate model errors to rationally gauge stochastic terms.

• Ensemble can be used to solve variational problem . Can this be generalized for small ensemble-large dimensions ?