ECMWF Stochastic representations of model uncertainty: Glenn Shutts March 2009 Stochastic representations of model uncertainty Glenn Shutts ECMWF/Met Office.

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Stochastic representations of model Stochastic representations of model uncertaintyuncertainty

Glenn Shutts Glenn Shutts

ECMWF/Met OfficeECMWF/Met Office

Acknowledgements : Judith Berner, Martin Leutbecher

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

OutlineOutline• Ensemble model spread

• The nature of ‘model error’

• The ‘stochastic physics scheme’ (perturbing parametrized tendencies)

• The spectral stochastic backscatter scheme

• Calibrating the schemes by coarse-graining

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Ensemble Forecast for Thurs 15Ensemble Forecast for Thurs 15thth 2007 2007

Representing initial state uncertainty by an Representing initial state uncertainty by an ensemble of statesensemble of states

2t

0t

1t

analysis

spread

RMS error

ensemble mean

Represent initial uncertainty by ensemble of atmospheric flow states Flow-dependence:

Predictable states should have small ensemble spread Unpredictable states should have large ensemble spread

Ensemble spread should grow like RMS error

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Buizza et al., 2004

Systems

Under-dispersion of the ensemble systemUnder-dispersion of the ensemble system

-------------- spread around ensemble meanspread around ensemble mean

RMS error of ensemble meanRMS error of ensemble mean

The RMS error grows faster than the spread

Ensemble is under-dispersive

Ensemble forecast is over- confident

Under-dispersion is a form of model error

Forecast error = initial error + model error

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Manifestations of model errorManifestations of model error

In medium-range:Under-dispersion of ensemble system (Over-confidence)Can extreme weather events be captured?

On seasonal to climatic scales:Not enough internal variabilityTo what degree do detection and attribution studies for

climate change depend on a correct estimate of internal variability?

Underestimation of the frequency of blocking Tropical variability, e.g. MJO, wave propagation Systematic error in T, Precip, …

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Causes of model error : Unrepresented Causes of model error : Unrepresented processes in weather and climate modelsprocesses in weather and climate models

• Systematic versus random error

• physical parametrization delivers ensemble-mean or ‘most likely’ tendencies ?

• random model error can be associated with :

(i) statistical fluctuations in sub-grid (or filter-scale) transport processes (e.g. convective mass flux)

(ii) unrepresented statistical physical process e.g. ‘turbulent backscatter’

• different systematic errors associated with model framework (e.g. gridpoint vs spectral) and parametrization choices can be used to create an ensemble

forecast system (e.g. multi-model ensemble; Hadley Centre QUMP)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Kinetic energy spectra from aircraftKinetic energy spectra from aircraft

Nastrom and Gage, 1985

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Kinetic Energy spectrum in the ECMWF IFSKinetic Energy spectrum in the ECMWF IFS

Wavelength ~ 600 km Missing mesoscale

energy

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Representing UncertaintyRepresenting Uncertainty

within conventional parameterization schemes

Stochastic parameterizations (Buizza et al, 1999, Lin and Neelin, 2000)

Multi-parameterizations approaches (Houtekamer, 1996)

Multi-parameter approaches (e.g. Murphy et al,, 2004; Stainforth et al, 2004)

Multi-models (e.g. DEMETER, ENSEMBLES, TIGGE, Krishnamurti)

outside conventional parameterisation schemes

Nonlocal parameterizations, e.g., cellular automata pattern generator (Palmer, 1997, 2001)

Stochastic kinetic energy backscatter (Shutts and Palmer 2004, Shutts 2005; Bowler et al, 2009)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Stochastic parameterizations have the potential Stochastic parameterizations have the potential to reduce model errorto reduce model error

Weak noise

Multi-modal

Strong noise

Unimodal

Stochastic parameterizations can change the mean and variance of a PDF

Impacts variability of model (e.g. internal variability of the atmosphere)

Impacts systematic error (e.g. blocking, precipitation error)

Potential

PDF

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Spectral stochastic perturbed tendency scheme

(‘New Stochastic Physics’)Revised form of the scheme due to Buizza et al (1999)

use a spectral pattern generator based on triangularly-truncated spherical harmonic expansions to represent a global field ‘multiplier’

at any spatial point the multiplier has a mean of 1 and prescribed variance

the field has Gaussian horizontal auto-correlation function with an adjustable correlation scale (e.g. 500 km)

Each spectral component in the pattern evolves in time according to a first-order autoregressive process with prescribed decay time (e.g. 6 model steps)

model parametrization tendencies are multiplied by the pattern field(excluding boundary layer and stratosphere)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

New stochastic physics pattern generatorNew stochastic physics pattern generator

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Decrease in ensemble mean error Decrease in ensemble mean error

x

Ensemble members

x

Ensemble mean error

Analysisx

Ensemble mean

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Continuous Ranked Probability Skill Score

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

r.m.s. error of 850 hPa temperature in the tropics versus spread for the ensemble-mean

(Crosses are for r.m.s. error)

Under-dispersion

Spread increased with newStochastic physics

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Continuous Ranked Probability Skill Score

Spectral Backscatter SchemeSpectral Backscatter Scheme

Rationale: A fraction of the dissipated energy is scattered upscale and acts as streamfunction forcing for the resolved-scale flow (LES, CASBS: Shutts and Palmer 2004, Shutts 2005); New: spectral pattern generator

Total Dissipation rate from Total Dissipation rate from numerical dissipation, convection, numerical dissipation, convection, gravity/mountain wave drag.gravity/mountain wave drag.

Forcing pattern: temporal and Forcing pattern: temporal and spatial correlations prescribedspatial correlations prescribed

D F

* D F

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Spectral Backscatter Scheme (SPBS)Spectral Backscatter Scheme (SPBS)

Spectral pattern generator:

where

and fjm,n are the complex spectral amplitudes at step j and

are associated Legendre functions| |mnP

( * denotes the complex conjugate)

Rationale: A fraction of the dissipated energy is backscattered upscale and acts as streamfunction forcing for the resolved-scale flow ( Shutts and Palmer 2004, Shutts 2005, Berner et al (2009)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

11stst-order autoregressive process-order autoregressive processfor horizontal pattern generationfor horizontal pattern generation

(n) is a scale-dependent parameter that sets the decorrelation time Currently ~ 0.07 for all nand is chosen so that iss.

g(n) sets the amplitude of the random number noise rjm,n

based on coarse-graining calculations using a big-domain cloud-resolving model g(n) is (1+n)

where j is the step number

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Power spectrum of coarse-grained streamfunction forcing at Power spectrum of coarse-grained streamfunction forcing at z=11.5 km computed from a cloud-resolving modelz=11.5 km computed from a cloud-resolving model

k-1.54

Log(E)

Log(k)

g(n) ~ k-1.27

E~ n g(n)2

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Streamfunction forcingStreamfunction forcing

Streamfunction forcing

Backscatter ratio

Total KE dissipation rate Pattern generator

Dtot = numerical dissipation +

gravity/mountain wave drag dissipation + deep convective production of KE

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Smoothed total ‘dissipation rate’Smoothed total ‘dissipation rate’ D*tot

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Numerical dissipation RateNumerical dissipation Rate

where is the relative vorticity and K is the biharmonicdiffusion coefficient.

Dnum is augmented by a factor of 3 to account for thekinetic energy loss that occurs as a result of interpolationof winds to the departure point in the semi-Lagrangianadvection step

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Gravity wave/orographic dragGravity wave/orographic drag

22 /C dD M w

u and v increments from the orographic drag parametrizationmultiplied by u and v to give a KE increment i.e.

Deep convection KE production

Md is the mass detrainment rate; w is a mean convective updraught speed and is the density

gwdgwd gwd

u vD u v

t t

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Smoothed total ‘dissipation’ rateSmoothed total ‘dissipation’ rate

2

2tot num gwd c

fD D D D

Dtot is smoothed to T30 using a tapered spectral filter

Boundary layer dissipation is omitted on the assumptionthat turbulent eddies of scale < 1 km will not project sufficiently on quasi-balanced, meso->synoptic scale motions

The bracketed term multiplying Dc is the absolute vorticitynormalized by twice the Earth’s angular rotation rate. This represents the dependence of balanced flow production on background rotation.

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Impacts on probability skill scoresImpacts on probability skill scores

Continuous Ranked Probability Skill Score for temperature at 850 hPa (20-90 degrees N)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for temperature at 850 hPa (Tropics)temperature at 850 hPa (Tropics)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

ContinuousContinuous Ranked Probability Skill Score for Ranked Probability Skill Score for u at 850 hPa (20 – 90 degrees N)u at 850 hPa (20 – 90 degrees N)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for u at 850 hPa (Tropics)u at 850 hPa (Tropics)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

ContinuousContinuous Ranked Probability Skill Score for Ranked Probability Skill Score for u at 200 hPa (20 – 90 N)u at 200 hPa (20 – 90 N)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for u at 200 hPa (Tropics)u at 200 hPa (Tropics)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Continuous Ranked Probability Skill Score for Continuous Ranked Probability Skill Score for geopotential height at 850 hPa (20 – 90 degs N)geopotential height at 850 hPa (20 – 90 degs N)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Rms error of the ensemble mean versus spread about Rms error of the ensemble mean versus spread about the ensemble mean for T at 850 hPa (20-90 N)the ensemble mean for T at 850 hPa (20-90 N)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Rms error of the ensemble mean versus spread about Rms error of the ensemble mean versus spread about the ensemble mean of T at 850 hPa (tropics)the ensemble mean of T at 850 hPa (tropics)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Rms error of the ensemble mean versus spread about the Rms error of the ensemble mean versus spread about the ensemble mean of u at 200 hPa (20-90 N)ensemble mean of u at 200 hPa (20-90 N)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Rms error of the ensemble mean versus spread about Rms error of the ensemble mean versus spread about the ensemble mean of u at 200 hPa (tropics)the ensemble mean of u at 200 hPa (tropics)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Experimental Setup for Seasonal RunsExperimental Setup for Seasonal Runs

“Seasonal runs: Atmosphere only” Atmosphere only, observed SSTs 40 start dates between 1962 – 2001 (Nov 1) 5-month integrations One set of integrations with stochastic

backscatter, one without Model runs are compared to ERA40 reanalysis

(“truth”)

No StochasticBackscatterNo StochasticBackscatter Stochastic BackscatterStochastic Backscatter

Reduction of systematic error of z500 over Reduction of systematic error of z500 over North Pacific and North AtlanticNorth Pacific and North Atlantic

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Increase in occurrence of Atlantic and Increase in occurrence of Atlantic and Pacific blockingPacific blocking

ERA40 + confidence ERA40 + confidence intervalinterval

No StochasticBackscatterNo StochasticBackscatter

Stochastic BackscatterStochastic Backscatter

Wavenumber-Frequency SpectrumWavenumber-Frequency Spectrum Symmetric part, background removed Symmetric part, background removed

(after Wheeler and Kiladis, 1999)(after Wheeler and Kiladis, 1999)

No Stochastic BackscatterNo Stochastic BackscatterObservations (NOAA)Observations (NOAA)

Improvement in Wavenumber-Frequency Improvement in Wavenumber-Frequency SpectrumSpectrum

Stochastic BackscatterStochastic BackscatterObservations (NOAA)Observations (NOAA)

Backscatter scheme reduces erroneous westward propagating modes

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Coarse-graining as a method of computing Coarse-graining as a method of computing model errormodel error

Cloud-Resolving Model (CRM) approach

1) Spatially-average model fields and tendencies to a coarse grid2) Compute tendencies implied by the coarse-grained model fields3) Subtract the tendencies computed in 2) from the coarse-grained

CRM tendendies

Forecast model method

1. Run a very high resolution forecast model e.g. IFS at T12792. Coarse-grain the tendency fields to a lower resolution e.g. T1593. Run a forecast at the lower resolution and subtract tendency field early in the forecast from the tendency field computed in 2)

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Computing the streamfunction forcingComputing the streamfunction forcing

1) Run a T1279 forecast for 2 hours and compute the total vorticity tendency from increments of u and v.

2) Smooth to T159 and take the inverse Laplacian to obtain streamfunction tendency

3) Run a T159 forecast for 2 hours.

4) Repeat 1) and 2) without smoothing

5) Compute the difference in the two streamfunction forcing functions

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Vertical section of the difference in u between Vertical section of the difference in u between T1279 run and T159 run at t+8 hrsT1279 run and T159 run at t+8 hrs

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

Streamfunction forcing estimated by Streamfunction forcing estimated by coarse-graining approachcoarse-graining approach

ECMWFStochastic representations of model uncertainty: Glenn Shutts March 2009

SummarySummary

• Insufficient ensemble model spread indicates the need to account for the statistical aspects of model error

• The true nature of this model error is not fully understood !

• Statistical fluctuations ignored in conventional physical parametrization maybe included

• Energy backscatter from unresolved flow structures may perturb the balanced flow dynamics

• The coarse-graining methodology provides a method for calibrating/validating assumptions inherent in stochastic parametrization