Climate Forecasting Unit Unreliable climate models overestimate attributable risk of extreme events Omar Bellprat and Francisco Doblas-Reyes (IC3, Barcelona) Our Common Future Conference, Paris, 2015
Climate Forecasting Unit
Unreliable climate models
overestimate attributable risk of
extreme events
Omar Bellprat and Francisco Doblas-Reyes (IC3, Barcelona)
Our Common Future Conference, Paris, 2015
Climate Forecasting Unit
An event attribution
Variable
Pro
ba
bili
ty
A model: Variability
Mean
Observations
A hindcast for the event
FAR=1-
P ANT
P NAT
No climate change
With climate change
Climate Forecasting Unit
An event attribution
Extreme event
Pro
ba
bili
ty
A model: Variability
Mean
A different hindcast
Observations
FAR=1-
P ANT
P NAT
No climate change
With climate change
Climate Forecasting Unit
An event attribution
Extreme event
Pro
ba
bili
ty
A model: Variability
Mean
Yet a different one
Observations
FAR=1-
P ANT
P NAT
No climate change
With climate change
Climate Forecasting Unit
An event attribution
Extreme event
Pro
ba
bili
ty
A model: Variability
Trend
Mean
Observations
No climate change
With climate change
How can we put trust into ? P
ANT P
NAT
Climate Forecasting Unit
Forecast reliability
In cases when above normal Temperature is predicted with probability
80% the frequency is indeed close to 80%.
Weisheimer and Palmer (2014). See also SPECS Fact-sheet: Climate forecast reliability
Does reliability matter for event attribution?
We don’t know, but assume.
Long hindcasts required
Large ensemble sizes
Few extreme events in the past
Climate Forecasting Unit
A statistical approach
= + + + Predict-
ability Trend
Hindcast
error
En
sem
ble
Perfect climatology: Variability Mean
Generate very long hindcasts with multi-thousand members
Hindcast
Attribution
Weigel et al. (2010)
Small, 10 % predictable
Climate Forecasting Unit
Variable reliability
Reliability can be varied at any level, 0=no reliability, 1=perfect
Climate Forecasting Unit
Attribution of extreme events Attributalbe risk increases with low reliability
Climate Forecasting Unit
Why does that happen?
Ratio of probabilities
is not stable for varying
hindcast spread
Same behavior if:
• Model quantile used
instead of threshold
• Change in the higher
moments included
• Non-Gaussian tails are
considered
Climate Forecasting Unit
Does that play a role? Becomes important if hindcasts get marginally useful
Reliability of ECMWF System 4 for a) Dry Winter b) Wet Winter c) Dry Summer b) Wet Summer
Weissheimer and Palmer (2014)
Climate Forecasting Unit
What is the role of predictability?
= + + + Predict-
ability Trend
Forecast
error
En
sem
ble
Perfect climatology: Variability Mean
Hindcast
Event attribution carried out with models with and without predictability
Climate Forecasting Unit
Predicting heat extremes
Soil moisture helps to predict the heat wave 2010, but not much in 2003
Climate Forecasting Unit
Conclusions
• Forecast reliability describes the trust we can put into
probability estimates for the occurrence of extremes
• Attributable risk increases in unreliable probability estimates
because ratio of probabilities is not stable
• Reliability matters and should become a standard
assessment in attribution studies (also when using GCMs)
Climate Forecasting Unit
Alternative measure of reliability
Climate Forecasting Unit
Varying assumptions
Climate Forecasting Unit
Hindcast error
True uncertainty Hindcast
Hindcast error
Overconfidence
Perfect reliability if model spread samples model error
Climate Forecasting Unit
Observations and model
𝐹𝑡 = 𝛼𝑥′𝑡 + 𝜖𝛽 + 𝑠𝑡 + (𝜖1, . . . , 𝜖𝑀)
𝑥𝑡 = 𝑥’𝑡 + 𝑠𝑡 Observations
Model
𝜎𝐸2 =1
𝑇 𝑥𝑡 − 𝐹𝑡
2𝑇
𝑡=1= (1 − 𝛼)
2+ 𝛽2 Model error
Model spread 𝜎𝑀 = 1 − 𝛼² − 𝛽²
Climate Forecasting Unit
Model reliability
𝜎𝑀2 = 𝜎𝐸
2 → (1 − 𝛼) 2+𝛽𝑟2 = 1 − 𝛼2 − 𝛽𝑟
2
𝛽𝑟 = 𝛼 − 𝛼2
A reliable model must sample its hindcast error
Perfect reliability is a function of the predictability