ECMWF User Meeting 14-16 June 2006 The use of ECMWF ensemble and lagged deterministic forecasts for 3-30 day outlooks in Sweden 1.Monthly instead of seasonal.

ECMWF User Meeting 14-16 June 2006

The use of ECMWF ensemble and lagged deterministic forecasts for 3-30 day outlooks in Sweden

1. Monthly instead of seasonal forecasting2. The used of lagged forecasts (as a complement to the EPS)3. Problems with weighting together different forecast systems

For details see :http://www.ecmwf.int/newsevents/meetings/forecast_products_user/Presentations2006/index.htm

http://www.ecmwf.int/newsevents/meetings/








1.The seasonal forecasts

Not used, partly because the forecasts seem to repeat themselves


Warmerthan normal

2003 2004

2005 2006

The last four years’ ECMWF summer forecasts (issued in April)

Warmerthan normal

Warmerthan normal

Warmerthan normal

Colder than normal

Not warmerthan normal

Colder than normal

Colder than normal


2. The monthly forecast

Used and found skilful, but tendencies of “jumpiness” in transitional periods


Max +5 to 10

Max +10 to 15

”Jumpiness” experiencedat a specific location

850 hPa temperatureplume for Norrköping,southern Sweden

clim

clim


3. The 21 day forecast (+9 days) Using the last three days 21d forecasts enables us to inder the trends beyond day 10, even beyond day 15










10 d 20 d 30 d

normal

temperature

statistics

Last days ECMWF fcand EPS

Last days Control +21 d forecasts

Main method since summer 2003

ECMWF monthly forecast


4. The problem of verification










Level of useful

forecasts

Introduction of more

ECMWF data


What to do?Two ways to go:

1.Political (cover up, play illusionist tricks or change the norms)

2.Scientific (go to the roots of the problem)


ACC=98%Slope=0.8

“I just happen to have some fresh verifications here,

depicting the results during the first half of this year...”

Political trick: Selective sampling


Verifikation Månadsprognos Svealand Jan 2000 t.o.m Feb 2006

-6-5,5

-5-4,5

-4-3,5

-3-2,5

-2-1,5

-1-0,5

00,5

11,5

22,5

33,5

44,5

55,5

6

Jan-

00 Apr Jul

Okt

Jan-

01 Apr Jul

Okt

Jan-

02 Apr Jul

Okt

Jan-

03 Apr Jul

Okt

Jan-

04 Apr Jul

Okt

Jan-

05 Apr Jul

Okt

Jan-

06 Apr Jul

Okt

Månad

Te

mp

av

vik

els

e

Obs

Prognos

Anomaly correlation of monthly forecast for Stockholm (2 m temperature)

More ECMWF input

Verif

Prog

But it didn’t look that bad….


Scientific approach:The conventional verification disregarded three factors:

1. Variable range of variation between 2002 and 2003

2.More than one verification method should be used

3.Twelve forecasts per year is a too small sample










2003 Lower correlationSmaller errors

2002 Higher correlationLarger errors


Introduction of more

ECMWF datareduced

the errors!

Another verification method

RMSE

MABSE


Verifying two years at

a time

(Lagged) verification over 24 monthscompared to over 12 months

12 months

24 months


5. Swedish concerns about the quality of the centre’s EPS

1. Forecasters at SMHI and the Air Force do not find much use of the deterministic EPS compared to an elaborate use of the deterministic model

2. The scientists at SMHI and the MISU (Univ. Stockholm) are critical about the perturbations + (recently) the stochastic physics

3. My impression is not that the EPS is bad or has become worse, but has had problems to keep pace with the improvements of the deterministic model


511255

159

42

The size of the T42 EPS perturbations is very large

The picture depicts the status before 1 February 2006. Since then the resolution of the deterministic system has increased by 50%, but the EPS perturbations which remain at their 1995 level of T42


Before 2001 there was little quality difference between perturbed and non-perturbed forecasts, amounting beyond D+5 to an ACC

difference. Since then it has increased to 10-15%

?

Difference in ACC between the unperturbed Control and a randomly selected EPS member


1.Over spreading in during the first 24-48 hours made it difficult to use the EPS as BC for the HIRLAM

2.In cases of extreme or interesting events the signals often come 1-2 earlier in the T799 lagged system

3.In cases of consistent and skilful T799 performance the EPS keep the forecaster uncertain too long

For more details see presentation at the OD Workshop November 2005


The RMSE of individual EPS members The 2 m temperature forecasts for London Feb-April 2006

perturbed m

embers

1 day

EPS mean

”Lagged”

Unperturbed Control


Figure 2.1: Schematic image of the RMS error of the ensemble members, ensemble mean, and control forecast as a function of lead-time. The asymptotic predictability range is defined as the average difference between two randomly chosen atmospheric states. In a perfect ensemble system the RMS error of the ensemble members is a factor larger than the RMS error of the ensemble

mean.Courtesy, L. Bengtsson, MISU

climate

RMSE(pert member)= 1.414 (=sqrt2) RMSE (ensemble mean)

Perturbed member

Ensemble mean

Control

“Tim Palmer’sLaw”


6. Use of the last T799 runs forming lagged ensembles (work under development)


EPS Mean and “Lagged Mean” 24 March 00 UTC + 84h


Valid Sun 9 April 12 UTC

EPS Mean and lagged ECMWF T799 4 April 00 UTC + 132h


7. The public 6-10 day forecasts

Once a week, “four out of five forecasts verify”


6-10 day forecast presented on TV 26 January 2006



Epsogram forStockholm


8. When does it pay to weight together forecasts?


),( 212

1 EEcorrE

E

E1< E2









9. How should x and y, the weights, be calculated taking the forecast error correlation into account?


),(

),(

21212

2

21212

1

EEcorrEEE

EEcorrEEE

y

x

E1< E2









Certain combinations of forecasts will not yield an improved weighted mean

),(0 212

1 EEcorrE

Eforx

E1< E2


Hypothetical error correlations

50%

100%

0%D+0 D+15

T799 vs T399

T799(T399) vs UKMOor an arbitrary eps-member

If all three models have the sameerror magnitude and correlation then the weights are 33.3%

But if the errors of T799 and T399 are more correlated than the errors of T799 (T399) versus UKMO the UKMO should be weighted the most

Extension to three models??


10. Future challenges

Extending the monthly forecasts by including precipitation and provide forecasts separately form week1, week2 and week34 - and much more….

ECMWF User Meeting 14-16 June 2006 The use of ECMWF ensemble and lagged deterministic forecasts for 3-30 day outlooks in Sweden 1.Monthly instead of seasonal.

Documents

ecmwf user meeting

ecmwf data slide

ecmwf monthly forecast

normal slide

verif slide

problem slide

cos slide

days ecmwf fc