A.Montani; The COSMO-LEPS system. Present status and future plans of the COSMO-LEPS system Andrea Montani, D. Cesari, T. Diomede, C. Marsigli, T. Paccagnella.

A.Montani; The COSMO-LEPS system.

Present status and future plans of the

COSMO-LEPS system

Andrea Montani,

D. Cesari, T. Diomede, C. Marsigli, T. Paccagnella

ARPA-SIMC

HydroMeteoClimate Regional Service of Emilia-Romagna, Bologna,

Italy

COSMO Genreal meetingMoscow


Outline

• Introduction: present status of COSMO-LEPS; migration to the 7-km system.

• Performance of the system: time-series verification of COSMO-LEPS using SYNOP;

• Present activity:COSMO-LEPS for TIGGE-LAM; multi-model clustering.

• Future plans.

Many regards from ECMWF Seminars and greetings for a good meeting to all WG4 members!!!


COSMO-LEPS (developed at ARPA-SIMC)

• What is it?It is a Limited-area Ensemble Prediction System (LEPS),

based on COSMO-model and implemented within COSMO (COnsortium for Small-scale Modelling, including Germany, Greece, Italy, Poland, Romania, Russia, Switzerland).

• Why?It was developed to combine the advantages of global-

model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of high-impact and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)

generation of COSMO-LEPS to improve the forecast of high-

impact weather in the short and early-medium range (up

to fc+132h)


Dim 2

Initial conditions Dim 1 Dim 2

Possible evolution scenarios

Dim 1 Initial conditions

ensemble size reduction

Cluster members chosen as representative members (RMs)

LAM integrations driven byRMs

LAM scenario

LAM scenario

LAM scenario

COSMO-LEPS methodologyCOSMO-LEPS methodology


COSMO-LEPS suite @ ECMWF: present status

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

older EPSolder EPS

younger EPSyounger EPS

clustering clustering periodperiod

0000

1212

Cluster Analysis and RM identificationCluster Analysis and RM identification

4 variables4 variables

Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa

2 2 time time stepssteps


European European areaarea

Complete Complete LinkageLinkage

1616 Representative Representative Members driving the Members driving the 1616

COSMO-model COSMO-model integrations (weighted integrations (weighted according to the cluster according to the cluster

populations)populations)

employing either employing either Tiedtke or Kain-Fristch Tiedtke or Kain-Fristch

convection scheme convection scheme (randomly choosen) (randomly choosen)

++perturbations in perturbations in

turbulence scheme and turbulence scheme and in physical in physical

parameterisationsparameterisations

COSMO-LEPS

clustering area

• suite runs as a “time-critical application” managed by ARPA-SIMC;

• Δx ~ 7 km; 40 ML; fc+132h;• COSM0 v4.12 since Jul09;• computer time (14.0 million

BU for 2010) provided by the COSMO partners which are ECMWF member states.

COSMO-LEPS

Integration Domain


Upgrades during the “COSMO year”

• 16 November 2009: – archive of large-scale precipitation (62.2) for both COSMO-LEPS members and LM-DET;

• 30 November 2009:– implementation of COSMO-LEPS at 7 km (new domain, new perturbations (in types and values), lsso=.true., lforest=.true. );– upgrade of COSMO to model version 4.8;

• 12 July 2010:– upgrade of INT2LM to model version 1.12;– upgrade of COSMO to model version 4.12.


Outline

• Introduction:

migration to the 7-km system.

COSMO-LEPS 10 km (old)

COSMO-LEPS 7 km (new)


Old systemx = 10 kmz = 40 MLt = 90 sngp = 306x258x40 = 3.157.920fcst range = 132hinitial conditions: interpolated

from EPS members

perturbations: type of convection scheme; tur_len; pat_len.

Implementation of COSMO-LEPS at 7 km

New system (COSMO-LEPS_7)

x = 7 kmz = 40 MLt = 60 sngp = 511x415x40 =

8.482.600fcst range = 132hinitial conditions: interpolated

from EPS members merged with surface and soil-layer fields produced at DWD for COSMO-EU

perturbations: type of convection scheme; tur_len; pat_len; crsmin; rat_sea; rlam_heat.

COSMO-LEPS_7 tested from May to November 2009 (no merging yet)

- to improve the forecast of near-surface parameters- to keep an “advantage” vs ECMWF EPS (running at F 25

km)

Why?


COSMO-LEPS_10 (Old) vs COSMO-LEPS_7 (New)

• Deterministic verification of T2M ensemble mean

Variable: 2-metre temperature. Period: from June to November 2009. Forecast ranges: fc+6h, fc+12h, …, fc+132h. Scores: root-mean-square error, bias.

• Probabilistic verification of 12-hour cumulated precipitation

Variable:12h cumulated precipitation (18-06, 06-18 UTC). Period: from June to November 2009. Forecast ranges: fc 6-18h, fc 18-30h, …, fc 114-126h. Scores: ROC area, BSS, RPSS, Outliers. Thresholds: 1, 5, 10, 15, 25, 50 mm/12h.

Observations: SYNOP reports over either MAP D-PHASE region (450 reports/day) or the FULL-DOMAIN (1400 reports/day).

Method: nearest grid point; no-weighted fcst.


Bias and rmse of T2M Ensemble Mean

Consider bias and rmse for 3 months (24/5 24/8/2009) over MAPDOM (∼ 450 synop). T2m forecasts are corrected with height.

Bias closer to zero and lower rmse for the 7-km suite. Improvement is not “massive”, but detectable for all forecast ranges, especially for day-time

verification. Similar results over MAPDOM and over FULLDOM (not shown). The signal is stable (same scores also for 6-month verification).

---- OLD rmse (10 km)---- NEW rmse (7 km)—— OLD bias (10 km)—— NEW bias (7 km)


ROC area, BSS for 12-hour tp

Consider the event “10 mm of precipitation in 12 hours” for ROC area and BSS (from Jun to Nov 2009).

Better results for the 7-km suite, both for ROC area and BSS values. The impact is more evident for BSS. Reduction of 12-h cycle in 7-km runs. The improvement is detectable for all forecast ranges and for both MAPDOM and FULLDOM.

ROC area (both FULLDOM and MAPDOM)

BSS (both FULLDOM and MAPDOM)


RPSS, OUTL for 12-hour tp

Consider scores not-dependent on one single threshold (from Jun to Nov 2009).

Better results for the 7-km suite in terms of RPSS. The improvement is detectable for all forecast ranges and for both MAPDOM and FULLDOM. The Percentage of outliers is only slightly reduced in the 7-km suite (solid lines), but the gap is

very small. The 7-km system has a positive impact in the reduction of the outliers BELOW THE MINIMUM for

the MAPDOM (the same holds for FULLDOM, although not shown).

% of outliers (only MAPDOM)

RPSS (both FULLDOM and MAPDOM)

COSMO-LEPS_7 implemented operationally on 1 December 2009


Outline

• Performance of the system: time-series verification of COSMO-LEPS using SYNOP;


– SYNOP on the GTS

Time-series verification of COSMO-LEPS

Main features:

variable: 12h cumulated precip (18-06, 06-18

UTC);

period : from Dec 2002 to Jul 2010;

region: 43-50N, 2-18E (MAP D-PHASEPHASE

area);

method: nearest grid point; no-weighted fcst;

obs: synop reports (about 470

stations/day);

fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h;

thresholds: 1, 5, 10, 15, 25, 50 mm/12h;

system: COSMO-LEPS;

scores: ROC area, BSS, RPSS, Outliers, …

both monthly and seasonal scores were computed


Time series of ROC area Area under the curve in the HIT rate vs FAR diagram; the higher, the better … Valuable forecast systems have ROC area values > 0.6.

Improvement of skill detectable for all thresholds along the years.

Poor performance of the system in Spring and Summer 2006 (both particularly dry), despite system upgrades.

Best performance in 2007 during DOP (D-PHASE Operation Period).

fc 30-42h: ROC area above 0.8 since mid-2007 and good scores in 2010.

fc 78-90h: ROC area ALSO above 0.8 in the last 10 months.


Seasonal scores of ROC area Performance of the system assessed for the last 4 winters (DJF) and the last 5 springs

(MAM). Consider the “event” 10 mm/12h; valuable forecast systems have ROC area values > 0.6. Need to take into

account the different statistics for each season (e.g. MAM 2006 more rainy than the others).

Best performance for the last winter (with COSMO-LEPS at 7 km), for all forecast ranges.

As for spring, no particular impact of the new system. Scores better than in MAM 2009.

Faster decay of system performance in MAM.


Outliers: time series + seasonal scores How many times the analysis is out of the forecast interval spanned by the ensemble members. … the lower the better … Performance of the system assessed as time series and for the last 4 winters.

Evident seasonal cycle (more outliers in winter).

Overall reduction of outliers in the years up to 2007; then, again in 2009 and 2010.

Need to take into account the different statistics for each season.

Best results for winter 2007-2008, with good performance of the new 7-km system.

Outliers before 10% from day 2 onwards.


Time series of Brier Skill Score BSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if

BSS > 0. BS measures the mean squared difference between forecast and observation in probability

space. BS equivalent to MSE for deterministic forecast. The improvement of

model performance is detectable for all thresholds along the years.

Fewer and fewer problems with high thresholds for shorter ranges.

Very good scores in 2010!!!.

fc 30-42h: BSS positive for all thresholds since April 2009.

fc 78-90h: good trend in 2010.


Ranked Probability Skill Score: time series + seasonal scores

A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.

Useful forecast systems, if RPSS > 0. Performance of the system assessed as time series and for the last 4 winters (DJF).

the improvement of the system performance is detectable for all forecast ranges along the years;

RPSS positive for all forecast ranges since May 2008; encouraging trend in the last year. Very good results for DJF 2009-2010 with COSMO-LEPS at 7 km!


Outline

• Present activity:COSMO-LEPS for TIGGE-LAM;


Products:• “high-priority” parameters (tp, lsp, t2m, td2m, u10, v10, gust10,

mslp, orog, lsm) operationally generated for each ensemble member, from fc+0h to fc+132h every 3h, in GRIB2 format;

• produced at ECMWF;• archived at ARPA-SIMC on the “native” rotated grid (0.0625 x

0.0625).

COSMO-LEPS for TIGGE-LAM

grib2 (TIGGE-LAM regular grid)

grib1 (original rotated grid)


Main results

Time-series verification scores.• It is difficult to disentangle improvements related to COSMO-LEPS upgrades

from those due to better EPS boundaries; nevertheless, positive trends can be identified: increase in BSS and ROC area scores reduction in outliers percentages; system upgrades of Dec 2007 brought small but positive impact; the increase in horizontal resolution had a clear positive impact last winter (also ECMWF

EPS did well anyway …).

Implementation of COSMO-LEPS_7km.• The new system was tested in parallel suite for 6 months:

higher BSS and ROC area values for the probabilistic prediction of 12-h precipitation with respect to the operational one,

lower T2M errors of the ensemble mean, positive impact of the introduction of the new perturbations.

• COSMO-LEPS_7km was implemented on 1 December 2009.


• COSMO-LEPS_7km:

– use the soil moisture analysis fields provided by DWD;

– save COSMO-LEPS output files on model levels (up to fc+48h) for further downscaling;

– test modifications of clustering methodologies:

• always select control runs by ECMWF EPS;

• consider shorter forecast ranges for clustering intervals (48-72h, 72-96h);

– follow the outcome of ECMWF TAC-subgroup on BC project possible modifications of the COSMO-LEPS suite;

• COSMO-LEPS for TIGGE-LAM:

– develop coding, post-processing and archiving of COSMO-LEPS output files in GRIB2 format (test Fieldextra);

– assistance to users.

Future plans (1)


Future plans (2)

• Support calibration and verification.

• Carry on collaboration within research project (e.g. SAFEWIND, IMPRINTS).

ECMWF Seminars 2011:

tentative dates are 12-15 September 2011 …..

No more overlap with COSMO Meeting, please!


Thank you !

European Conference on Applied Climatology / EMS annual meeting

13 – 17 September 2010, Zurich (Switzerland)

Session NWP2: Ensemble forecasting

http://meetings.copernicus.org/ems2010


Operational set-up

Core products: 16 perturbed COSMO-model runs (ICs and 3-

hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h;

Additional products: 1 deterministic run (ICs and 3-hourly BCs from the

high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h;

1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h.


Score dependence on the domain size (1)

Verification of COSMO-LEPS against synop reports over the MAP D-PHASE area (~ 470 stations; MAPDOM) and the full domain (~ 1500 stations; fulldom):

different statistics of the verification samples; up to now, performance of the system over the 2 domains assessed only for 6

months (March-August 2007). difficult to draw general conclusions


Score dependence on the domain size (2)

RPSS

RPSS score… the higher the better… (and positive).

ROC area… the higher the better… (and above 0.6). Smoother transitions from month to month in “fulldom” scores.

Slightly better performance of COSMO-LEPS over the MAPDOM, but the signal varies from month to month.

Higher predictability with orographic forcing?

Need to check individual regions and/or to stratify for type of stations.

OUTL

ROC

Outliers percentage … the lower the better.


Bias and rmse of T2M Ensemble Mean

Consider bias (the closer to zero, the better) and rmse (the lower the better).

Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite. The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time

verification. The signal is stable (similar scores for 1-month or 3-month verification). Need to correct T2M forecasts with height to assess the impact more clearly.


The COSMO-LEPS suite @ ECMWFNovember 2002 – May 2004

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

oldest EPSoldest EPS

1212

middle EPSmiddle EPS

oldest EPSoldest EPS

youngest EPSyoungest EPS


0000

1212



Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa





5 Representative 5 Representative MembersMembers

Driving the 5Driving the 5COSMO-model COSMO-model integrationsintegrations

COSMO-LEPS

Integration Domain

COSMO-LEPS

clustering area


The COSMO-LEPS suite @ ECMWFJune 2004 – January 2006

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

middle EPSmiddle EPS

youngest EPSyoungest EPS


0000

1212



Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa





COSMO-LEPS

Integration Domain

10 10 Representative Representative

MembersMembersdriving the 10driving the 10COSMO-model COSMO-model integrationsintegrations

employing either employing either Tiedtke or Kain-Tiedtke or Kain-Fristch scheme Fristch scheme

randomly randomly choosenchoosen

COSMO-LEPS

clustering area

• Suite running in real time at ECMWF managed by ARPA-SIM;

• Δx ~ 10 km• Fc length: 120h


COSMO-LEPS (developed at ARPA-SIM)

• What is it?It is a Limited-area Ensemble Prediction System

(LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland).

• Why?Because the horizontal resolution of global-model

ensemble systems is limited by computer time constraints and does not allow a detailed description of mesoscale and orographic-related processes.

The forecast of heavy precipitation events can still be inaccurate (in terms of both locations and intensity) after the short range.


COSMO-LEPS project

combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of intense and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)

generation of COSMO-LEPS in order to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of the so-

called “severe weather events”.


Semi-diurnal cycle in COSMO-LEPS scores

BSS score … the higher the better … Performance of the system assessed for 5 different Summers (JJA).

BSS Evident 12-hour cycle in

BSS scores (the same holds for RPSS, while less evident for ROC area scores).

Better performance of the system for “night-time” precipitation, that is for rainfall predicted between 18Z and 6Z (ranges 30-42h, 54-66h, …).

The amplitude of the cycle is somewhat reduced throughout the years and with increasing forecast range.

The bad performance in Summer 2006 is confirmed.

A.Montani; The COSMO-LEPS system. Present status and future plans of the COSMO-LEPS system Andrea Montani, D. Cesari, T. Diomede, C. Marsigli, T. Paccagnella.

Documents

cosmoleps members

old cosmoleps

new system cosmoleps

upgrade of cosmo

implementation of cosmo

cosmo year16

cosmo consortium

generation of cosmo