A.Montani; The COSMO-LEPS system. COSMO-LEPS: present status and plans Andrea Montani, C. Marsigli, T. Paccagnella ARPA-SIMC HydroMeteoClimate Regional.

A.Montani; The COSMO-LEPS system.

COSMO-LEPS:

present status and plans

Andrea Montani,

C. Marsigli, T. Paccagnella

ARPA-SIMC

HydroMeteoClimate Regional Service of Emilia-Romagna, Bologna,

Italy

COSMO General meetingSibiu, 2-5 September 2013


Outline

• Present status of COSMO-LEPS: about the operational verification, about the calibrated precipitation, about the convection schemes, about the clustering technique, about the future plans.


COSMO-LEPS suite @ ECMWF: present status

d-1d-1 dd d+5d+5d+1d+1 d+2d+2 d+4d+4d+3d+3

older EPSolder EPS

younger EPSyounger EPS

clustering clustering periodperiod

0000

1212

Cluster Analysis and RM identificationCluster Analysis and RM identification

4 variables4 variables

Z U V QZ U V Q

3 levels3 levels

500 700 850 hPa500 700 850 hPa

2 2 time time stepssteps

Cluster Analysis and RM identificationCluster Analysis and RM identification

European European areaarea

Complete Complete LinkageLinkage

16 Representative Members driving the 16

COSMO-model integrations (weighted according to the cluster

populations)

using either Tiedtke or Kain-Fristch convection scheme (members 1-8 T,

members 9-16 KF) +

perturbations in turbulence scheme and

in physical parameterisations

COSMO-LEPS

clustering area

• suite runs twice a day (00 and 12UTC) as a “time-critical application” managed by ARPA-SIMC;

• Δx ~ 7 km; 40 ML; fc+132h;• COSM0 v4.26 since Jan 2013;• computer time (50 million

BUs for 2013) provided by the ECMWF member states in COSMO.

COSMO-LEPS

Integration Domain


Main changes during the COSMO year

• 16 January 2013: COSMO upgrade: 4.21 4.26;

int2lm upgrade: 1.18 1.20.

• 17 January 2013: operational dissemination implemented for ARPA-Veneto.

• 22 January 2013, technical changes at ECMWF:

• change of ECMWF super-computer and of the user running the suite: itm zcl;

• introduction of a new “dissemination stream” for COSMO-LEPS: “ad-hoc” initial and boundary conditions do not have to be retrieved any more, but are prepared on a dedicated file system; product dissemination starts about 40 minutes earlier than before (at 9UTC and 21UTC).

• 7 May 2013: enriched test dissemination implemented for HNMS.

• 13 May 2013: in the framework of GEOWOW research project, COSMO-LEPS was the first system to populate TIGGE-LAM archive at ECMWF (high-priority parameters in grib2 format).

• 25 June 2013: tests with Fieldextra 11.1.0 started.


Outline

• Present status of COSMO-LEPS: about the operational verification,


– SYNOP on the GTS

Time-series verification of COSMO-LEPS

Main features:

variable: 12h cumulated precip (18-06, 06-18

UTC);

period : from Dec 2002 to Jul 2013;

region: 43-50N, 2-18E (MAP D-PHASEPHASE

area);

method: nearest grid point; no-weighted fcst;

obs: synop reports (about 470

stations/day);

fcst ranges: 6-18h, 18-30h, …, 102-114h, 114-126h;

thresholds: 1, 5, 10, 15, 25, 50 mm/12h;

system: COSMO-LEPS;

scores: ROC area, BSS, RPSS, Outliers, …

both monthly and seasonal scores were computed


Brier Skill Score: time series and seasonal scores

BSS is written as 1-BS/BSref. Sample climate is the reference system. Useful forecast systems if BSS > 0.

BS measures the mean squared difference between forecast and observation in probability space.

Performance of the system assessed as time series (6-month running mean) and for the last 4 springs (MAM).

Seasonal scores for a fixed event (“12h precip > 10mm”): need to take into account the different statistics for each season (last MAM was the wettest).

Best performance for the last spring, with BSS positive for all forecast ranges.

Time series: high month-to-month variability, but a positive trend can be noticed.

Good scores in 2012 and 2013; BSS positive for all thresholds since April 2009; fewer and fewer problems with high thresholds.


Outline

• Present status of COSMO-LEPS: about operational verification (time-series scores show improvements),

about the calibrated precipitation;


about calibrated precipitation

• For each COSMO-LEPS member, calibrated precipitation is operationally generated over Germany, Switzerland and Emilia-Romagna; the calibration technique is based on CDF-based corrections, making use of COSMO-LEPS reforecast.

• For MAM2013, inter-comparison between raw and calibrated 24h TP forecast.Main features:

variable: 24h cumulated precip (06-06 UTC);

period : DJF 2012-13 and MAM 2013;

region: Germany, Switzerland, Emilia-

Romagna;

method: nearest grid point; no-weighted fcst;

obs: synop reports (about 300 stations/day);

fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h;

thresholds: 1, 5, 10, 15, 25, 50 mm/12h;

system: opecleps and Calibcleps;

scores: ROC area, BSS, RPSS, Outliers, RelDiag, …


opecleps vs Calibclepsfc 42-66h; 10mm/24h


Outline


about calibration (positive impact, especially over Emilia-Romagna);

about convection schemes; about the clustering technique;

about the future plans.


• Adapt COSMO-LEPS suite to ECWMF forthcoming upgrades:– increase of vertical resolution in ECMWF-EPS: 62 91;

– change of Member-State server: ecaccess ecgb;

– change of super-computer: IBM Cray.

• Keep an eye (possibly, two) to the performance of ECMWF EPS.

• Carry on study about the clustering methodology.

about the future plans

• Support verification with Versus.

• Analysis of the performance of COSMO-HYBEPS (COSMO-LEPS + 2-3

COSMO runs nested on IFS/GME/GFS): tests ongoing.

• Increase of COSMO-LEPS vertical resolution (40 50ML): tests start in

October.

• Use of high-resolution ECMWF-EPS boundaries (LAMEPS_BC project): tests

start by the end of 2013.


Thank you for the attention !


Extra slides on configuration

• European Conference on Applications of Meteorology / EMS annual meeting

• 09 – 13 September 2013, Reading (UK)

• Session NWP4 (on 13 September): Probabilistic and ensemble forecasting at short and and medium-

range

• http://www.ems2013.net/home.html


Dim

2

Initial conditions Dim 1

Dim

2

Possible evolution scenarios

Dim 1 Initial conditions

ensemble size reduction

Cluster members chosen as representative members (RMs)

LAM integrations driven byRMs

LAM scenario

LAM scenario

LAM scenario

COSMO-LEPS methodologyCOSMO-LEPS methodology


COSMO-HYBrid Ensemble Prediction System

From the results of CONSENS PP, come to a synthesis with the different ensemble systems / strategies, considering scientific, implementation, solidity aspects.

Generate 20-member hybrid ensemble (COSMO-HYBEPS) , where:

a) 16 members comes from COSMO-LEPS,

b) 1 member is nested on IFS (uses Tiedtke scheme),

c) 1 member is nested on IFS (uses Kain-Fritsch scheme),

d) 1 member is nested on GME,

e) 1 member is nested on GFS.

already existing taken from CONSENS.

All members have Δx ~ 7 km; 40 ML; fc+132h;

Study performance of different members’ combinations with the same ensemble size.

“20-members esuite” implemented on 7/9/2012;

will be run up to the end of the year


COSMO-LEPS (developed at ARPA-SIM)

• What is it?It is a Limited-area Ensemble Prediction System (LEPS),

based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland).

• Why?It was developed to combine the advantages of global-

model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of severe and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …)

generation of COSMO-LEPS to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of

severe weather events.


Operational set-up

Core products: 16 perturbed COSMO-model runs (ICs and 3-

hourly BCs from 16 EPS members) to generate, “via weights”, probabilistic output: start at 12UTC; t = 132h;

Additional products: 1 deterministic run (ICs and 3-hourly BCs from the

high-resolution deterministic ECMWF forecast) to “join” deterministic and probabilistic approaches: start at 12UTC; t = 132h;

1 hindcast (or proxy) run (ICs and 3-hourly BCs from ECMWF analyses) to “downscale” ECMWF information: start at 00UTC; t = 36h.


Types of perturbations

As for types and values, the results from CSPERT experimentation

were followed (* denotes default values for COSMO v4.26 ):

•convection_scheme: Tiedtke* (members 1-8), Kain-Fritsch (members 9-16),

•tur_len (either 150, or 500*, or 1000),

•pat_len (either 500*, or 2000),

•crsmin (either 50, or 150*, or 200),

•rat_sea (either 1, or 20*, or 40),

•rlam_heat (either 0.1, or 1*, or 5),

•mu_rain : either 0.5* (with rain_n0_factor =0.1) or 0 (with rain_n0_factor =1.0),

•cloud_num (either 5x10^8* or 5x10^7).

• convection scheme: T=Tiedtke KF=Kain-Fritsch;

• tur_len: maximal turbulent length scale (default 500m); this parameter is used mainly in the

calculation of the characteristic length scale for vertical mixing and thus into the calculation of

the vertical transport momentum coefficient;

• pat_len: length scale of thermal surface patterns (default 500m); this parameter is mainly

used in the calculation of the large-scale part of the equation addressing the heat flux

parameterisation; horizontal length;

• rlam_heat: scaling factor of the laminar layer depth (default 1); it defines the layer with non-

turbulent characteristics (molecular diffusion effects only);

• rat_sea: ratio of laminar scaling factors for heat over sea (default 20);

• crsmin: minimal stomata resistance (default 150);

• Cloud_num: Cloud droplet number concentration;

• Mu_rain: Exponent of the raindrop size distribution;

•( gscp: Switch on/off of the graupel scheme).


Main results

Time-series verificationECMWF EPS changed substantially in the last years (more and more weight to EDA-

based perturbations) and it is hard to disentangle improvements related to COSMO-

LEPS upgrades from those due to better boundaries; nevertheless:– high values of BSS and ROC area for the probabilistic prediction of 12-h precipitation for

autumn 2011;

– poor performance in the first months of 2012, then recovery. Need to investigate what

happened.

Case-study verificationConsistent signal for different forecast ranges of a high-impact weather event for the

snowfalls of February 2012.


Extra slides on verification


Time series of ROC area (6-month running mean)

Area under the curve in the HIT rate vs FAR diagram; the higher, the better … Valuable forecast systems have ROC area values > 0.6.

Highest scores in the 2nd part of 2011 and, for the highest threshold, in 2013. Positive trend through the years can be noticed.

Drier seasons during 2011 and 2012 with few heavy-precipitation events: limited significance of the results for the 15mm threshold.

Limited loss of predictability with increasing forecast range (not shown).


Seasonal scores of ROC and BSS: last 4 springs

Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges.

Valuable forecast systems have ROC area values > 0.6 and BSS > 0.

Need to take into account the different statistics for each season (MAM 2011 was the driest).

Best performance for the spring 2011 and 2013, but less marked diurnal cycle in 2013.

Spring 2013: BSS is positive for all forecast ranges

Similar results for the other thresholds (not shown).


Outliers: time series + ………seas scores (DJF)?

How many times the analysis is out of the forecast interval spanned by the ensemble members. … the lower the better … Performance of the system assessed as time series and for the last 4 winters.

Evident seasonal cycle (more outliers in winter).

Overall reduction of outliers in the years up to 2007; then, again in 2009 and 2010, but later.

Need to take into account the different statistics for each season.

In the short range, best results for winter 2010-2011.

For longer ranges, the performance of the system is “stable”.

Outliers before 10% from day 3 onwards.


Seasonal scores of BSS: ……last 4 winters

Fixed event (“12h precip > 10mm”): consider the performance of the system for increasing forecast ranges.

Fixed forecast range (fc 30-42h): consider the performance of the system for increasing thresholds. Need to take into account the different statistics for each season (last DJF was the driest).

Fixed event: best performance for the last two winters (ECMWF EPS had a record performance for winter 2009-2010): BSS positive for all forecast ranges.

Fixed forecast range: similar results as before.

Similar results for longer forecast ranges and for higher thresholds.


Ranked Probability Skill Score: time series + …….. seasonal scores (MAM)

A sort of BSS “cumulated” over all thresholds. RPSS is written as 1-RPS/RPSref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation.

Useful forecast systems for RPSS > 0. Performance of the system assessed as time series and for the last 4 springs (MAM).

the increase of the COSMO-LEPS skill is detectable for 3 out of 4 forecast ranges along the years, BUT

low skill in the first months of 2012 (the problem comes from MAM), then recovery. Best results for MAM 2011; quick decrease of RPSS with fcst range for MAM 2012.


Bias and rmse of T2M Ensemble Mean

Consider bias (the closer to zero, the better) and rmse (the lower the better).

Bias closer to zero (0.5 °C of decrease) and lower rmse for the 7-km suite. The improvement is not “massive”, but detectable for all forecast ranges, especially for day-time

verification. The signal is stable (similar scores for 1-month or 3-month verification). Need to correct T2M forecasts with height to assess the impact more clearly.


Overestimation of Td2m and soil moisture (1)

Verification period: MAM07 and MAM08.

Obs: synop reports (about 470 stations x day).

Region: 43-50N, 2-18E (MAP D-PHASE area).

Larger bias and larger rmse in MAM08 rather than in MAM07 for COSMO-LEPS deterministic run (in 2007, no multi-layer soil model).


Extra slides on LAMEPS-BC


Test data for LAMEPS Boundary Conditions


Outline

• Introduction:

migration to the 7-km system.

COSMO-LEPS 10 km (old)

COSMO-LEPS 7 km (new)


COSMO-LEPS

16-MEMBER EPS

51-MEMBER EPS

tp > 1mm/24h

tp > 5mm/24h

Average values (boxes 0.5 x 0.5)MAM06

As regards AVERAGE precipitation above these two threshols, the 3 systems have similar performance.


opecleps vs Calibcleps

fc 42-66h; 1mm/24h

fc 42-66h; 10mm/24h


Outline


about calibration (positive impact, especially over Emilia-Romagna!);

about convection schemes,

members 1-8 use Tiedtke convection scheme (8TD),

members 9-16 use Kain-Fritsch (8KF).

MAM 2013: compare cleps16, 8TD, 8KF.


about the convection scheme

BSS, tp > 1mm

ROC, tp > 10mm

ROC, tp > 1mm

BSS, tp > 10mm

• As expected, best performance by the full ensemble (cleps16).• Tiedtke-members better than Kain-Fritsch members, but NOT for

all scores.

___ cleps16___ 8TD ___ 8KF


about the clustering technique

AIM: provide limited-area ensembles (either convection-parameterised or convection-permitting) with the best set of boundary conditions.

Study different types of clustering analyses (over the same area, grid):• ope: use two 12-hourly-lagged EPS, steps=96/120 (108/132) for the younger

(older) EPS, variables=Z/U/V/Q, levels=500/700/850 hPa;• rnd0: use the younger EPS and always select members 0 to 15 (0 denotes the

control member)• rnd1 : like rnd0, but select members 1 to 16.• ...

Analyse properties (e.g. spread, skill) of the 16-member global ensembles for several upper-air variables

Outcome: modifications to the number of clusters / number of EPS considered / clustering intervals.


Test modifications of clustering methodology

• Consider distances between ECMWF EPS members according to “COSMO-LEPS metric” (Z, U, V, Q in the mid-lower troposphere over the clustering domain).

• Look at distances between pairs of ECMWF EPS members; to what extent these distances grow with forecast range, using “COSMO-LEPS metric”?

• Study a number of seasons. Outcome: modifications to the number of clusters / number of EPS

considered / clustering intervals.


Extra slides on COSMO-S14-EPS


Important ingredients (from 1st and 2nd FROST meetings)

1. Provide reasonable “numbers”. addressed

2. Develop experience with probabilities. ?

3. Feedback on the top-priority products. being addressed

4. Snow analysis. ?

5. Soil-field initialisation. addressed

6. High-res obs to assess the quality of the system. being addressed

7. Computer time. addressed

8. Timeliness in product delivery. addressed

9. ......

anything to add/remove?


FROST-2014 vs SOCHMEL

1) Introduction to FROST-2014:

a) What is it?

b) What has to do with COSMO?

2) COSMO ensemble activities within FROST-2014:

a) introduction to SOCHMEL (the SOCHi-targeted Mesoscale EnsembLe system)

b) methodology;

c) phases of development;

d) planned activity.

3) Final remarks.

A.Montani; The COSMO-LEPS system. COSMO-LEPS: present status and plans Andrea Montani, C. Marsigli, T. Paccagnella ARPA-SIMC HydroMeteoClimate Regional.

Documents

cosmoleps system

cosmoleps suite

cosmo upgrade

cosmo year16

cosmoleps integration

outlinepresent status

cosmomodel integrations

reference system