29th EWGLAM Meeting HIRLAM-A Verification Xiaohua Yang with contributions from Kees Kok, Sami Niemela, Sander Tijm, Bent Sass, Niels W. Nilsen, Flemming.

29th EWGLAM Meeting

HIRLAM-A Verification

Xiaohua Yang

with contributions from

Kees Kok, Sami Niemela, Sander Tijm, Bent Sass, Niels W. Nilsen, Flemming Vejen

Challenges in Meso-scale Verification

• Increasing needs of new methods for routine meso-scale verification– Large number of routine HIRLAM runs at 5 km resolution– Almost all HIRLAM services now runs real-time 2.5 km HARMONIE

• In lack of mature method, currently routine verification are made with traditional tools, thus mainly of monitoring nature– Traditional (point or event based) verification remains to have

value for some parameters, such as mslp, W10m, T2m; but no more applicable for precipitation

• Increasingly difficult with verification of high resolution model, especially for precipitation– Model resolution now better than synoptic network– Rich amount of asynoptic data but not in routine

verification– Only limited predictability for km scale convective events

Meso-scale verification requires use of asynoptic observations and thereby new method

Need for use of asynoptic information

• Example of a very local, strongly convective case along Danish-German border in the evening of August 20, with a life time of 1-2 hours, with dramatic scenes seldom seen in Scandinavia… …

No trace to suggest the dramatic event from the synoptic obs-

network!

Estimated total precipitation from radar data (Flemming Vejen, DMI)

Gauge vs radar retrieval rain(Flemming Vejen, DMI)

Simulated radar data for verification of AROME forecast

(Sami Niemela, FMI)

Radar simulator (Haase and Crewell 2000)

3D prognostic:TEMPERATURE

HUMIDITYCLOUD CONDENSATE

RAINSNOW

(GRAUPEL)

Radar simulator

Radar reflectivity (dBZ) from the

model

COMPARISON WITH OBSERVED dBZ IN OBSERAVTION SPACE!

Meso-scale Convective Systems

AROME OBSERVATIONS+ 13 h

Reflectivity Frequency Distribution

Large hails detected

Areas of strong precipitation overestimated

AROME OBSERVATIONS+ 9 h

Frontal and convective rain

Strong convection case

Completely missed by AROME... and the host model (HIRLAM - RCR)!

RTTOV 8.5 is used to derive from model data clear/cloud multi-level infrared, to be compared to upscaled satellite data

Entity based verification (Ebert & McBride, 2000)

– Overcome double penalty dilemma– Error decomposition

MSEtot = MSEdispl + MSEvol + MSEpat

Simulated satellite dataIn verification and process studies

(Zingerler, FMI)

Assessing Model Predictive Potential with Statistical Post-Processing

(Kok, Schereur, Vogelezang)

Model1(LRM)

Model2(HRM)

statistical post-processing

Probablistic Forecast equations

potentialpredictors

DMO verification

Probablistic verification

Source: 200602-200707 ECMWF Operational vs EPS Control 12 UTC forecasts for 3 hourly accumulated precipitation (+3, +6, … ,+72)

Oper (HR, N400, 0.225° ) vs control (LR, N200, 0.450° )

Probabilistic information is extracted from these data using MOS, and verified in parallel to DMO, for precipitation data at station De Bilt. This provide an added measure of predictive potential of a model.

Illustration in Comparative Verification

DMO verification:HR performs worse than LR

* central grid point value* extent of rain area, distance to rain area

on different sized neighbourhoods around central station:(25km, 50km, 100km, .., 250km):

* mean and maximum precipitation* fraction of grid-points with precip

distance-weighted predictors:* maximum precip. weighted with distance

Potential Predictors in MOS Evaluation

Probablistic verification using post-processed model output:

No significant difference between HR ad LR

Summaries

• Fine resolution modelling at 2-5 km scales are now wide-spread in HIRLAM services. Traditional verification methods based on point verification is insufficient to verify convective-scale events. – To verify model results that has higher resolution

than synoptic network– To retrieve and compare to asynoptic data

• Precipitation verification are focus in several ongoing development for new verification method. In view of limited predictability in short range, use of probablistic approach for deterministic model seems logical

• Meso-scale verification has still many development potential. Hirlam has benefited greatly from pioneering efforts in other consortia on meso-scale verification, and look forward to continue collaboration on this area

Recent R & D on Verification in HIRLAM-A

Synoptic HIRLAM (5-20 km)• Common verification for operational HIRLAM and harmonisation of verification package for

HIRLAMD and HARMONIE• Verification using observed and simulated satellite data (Zingerler) • Entity based verification (Zingerler)• Inclusion of variance measure in model evaluation and interpretation of verification scores

(Persson)Multi-model synoptic scale EPS (GLAMEPS, HIREPS)• BMA for probablistic forecast (Alkemade, Schreur, Kok)• Adaptation of ECMWF EPS post-processing and verification package for multi-model SREPS (INM)• NORLAMEPS post-processing and verification package (met.no)

– Upscaling of model precipitation for comparison with gauge dataHARMONIE system (2-10 km, AROME, ALADIN, ALARO, HIRALD)• Joint meso-scale verification working group on sanity check for model physics• Verification using observed and simulated Radar data (Niemela)• Calibrated precipitation retrieval from radar data (SMHI, DMI…)• Statistical post-processing and verification of deterministic forecast with probablistic approach

(Kok, Scherur, Vogelezang)