Subjective evaluation of different versions of ALADIN/HU

ALADIN Workshop 2004, Innsbruck

1

Subjective evaluation of different versions of ALADIN/HU

Helga Tóth, HMS


2

Contents

• Motivation• Method• Results• Comparison to objective scores• Conclusions


3

Motivations• Complex view of the models behavior in different synoptic situation• Grey zone problem is exist on 6.5 km resolution, or is not?• 3D-VAR vs. dynamical adaptation comparison not only objectively but also subjectively• To have more information about that variables which are not included in the objective verification system


4

Methods• From 1. Febr. 2004.• Subjective verification of the previous

day :the Sunday fcst. on Monday the Thursday fcst. on Friday

• On the territory of Hungary:• Discussion and classification (1 bad 5 excellent) at 11:30 am


5

Methods II• Comparison of different models Previous 00 runs by:

- ALADIN/HU dyn. ad. on 6.5 km res.- ALADIN/HU dyn. ad. on 12 km res.- ALADIN-3D-VAR on 12 km res.

12 UTC run two days before by:- ECMWF


6

Methods III• Participants: Gabriella Csima, Edit

Hágel, István Ihász, Gabriella Szépszó, Helga Tóth, (Regina Szoták)

• Verified parameters:- 2m Temperature- Precipitation- Total cloudiness- 10m Wind


7

Results• Total mean and standard deviation

Total average

3,9

3,95

4

4,05

4,1

4,15

AL6 AL12 3D-V ECM

Total standard deviation

0,55

0,6

0,65

0,7

0,75

AL6 AL12 3D-V ECM

3D-VAR got the worst marks, the dyn. ad.-s are better


8

3,2

3,4

3,6

3,8

4

4,2

4,4

T2m Precip. Wind Cloud.

AL6

AL12

3D-V

ECM

-weak forecast: - cloudiness: for all ALADINs

- T2m: for 3D-VAR

-good and similar: Wind and the precipitation


9

T2m AL-3DV1210%

31%

41%

18% rate of 1,1/2, 2, 2/3

rate of 3,3/4rate of 4

rate of 4/5,5

T2m AL-DYN6

21%

57%

17%5%

rate of 1,1/2, 2, 2/3 rate of 3,3/4rate of 4

rate of 4/5,5

T2m AL-DYN125%

22%

17%

56%


rate of 4/5,5

T2m ECMWF6%

19%

17%

58%

rate of 1,1/2, 2, 2/3

rate of 3,3/4

rate of 4

rate of4/5, 5

3D-VAR produced 2 times more middle-class 2mT forecasts than the others


10

Precip. AL-DYN6

22%

24%

49%

5%


rate of 4/5,5

Precip. AL-3DV12

16%

26%

53%

5%rate of 1,1/2, 2, 2/3 rate of 3,3/4rate of 4

rate of 4/5,5

Precip. AL-DYN122%

22%

29%

47%


rate of 4/5,5

Precip. ECMWF

21%

29%

50%

0% rate of 1,1/2, 2,2/3 rate of 3,3/4

rate of 4

rate of4/5, 5

3D-VAR got the most excellent marks and the least middle-class


11

Cloud. AL-DYN62%

35%

46%

17% rate of 1,1/2, 2, 2/3rate of 3,3/4

rate of 4

rate of 4/5,5

Cloud. AL-DYN122%

35%

44%

19% rate of 1,1/2, 2, 2/3

rate of 3,3/4

rate of 4

rate of4/5, 5

Cloud. AL-3DV123%

39%

42%

16% rate of 1,1/2, 2, 2/3

rate of 3,3/4

rate of 4

rate of 4/5,5

Cloud. ECMWF

25%

36%

37%

2%rate of 1,1/2, 2, 2/3

rate of 3,3/4

rate of 4

rate of 4/5,5

Too many middle-class forecasts of ALADINs and Too few excellent


12

Mean precipitation-mark on rainy days

3,53,553,6

3,653,7

3,753,8

3,853,9

AL6 AL12 3D-V ECM

- rainy day > 5mm (~23 days)

- Order is the same as the for the full period

- Not neglectable diff. AL6 and ECMWF


13

Objective scores on the surface2mT-Opposite results

for some parameters than got by the subjective evaluation

- But the the scores are calculated on the whole domain


14

Wind direction

Wind speed

Subjective and objective verif. gave similar results for wind


15

rel.hum

Geopot.

ECMWF produced better scores for the 2Rh and geopot.


16

Regular deficiency• Cloudiness DYN. 12

DYN. 6.5

3D-V

ECM

ALADIN models generally forecast to much total cloudiness, which is not informative in the oper. practice.


17

• Temperature I

- Smaller max. temperature in ALADINs on the spring time

- 3D-VAR the coldest model


18

• Temperature II

‘Bean-shape’ cold spot in the 2mT and Tsurf in 3D-VAR and the guess

after a time disappeared

? modification in 3D-VAR

? By accident


19

Case study (2004. 02. 22)• Strong inversion sleet• Temperature structure at the initial time:

— Temp— AL6— AL12— 3D-V

- DYN. AD-s contain the inversion

- in 3D-VAR too weak (nothing in the guess)


20

•Temperature structure after 12 h integration:

— Temp

— AL6

— AL12

— 3D-VNo inversion at all


21

-Too warm 2mT, ECWMF is better (below 0 in the North few degree SE)

- Few degree wind-direction error which can be the reason of the misfcst.


22

Conclusion• Small differences between the two kind of

dynamical adaptation (no grey zone?)• 3D-VAR has a positive impact on the

precipitation, but negative on the 2mT and cloud.

• Inconsistency between the subjective and objective evaluation maybe because of the domain differences Scores on Hungarian territory by the help of Slovenian colleague

Subjective evaluation of different versions of ALADIN/HU

Documents

hmsaladin workshop

windaladin workshop

ecmwfaladin workshop

amaladin workshop

betteraladin workshop

aladinhu dyn

subjective verification

standard deviation3dvar