15/11/2005 Juha Kilpinen 1 Comparison of ECMWF and HIRLAM wind forecasts in the Baltic Sea Juha Kilpinen, AnnaKaisa Sarkanen, Pertti Nurmi and Sigbritt Näsman Finnish Meteorological Institute (FMI) • FMI news – New building (one floor for Finnish Marine Research Institute) – New supercomputer • Meteorological visualisation applications (EGOWS 2005) • Use and interpretation of medium and extended range forecast guidance (the title above)
36
Embed
New building (one floor for Finnish Marine Research ... · Juha Kilpinen 715/11/2005 Comparison of ECMWF and HIRLAM wind forecasts in the Baltic Sea Juha Kilpinen, AnnaKaisa Sarkanen,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
15/11/2005Juha Kilpinen 1
Comparison of ECMWF and HIRLAM wind forecasts in the Baltic Sea
Juha Kilpinen, AnnaKaisa Sarkanen, Pertti Nurmi and Sigbritt NäsmanFinnish Meteorological Institute (FMI)
• FMI news– New building (one floor for Finnish Marine Research Institute)
– New supercomputer• Meteorological visualisation applications
(EGOWS 2005)
• Use and interpretation of medium and extended range forecast guidance (the title above)
15/11/2005Juha Kilpinen 2
15/11/2005Juha Kilpinen 3
15/11/2005Juha Kilpinen 4
15/11/2005Juha Kilpinen 5
New supercomputer
• SGI Altix 3700 BX2– 304 processors– 304 Gb of memory– 30 times faster than the
previous IBM – HIRLAM and other
applications (ECHAM5)
15/11/2005Juha Kilpinen 6
Meteorological visualisation applications
• Last EGOWS meeting in Exeter– http://www.metoffice.gov.uk/egows2005/programme.html
15/11/2005Juha Kilpinen 7
Comparison of ECMWF and HIRLAM wind forecasts in the Baltic Sea
Juha Kilpinen, AnnaKaisa Sarkanen, Pertti Nurmi and Sigbritt NäsmanFinnish Meteorological Institute (FMI)
• This study is a part of two frameworks– evaluation of ECMWF EPS forecasts in FINLAND– evaluation of warning criteria for forecasting probability of near gale
force wind in the Baltic Sea (Finland and Sweden issue near galewarnings and storm warnings for same areas, but with different criteria)
• Both ECMWF and HIRLAM data is used• Period of data is one winter (~ September 2004 – April 2005) • Observations from 21 coastal stations• Both deterministic and probabilistic forecasts are verified• Different methods for producing probability forecasts from
deterministic data has been tested • Different calibration methods has been tested and verified
15/11/2005Juha Kilpinen 8
Data (ECMWF model data)• ECMWF MARS data
– u10m, v10m->speed10m• Period 1.10.2004-30.4.2005
(Finnish stations)• Period 1.10.2004-31.3.2005
(Swedish stations)• Period 1.11.2004-31.3.2005
(Danish and Norwegian stations)
• forecasts valid at 00, 06, 12 and 18 UTC
• forecast lead time from +12h-+144h
• Operational data and Control data (interpolated to 0.5x 0.5 degree resolution)
• EPS data (interpolated to 0.5x 0.5 degree resolution)
15/11/2005Juha Kilpinen 9
Data (HIRLAM Model data)• HIRLAM RCR
– 10m wind speed & additional parameters
• Grid length about 22 km• forecast lead time from +6h-
+48h• HIRLAM MBE
– 10m wind speed & additional parameters
• Grid length 9 km• forecast lead time from +1h-
+24h
Different grids of HIRLAM (RCR (above) and BME)
0.5 deg
1.0 deg
02981 (Utö)
02987 (Kalbådagrund)
EC grid 0.5x0.5 or 1.0x1.0
RCR
MBE
15/11/2005Juha Kilpinen 10
Different grids of HIRLAM (RCR (above) and BME)
0.5 deg
1.0 deg
02981 (Utö)
02987 (Kalbådagrund)
EC grid 0.5x0.5 or 1.0x1.0
RCR
MBE
15/11/2005Juha Kilpinen 11
Observation stations:01427014480146701482
060810605206041060790617906193
0251702616026440268002584
028730291002980029810297902987
15/11/2005Juha Kilpinen 12
Stations
Hailuoto 02873
Valassaaret 20910
Nyhamn 02980
Utö 02981
Bogskär 02979
Kalbådagrund 02987 31 m
15/11/2005Juha Kilpinen 13
Stations
Hailuoto 02873
Valassaaret 20910
Nyhamn 02980
Utö 02981
Bogskär 02979
Kalbådagrund 02987
32 m
15/11/2005Juha Kilpinen 14
The methods for producing probabilistic forecasts for near gale force winds
• Deterministic data• Method 1a: observed error distribution (dependent sample) is
used to produce probability forecasts from operative deterministic forecasts (“error dressing”)
• Method 1b: as above, but observed error distribution is approximated with Gaussian distribution (µ , σ)
• Method 2: as above but σ is derived from model “stability”(temperature difference of two model levels) with an empirical formula (by Håkan Hultberg, SMHI)
• Method 3: operational forecast is Kalman filtered and σ (a bi-product of Kalman filter) is used to estimate the probability (also Gaussian approximation)
15/11/2005Juha Kilpinen 15
The methods for producing probabilistic forecasts for near gale force winds
• Deterministic data (cont.)Method 4: Uncertainty method (also called Neighbourhood
method by e.g. Susanne Theis)– Spatial (Fig.) and/or temporal – neighbouring grid points– “Unc”
• EPS data (51 members): the probability of near gale wind (speed > 14 m/s or >13.9 m/s)
15/11/2005Juha Kilpinen 16
The methods for producing probabilistic forecasts for near gale force winds
• Calibrated EPS data: the calibration is made by Kalman filtering the EPS mean and then all EPS member are transformed with the same relationship to provide a new 51 member ensemble.
• Deterministic operative data: Kalman filtering of operative wind forecasts and application of Kalman filter residuals to provide the error distribution for probability of near gale. Also other alternatives exits but they have not yet been tested.
15/11/2005Juha Kilpinen 17
Some examples of theoretical pdf’s with different parametersLimit (upper) Probability
of the phenomenon
15/11/2005Juha Kilpinen 18
Examples of individual ensembles (station 02680 +24h (51 members) )
Error distribution of Controlforecasts (sample of 180 cases)and fitted Gaussian distribution
day 49 day 83
15/11/2005Juha Kilpinen 19
Examples of forecast/observation distributions (station 02680 +24h)
Error distribution of EPS meanforecasts (sample of 180 cases)and fitted Gaussian distributionBLUE line corresponds to meanspread of EPS mean
Distribution of EPS mean Distribution of observations
Distribution of error
15/11/2005Juha Kilpinen 20
Calibration of EPS forecasts• Traditionally calibration of ensemble forecasts means that the
probabilities are relabelled using the information in reliability diagram (a large sample of past forecasts and observations are needed)
• In this study Kalman filtering is used to calibrate ensemble mean forecasts (and also operative as well as control). Then every individual ensemble member is transformed with the state vector. This calibrates at least the “mean” of distribution and hopefully also the “spread”.
• Kalman filtering is also used in traditional way to correct the deterministic forecasts and then to estimate the probabilities using observed error distribution
15/11/2005Juha Kilpinen 21
Verification measures• For deterministic forecasts (operative, control and EPS mean,
Kalman filtered operative)
– Mean Error ME– Root Mean Squared Error, RMSE– Standard Deviation of Error, STD
Results: Deterministic forecastsPooled results from 21 stations (12 UTC)
lead time (days)
ME
-0,1
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 2 3 4 5 6
ME_OPERME_CONTRME_EPSME_KAL
15/11/2005Juha Kilpinen 26
Results: Deterministic forecastsPooled results from 21 stations (12 UTC)
lead time (days)
RM
SE
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 2 3 4 5 6
RMSE_OPERRMSE_CONTRRMSE_EPSRMSE_KALSPREAD
15/11/2005Juha Kilpinen 27
Results: Deterministic forecastsPooled results from 21 stations (12 UTC)
lead time (days)
STD
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 2 3 4 5 6
STD_OPERSTD_CONTRSTD_EPSSTD_KALSPREAD
15/11/2005Juha Kilpinen 28
Relative Operating Characteristic
• To determine the ability of a forecasting system to discriminatebetween situations when a signal is present (here, occurrence of near-gale) from no-signal cases (“noise”)
• To test model performance relative to a specific threshold
• Applicable for probability forecasts and also for categorical deterministic forecasts
• Inhomogeneous observations a problem for verification• EPS forecasts are slightly under dispersive• The biases depend on station (height of anemometer etc.)• Kalman filtering is able reduce biases and produce better
probability forecasts for most stations in terms of ROC curve ROC area and Brier Skill Score
• “Dressing” of dependent sample: quality level hard to reach