Page 1
WMO
Dr. Lars Peter Riishojgaard, WIGOS Project Manager, WMO Secretariat
On the use of data assimilation to assess
the cost/benefit of meteorological satellites
EUMETSAT Meteorological Satellite Conference,
Side event at WMO, Geneva, September 23 2014
Page 2
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
Question: Can we put numbers on the monetary
value of (for instance) satellite observations used for
weather forecasting?
• Economics of meteorological observations
• Money and weather forecasting
• Weather forecasting and observational data
• Impact of observational data
• Cost of satellite data
• Role of data assimilation and NWP
• Caveats
• Final remarks
2
Page 3
Weather Prediction and the US
Economy; A Macroscopic View
• Department of Commerce: “20% of overall US economy is weather sensitive”: ~$3 trillion/year
– Impact to air and surface transportation, agriculture, construction, energy production and distribution, etc.
• Assume that half of this is “forecast sensitive”: $1.5 trillion/year
• Assume that the potential savings due to weather forecasting amount to 5% of the “forecast sensitive total”: ~$75B/year
(discussed during CBS TECO in Windhoek, 2010)
3
Page 4
… a Macroscopic View … (II)
• Define “perfect forecast information” as NWP output with useful
skill at two weeks!
• 0 h useful forecast range => $0 in savings
• 336 h useful forecast (two weeks maximum predictability) range
=> $75B in savings
• Assume now that the savings are distributed linearly over the
achieved forecast range for the global NWP system:
– $75B/336h ~ $223B/hr
• This implies that the value to the United States economy of
weather observations, dissemination, forecast products and
services is >$220M per hour of forecast range per year !
(discussed during CBS TECO in Windhoek, 2010)
4
Page 5
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
The global picture
• The amount of $75B/year is one estimate of the
magnitude of the total potential socioeconomic benefit
of weather prediction activities to the US economy
• Scaling exercise, using World Bank (2011) numbers:
• Annual GDP of United States: ~$15T
• Annual GDP of all nations combined: ~$70T
– Assuming on average (i) equal sensitivity to weather, and (ii)
equal potential benefits from ability to predict across all nations,
we get an estimated
$75B *($15T/$70T) = $350B as the total global potential
benefit of weather prediction activities (indicating a likely range of
$100B to $1T)
5
Page 6
What does this have to do with satellite
data? (NWP illustration) There is a need for a global
coverage of observational data,
irrespective of target location of
forecast!
6
Page 7
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
What is the cost of acquiring
satellite weather observations?
• Difficult to come up with accurate numbers
• Some agency budgets are publicly known, others are not
• Need to distinguish between systems developed for
research versus operational systems
• Even the latter provide value beyond “just” weather use of
data data, e.g. through impact on climate, air quality, other
environmental monitoring/prediction activities, value of
scientific research, technological advancements, national
prestige, …
• Overall annual expenditure on meteorological satellite data is
likely to be in $3-8B/year range
7
Page 8
What does NWP and data assimilation
have to do with this?
Beyond a range of 12-24h, NWP output is the
foundation of all weather forecast activities
NWP has objective, quantitative metrics:
Well-defined prediction problem with a “right” answer
(and an infinity of wrong ones)
Well-defined measures for quality of output
Well-established methodologies for assigning merit to
individual observing systems
One can actually somewhat meaningfully define
“perfect forecast information” in an NWP context
8
Page 9
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
Jung et al., WMO Impact Workshop in Sedona, May 2012
9
NWP tools for impact assessment
(I)
• OSEs (Observing System
Experiments) are based on
data denial (or addition)
• Impact focuses on the
medium to long range
• Results show the impact of
withdrawing (or adding)
certain data
• OSE results are absolute;
e.g. “observing system X
extends the useful forecast
range by N hours in the
NH”
Page 10
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
Gelaro et al, Fifth WMO Impact Workshop, Sedona 2012
10
• FSO (Forecast Sensitivity to Observations) are based on the adjoint of the model/analysis system or an ensemble approach
• Approach focuses exclusively on the short (quasi-linear) range
• Results show the impact of observations in the presence of all other observations
• FSO measures of impact are relative (e.g. often expressed in percentages that add up 100, even for poor forecasts or poor system performance)
NWP tools for impact assessment (II)
Page 11
Forecast impact experiment from Dec. 2010 to Jan. 2011
Impact Impact / Obs. number
WMO Workshop on the Impact of Various Observing Systems on NWP Sedona – 22-25 May 2012
Could we use this type of FSO information to
rank observing systems by impact per dollar?
19
Of course we can! Simply divide the impact by the
cost of running the system and come up with a third
“impact per dollar” bar chart!
11
Page 12
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
Answer: Yes, we can in fact assign some
form of monetary value to (e.g.)
meteorological satellite data
• Armed with the following three pieces of information
– Cost of acquiring the observations (difficult but not impossible to
estimate),
– Overall economic benefits of meteorological products derived from
observations (crudely estimated, or properly analyzed by trained
economists; most difficult part of the exercise),
– Individual contribution of observing systems to NWP skill, e.g. as
measured by FSO diagnostics,
it is (almost frighteningly) easy to devise an NWP-based
cost/benefit ratio for individual elements of the GOS, such as
the satellite components; involves a number of assumptions,
but eminently feasible
12
Page 13
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
Caveats
• We may be able to assign value, but should we?
– The respective contributions of the components of the GOS differ
between NWP systems, even at comparable levels of skill
– The contributions vary, depending on which other GOS
components are used in the experiments
– OSE results are expensive to acquire, and often inconclusive
– FSO impacts are relative; even if the overall forecast performance
is poor, some observing systems may stand out due to their large
share in a modest improvement
– The sum of the percentages of the contributions always add
up to 100, even with few observations and poor skill
13
Page 14
EUMETSAT Meteorological Satellite Conference, Geneva, 22-26 .9.2014
Final remarks
• The economic impact of weather is relatively well understood
• In contrast, the economic impact of weather prediction is generally not well studied and documented
• The cost/benefit of meteorological observations are a subject of intense interest among program managers and decision makers
• The costs are incurred (and known) mostly at the regional levels, the impact is realized and assessed globally, and the benefits accrue locally
• Tendency to focus too much on NWP diagnostics due to their compelling nature
• Impact of radars, geostationary satellites etc. will be underestimated
• Further work on cost/benefit is needed; capability is emerging and decision makers demand this type of information
14