Page 1 COST/ESF School: UTLS, Cargese, 3-15 October 2005 DA 12: Evaluation of future missions Author: W.A. Lahoz Data Assimilation Research Centre, University.

Page 1COST/ESF School: UTLS, Cargese, 3-15 October 2005

DA 12: Evaluation of future missions

Author: W.A. Lahoz

Data Assimilation Research Centre, University of Reading RG6 6BB, UK


•Why is it important to evaluate expensive future missions such as ESA’s Envisat

Quantify added value from new observations in comparison to Global Observing System

•Use of data assimilation:

Different approaches to evaluating future missions. Observing System Simulation Experiments (OSSEs) and Observing System Experiments (OSEs)

•Example of an OSSE: The proposed SWIFT instrument, measuring stratospheric winds and ozone

Topics:


Importance of evaluating future EO missions


Envisat: http://envisat.esa.int


You are given 2.3 BEuros for Envisat to observe the Earth System…

Hooray!

BUT…

How can you check if this is a good use of money?

How can you quantify value?


What do you need to consider?

NOT the value of Envisat

BUT added value of Envisat above what else will be available

-> INCREMENTAL VALUE

THIS IS TRUE FOR ANY ADDITION TO GOS


So…

What will the GOS be like?

Existing & planned satellite missions

What type of observations do we include?

Conventional: ground-based, sondes, aircraft

Satellites: operational, research


850 KM

3 5 8 0 0 K m SUBSATELLITE POINT

GOMS (Russian Federation)

76E

MSG

(EUMETSAT) 63 E

MTSAT (J apan)

140E

FY-2 (China)

105E

GOES-E (USA) 75W

NPOESS (USA)

GOES-W (USA) 135W

G E O S T

A T I O N A R Y

O R B I T

Global Earth Observing system for 2008-2010


Observation types used by Met Office for NWP


GOS:

Observing characteristics

Viewing geometries (limb/nadir; LEO/GEO)

Sonde & ground-based observations distribution

Aircraft corridors

Observation errors: random, systematic, representativeness

See DA 11


We have a description of future GOS

BUT…

How do we get values of a future instrument: obs & errors?

Need to sample the “truth” (nature)

-> Get from a model run, or analyses


• Lack of global observations of stratospheric winds in the current operational meteorological system:

• No sondes above 10 hPa (no global coverage anyway)• AMVs from satellites in troposphere• Wind information from temperature nadir sounders in extra-tropics

(troposphere/stratosphere)• But, thermal wind relation breaks down in tropics

• We have no good current estimates of state of the tropical stratosphere:

• Variability in the quasi-biennial oscillation (QBO) is underestimated• “Balanced” winds problematic for estimating varability of QBO Randel et al. (2002)

A current concern about GOS are winds


“Realistic” quasi-biennial oscillations in the MO Unified Model

MO observational analyses of equatorial winds for Nov 1992 - Jan 2000


• Recent past:

• UARS WINDII: mesospheric winds

• UARS HRDI: stratospheric winds, but impact marginal as observed winds not accurate enough compared to forecasts (Boorman et al. 2000)

• Future:

• ESA ADM-Aeolus: launch October 2007

• CSA SWIFT: launch 2010?

Missions measuring winds


ADM-Aeolus

Doppler Wind Lidar (DWL)

1 component global wind profiles up to ~30 kmN.B. need DA to get 2 components

•Better information to predict weather•Global wind profiles for the entire planet, including remote areas lacking any g-based weather station

Main objective:•Correct major deficiency in winds in current GOS

•Increased skill in NWP •Data needed to address WCRP key concerns:

Quantification of climate variabilityValidation & improvement of climate modelsProcess studies relevant to climate change

OSSEs done for ADM-Aeolus (e.g. Stoffelen)


Use of data assimilation


Philosophy:

Nature, Truth (T) -> sample to get all observations

Test impact of one observation (e.g. Envisat-like)

Control experiment (C): all observations less that of interest

Perturbation experiment (P): all observations

Compare C & P against T

QUANTIFY!


Space agencies (ESA, NASA, JAXA, CSA) invest a lot of money

on missions (e.g. ESA’s Envisat has cost 2.3BEuros) Important to evaluate beforehand possible benefits of

future missions, especially those involving satellites

Techniques for evaluating future missions:

A technique often used by the space agencies is the OSSE Information content (e.g. Prunet et al. QJ 98, IASI) Ensembles (e.g. Andersson et al. WMO 2005, ADM-Aeolus) OSSEs tend not be used as much by the met agencies

This is due to the shortcomings of OSSEs (see later) -> less attractive to the met agencies

Future EO missions


A technique often used by the met agencies to evaluate components of an existing observing system is the “Observing System Experiment” (OSE)

An OSE studies the impact of one observation type by removing it from the system under study

- test impact of satellite data for NH/SH - test impact of nadir/limb geometries

Note:


OSSE goal: evaluate if the difference P-T (measured objectively)is significantly smaller than the difference C-T

Simulated atmosphere (“truth”; T): using a model Simulated observations of instruments appropriate to the

study, including errors: using T Assimilation system: using a model Control experiment C: all observations except those under

study Perturbation experiment P: all observations

Structure of an OSSE


Note shortcomings of an OSSE:

Expensive (cost ~ assimilation system) -> alleviate problem: “reduced OSSE” (e.g. profiles instead of radiances)

Note: “reduced OSSE” generally only useful when observation of interest has relatively high impact (e.g. stratospheric winds)

Difficult interpretation (model dependence) -> alleviate problem: conservative errors, several methods to investigate impact

Incest -> alleviate problem: different models to construct “truth” & perform assimilation (BUT there could be bias between models)

Despite shortcomings, high cost of EO missions meansthat OSSEs often make sense to space agencies


Example of an OSSE


SWIFT: see Lahoz et al. QJ 2005

Based on UARS WINDII principle (Doppler effect)

2 wind components using 2 measurements at ~90o

Thermal emission (mid-IR) of ozone (1133 cm-1)

Technology difficult to implement

Global measurements of wind and ozone profiles (~20-40 km)

OSSE: evaluate proposed SWIFT instrument


1. Current observing system:

No operational observations of winds for levels above those of radiosondes (~10 hPa)

Note: indirect information on winds can be obtained from nadir soundings of temperature (thermal wind; but this breaks down in the tropics)

2. Science:

Help build climatologies of tropical winds

Transport studies (e.g. ozone fluxes)

Use assimilation to obtain 4-d quality-controlled datasets for scientific studies (e.g. climate change and its attribution)

Why SWIFT?


Establish basis for assimilating SWIFT observations (u, v; ozone) Investigate scientific merits of SWIFT observations

Models used: “Truth” (ECMWF directly, or forcing a CTM) Assimilation system (Met Office) (cf. incest)

Simulated observations:Operational: C {MetOP, MSG, sondes, balloons, aircraft,

surface}Temperature, winds, humidity, ozone

SWIFT; C+SWIFT = POzone, winds (stratosphere, conservative errors)

Several assimilation experiments; analyses evaluated. Qualitative & quantitative tests

Design of SWIFT OSSE


SWIFT: N - and S - observations (87°N-53°S, 53°N-87°S): non sun-synchronous orbit

- winds 16-50km, every 2km approximately

- ozone 16-44km, every 2km approximately

Errors: conservative; random; representativeness error considered to be relatively unimportant

SWIFT wind component error

0

10

20

30

40

50

60

0 5 10 15 20 25

error (m/s)

hei

gh

t (k

m)

SWIFT ozone error

10

20

30

40

50

0 5 10 15 20

error (as a percentage of the observation)

hei

gh

t (k

m)

SWIFT characteristics


Several tests -> robustness (cf. interpretation of the OSSE)

Qualitative (histograms, monthly means)

Quantitative (RMS statistics, significance tests)

Assumption:

We can discount the bias between the ECMWF and Met Office models because it is removed when comparing P-T and C-T

Same bias in both P-T and C-T differences which are compared

Evaluation of SWIFT impact


Y=Abs(C-T) -Abs(P-T); Zonal-wind (m/s); January 2000; Shaded:95% C.L. & Y>0. Similar results for April 2000.

10 hPa 1 hPa

Significance tests Areas > 5%

N.B. Some areas of -ve impact (new obs can degrade DA system) - not significant


SWIFT winds Significant impact in tropical stratosphere EXCEPT lowermost levels Can have significant impact in extra-tropics when:

SWIFT observations available Flow regime is variable (relatively fast changing)

Have scientific merit in that they improve: Information on tropical winds Wintertime variability

Useful for forecasting & producing analyses to help study climate change & its attribution:

better models, better initial conditions, model evaluation

SWIFT ozone Significant impact at 100 hPa & 10 hPa -> regions of relatively high vertical gradient

Conclusions


“Reduced OSSE”: Radiances used for AMSU-A, IASI at time of SWIFT launch

1. Expectation is that impact in tropics will not change2. Impact in the extra-tropics may remain unchanged (impact in

flow regimes which change relatively fast)

Thermal wind relationship does not hold for 1, and is not accurate in 2

Caveats


Higher horizontal resolution-> Less “thinning” of satellite data (reduce number of obs): AMSU-A, IASI

Would impact stratospheric wind analyses in the extra-tropics

Conclusions in tropics should be robust

Conclusions for ozone analyses (100 & 10 hPa) should not change

NOTE: the SWIFT OSSE did not evaluate forecasts (only analyses) and did not calibrate the OSSE by, e.g., removing simulated sonde data & replacing it with real sonde data

Caveats


You might think OSSEs are a load of… (fill in words)

If so help is at hand!

• Information content

• Ensembles

Alternatives


Aim: evaluate the potential benefit of future sensors compared to other available sensors

Prunet et al. (1998) used approach to quantify impact of information content in simulated IASI radiances vs information content in TOVS radiances.

Impact of IASI radiances estimated by comparison of analysed errors (which include TOVS or IASI data) vs those of a background field (from model forecast excluding both TOVS and IASI data).

If observation type of interest has positive impact, analysed errors should be smaller than background errors.

By comparing errors of analyses including TOVS or IASI data, relative information content in these data can be evaluated, and assessment made of their relative benefit.

Information content


In principle, information content approach is simpler & less expensive to apply than an OSSE.

However, information content approach requires a realistic characterization of background & observation errors, which could be difficult to achieve.

Furthermore, it could be argued that OSSE approach provides a more complete test of the future sensors.

Information content


Ensembles Courtesy Andersson et al.ECMWF





According to Andersson et al:

Need “truth” for simulated data


Important to quantify value of future missions

-> participation of all actors: multi-disciplinary

-> quantify benefits: OSSEs & other methods

-> caveats: set up experiments carefully

Should there be a dedicated OSSE facility?

Way forward:

Page 1 COST/ESF School: UTLS, Cargese, 3-15 October 2005 DA 12: Evaluation of future missions Author: W.A. Lahoz Data Assimilation Research Centre, University.

Documents

costesf school

gos slide

research slide

nwp slide

value of envisat

expensive future missions

future eo missions

future instrument