Slide 1 Evaluation of observation impact and observation error covariance retuning Cristina Lupu, Carla Cardinali, Tony McNally ECMWF, Reading, UK WWOSC.

Slide 1

Evaluation of observation impact and

observation error covariance retuning

Cristina Lupu, Carla Cardinali, Tony McNally

ECMWF, Reading, UK

WWOSC 2014, Montréal, Canada

Slide 2

Outline

1. Motivation

2. Estimating observation error variances

3. Assimilation experiments with an updated diagonal R

4. Summary


Slide 3

Motivation• Current operational ECMWF system is quite complex: ~ 40 millions observations from 60 instruments are daily assimilated.

• The assumed R and B play an important role in determining the weight of a given observation in the assimilation system.

• The estimation of the error covariances remains a significant challenge.

• In assimilation systems, the observation error covariance R describes errors in the observations as well as the forward model;

• We assume a diagonal R.

• Diagnostics tools are used to quantify the impact of all observations in ECMWF system both in analysis and forecasts.


Slide 4

Ways to estimate R • Diagnostics based on output from DA systems:

• Desroziers method (Desroziers et al., 2005) An estimate of the observation error variances may be obtained a posteriori from the statistical analysis of the observation residuals.

• Adjoint-based methods: makes forecast sensitivity to data assimilation system input parameters [ y, R, xb , B] possible:

• Forecast sensitivity to observations (FSO) – is used to monitor the impact of observations to reduce short-range forecast errors.

• Forecast R-sensitivity (Daescu & Todling, 2010; Daescu & Langland, 2013)may be used to provide guidance to error covariance tuning procedures.The sensitivity of a scalar measure of forecast error is computed with respect to changes to a set of covariance parameters .

• … WWOSC 2014, Montréal, Canada

Slide 5

Initial assimilation experiment

• Aim: Investigate the benefits of an updated (diagonal) R compared to the operational assimilation of IASI/Metop-A

• Baseline experiment assimilating only conventional observations and IASI /Metop-A with R diagonal as in ECMWF operations.

• Setup: 4D-Var, T511 (~ 40 km resolution), 137 vertical levels; Period: 8 June – 30 July 2012

• The simplified (in terms of observation usage) experiment intends to provide the backbone process for observation error variances tuning.


Slide 6

Desroziers diagnostic for σo for IASI

• The observation error standard deviations (σo) assumed in our system are strongly inflated.


Slide 7

Adjoint-based methods

FSO R-sensitivity (FSR)


FSO: The impact of observation is beneficial in each analysis cycle and reduces 24-h forecast error over the global domain by an average of 23.06%; IASI and AIREP observations are contributing the most to 24-h forecast impact.

FSR: Positive sensitivities: identify those observation types whose error variance deflation (decreasing the σo ) is of potential benefit to the 24-h forecast;

Slide 8

FSO

IASI (σo)2 – sensitivity guidance


IASI channels:

Positive sensitivities: Long-wave CO2 temperature-sounding channels;

Negative sensitivities: O3 band (range 1479-1671 )

• Inflation of the assigned observation error σo is of potential benefit to the forecast

• An observation error σo specification according with Desroziers estimates may have a detrimental forecast impact.

FSO, ch. 1671, O3 band

Slide 9

Adjusting (σo)2 for selected IASI channels


Use of adjoint-methods for tuning of observation error involved two steps:

•Use FSR to identify/select IASI channels where observation error standard deviations (σo) should be decreased/increased.

•For selected channels, use Desroziers estimates of (σo) to quantify how much this should be changed.

Slide 10

Experiments with an updated R

• Aim: Investigate the benefits of an updated R compared to the operational assimilation of IASI/Metop-A

• Baseline: assimilating only conventional observations and IASI /Metop-A with R diagonal as in ECMWF operations.

• Exp.1: As Baseline, but with updated diagonal R for all IASI channels as derived from Desroziers diagnostic;

• Exp.2: As Baseline, but with updated diagonal R for 33 selected IASI channels (temperature-sounding channels 173-254 and WV channels 2889-5480).


Slide 11

IASI AIREP-T

Impact on FG-departures


Normalised by Baseline, 95% confidence interval

WIND-U,V

Exp.1Exp.2

Slide 12

Forecast scores: geopotential


Normalised change in RMS geopotential forecast error at 500 hPaVerified against operational analysis; 95% confidence error bars

54 days summer 2012.

Nor

mal

ized

diff

eren

ce

Exp.1 - Baseline

Exp.2 - Baseline

Better than Baseline

Worse than Baseline

Forecast dayForecast day

Slide 13

Total dry energy error norm


• The energy norm evaluates the entire model volume of the atmosphere and calculates a combined error from four meteorological variables.

N. Hem. S. Hem.

• Using the Desroziers diagnosed σo for all IASI channels results in a degradation of analysis and subsequent forecasts .

Slide 14

Analysis sensitivity to observations


Analysis sensitivity Baseline Exp.2

Global Observation influence 9.3% 10.3%

Background influence 90.7% 89.7%

Slide 15

Jo – statistics per channel: IASI


Current obs. errors(Baseline)

Based on effective departure: deff = (y – H(x))T R-½

New obs. errors(Exp.2)

Slide 16

Forecast R- and B-sensitivities


•Positive R-sensitivities for all observation types : decreasing σo for all obs.

•The B-sensitivity provides guidance on weighting in the assimilation system between the background state and the whole observing system: background error covariance inflation.

•An optimal weighting between B and R information may be explained through a single covariance weight coefficient.

Slide 17

Summary• Results of a study aimed at tuning observation errors variances for IASI/Metop-A based on two methods: a posteriori diagnosis and adjoint-based R-sensitivity.

• Using the Desroziers diagnosed σo for all IASI channels results in a degradation in analysis and subsequent forecasts .

• Forecast R-sensitivity: found to be promising for providing guidance on IASI channel selection, but does not provide the amount of how much the observation-error variances should be changed.

• Beneficial forecast impact of geopotential, wind, temperature over the operational R.

• Forecast R- and B-sensitivities can provide guidance toward the real covariance matrices. The method may show if background information is being over (or under) weighted. In this case it appears the EDA based background errors are overweighting the background.


Slide 18

Open issues • Using Forecast sensitivity to R (FSR) to tune R in the current operational ECMWF system is a challenge:

• very large number of assimilated observations• what modifications to R do we need and why?

• In the ECMWF system, an ensemble of data assimilations is used to specify background errors.• The assumed R is used in the ensemble to perturb observations.• Need to investigate the impact of the new R on background error estimate.


Slide 19

Thank you!Questions?


Slide 1 Evaluation of observation impact and observation error covariance retuning Cristina Lupu, Carla Cardinali, Tony McNally ECMWF, Reading, UK WWOSC.

Documents

canada slide

h forecast slide

h forecast error

forecast sensitivity

r diagonal

given observation

observation residuals

canada fso