Optimal solution error covariances in nonlinear problems of variational data assimilation

1

Optimal solution error covariances in nonlinear problems of variational data assimilation

Victor Shutyaev

Institute of Numerical Mathematics, Russian Academy of Science, Moscow

Igor Gejadze Department of Civil Engineering, University of Strathclyde, Glasgow, UK

F.-X. Le Dimet, LJK, University of Grenoble, France

2

u

Ttxxt

fFt

t 0

,0,,,

)(

Problem statement

)(inf)( vJuJv

Model of evolution process:

Objective function (for the initial value control):

Control problem:

XbbbYooo uuuuVCCVuJo

),(),()( 11

- nonlinear differential operator

Unknown initial condition (analysis)

BackgroundObservations Inverse of the

background covariance matrix

Observationoperator

Inverse of theobservationcovariance matrix

)(F

u

Tt

0t

uuu Optimal solution (analysis) error:

True state

3

Optimal solution error via errors in input data

oobbob VuuRVuuuRVuuRV 1111 ,,,

In the nonlinear case the optimal solution error and input data errors are related via the nonlinear operator equation [1]:

Background error Observation erroruubb

oo C

What interpretation for being of random nature?bo ,

For example, for each sensor the observation error is a random time series.oFor the background error it can be seen as an error in expert guesses.b

XbbbY uuuuVuJo

),()()( 1

XY uuVuJo

0),0(~

2)()( 1

Variational DA:

Tikhonov regularization:

Estimates in variational DA and Tikhonov’s method have different statistical properties: in particular, Tikhonov’s estimates are not consistent (biased)

1,0

CvuuR

v

uuFt

t

,

0,

0

4

Statistical properties of the optimal solution error

0][ 0 EuE uT VuuE ][

Covariance matrix

If we consider as random errors, then optimal solution error is a random error. Moreover we assume that it is subjected to the multivariate normal distribution and can be quantified by: Expectation

bo ,

What are reasons to believe that?

Some classics from nonlinear regression:Estimate is consistent and asymptotically normal if is i.i.d. with

and has certain regular properties, Gennrich (1969). This result follows from strong law of large numbers. Extended to multi-variate case and for certain classes of dependent observations, Amemiya, (1984). In reality the number of observation is always finite, thus the concept of ‘close-to-linear’ statistical behaviour, Ratkowski (1984).

u o

)(F )(,0)( T

ooo EE )( T

Are the results above valid for complete error equation ?It requires that and it must be normally distributed;A difficulty is that full equation might have many solutions; however, if among them we choose the one which corresponds to the global minimum of the cost functional, then we should also achieve consistency and asymptotic normality.

bTbbb VEE )(,0)(

bbV 1

TYouJ ,0)( X

ooo VuuRuuuRVuuR 11 ,,, 1bV

5

Covariance and the inverse Hessian

RVRVHuJFF ob11)(

Linear case and normal input data:H -Hessian, also: Fisher information matrix, grammian, …

Nonlinear case and normal input data:

With the following approximationsone obtains

0,,),0,(, uRuuRuRuuR

1uV H u

The sufficient condition for these approximations to be valid is called the tangent linear hypothesis:This condition means that even though the dynamics is nonlinear, evolution of errors is well described by the tangent linear model.

FFF

As most sufficient conditions the tangent linear hypothesis is overly restrictive. In practice, the above formula is valid if the linearization error is not cumulative in a probabilistic sense.

1HV u

oobbob VuuRVuuuRVuuRV 1111 ,,,

1 1( ) : ( ,0) ,0b oH u V R u V R u

6

On errors

1. Consider function as the exact solution to the problem

ul ul u

ub u b,l , obs Co,l

b,l , o,l

3. End ensemble loop.4. Compute the sample covariance

ˆ V u 1

Lulul

T

l1

L

2. Start ensemble loop 2.1 Generate using Monte-Carlo 2.2 Compute 2.3 Solve the original nonlinear DA problem with perturbed data and find 2.4 Compute

l1,...,L

Fully nonlinear ensemble method (Monte Carlo)

Two types of error are presented in the formula .Error due to approximationscan be called the ‘linearization error’.However, the true state is not usually known (apart from the identical twin experiment setup) and one must use its best available estimate as the Hessian origin. Hence, another error called the ‘origin error’. This error cannot be eliminated, however its possible magnitude can be estimated.

0,1 uHV u

0,,),0,(, uRuuRuRuuR

uu

luu

7

Iterative methods for the inverse Hessian computation

1. Inverse Hessian by the Lanczos and Arnoldi methodsThe Lanczos and Arnoldi methods compute a set of leading Ritz values/vectors which approximate the eigen-pairs of the preconditioned Hessian using the Hessian-vector product:

xvBBVvuH

xTCCVFt

BvxFt

Tb

o

,0

0,,

,0,0

1

1

ufFt t 0,)(

oYoooXbbb

w

CCVvBvBVvJ

wJvJ

BvxFt

)),(())(),(()(

)(inf

,0,0)(

111

11

2. Inverse Hessian by the BFGS methodThe BFGS forms the inverse Hessian in course of solving the auxiliary control problem:

Iterative methods allow us to compute a limited-memory approximation of the inverse Hessian (at a limited computational cost) without a need to compute the Hessian matrix. These methods require efficient preconditioning (B).

8

Model (non-linear convection-diffusion):

0

,1,0

)0,(

,0,1,0

x

t

x

t

ux

Ttx

Qx

kxx

vt

Example 1: Initialization problemNonlinear diffusion coefficient Field evolution

and ensemble variance

diag H 1

H 1and ensemble covariance

9

When the main result is not validIn a general nonlinear case one may not expect the inverse Hessian to always be a satisfactory approximation to the optimal solution error covariance.

0,1 uHV u

Model: 1D Burgers with strongly nonlinear dissipation term

2

10

2

,0,1

,0,0

0,

,0,1,0

0,2

1

xkkk

x

t

x

t

ux

Ttx

xxk

xxt

Field evolution: case A and case B

and ensemble variance for initialization problem ))0,(( 1 uHdiag

Case A: sensors at )65,0,6.0,45.0,4.0(kx Case B: sensors at )6.0,5.0,4.0(kx

In Figures: inverse Hessian – solid line, ensemble estimate – marked line, background variance – dashed line

10

Effective Inverse Hessian method (EIH): main idea

uuRVVuuRVVgg

uugguuEuuEVT

oTooob

Tbbb

T

TTu

,,

],,,,[][1111

11

],[ 1 uuHEVV u

L

lluuH

LV

1

1 ,1

dvvfvuHV u,1

dvvVvvuH

VV T

M

11212 2

1exp,

)2(

1

Exact nonlinear operator equation

Exact optimal solution error covariance (by definition)

Resulting from series of assumptions the equation above reduces to the form:

I. Computing the expectation by Monte Carlo:

uuu ll

l-th optimal solution

II. Computing the expectation by definition:

As we assume that - normal, then uf

v is dummy argument !

Assumes nonlinear dynamics, but asymptotic normality and ‘close-to-linear’ statistical behavior (Ratkowski, 1983 )

oobbob VuuRVuuuRVuuRV 1111 ,,,

11

EIH method: implementation

Preconditioning BHBHHBBH 11 ~~

21bVB

BdvvVvvuHVBV TM 11212

2

1exp,

~)2(

)0,(~ 2121 uHVB b

1-level preconditioning:

2-level preconditioning:

Iterative process

BuuHL

BV

VuuEuEp

uHV

L

l

pl

p

pTppp

1

11

10

),(~1

])([,0][,...1,0

0,

Monte Carlo (MC) for integration

,...1,0),0,(

,)(2

1exp,

~)2(

10

112121

puHV

BdvvVvvuHVBV pTpMp

This integral is a matrix which can be presented in a compact form !

For integration instead of MC one can use quasi-MC or multi-pole method for faster convergence (smaller L) !

12

EIH method: example - 1Relative error in the variance estimateby the ‘effective’ IH (asymptotic) and IH

Mi

VVVV iiiiLiiiiii

,...,1

1ˆ)ˆ(ˆ,1ˆ,,,,

- reference covariance (sample covariance with large L=2500 )

Envelope for relative error in the sample variance estimate for L=25(black) and L=100 (white)

V̂

IVIIV

- based on a set of optimal solutions- does not require optimal solutions

Can be improved using ‘localization’, but requiresoptimal solutions!

Envelope for by the ‘effective IH’,L=25(black) and L=100(red)

13

EIH method: example - 2

Relative error in the variance estimateby the ‘effective’ IH (asymptotic) and IH

Mi

VVVV iiiiLiiiiii

,...,1

1ˆ)ˆ(ˆ,1ˆ,,,,

- reference covariance (sample covariance with large L and after ‘sampling error compensation’ procedure )

Envelope for relative error in the sample variance estimate for L=25(black) and L=100 (white)

V̂

IVIIV

- based on a set of optimal solutions- does not require optimal solutions

Can be improved using ‘localization’, but requiresoptimal solutions!

Envelope for by the ‘effective IH’,L=25(black) and L=100(red)

14

EIH method: examples 1-2, correlation matrix

Example 1

Example 2

Reference correlation matrix

Error in the correlation matrix by IH method

Error in the correlation matrix by EIH method

15

On a danger of the origin error

Each is a likely optimal solution given .For each the likelihood region is defined by its covariance approximated by , which may significantly differ. Dependent on what optimal solution actually implemented (and considered as an origin), the covariance estimates may not approximate at all.

,, ll uuuu

uVu ,iu

iu luuH ,1

uVThus, solutions of such nonlinear systems cannot by verified in principle. Difference in mutual probabilities can be considered as an indicator of verifiability.

16

Conclusions

For an exact origin:

the inverse Hessian is expected to approximate well the optimal solution error covariance if the tangent linear hypothesis (TLH) is valid. In practice, this approximation can be sufficiently accurate even though the TLH breaks down;

if the nonlinear DA problem tends to be at least asymptotically normal or (better) exhibits a ‘close-to-linear’ statistical behavior, then the optimal solution error covariance can be approximated by the ‘effective inverse Hessian’.

For an approximate origin:

the likely magnitude of the origin error can be revealed by a set of variance vectors generated around an optimal solution; based on this information the verifiability of the optimal solution can be analysed.

the upper bound of the set can be chosen to achieve reliable (robust) state estimation

In the linear case the optimal solution error covariance is equal to the inverse Hessian

In the nonlinear case, one must distinguish the linearization error (originates from linearization of operators around the Hessian origin) and the origin error (originates from the difference between the best known and a true state)

In an extremely nonlinear case the posterior covariance Does not represent the pDF (though locally!)

reasonably nonlinear case

17

References

Gejadze, I., Copeland, G.J.M., Le Dimet, F.-X., Shutyaev, V.P. Computation of the optimal solution errror covariance in variational data assimilation problems with nonlinear dynamics. J. Comp. Physics. (2011, in press)

Gejadze, I., Le Dimet, F.-X., Shutyaev, V.P. On optimal solution error covariances in variational data assimilation problems. J. Comp. Physics. (2010), v.229, pp.2159-2178

Gejadze, I., Le Dimet, F.-X., Shutyaev, V.P. On analysis error covariances in variational data assimilation. SIAM J. Sci. Comput. (2008), v.30, no.4, 1847-1874