Optimized localization and hybridization to lter ensemble ......Introduction Linear ltering Joint optimization Results Conclusions Linear ltering of sample covariances Localization

Optimized localization and hybridizationto �lter ensemble-based covariances

Benjamin Ménétrier and Tom AulignéNCAR - Boulder - Colorado

Roanoke - 06/04/2015

Acknowledgement: AFWA

Introduction Linear �ltering Joint optimization Results Conclusions

Introduction

Context:

• DA often relies on forecast error covariances.

• This matrix can be sampled from an ensemble of forecasts.

• Sampling noise arises because of the limited ensemble size.

• Question: how to �lter this sampling noise?

Usual methods:

• Covariance localization→ tapering with a localization matrix

• Covariance hybridization→ linear combination with a static covariance matrix

1


Introduction

Context:





Usual methods:



1


Introduction

Context:





Usual methods:



1


Introduction

Context:





Usual methods:



1


Introduction

Context:





Usual methods:



1


Introduction

Context:





Usual methods:



1


Introduction

Context:





Usual methods:



1


Introduction

Context:





Usual methods:



1


Introduction

Questions:

1. Can localization and hybridization be considered together?

2. Is it possible to optimize localization and hybridizationcoe�cients objectively and simultaneously?

The method should:

• use data from the ensemble only.

• be a�ordable for high-dimensional systems.

3. Is hybridization always improving the accuracy of forecasterror covariances?

2


Introduction

Questions:



The method should:




2


Introduction

Questions:



The method should:




2


Introduction

Questions:



The method should:




2


Introduction

Questions:



The method should:




2


Introduction

Questions:



The method should:




2


Introduction

Questions:



The method should:




2


Introduction

Questions:



The method should:




2


Outline

Introduction

Linear �ltering of sample covariances

Joint optimization of localization and hybridization

Results

Conclusions

3


Outline

Introduction



Results

Conclusions

4



An ensemble of N forecasts {x̃bp} is used to sample B̃:

B̃ =1

N−1

N

∑p=1

δ x̃b(δ x̃b

)T

where: δ x̃bp

= x̃bp−〈x̃b〉 and 〈x̃b〉=

1

N

N

∑p=1

x̃bp

Asymptotic behavior: if N → ∞ , then B̃→ B̃?

In practice, N < ∞ ⇒ sampling noise B̃e = B̃− B̃?

Theory of sampling error:

E[B̃2ij

]=

N(N−3)

(N−1)2E[B̃?2ij

]− 1

(N−1)(N−2)E[B̃iiB̃jj

]+

N2

(N−1)2(N−2)E[Ξ̃ijij

]

5




B̃ =1

N−1

N

∑p=1

δ x̃b(δ x̃b

)T

where: δ x̃bp


1

N

N

∑p=1

x̃bp




E[B̃2ij

]=

N(N−3)

(N−1)2E[B̃?2ij

]− 1


]+

N2


]

5




B̃ =1

N−1

N

∑p=1

δ x̃b(δ x̃b

)T

where: δ x̃bp


1

N

N

∑p=1

x̃bp




E[B̃2ij

]=

N(N−3)

(N−1)2E[B̃?2ij

]− 1


]+

N2


]

5




B̃ =1

N−1

N

∑p=1

δ x̃b(δ x̃b

)T

where: δ x̃bp


1

N

N

∑p=1

x̃bp




E[B̃2ij

]=

N(N−3)

(N−1)2E[B̃?2ij

]− 1


]+

N2


]

5




B̃ =1

N−1

N

∑p=1

δ x̃b(δ x̃b

)T

where: δ x̃bp


1

N

N

∑p=1

x̃bp




E[B̃2ij

]=

N(N−3)

(N−1)2E[B̃?2ij

]− 1


]+

N2


]5



Localization by L (Schur product)

Covariance matrix

B̂ = L◦ B̃

Increment

δxe =1√N−1

N

∑p=1

δ x̃bp◦(L1/2vα

p

)Localization by L + hybridization with B

Covariance matrix

Increment

δx = βe

δxe + βc B1/2vc

Localization + hybridization = linear �ltering of B̃

Lh and βc have to be optimized together

6




Covariance matrix

B̂ = L◦ B̃

Increment

δxe =1√N−1

N

∑p=1

δ x̃bp◦(L1/2vα

p

)

Localization by L + hybridization with B

Covariance matrix

Increment

δx = βe

δxe + βc B1/2vc



6




Covariance matrix

B̂ = L◦ B̃

Increment

δxe =1√N−1

N

∑p=1

δ x̃bp◦(L1/2vα

p


Covariance matrix

Increment

δx = βe

δxe + βc B1/2vc



6




Covariance matrix

B̂ = L◦ B̃

Increment

δxe =1√N−1

N

∑p=1

δ x̃bp◦(L1/2vα

p


Covariance matrix

B̂h = (βe)2 L◦ B̃+ (β

c)2 B

Increment

δx = βe

δxe + βc B1/2vc



6




Covariance matrix

B̂ = L◦ B̃

Increment

δxe =1√N−1

N

∑p=1

δ x̃bp◦(L1/2vα

p


Covariance matrix

B̂h = (βe)2 L︸︷︷︸

Gain Lh

◦ B̃+ (βc)2 B︸︷︷︸

Offset

Increment

δx = βe

δxe + βc B1/2vc



6




Covariance matrix

B̂ = L◦ B̃

Increment

δxe =1√N−1

N

∑p=1

δ x̃bp◦(L1/2vα

p


Covariance matrix

B̂h = (βe)2 L︸︷︷︸

Gain Lh

◦ B̃+ (βc)2 B︸︷︷︸

Offset

Increment

δx = βe

δxe + βc B1/2vc



6


Outline

Introduction



Results

Conclusions

7


Joint optimization: step 1

Step 1: optimizing the localization only, without hybridization

Goal: to minimize the expected quadratic error:

e = E[‖ L◦ B̃︸︷︷︸

Localized B̃

− B̃?︸︷︷︸Asymptotic B̃

‖2]

(1)

Light assumptions:

• The unbiased sampling noise B̃e = B̃− B̃? is not correlatedwith the asymptotic sample covariance matrix B̃?.

• The two random processes generating the asymptotic B̃? andthe sample distribution are independent.

An explicit formula for the optimal localization L is given inMénétrier et al. 2015 (Montly Weather Review).

8





e = E[‖ L◦ B̃︸︷︷︸

Localized B̃


‖2]

(1)

Light assumptions:




8





e = E[‖ L◦ B̃︸︷︷︸

Localized B̃


‖2]

(1)

Light assumptions:




8





e = E[‖ L◦ B̃︸︷︷︸

Localized B̃


‖2]

(1)

Light assumptions:




8





e = E[‖ L◦ B̃︸︷︷︸

Localized B̃


‖2]

(1)

Light assumptions:




8





e = E[‖ L◦ B̃︸︷︷︸

Localized B̃


‖2]

(1)

Light assumptions:




8



This formula of optimal localization L involves:

• the ensemble size N

• the sample covariance B̃

• the sample fourth-order centered moment Ξ̃

Lij

=(N−1)2

N(N−3)

− N

(N−2)(N−3)

E[Ξ̃ijij

]E[B̃2ij

]+

N−1

N(N−2)(N−3)

E[B̃iiB̃jj

]E[B̃2ij

]

9



This formula of optimal localization L involves:

• the ensemble size N

• the sample covariance B̃

• the sample fourth-order centered moment Ξ̃

Lij

=(N−1)2

N(N−3)

− N

(N−2)(N−3)

E[Ξ̃ijij

]E[B̃2ij

]+

N−1

N(N−2)(N−3)

E[B̃iiB̃jj

]E[B̃2ij

]9



Step 2: optimizing localization and hybridization together

Goal: to minimize the expected quadratic error

eh = E[‖ Lh ◦ B̃+ (β

c)2B︸︷︷︸Localized / hybridized B̃


‖2]

Same assumptions as before.

Result of the minimization: a linear system in Lh and (βc)2

Lhij

= Lij−

E[B̃ij

]E[B̃2ij

]Bij

(βc)2 (2a)

(βc)2 =

∑ijB

ij

(1−Lh

ij

)E[B̃ij

]∑

ijB2

ij

(2b)

10





eh = E[‖ Lh ◦ B̃+ (β



‖2]



Lhij

= Lij−

E[B̃ij

]E[B̃2ij

]Bij

(βc)2 (2a)

(βc)2 =

∑ijB

ij

(1−Lh

ij

)E[B̃ij

]∑

ijB2

ij

(2b)

10





eh = E[‖ Lh ◦ B̃+ (β



‖2]



Lhij

= Lij−

E[B̃ij

]E[B̃2ij

]Bij

(βc)2 (2a)

(βc)2 =

∑ijB

ij

(1−Lh

ij

)E[B̃ij

]∑

ijB2

ij

(2b)

10





eh = E[‖ Lh ◦ B̃+ (β



‖2]



Lhij

= Lij−

E[B̃ij

]E[B̃2ij

]Bij

(βc)2 (2a)

(βc)2 =

∑ijB

ij

(1−Lh

ij

)E[B̃ij

]∑

ijB2

ij

(2b)

10


Hybridization bene�ts

Comparison of:

• B̂ = L◦ B̃, with an optimal L minimizing e

• B̂h = Lh ◦ B̃+ (βc)2 B, with optimal Lh and β

c minimizing eh

We can show that:

eh− e =−(βc)2∑

ij

B2ijVar(B̃ij

)E[B̃2ij

]︸︷︷︸≤0

(3)

With optimal parameters, whatever the static B:Localization + hybridization is better than localization alone

11



Comparison of:



c minimizing eh

We can show that:

eh− e =−(βc)2∑

ij

B2ijVar(B̃ij

)E[B̃2ij

]︸︷︷︸≤0

(3)


11



Comparison of:



c minimizing eh

We can show that:

eh− e =−(βc)2∑

ij

B2ijVar(B̃ij

)E[B̃2ij

]︸︷︷︸≤0

(3)


11


Outline

Introduction



Results

Conclusions

12


Practical implementation

An ergodicity assumption is required to estimate the statisticalexpectations E in practice:

• whole domain average,

• local average,

• scale dependent average,

• etc.

→ This assumption is independent from earlier theory.

Localization Lh and hybridization coe�cient βc can be computed:

• from the ensemble at each assimilation window,

• climatologically from an archive of ensembles.

13





• local average,


• etc.





13





• local average,


• etc.





13





• local average,


• etc.





13





• local average,


• etc.





13


Experimental setup

• WRF-ARW model, large domain, 25 km-resolution, 40 levels

• Initial conditions randomized from a homogeneous static B

• Reference and test ensembles (1000 / 100 members)

• Forecast ranges: 12, 24, 36 and 48 h

Temperature at level 7 (∼ 1 km above ground), 48 h-range forecasts

Standard-deviation (K) Correlations functions

14


Experimental setup







14


Experimental setup







14


Experimental setup







14


Experimental setup







14


Localization and hybridization

• Optimization of the horizontal localization Lh

horand of the

hybridization coe�cient βc at each vertical level.

• Static B = horizontal average of B̃

•

15




horand of the



•

15




horand of the



• Localization length-scale:

15




horand of the



• Hybridization coe�cients for zonal wind:

15




horand of the



• Impact of the hybridization:

• B̃?is estimated with the reference ensemble

• Expected quadratic errors e and ehare computed

Error reduction from e to eh for 25 members

Zonal wind Meridian wind Temperature Speci�c humidity

4.5 % 4.2 % 3.9 % 1.7 %

→ Hybridization with B improves the accuracy of theforecast error covariance matrix

15




horand of the








4.5 % 4.2 % 3.9 % 1.7 %


15




horand of the








4.5 % 4.2 % 3.9 % 1.7 %


15




horand of the








4.5 % 4.2 % 3.9 % 1.7 %


15




horand of the








4.5 % 4.2 % 3.9 % 1.7 %


15


Outline

Introduction



Results

Conclusions

16


Conclusions

1. Localization and hybridization are two joint aspects of thelinear �ltering of sample covariances.

2. We have developed a new objective method to optimizelocalization and hybridization coe�cients together:

• Based on properties of the ensemble only

• A�ordable for high-dimensional systems

• Tackling the sampling noise issue only

3. If done optimally, hybridization always improves the accuracyof forecast error covariances.

Ménétrier, B. and T. Auligné: Optimized Localization andHybridization to Filter Ensemble-Based CovariancesMonthly Weather Review, 2015, accepted

17


Conclusions








17


Conclusions








17


Conclusions








17


Conclusions








17


Conclusions








17


Conclusions








17


Conclusions








17


Perspectives

Already done in the paper:

• Extension to vectorial hybridization weights:

δx = βe ◦δxe +βc ◦δxc

→ Requires the solution of a nonlinear system A(Lh,βc) = 0,performed by a bound-constrained minimization.

• Heterogeneous optimization: local averages over subdomains

• 3D optimization: joint computation of horizontal and verticallocalizations, and hybridization coe�cients

To be done:

• Tests in a cycled quasi-operational con�guration

• Extension of the theory to account for systematic errors in B̃?

(theory is ready, tests are underway...)

18


Perspectives







To be done:




18


Perspectives







To be done:




18


Perspectives







To be done:




18


Perspectives







To be done:




18


Perspectives







To be done:




18


Perspectives







To be done:




18


Perspectives







To be done:




18


Perspectives







To be done:




18

Thank you for your attention!

Any question?

Optimized localization and hybridization to lter ensemble ......Introduction Linear ltering Joint optimization Results Conclusions Linear ltering of sample covariances Localization

Documents