Copper Slides

8/10/2019 Copper Slides

1/23

Fast Algorithms for the Geostatistical approach for solvingLinear Inverse Problems based on Hierarchical matrices

Arvind Saibaba1

Peter K. Kitanidis2,1

1Institute for Computational and Mathematical Engineering2Department of Civil and Environmental Engineering.

Stanford University

March 30, 2012

Arvind Saibaba Peter K. Kitanidis Geostatistical approach to Inverse problems 1 / 21


2/23

Outline

1 Motivation

2 Bayesian approach to Inverse problems

3 Hierarchical matrices

4 Solving the system

5 Contaminant Source Identification



3/23

Contaminant Source Identification

Figure: Stockie SIAM Review 2011

Figure: Flath et al. SISC 2011Arvind Saibaba Peter K. Kitanidis Geostatistical approach to Inverse problems 3 / 21


4/23

Random Field

Model unknowns as a Gaussian random field

E[s] =X E[(s X)(s X)T] =Q

Figure: Three realizations of a Gaussian random field with exponential covariance

Storage and computational costs for Qij =(xi, xj) i,j= 1, . . . ,m high.

Examples of(, ) : Matern family,Gaussian,Exponential.



5/23

Bayesian viewpoint

Consider the measurement equation

y= h(s) +v v N(0,R)

where,

y := observations or measurements - given.s := model parameters, we want to estimate.

h(s) := parameter-to-observation map - given. := (unknown) drift coefficients

Using Bayes rule, the posterior pdf is

p(s,|y) p(y|s,)p(s,)

exp

1

2s XQ1

1

2yHsR1



6/23

Best Estimate

Maximum a posteriori estimate:

arg mins,

1

2sXQ1 +

1

2y HsR1

Solution:

HQHT +R HX(HX)T 0 = y0 s= X+QHTOperation costs:

n : number of measurements 103

m : number of unknowns 105

p : number of drift coefficients 1

Construction: O(m2n+mn+mnp) Solving: O(n+p)3



7/23

Fast summation algorithms

Various schemes for computing

(Qv)i =N

j=1

(xi, xj)vj i,j= 1, . . . ,N

FFT based algorithms

Restricted to regular grids.O(Nlog N) complexity.



8/23



(Qv)i =N

j=1

(xi, xj)vj i,j= 1, . . . ,N



Fast Multipole Method - Greengard and Rokhlin.N-body problem, Boundary Element Method, Radial basis interpolation.O(Nlog N) orO(N) complexity.Several black-box versions exist.



9/23



(Qv)i =N

j=1

(xi, xj)vj i,j= 1, . . . ,N



Fast Multipole Method - Greengard and Rokhlin.N-body problem, Boundary Element Method, Radial basis interpolation.O(Nlog N) orO(N) complexity.Several black-box versions exist.

Hierarchical matrices - Hackbusch and co-workers.Boundary Element Method, preconditioners for sparse pde systems.O(Nlog N) orO(N) complexity.Other matrix operations possible - adding, multiplying, inversion,factorizations.



10/23

H-matrix formulation: An Intuitive Explanation.

Consider : [0, 1]2 R, for xi, yi= (i 1) 1N1

, i= 1, . . . ,N

(x, y) = exp(|x y|)

Figure: blockwise rank- = 106, N=M= 256



11/23

H-matrix formulation

Key features of Hierarchical matrices:

A hierarchical separation of space.An acceptable tolerance that is specified.

Low-rank approximation ofadmissible sub-blocks.

Valid forasymptotically smoothkernels.

DefinitionA cluster pair (, ) is consideredadmissibleif

min{diam(X), diam(X)} dist(X,X)

Definition

A kernel is calledasymptotically smooth, if there exist constants cas1 , cas2 and a

real number g 0 such that for all multi-indices Nd0 it holds thaty K(x, y) cas1 p!(cas2 )p(|x y|)gp, p= ||



12/23

H-matrix

Storage and Multiplication is almost linear complexity.

Low rank approximation computed by Adaptive Cross Approximation.Well suited for Krylov Subspace methods (eg. GMRES, CG).

102

103

104

105

102

101

100

101

102

103

N

time(sec)

HierMatrix

Direct

Figure: left: A typical H-matrix rank structure and right: Time for matrix vectorproduct for exponential covariance function = 106



13/23

Iterative solver

Krylov subspace methods for solving Ax= b, at the i-th iteration satisfy

ri span{r0,Ar0,A2r0, . . . ,A

i1r0} = (A)r0, Pi

where, ri =b Axi, is the residual at the i-th iteration.Minimal residual methods such as MINRESorGMREStry to compute apolynomial such that

ri = minPi

(A)r0

We apply it to the systemHQHT +R HX

(HX)T 0

A

=

y0

A is not constructed explicitly, rely only on matrix vector products. For eg.

HQHT +R

x= H

Q HTx +Rx

Construction: O(p+k2m log m) Matrix-vector product: O(km log m+2+np+n)



14/23

Eigenvalue decay of Covariance kernel (,)

Consider the integral eigenvalue problemD

(x, y)(y)dy= (x)

Smoother the kernel is, the faster {m} 0.

IfD Rd and if the kernel is 1

piecewise Hr m c1mr/d

piecewise smooth m c2mr for any r>0

piecewise analytic m c3expc4m

1/d

We expect the eigenvalues of the Qij=(xi, xj) to behave similarly.

1Schwab and Todor (2006).Arvind Saibaba Peter K. Kitanidis Geostatistical approach to Inverse problems 12 / 21

S P di i


15/23

Steps to compute Preconditioner

= HQHT +R= R1/2(R1/2HQHTR1/2 +I)R1/2

Compute low-rank representation using Thick-restart Lanczos algorithm

Q VrrVTr

Form the matrix

M= R

1/2

HVr1/2

r

Compute the singular value decomposition (SVD) of the matrix M

M= UVT

Use Sherman-Morrison-Woodbury update to compute the inverse of

(MMT +I)1 =I UDrUT Dr= diag

2i

1 +2i

Compute the approximate inverse of, which we denote by 1 as

1 =R1 R1/2UDrUTR1/2


S P di i i l l i


16/23

Steps to compute Preconditioner - computational complexity

Assumptions: Hx has complexity and R is diagonal.

1 Q VrrVTr O(rkm log m)

2 M= R1/2HVr1/2r O(r+r)

3 M= UVT O(nr2)4 (MMT +I)1 =I UDrU

T O(r)

5 1 =R1 R1/2UDrUTR1/2 O(nr)

Finally, 1x only costs O(nr).


Wh d it k?


17/23

Why does it work?

Theorem (Bauer-Fike)

LetA be a diagonalizable matrix, andV be the non-singular eigenvector matrixsuch thatA= VV1. If is an eigenvalue ofA+A, then an eigenvalue (A) exists such that

| | cond(V)A

Applying this result to the matrix A= 1, we have

|1 (1)| rQH21

gives us an explicit bound on the spectral radius of the matrix.


A li ti C t i t S Id tifi ti


18/23

Application: Contaminant Source Identification

Forward problem

u

t v.u+ D2u = 0, [0,T]

u = u0, {t= 0}

u = 0 D [0,T]

Measurement Operator

h(s) =Hs= H

Sensors A1

Forward Propagation T

Prolongations


Reconstruction


19/23

Reconstruction

Figure: Reconstruction of a Gaussian initial condition 2 exp(30xxc) which iscentered at xc = (0.25, 0.5). The unknowns are discretized on a 100 100 grid in

space and 20 time steps, in the domain with L= T= 1. The measurements arecollected in a 101010 grid, i.e. nm = 100, nt= 10. (left) reconstructed field and(right) true field. The relative error in the reconstruction was 0.063.


Results: with preconditioner


20/23


All tests performed with

K(x, y) = (1 +r+ (r)2/3) exp(r) r= x y

Sensors Unknowns Iterations Rel. Err

8 8 100 100 30 0.0941200 200 32 0.0949300 300 33 0.0953

10 10 100 100 38 0.0669200 200 39 0.0675300 300 41 0.0679

12 12 100 100 136 0.0495200 200 196 0.0503300 300 200 0.0500

Table: The performance of the iterative scheme for the contaminant sourceidentification problem. In each case the number of time measurements were nt= 10,with L= T= 1 and t= 0.05. For the preconditioner, we used r= 100.




21/23


All tests performed with

K(x, y) = (1 +r+ (r)2/3) exp(r) r= x y

r Iterations r Rel. err.

129 300 2.45 105 0.0148

201 196 4.76 106

0.0146278 96 1.75 106 0.0145355 41 7.09 107 0.0144

Table: Performance of iterative scheme with increasing r for grid size 100100 andnumber of sensors 2525, so that number of measurements are 6, 250. . indicatesthat it reached the maximum number of iterations 300, without converging to the

desired solver tolerance.


Spectrum of preconditioned matrix


22/23

Spectrum of preconditioned matrix

100

101

102

103

101

100

101

102

i

|i|

m=400

m=1600

m=10000

106

104

102

100

102

100

101

102

103

x = |i1|

#

|i

1|>x

m=400

m=1600

m=10000

Figure: (left) Eigenvalues of the preconditioned operator (right) Plot of#{i :|i1| > x} against x for all eigenvalues. For these plots, we assumed thenumber of observations to be 101010, the number of unknowns varied from2020 to 100100.


Conclusions


23/23

Conclusions

Our contributions

Ascalablematrix-free approach to solving linear inverse problems.

A preconditioner that is expensive to compute but performs well.

Quantifying uncertainty - generating conditional realizations.Unconditional realizations using Chebyshev matrix polynomials.

AcknowledgementThe authors were supported by NSF Award 0934596,Subsurface Imaging and Uncertainty Quantification.


Copper Slides

Documents