Inverse Problems - Applications and Solution Strategies · 2014. 12. 10. · Inverse Problems - Applications and Solution Strategies Inverse Problems Determine causes for identi cation

Inverse Problems - Applications and Solution Strategies

Inverse Problems -Applications and Solution Strategies

Barbara Kaltenbacher, Alpen-Adria Universitat Klagenfurt

AD12, Fort Collins, July 2012


Outline

I inverse problems

I some applicationsI regularization of nonlinear problems

I Tikhonov regularizationI iterative methodsI Kaczmarz methodsI expectation maximization


Inverse Problems

Determine causes for

identification

observedor

desired

optimization

effects.

Inverse problems are often unstable:Small perturbations in the data can lead to large errors in the solution!→ regularization necessary

Identifiability question:Are the searched for quantities uniquely determined by the given data?


Inverse Problems


identification

observedor

desired

optimization

effects.




Inverse Problems


identification

observedor

desired

optimization

effects.




Some Application Examples


Material Characterization for Magnetic Flux Measurement

measurement principle:

Faraday’s Law:

moving conductor

in magnetic field

induces electric voltage.

→ identification of the space-dependentmagnetic permeability of the coil core;

→ hysteresis modelling

• Endress + Hauser, Reinach, CH;

• Inst. theor. Elektrotechnik, Univ. Stuttgart;

• Inst. Mechanik u. Mechatronik, TU Wien


Virtual Material Development

cross section of piezoceramicpolarization hysteresis strain hysteresis

→ micromechnaical modelling of ferroelectric polycrystals;→ hysteresis modelling

• joint project COMFEM (Bosch, Siemens, PI Ceramic, CeramTec,Fraunhofer Inst. f. Werkstoffmechanik, KIT)

• Inst. Mechanik u. Mechatronik, TU Wien


Parameter Identification in Systems Biology

→ determination of parameters in technical and biological systems→ application to identification of gene networks (activation/inhibition)

gene network dependency matrix

• Cluster of Excellence SimTech project with Jun.Prof.Dr. Nicole Radde,

Univ. Stuttgart


Identification of Cracks in Piezoceramics

current/voltage measurementsat surface electrodes

potential distributionin material with crack

→ localization of cracksinside piezoelectric devicesfrom surface measurements(nondestructive testing)

→ identifiability→ adaptive discretizetion

• DFG Project with Prof.Dr. Anna-Margarete Sandig, Univ. Stuttgart


All these applications lead toparameter identification problems in PDE/ODE models


Medical Imaging

I e.g., electrical impedance tomography (EIT)

−∇(σ∇φ) = 0 in Ω .

Identify conductivity σ = σ(x) from measurements of theDirichlet-to-Neumann map Λσ, i.e., all possible pairs(φ , σ∂nφ) on ∂Ω.

I e.g., quantitative thermoacoustic tomography (qTAT):

∇×(µ−1∇× E

)+ σ

∂

∂tE + ε

∂2

∂t2E = J in Ω .

Identify σ = σ(x) from measurements of the deposited energyσ|E|2 in Ω.


Medical Imaging

I e.g., electrical impedance tomography (EIT)

−∇(σ∇φ) = 0 in Ω .

Identify conductivity σ = σ(x) from measurements of theDirichlet-to-Neumann map Λσ, i.e., all possible pairs(φ , σ∂nφ) on ∂Ω.

I e.g., quantitative thermoacoustic tomography (qTAT):

∇×(µ−1∇× E

)+ σ

∂

∂tE + ε

∂2

∂t2E = J in Ω .

Identify σ = σ(x) from measurements of the deposited energyσ|E|2 in Ω.


Parameter identification in PDEs: Model problems

I e.g. “a-example” (transmissivity in groundwater modelling)

−∇(a∇u) = 0 in Ω .

Identify a = a(x) from measurements of u in Ω. [Adrian Sandu, Tue]

I e.g. “c-example” (potential in stat. Schrodinger equation)

−∆u + c u = 0 in Ω .

Identify c = c(x) from measurements of u in Ω.


Parameter identification in PDEs: Model problems

I e.g. “a-example” (transmissivity in groundwater modelling)

−∇(a∇u) = 0 in Ω .

Identify a = a(x) from measurements of u in Ω. [Adrian Sandu, Tue]

I e.g. “c-example” (potential in stat. Schrodinger equation)

−∆u + c u = 0 in Ω .

Identify c = c(x) from measurements of u in Ω.


Forward operator F

I EIT: F : σ 7→ Λσwhere Λσ : φ 7→ σ∂nφ and −∇(σ∇φ) = 0 in Ω .

I qTAT: F : σ 7→ σ|E|2

where ∇×(µ−1∇× E

)+ σ ∂

∂t E + ε ∂2

∂t2 E = J in Ω + bndy.cond.

I a-example: F : a 7→ uwhere −∇(a∇u) = 0 in Ω + boundary conditions

I c-example: F : c 7→ uwhere −∆u + c u = 0 in Ω + boundary conditions

forward operator F often involves solution of PDE,F is usually nonlinear and sometimes also nonsmooth


Forward operator F




)+ σ ∂

∂t E + ε ∂2






Forward operator F




)+ σ ∂

∂t E + ε ∂2






Forward operator F




)+ σ ∂

∂t E + ε ∂2






Forward operator F




)+ σ ∂

∂t E + ε ∂2






Forward operator F




)+ σ ∂

∂t E + ε ∂2






Differentiating F

I large numbers of dependent and independent variables

I directional derivative of state u wrt. parameter often obeyssimilar PDEs to those for the state itself.e.g. a-example −∇(a∇u) = 0; v = ∂u(a; b) solves

−∇(a∇v) = ∇(b∇u)

I derivatives of (cost) functionals wrt. parameters can becomputed from adjoint PDEs derived “by hand”

I . . . but sometimes one would not want to set up and solvethose PDEs

I . . . and correspondence (F ′(a)∗)h = (F ′(a)h)∗ only holds forspecific discretizations [Emre Ozkaya, Tue]


Differentiating F



−∇(a∇v) = ∇(b∇u)





Differentiating F



−∇(a∇v) = ∇(b∇u)





Differentiating F



−∇(a∇v) = ∇(b∇u)





Differentiating F



−∇(a∇v) = ∇(b∇u)





An Example: Ferroelectric Hysteresismicromechanical switching models, e.g. [Huber&Fleck 2001]:(

S

D

)=

(M∑I=1

([sE ]I [d]tI

[d]I [εσ]I

)ξI

)(σ

E

)+

M∑I=1

(SiI

PiI

)ξI

S. . . mechanical strain, D. . . dielectric displacementσ. . . mechanical stress, E. . . electric fieldSiI irreversible strains, Pi

I polarization

M . . . number of polarization directions ξI . . . volume fraction for variant I

ξI =K∑

J=1,J 6=I

(cJIψ(ξJ)− cIJψ(ξI )) . (1)

cIJ = φ(fIJ) fIJ = E ·∆Pi + σt∆Si (driving force)where ∆Pi = Pi

J − PiI ; ∆Si = Si

J − SiI .

ρu− DIV(

[cE ]effDIVTu− Si − [e]effgradϕ)

= 0 (2)

div([e]eff

(Bu− Si

)− [εS ]effgradϕ− Pi

)= 0 , (3)


Inverse Problems as Operator Equations


Nonlinear ill-posed problems

nonlinear operator equation

F (x) = y

!! from now on x is not space variable but parameter function= independent variables


Nonlinear ill-posed problems

nonlinear operator equation

F (x) = y

F : D(F )(⊆ X )→ Y . . . nonlinear operator;F not continuously invertible;

X , Y . . . Hilbert/Banach spaces;

y δ ≈ y . . . noisy data, ‖y δ − y‖ ≤ δ. . . noise level.

regularization necessary


Regularization of Inverse Problems

• unstable operator equation: F (x) = y with F : x 7→ y

• solution x = F−1(y) does not depend continuously on yi.e.,

(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

)

• only noisy data y δ ≈ y available (measurements): ‖y δ − y‖ ≤ δ

• making∥∥F (x)− y δ

∥∥ small 6⇒ good result for x!





(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)

)• only noisy data y δ ≈ y available (measurements): ‖y δ − y‖ ≤ δ







(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)








(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)





An ExampleIdentification of a source term x in a 1-d differential equation y ′′ = x , δ = 1%

exact and noisy data y , yδ exact x vs x with∥∥F (x)− yδ

∥∥ = 1.e − 14

exact and noisy data y , yδ exact x vs x with∥∥F (x)− yδ

∥∥ = 2δ





(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)




• regularization means approaching solution along stable path:given (yn), yn → y construct xn := Rαn(yn) such thatxn = Rαn(yn)→ x = F−1(y)

• regularization parameter choice:trade-off between approximation (small α) and stability (large α)





(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)










(∀(yn), yn → y 6⇒ xn := F−1(yn)→ F−1(y)







Reference to noise level

Recall: We wish to solve

F (x) = y given y with∥∥∥y − y δ

∥∥∥ ≤ δ(measurement noise level)

I accuracy requirements for forward evaluation:‖F (x)− Fh(x)‖ ∼ δ to avoid loss of information during inversion.

I On the other hand it does not make sense to make the datamisfit

∥∥F (x)− y δ∥∥ smaller than the noise level δ.

discrepancy principle for choosing α = α∗:∥∥∥F (xδα∗)− y δ∥∥∥ ∼ δ




















Parameter identification in a PDE as anonlinear operator equation

F (x) = y

F . . . forward operator: F (x) = (C S)(x) = C (u)where u = S(x) solves

D(x , u) = f . . . PDE

Hilbert spaces X , V , Y : x ∈ XS→ u ∈ V

C→ y ∈ Y


Regularization Methods


Tikhonov Regularization

Minimize jα(x) =∥∥F (x)− y δ

∥∥2+ α ‖x‖2 over x ∈ X ,

or equivalently

Minimize Jα(x , u) =∥∥C (u)− y δ

∥∥2+ α ‖x‖2 over x ∈ X , u ∈ V

under the constraint D(x , u) = f

(one shot formulation)

PDE constrained optimization [Johannes Lotz, Mon], [Andrea Walther, Tue]




∥∥2+ α ‖x‖2 over x ∈ X ,

or equivalently


∥∥2+ α ‖x‖2 over x ∈ X , u ∈ V







∥∥2+ α ‖x‖2 over x ∈ X ,

or equivalently


∥∥2+ α ‖x‖2 over x ∈ X , u ∈ V





Tikhonov Regularization with the Discrepancy Principle

Minimize jα(x) =∥∥∥F (x)− y δ

∥∥∥2+ α ‖x‖2 over x ∈ X ,

Choice of α: discrepancy principle (fixed constant τ ≥ 1)∥∥∥F (xδα∗)− y δ∥∥∥ = τδ

nonlinear 1-d equation φ(α) = 0 for α;evaluation of φ requires minimization of Tikhonov functional

Convergence analysis:

[Engl& Hanke& Neubauer 1996] and the references therein


Tikhonov Regularization and AD

I different levels on which AD might be used(as valid generally in PDE constrained optimization)

I tangential stiffness matrix in FEM solution of nonlinear PDEsfor forward evaluation of F ;

I gradients (Hessians) of cost function Jα and equalityconstraints (=PDE) wrt. parameter x and state u in one shotformulation of Tikhonov regularization;

I gradients (Hessians) of reduced cost function jα wrt.parameter x

I derivative of discrepancy function φ wrt. α.

I convergence with optimal rates requires(F ′(x)∗)h = (F ′(x)h)∗ + O(δ2)when solving via discretized version of Euler equationF ′(x)∗(F (x)− y δ) + α(x − x0) = 0


Tikhonov Regularization and AD

I different levels on which AD might be used(as valid generally in PDE constrained optimization)

I tangential stiffness matrix in FEM solution of nonlinear PDEsfor forward evaluation of F ;

I gradients (Hessians) of cost function Jα and equalityconstraints (=PDE) wrt. parameter x and state u in one shotformulation of Tikhonov regularization;

I gradients (Hessians) of reduced cost function jα wrt.parameter x

I derivative of discrepancy function φ wrt. α.

I convergence with optimal rates requires(F ′(x)∗)h = (F ′(x)h)∗ + O(δ2)when solving via discretized version of Euler equationF ′(x)∗(F (x)− y δ) + α(x − x0) = 0


Further aspects of Tikhonov RegularizationI problem adapted regularization terms, e.g.:

I total variation for sharp edges in imagesI l1 for sparsity

I problem adapted data misfit terms, e.g.:I Kullback-Leibler divergence for modelling Poisson noiseI l1 for robustness wrt. outliers

potential nonsmoothness! [Kamil A. Khan, Mon], [Markus Beckers, Tue], [Andreas Griewank, Tue], [Sabrina Fiege, Tue]. . .

I other parameter choice rules, e.g. a priori choice, generalizedcross-validation, L-curve, balancing principles, quasioptimality, . . .

I stochastic noise modelsI relation to Bayesian estimation via MAP estimatorI iterated Tikhonov regularizationI . . .







I stochastic noise models

I relation to Bayesian estimation via MAP estimatorI iterated Tikhonov regularizationI . . .







I stochastic noise modelsI relation to Bayesian estimation via MAP estimator

I iterated Tikhonov regularizationI . . .
















Literature on Tikhonov regularization

I stability and convergence: [Seidman&Vogel 1989]

I convergence rates [Engl&Kunisch&Neubauer 1989] [Neubauer 1999]

[Hofmann&Scherzer 1998]

I analysis in Banach space: [Burger&Osher 2004],[Hofmann&BK&Poschl&Scherzer 2007] [BK&Hofmann 2010],[Hein&Hofmann&Kindermann&Neubauer&Tautenhahn 2009], [. . . ]

I adaptive discretization based on goal oriented error estimators[BK&Kirchner&Vexler 2011], [Don Estep, Thu]

I . . .


Gradient type methodsGradient descent for the minimization of

min 12 ‖F (x)− y‖2 over D(F ) :

xδk+1 = xδk + ωδkF′(xδk )∗(y δ − F (xδk ))

I Landweber iteration: ωδk ≡ 1

I steepest descent and minimal error method:

ωδk :=‖sδk‖

2

‖F ′(xδk )sδk‖2 , ωδk :=

‖yδ−F (xδk )‖2

‖sδk‖2 ,

where sδk = F ′(xδk )∗(y δ − F (xδk )).

I iteratively regularized Landweber

xδk+1 = xδk + F ′(xδk )∗(y δ − F (xδk ))+βk(x0 − xδk )


Gradient type methods with the Discrepancy Principle

xδk+1 = xδk + ωδkF′(xδk )∗(y δ − F (xδk ))

I stopping index k∗ acts as a regularization parameter

I discrepancy principle:∥∥∥y δ − F (xδk∗)∥∥∥ ≤ τδ < ∥∥∥y δ − F (xδk )

∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ δ−1


Literature on gradient type methods

I Landweber for nonlinear inverse problems[Hanke&Neubauer&Scherzer 1995]

I steepest descent and minimal error method [Scherzer 1996],[Neubauer&Scherzer 1995]

I iteratively regularized Landweber [Scherzer 1998]

I generalization to Banach space setting:[Schopfer&Louis&Schuster 2006, Schopfer&Schuster&Louis 2008, BK

&Schopfer&Schuster 2009]

I . . .


Newton type methods

F ′(xδk )(xδk+1 − xδk ) = y δ − F (xδk )

formulation as least squares problem:

minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

ill-posedness apply Tikhonov regularization:Levenberg-Marquardt method:

minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk

∥∥∥x − xδk

∥∥∥2

Iteratively regularized Gauss-Newton method (IRGNM)

minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk ‖x − x0‖2

Both methods differ by choice of sequence αk and convergence analysis.


Newton type methods



minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2


minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk

∥∥∥x − xδk

∥∥∥2


minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk ‖x − x0‖2



Newton type methods



minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2


minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk

∥∥∥x − xδk

∥∥∥2


minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk ‖x − x0‖2



Newton type methods



minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2


minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk

∥∥∥x − xδk

∥∥∥2


minx∈D(F )

∥∥∥y δ − F (xδk )− F ′(xδk )(x − xδk )∥∥∥2

+ αk ‖x − x0‖2



Levenberg-Marquardt

xδk+1 = xδk + (F ′(xδk )∗F ′(xδk ) + αk I )−1F ′(xδk )∗(y δ − F (xδk )) ,

Choice of αk :∥∥∥y δ − F (xδk )− F ′(xδk )(xδk+1(αk)− xδk )∥∥∥ = q

∥∥∥y δ − F (xδk )∥∥∥

for some q ∈ (0, 1) inexact Newton method.

Choice of stopping index k∗: discrepancy principle:∥∥∥y δ − F (xδk∗)∥∥∥ ≤ τδ < ∥∥∥y δ − F (xδk )

∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ log(δ−1)

[Hanke 1996]


Levenberg-Marquardt



∥∥∥y δ − F (xδk )∥∥∥



∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ log(δ−1)[Hanke 1996]


Levenberg-Marquardt



∥∥∥y δ − F (xδk )∥∥∥



∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ log(δ−1)[Hanke 1996]



xδk+1 = xδk+(F ′(xδk )∗F ′(xδk )+αk I )−1(F ′(xδk )∗(y δ−F (xδk ))+αk(x0−xδk )) .

a-priori choice of αk :

αk > 0 , 1 ≤ αk

αk+1≤ r , lim

k→∞αk = 0 ,

for some r > 1.

(a-priori or) a posteriori choice of k∗∥∥∥y δ − F (xδk∗)∥∥∥ ≤ τδ < ∥∥∥y δ − F (xδk )

∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ log(δ−1)

[Bakushinski 1992], [Bakushinski&Kokurin 2004];[BK&Neubauer&Scherzer 1997], [Hohage 1997], [BK& Neubauer&Scherzer 2008]





αk > 0 , 1 ≤ αk

αk+1≤ r , lim

k→∞αk = 0 ,

for some r > 1.


∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ log(δ−1)[Bakushinski 1992], [Bakushinski&Kokurin 2004];[BK&Neubauer&Scherzer 1997], [Hohage 1997], [BK& Neubauer&Scherzer 2008]





αk > 0 , 1 ≤ αk

αk+1≤ r , lim

k→∞αk = 0 ,

for some r > 1.


∥∥∥ , 0 ≤ k < k∗ ,

k∗ ∼ log(δ−1)[Bakushinski 1992], [Bakushinski&Kokurin 2004];[BK&Neubauer&Scherzer 1997], [Hohage 1997], [BK& Neubauer&Scherzer 2008]


Further Literature on Newton type methods

I generalization to regularization methods Rα(F ′(x)) ≈ F ′(x)†

in place of Tikhonov [BK 1997], [Rieder 2001]

xδk+1 = x0 + Rαk(F ′(xδk ))(y δ − F (xδk )− F ′(xδk )(x0 − xδk )) .

I continuous version (artificial time) [BK&Neubauer&Ramm 2002]

I projected version for constrained problems [BK&Neubauer 2006]

I analysis with stochastic noise [Bauer&Hohage&Munk 2009]

I analysis in Banach space [Bakushinski&Konkurin 2004], [BK&

Schopfer&Schuster 2009], [BK& Hofmann 2010]

I preconditioning [Egger 2007], [Langer 2007]

I quasi Newton methods [BK 1998]


Iterative Regularization and AD

I require Jacobian-vector products F ′(x)vand adjoint-vector products F ′(x)∗w

I convergence with optimal rates requires(F ′(x)∗)h = (F ′(x)h)∗ + O(δ2)

I meaning of adjoint ∗: contains transpose but depending onchoice of spaces might additionally involve [Don Estep, Thu]

I solution of PDEs (Sobolev spaces)I nonsmoothness (Lp spaces, p 6= 2)














Kaczmarz methods


Kaczmarz methods for systems of nonlinear operatorequations

Fi (x) = yi , i = 0, . . . ,N − 1 ,

noisy data‖y δi − yi‖ ≤ δ , i = 0, . . . ,N − 1 ,

e.g. x . . . coefficient in a PDE,F(x) = (F0(x), . . . ,FN−1(x)). . . discr. Dirichlet-to Neumann map

Kaczmarz methods (algebraic reconstruction technique):cyclic iteration over subproblems [Kaczmarz’93], [Natterer ’97]

+ perform iterations for several smaller subproblems Fi (x) = yiinstead of one large problem F(x) = y

+ easy to implement especially if Fi are similar



Fi (x) = yi , i = 0, . . . ,N − 1 ,








Fi (x) = yi , i = 0, . . . ,N − 1 ,








Fi (x) = yi , i = 0, . . . ,N − 1 ,







Landweber iteration for a single operator equation

xδk+1 = xδk − F′(xδk )∗(F(xδk )− yδ)

Discrepancy principle:stop the iteration as soon as ‖F(xδk )− yδ‖ ≤ τδ[Hanke Neubauer Scherzer ’94]


Landweber Kaczmarz iteration

xδk+1 = xδk − F ′[k](xδk )∗(F[k](x

δk )− y δ[k])

[k] = k modN

Discrepancy principle:stop the iteration as soon as ‖F[k](x

δk )− y δ[k]‖ ≤ τδ

[Kowar Scherzer ’04]


Loping Landweber Kaczmarz

xδk+1 = xδk − ωkF′[k](x

δk )∗(F[k](x

δk )− y δ[k])

ωk :=

1 if ‖F[k](x

δk )− y δ[k]‖ ≥ τδ

0 otherwise.

Discrepancy principle:stop the iteration as soon as ‖Fi (xδk )− y δi ‖ ≤ τδ ∀i ∈ 0, . . . ,N − 1i.e., kδ∗ := minjN ∈ IN : xδjN = xδjN+1 = · · · = xδjN+N[Haltmeier Leitao Scherzer’07], [De Cesaro Haltmeier Leitao

Scherzer’08], [Haltmeier’09]


Levenberg-Marquardt for a single operator equation

xδk+1 = xδk − (F′(xδk )∗F′(xδk ) + αk I )−1F′(xδk )∗(F(xδk )− y δ)

Choice of αk : (inexact Newton) ρ ∈ (0, 1)‖F′(xδk )(xδk+1(α)− xδk ) + F(xδk )− y δ‖ = ρ‖F(xδk )− y δ‖Discrepancy principle:stop the iteration as soon as ‖F(xδk )− yδ‖ ≤ τδ[Hanke’96], [Rieder’99], [Hanke’09]


Levenberg-Marquardt Kaczmarz iteration

xδk+1 = xδk + (F ′[k](xδk )∗F ′[k](x

δk ) + αk I )

−1F ′[k](xδk )∗(y δ[k] − F[k](x

δk ))

Choice of αk : (inexact Newton) ρ ∈ (0, 1)‖F ′[k](x

δk )(xδk+1(α)− xδk ) + F[k](x

δk )− y δ[k]‖ = ρ‖F[k](x

δk )− y δ[k]‖

Discrepancy principle:stop the iteration as soon as ‖F[k](x

δk )− y δ[k]‖ ≤ τδ

[Burger BK’04] (also IRGN-Kaczmarz), [Baumeister BK Leitao’09]


Example 1

Reconstruction from Dirichlet-Neumann Map:Estimate space-dependent coefficient q ≥ 0

−∆u + qu = 0, in Ω,

u = f on ∂Ω,

from N Dirirchlet-Neumann pairs (fi ,∂ui∂ν |∂Ω).

Ω = (0, 1)2

fi ≈ δ(· − x i ) , x i uniformly spaced on ∂ΩN = 20q∗ = 3 + 5 sin(πx) sin(πy)q0 ≡ 3

Results with Levenberg-Marquardt-Kaczmarz

Difference q∗ − qk at iterates 1, 2, 3, 5, 10, and 100.

Convergence with exact data

Semi-logarithmic plot of error (left) and residual (right) vs.iteration number

Semiconvergence with noisy data

Semi-logarithmic plot of error (left) and residual (right) vs.iteration number, δ = 1%


Loping Levenberg-Marquardt Kaczmarz iteration

xδk+1 = xδk +ωk(F ′[k](xδk )∗F ′[k](x

δk ) +αI )−1F ′[k](x

δk )∗(y δ[k]−F[k](x

δk ))

ωk :=

1 if ‖F[k](x

δk )− y δ[k]‖ ≥ τδ

0 otherwise.

Discrepancy principle:stop the iteration as soon as ‖Fi (xδk )− y δi ‖ ≤ τδ ∀ii.e., kδ∗ := minjN ∈ IN : xδjN = xδjN+1 = · · · = xδjN+N[Baumeister BK Leitao’09]


Inverse doping problem for semiconductorsReconstruct γ in

div (µnγ∇u) = 0 in Ωu = −U(x) on ∂ΩD

∇u · ν = 0 on ∂ΩN

div (µp1γ∇v) = 0 in Ω

v = U(x) on ∂ΩD

∇v · ν = 0 on ∂ΩN

from N Dirirchlet-Neumann pairs (Ui ,Λ(Ui ))where Λ(U) =

∫Γ1

(µnγuν − µpγ−1vν) ds,u, v . . . concentration of electrons and holesµn, µp . . . mobility of electrons and holes

C(x) = γ(x)− γ−1(x)− λ2∆(ln γ(x))

Ω = (0, 1)2, N = 9, xi uniformly spaced in [0, 1], i = 0, . . .N − 1Γ1 := (x , 1) ; x ∈ (0, 1) , Γ0 := (x , 0) ; x ∈ (0, 1) ,∂ΩN := (0, y) ; y ∈ (0, 1) ∪ (1, y) ; y ∈ (0, 1) .

Ui (x) :=

1, |x − xi | ≤ 2−4

0, elsei = 0, . . . ,N − 1 ,


Inverse doping problem for semiconductorsReconstruct γ in

div (µnγ∇u) = 0 in Ωu = −U(x) on ∂ΩD

∇u · ν = 0 on ∂ΩN

div (µp1γ∇v) = 0 in Ω

v = U(x) on ∂ΩD

∇v · ν = 0 on ∂ΩN

from N Dirirchlet-Neumann pairs (Ui ,Λ(Ui ))where Λ(U) =

∫Γ1

(µnγuν − µpγ−1vν) ds,u, v . . . concentration of electrons and holesµn, µp . . . mobility of electrons and holes

C(x) = γ(x)− γ−1(x)− λ2∆(ln γ(x))

Ω = (0, 1)2, N = 9, xi uniformly spaced in [0, 1], i = 0, . . .N − 1Γ1 := (x , 1) ; x ∈ (0, 1) , Γ0 := (x , 0) ; x ∈ (0, 1) ,∂ΩN := (0, y) ; y ∈ (0, 1) ∪ (1, y) ; y ∈ (0, 1) .

Ui (x) :=

1, |x − xi | ≤ 2−4

0, elsei = 0, . . . ,N − 1 ,

Exact coefficient and PDE solution for one voltage source

exact coefficient γ to be identified (left);typical voltage source Ui and corresponding solution u (right)

Exact coefficient and initial guess

exact coefficient γ to be identified (left);initial guess (right)

Comparison of loping Levenberg-Marquardt-Kaczmarz withLandweber-Kaczmarz

Numerical experiment with noisy data (5%):error obtained with l-LMK after 24 cycles (left);error obtained with l-LWK after 205 cycles (right)

Comparison of loping Levenberg-Marquardt-Kaczmarz withLandweber-Kaczmarz

Numerical experiment with noisy data (5 per cent):number of non-loped inner steps in each cycle for l-LMK (solidred) and l-LWK (dashed blue), respectively.


Expectation Maximization (EM) algorithms


EM (Richardson-Lucy) algorithm for linear problems

for image reconstruction with nonnegativity constraints:[Bertero 1998], [Natterer&Wuebbeling 2001], [Dempster&Laird&Rubin 1977]

F : L1(Ω)→ L1(Σ) linear operator with F ∗1 = 1 (scaling)

xδk+1 = xδkF∗(

y δ

Fxδk

) multiplicative fixed-point scheme. well-suited for multiplicative noise models (e.g. Poisson models)

F , F ∗ positivity preserving, xδ0 ≥ 0, y δ ≥ 0 ⇒ ∀k ∈ IN : xδk ≥ 0


Derivation

(EM) xδk+1 = xδkF∗(

y δ

Fxδk

)is descent method for the functional

J(x) :=

∫Σ

[y δ log

(y δ

Fx

)− y δ + Fx

]dσ ,

Kullback-Leibler divergence (relative entropy) between Fx and y δ.optimality condition

x

(−F ∗

(y δ

Fx

)+ F ∗1

)= 0 .

with operator scaling F ∗1 = 1 (EM)

[Multhei&Schorr89], [Natterer&Wuebbeling 2001], [Resmerita&Engl&Iusem

2007], [Bissantz&Mair&Munk]


EM algorithm for nonlinear problems

nonlinear operator F : L1(Ω)→ L1(Σ), no scaling fixed-pointequation

xF ′(x)∗1 = xF ′(x)∗(y δ

Fx

).

nonlinear EM algorithm

xδk+1 =xδk

F ′(xδk )∗1F ′(xδk )∗

(y δ

F (xδk )

).

[Haltmeier&Leitao&Resmerita 2009]


Conclusions/Outlook

I derivatives are needed at different levels: solution of nonlinearPDE models, minimization of Tikhonov functional, evaluationof Jacobians and their adjoints in iterative methods, . . .

I complexity of models often prohibitive for derivativecomputation via adjoint PDE (“by hand”);

I regularization methods require (F ′(x)∗)h = (F ′(x)h)∗+O(δ2);

I nonsmoothness (Lipschitz continuity, piecewise C1) often playsa role;


Thank you for your attention!


Thank you for your attention!

26th IFIP TC7 Conference 2013 on

System Modelling and Optimization

September 9-13, 2013, Klagenfurt, Austria

http://ifip2013.uni-klu.ac.at/

Inverse Problems - Applications and Solution Strategies · 2014. 12. 10. · Inverse Problems - Applications and Solution Strategies Inverse Problems Determine causes for identi cation

Documents