Introduction to the theory, numerical methods and applications of ill-posed problems ... · 2016-05-06 · 1 Introduction to the theory, numerical methods and applications of ill-posed

1

Introduction to the theory, numerical methods and

applications of ill-posed problems

(lecture course)

Anatoly Yagola

Professor, Dr. Sc., Department of Mathematics, Faculty of

Physics, Moscow State University, Moscow 119991,

Russia, e-mail: [email protected]

Orebro, May 2-4 , 2016

mailto:[email protected]

CONTENTS

Introduction

Well-posed and ill-posed problems

Elements of functional analysis

Examples of ill-posed problems

Definition of the regularizing algorithm

Ill-posed problems on compact sets

2

CONTENTS

Extremal problems statement

Solvability of an extremal problem.

Simplest necessary and sufficient

conditions of minimum

Convex functionals

Solvability of a convex programming

problem

3

CONTENTS

Criteria of convexity and strong

convexity

Least square method. Pseudoinversion

Minimizing sequences

Some methods for solving one-

dimensional extremal problems

Method of steepest descent

4

CONTENTS

Conjugate gradient method

Newton’s method

Zero-order methods

Conditional gradient method

Projection of conjugate gradients

method

5

CONTENTS

Numerical methods for solving ill-posed

problems on special compact sets

Ill-posed problems with a source-wise

represented solution

Tikhonov’s regularizing algorithm

Generalized discrepancy principle

6

CONTENTS

Incompatible ill-posed problems

Numerical methods for the solution of

Fredholm integral equations of the first

kind

Convolution type equations

7

Introduction

In the lecture course we will describe

fundamentals of the theory of ill-posed

problems so as numerical methods for their

solution if different a priori information is

available. For simplicity, only linear

equations in normed spaces are considered.

8

Introduction

Although, it is clear that all similar

definitions can be introduced also for

nonlinear problems in more general metric

(even also topological) spaces.

Numerous inverse problems can be found in

different branches of physics (astrophysics,

geophysics, spectroscopy, nuclear physics,

etc.).

9

Introduction

Mostly, these inverse problems are ill-

posed. It means that small deviations of

input data (due to experimental errors) can

produce large errors in an approximate

solution.

10

Introduction

The modern theory of ill-posed problems

started seventy years ago (1943) when

famous Russian mathematician Andrey

Tikhonov published in Soviet Mathematics

Doklady magazine a paper “On stability of

inverse problems”.

11

Introduction

In 1963 Tikhonov formulated the definition

of a regularizing algorithm. He and two

other Russian mathematicians Mikhail

Lavrentiev and Valentin Ivanov became

founders of the modern theory of inverse

and ill-posed problems.

12

Introduction

13

Introduction

Now this theory is greatly developed

throughout the world. The most eminent

specialists in Sweden are Lars Elden from

Linkoping and Larisa Beilina from

Gothenburg.

14

Course literature

A.N.Tikhonov, A.V.Goncharsky,

V.V.Stepanov, A.G.Yagola. Numerical

methods for the solution of ill-posed

problems. - Kluwer Academic Publishers,

1995.

A.N.Tikhonov, A.S.Leonov, A.G.Yagola.

Nonlinear ill-posed problems. V. 1, 2 -

Chapman and Hall, 1998.

15

Course literature

L. Beilina, M.V. Klibanov. Approximate

global convergence and adaptivity for

coefficient inverse problems. – Springer,

2012.

16

Recommended reference literature

O.M. Alifanov, E.A. Artioukhine, S.V.

Rumyantsev. Extreme Methods for Solving

Ill-Posed Problems with Applications to

Inverse Heat Transfer Problems. – Begell

House Inc., 1995.

A. Bakushinsky, A. Goncharsky. Ill-posed

problems: Theory and applications. -

Kluwer Academic Publishers, 1994.

17


A.B. Bakushinsky, M.Yu. Kokurin.

Iterative Methods for Approximate Solution

of Inverse Problems. – Springer, 2005.

H.W. Engl, M. Hanke, A. Neubauer.

Regularization of Inverse Problems. –

Kluwer Academic Publishers, 1996.

18


V.K. Ivanov, V.V. Vasin, V.P. Tanana.

Theory of Linear Ill-Posed Problems and its

Applications. – VSP, 2002.

M.M. Lavrentiev, L.Ya. Saveliev. Operator

Theory and Ill-Posed Problems. – De

Gruyter, 2006.

V.V. Vasin, A.L. Ageev. Ill-Posed

Problems with A Priori Information. – VSP,

1995.19


Let us consider an operator equation:

Az=u,

where is a linear operator acting from a

Hilbert space Z into a Hilbert space U. It is

required to find a solution of the operator

equation z corresponding to a given

inhomogeneity (or right-hand side) u.

20


The problem of solving the operator

equation is called to be well-posed

(according to Hadamard) if the following

three conditions are fulfilled:

21


Usually, a choice of the space of solutions

(including a choice of the norm) is

determined by requirements of an applied

problem. A mathematical problem can be

ill-posed or well-posed depending on a

choice of a norm in a functional space.

22


Lavrentiev’s example

Condition 3) is fulfilled if

||z||A = ||Az||,

A is an injective operator.

23


Linear spaces

Metric spaces

Normed spaces

Banach spaces

Euclidean spaces

Hilbert spaces

24


Example. A finite-dimensional vector space

Rn that is very well known from Linear

algebra.

25


Example. A space C[a,b] of (real)

functions defined and continuous on a

segment [a,b] with the norm:

||y||C[a,b]=max{|y(s)|, s[a,b]}.

Convergence in this space is called uniform

convergence.

26


Definition. A sequence xnN, n=1, 2, …; is

called fundamental sequence if for any 0

there exists a natural number K such that for

any nK and any natural number p ||xn+p-

xn||.

If any fundamental sequence converges then

the normed space is called the complete

space (or Banach space).

27


Example. Rn and C[a,b] are Banach

spaces.

28


29


Definition. A linear space is called the Euclidean

space if for any two elements x, yE is defined a

real number (x,y) (a scalar product) such that:

1) for any x, yE (x,y)=(y,x);

2) for any x1,x2, yE (x1+x2,y)=( x1,y)+(x2,y);

3) for any x, yE and any real (x,y)=

(x,y);

4) for any xE (x,x)0, and (x,x)=0 if and

only if x=0.

30


Example. A finite-dimensional vector space

Rn is an Euclidean space with a scalar

product (x,y)=x1y1+…+xnyn, where x1,…,xn;

y1,…,yn; are components of vectors x and y

respectively.

31


32


33


34


Linear operators

35


36


37


Definition. A sequence yn,, n=1,2,…, of

elements of a normed space N is called

compact if any its subsequence has a

convergent subsequence. If space N is not

Banach space, then it has a fundamental

subsequence.

38


Definition. A linear operator А: N N is

called compact if for any bounded

sequence i=1, 2,…; elements of the

sequence zn=Ayn of elements in is

compact.

Definition. A linear compact operator is

completely continuous.

39


Theorem. Any completely continuous

operator is bounded (consequently,

continuous).

In finite-dimensional spaces any linear

operator is completely continuous. It is not

true in infinite-dimensional spaces.

40


Example. Let us consider the identity

operator I: , Iy=y for any y. It is bounded

but not compact.

Theorem. Let А be the Fredholm integral

operator mapping from L2[a,b] into L2[a,b].

Then А is completely continuous.

41


Bb

42

The integral operator A is completely continuous (compact and continuous) while

acting from ],[2 baL into ],[2 dcL


Let us suppose that the inverse operator is

continuous. It is very easy to arrive to a

contradiction. If A is an injective operator

then an inverse operator exists. Evidently, if

an operator B is continuous and an operator

A is completely continuous then the

operator BA is completely continuous.

43


44

If inverse to A operator is bounded then

any bounded sequence is compact! It is not

true in infinite-dimensional spaces.


Infinite-dimensional range of a completely

continuous operator R(A) is not closed! It is

easy to prove using Banach’s Theorem. If

the injective linear operator A is

continuously mapping an infinite-

dimensional Banach space on an infinite-

dimensional Banach space then the inverse

operator is contnuous.

45


46

Definition of the regularizing

algorithm

47


algorithm

48


algorithm

A problem of solving an operator equation

is called to be regularizable if there exists at

least one regularizing algorithm. Directly

from the definition it follows that if there

exists one regularizing algorithm then

number of them is infinite.

49


algorithm

At the present time, all mathematical

problems can be divided into following

classes:

well-posed problems;

ill-posed regularizable problems;

ill-posed nonregularizable problems.

50


algorithm

Not all ill-posed problems are regularizable,

and it depends on a choice of spaces Z, U.

Russian mathematician L.D. Menikhes

constructed an example of an integral

operator with a continuous closed kernel

acting from C[0,1] into L2[0,1] such that an

inverse problem is nonregularizable.

51


algorithm

52


algorithm

53


algorithm

54


algorithm

55


algorithm

The next very important property of ill-

posed problems is impossibility of error

estimation for a solution even if an error of

a right-hand side of an operator equation or

an error of an argument in a problem of

calculating values of an operator is known.

This basic result was also obtained by A.B.

Bakushinsky for solving operator equations.

56


algorithm

57


algorithm

From the definition of the regularizing

algorithm it follows immediately if one

exists then there is infinite number of them.

While solving ill-posed problems, it is

impossible to choose a regularizing

algorithm that finds an approximate solution

with the minimal error.

58


algorithm

It is impossible also to compare different

regularizing algorithms according to errors

of approximate solutions. Only including a

priori information in a statement of the

problem can give such a possibility, but in

this case a reformulated problem is well-

posed in fact. We will consider examples

below.

59


algorithm

Regularizing algorithms for operator

equations in infinite dimensional Banach

spaces could not be compared also

according to convergence rates of

approximate solutions to an exact solution

as errors of input data tend to zero. The

author of this principal result is V.A.

Vinokurov.

60

61

Consider the results obtained by Vinokurov.

Let be a linear continuous injective operator

acting in Banach space and the inverse operator

. be unbounded on . Suppose that

. is an arbitrary positive function such that

. as , and is an arbitrary method

to solve the problem.

The following equality holds for elements except

maybe for a first category set in :

A uniform error estimate can only exist on a first

category subset in .

AZ

1A 1AD

R

z

Z

0 0

zR ,,suplim

0

Z


algorithm

62


algorithm

63

Ill-posed problems on compact

sets

64


sets

65


sets

Tikhonov (1943) “On stability of inverse

problems” (Soviet Mathematics Doklady)

66


sets

67


sets

68


sets

69


sets

70


71


72

Solvability of an extremal

problem

73


problem

74


problem

75


problem

76


problem

77


problem

78

Convex functionals

79

Convex functionals

80

Convex functionals

A classification of extremal problems:

81

Convex functionals

82

Convex functionals

83

Convex functionals

84

Convex functionals

Theorem. If (*) is a convex programming

problem then any point of a local minimum

is a point of the global minimum.

Convex functionals have no local minima.

85

Convex functionals

86

Solvability of a convex

programming problem

Theorem. A convex continuous (lower

semicontinuous) functional f(x) on a closed

bounded convex set of a Hilbert space H

has a minimum point (i. e. a convex

programming problem (*) is solvable).

Theorem (Weierstrass). A weakly lower

semicontinuous functional on a weak

compact has a minimum point.

87


programming problem

We need to prove two assertions only:

1. A convex continuous (lower

semicontinuous) functional is weakly lower

semicontinuous.

2. A closed bounded convex set of a

Hilbert space is a weak compact.

The theorem is true for reflexive Banach

spaces only.88


programming problem

89


programming problem

90


programming problem

91

Criteria of convexity and

strong convexity

92

Criteria of convexity and

strong convexity

93

Least square method.

Pseudoinversion

94


Pseudoinversion

95

System of normal equations


Pseudoinversion

96


Pseudoinversion

97


Pseudoinversion

98

Pseudoinverse to A operator


Pseudoinversion

99


100


101


102


103


104


105


106



107



108


109


110


111


112


113


114


115


116


117


118


119

Newton’s method

120

Newton’s method

121

Newton’s method

122

Newton’s method

123

Zero-order methods

Coordinate descent.

Hooke-Jeeves method.

Nelder-Mead method.

Random search.

Genetic algorithms.

124


125


126


127

Projection of conjugate

gradients method

128


gradients method

129


gradients method

130


gradients method

131


gradients method

132


gradients method

If we have linear constraints of inequality

type, the method can be applied also. It is a

bit more complicated.

133



134



135



136



137

Bounded monotonic functions

Let is a set of monotonic nonincreasing on

[a, b] functions is .

Lemma. is a compact in Lp[a,b], p>1.

The Lemma guarantees convergence of

quasisolutions (or η-quasisolutions) to the

exact solution in Lp[a,b], p>1.

138

Bounded monotonic functions

Theorem (Goncharsky, Yagola). If the exact solition of the

operator equation is a nonincreasing bounded monotonic

continuous function then quasisolutions (or η-

quasisolutions) converge to the exact solution uniformly

on any segment . If the exact solution is

piecewise continuous then the convergence is uniform on

any segment that doesn’t include the points of

discontinuity of the exact solution (so called piecewise

uniform convergence).

139



140



141



142



143



144



145



146



147

148

Ill-posed problems with a source-wise

represented solution

(1)

is a linear injective operator.

Assume the next a priori information: is

sourcewise represented with a linear compact

operator :

(3)

Here is a reflexive Banach space.

Suppose is injective, is known exactly, .

uzA

UZA :

z

ZVB :

vBz

V

B A uu

149

Set and define the set

Minimize the discrepancy on .

If , then the solution is

found. Denote . Otherwise, we change

to and reiterate the process.

If is found, then we define the approximate

solution of (1) as an arbitrary solution of the

inequality

1n

nvVvBvzZzZn ,,:

uAzzF

nZzuAz :min

nZ

n

1n

nn

n

nz

nZzuAz

150

Theorem 1: The process described above converges: . . There exists (generally speaking, depending on ) such that for . Approximate solutions strongly converge to . as .

Proof The ball is a bounded closed set in . The set is a compact in for any , since is a compact operator. Due to Weierstrass theorem the continuous functional attains its exact lower bound on .

Clearly, , where

. is the integer part of a number.

n 00

z 0 nn 0,0

nz

z 0

nvVvVn :

VnZ Z n

B zF

nZ

NZvBz

otherwise1

integerpositiveais

v

vvN

151

Therefore is a finite number and there is such that for any . The inequality for any is evident. Thus, for all the approximate solutions . belong to the compact set , and the method coincides with the quasisolutions method for all sufficiently small positive . The convergence follows from the general theory of ill-posed problems.

Remark 1: The method is a variant of the method of extending compacts (Ivanov, Dombrobskaya).

n0

0 nn 0,0 Nn 0

0,0

nz 0nZ

zzn

152

Theorem 2: For the method described above there exists an a posteriori error estimate. It means that a functional exists such that as and at least for all sufficiently small positive .

Remark 2: The existence of the a posteriori error estimation follows from the following. If by . we denote the space of sourcewise represented with the operator solutions of (1), then . Since is a compact set, then . is a -compact space.

,u 0, u

0 ,uzzn

ZZ

B

1n nZZ nZ

Z

153

An a posteriori error estimate is not an error estimate

in general meaning that is impossible in principle

for ill-posed problems. But it becomes an upper

error estimate of the approximate solution for

“small” errors , where depends on the

exact solution .0 0

z

154

155

The operators and are known with errors. Let

there be linear operators , such that

. , . Denote the vector of errors

by . For any integer define a

compact set .

Find a minimal positive integer number such

that the inequality

has a nonempty set of solutions.

Then the a posteriori error estimation is

A B

AhABhB

Ah hAAA

Bh hBBB

BA hh ,, n nvVvvBzZzZ

BB hhn ,,:,

nn

nhhAhBhuzA BAhBhAh ABA

}

,:max{,,, ,

nhhAhBhuzA

ZzzznhBAu

BAhBhAh

hnnBhh

ABA

BBA

156

A posteriori error estimation

For some ill-posed problems it is possible to find a

so-called a posteriori error estimation.

Let be an exact injective operator with closed

graph and be a -compact space.

Introduce a function such that . , , , :

The function is an a posteriori error

estimation for the problem (1), if as

A

Z ,u Zz

0 z ],0( z Uu uu

,, uuRz

,u

0, u

0

157

Tikhonov’s regularizing algorithm

Let be Hilbert spaces, be a closed

convex set of a priori constraints such that ,

. , be linear operators. On a set

introduce the Tikhonov's functional:

where is a regularization parameter.

(2)

For any , and bounded linear operator

. the problem (2) is solvable and has a unique

solution .

UZ , ZD

D0

A hA ,,uAh

22zuzAzM h

0

DzzM :inf

Uu 0

hA

Dz

158

A priori choice of

A regularizing algorithm using the extreme

problem (2) for : to construct

such that as .

If is an injective operator, and ,

. as , then as ,

i.e., there is the a priori choice of .

zM zz

0

A Dz 0

0

2

h0 zz

0

159

Generalized discrepancy principle

(a posteriori choice of )The incompatibility measure of (1) on :

Let it can be computed with an error , i.e.,

instead of there is such that

The generalized discrepancy:

The generalized discrepancy is continuous and

monotonically non-decreasing for .

D

DzuzAAu hh :inf,

0

hAu , hAu ,

hhh AuAuAu ,,,

222

, hh AuzhuzA

0

160

The generalized discrepancy principle to choose the

regularization parameter:

1) If the condition is not just,

then is an approximate solution of (1);

2) If the condition is just,

then the generalized discrepancy has a positive

zero and .

If is an injective operator, then .

Otherwise, , where is the normal

solution of (1), i.e., .

222, hAuu

0z

222, hAuu

**

zz

A zz

0lim

*

0lim zz

*z

uAzDzzz ,:inf*

161

If are bounded linear operators, is a closed

convex set, , , the generalized

discrepancy principle are equivalent to the

generalized discrepancy method:

find

inf

hAA, D

D0 Dz

222,,: hh AuzhuzADzz

162

Inverse problem for the heat conduction

equation

There is a function , we want to

find such that

as .

We can write that

0,

0,0

,0,02

tlw

tw

Tltxwaw xxt

lLTwu ,0, 2

lWxwxz ,00, 12 xzxz

0

l l

dxx

xzxzxzduu

0 0

22222

,

163

The problem may be written in the form of integral

equation

where is the Green function:

The problem is solved for the parameters

. , the function is taken

such that .

l

dxxzTxGu0

,,

txG ,,

1

2

2 expsinsin,,n

lt

l

na

l

nx

l

ntxG

0.1,1.0,0.1 lTa u

u 05.0

164

The exact solution ( ) and the approximate

solution ( ).

)(xz xz

165

The Euler equation

The Tikhonov's functional is a strongly convex functional in a Hilbert space.

The necessary and sufficient condition for to be a minimum point of on a set of a prioriconstraints is

If is an interior point of , then , or

We obtain the Euler equation.

zM

z

zM D

DzzzzM

0,

z D 0

zM

uAzzAA hhh

**

166

Error-free methods

As the first example we consider so-called the “L-

curve method” (P.C. Hansen). In this method the

regularization parameter in Tikhonov functional

is selected as a point maximum curvature of the L-

curve {(ln||Ahz - u||, ln||z||): 0}.

But this method cannot be used for the solution of

ill-posed problems because the L-curve doesn’t

depend on h and (see the theorem). Everybody

can easily prove that this method is inapplicable to

solving the simplest finite-dimensional well-posed

problems (e.g., equation z=1).

167

Another very popular “error free” method is GCV – the

generalized cross-validation method (G. Wahba), where

(Ah, u) is found as the point of the global minimum

of the function

G() = ||(AhAh* + I)-1u|| [tr(AhAh

* + I)-1]-1, 0.

This method is not applicable for the solution of ill-

posed problems including ill-posed systems of linear

algebraic equations (see the theorem above). It is

possible construct well-posed systems of linear

algebraic equations the GCV method failed for their

solution. Let Z = U = R2,

21

11,

1

2Au

168

Here h 0. Very easy to calculate the GCV solution

zgcv and prove that it converges to (-1/3, -1/3)* instead

of ze = (-3, 1)* when h 0.

A lot of other examples could be found in a paper

by

V. Titarenko and A. Yagola (2000) Vestnik

Moskovskogo Universiteta, ser. 3. Fizika.

Astronomia (4), 15 (in Russian).

169

Publications on error-free methods

A.S.Leonov and A.G.Yagola. Can an ill-posed

problem be solved if the data error is unknown? -

Moscow University Physics Bulletin, v.50, N 4, 1995,

pp.25-28.

C.R. Vogel. Non-convergence of the L-curve

regularization parameter selection method. - Inverse

Problems, 1996, v.12, pp. 535-547.

A.G.Yagola, A.S.Leonov, V.N.Titarenko. Data errors

and an error estimation for ill-posed problems. –

Inverse Problems in Engineering, 2002, v. 10, N 2,

pp. 117-129.

Incompatible ill-posed problems

170

171

172

173

174

175

176

177

Numerical methods for the solution of

Fredholm integral equations of the first

kind

178

179

180

181

Finite-dimensional approximation of Tikhonov’s functional

182

Convolution type equations

183

184

185

186

187

188

189

190

191

192

193

194

195

Examples and Applications

196

Functions convex along lines parallel to

coordinate axes

Consider an n-dimensional Euclidean space Rn, n < .

A set Rn is convex along all lines parallel to coordinate axes if i [1,n] x1,x2 such that

x1=(a1,…, ai-1, x1i, ai+1,…, an),

x2=(a1,…, ai-1, x2i, ai+1,…, an)

and (0,1): x3= x1 +(1-) x2.

197

A cross is an example of a set convex along

coordinate axes

198

A function z(x) on is convex downwards

along all lines parallel to an i-th coordinate

axis if x1,x2 such that

x1=(a1,…, ai-1, x1i, ai+1,…, an),

x2=(a1,…, ai-1, x2i, ai+1,…, an)

and (0,1):

z( x1 +(1-) x2 ) z(x1) + (1-) z(x2)

199

Let n* [0,n]. Consider functions z(x) given on .

By Mn*n ( ) define the set of functions z(x) that

are convex downwards along all lines parallel to

n* first coordinate axes and convex upwards along

all lines parallel to (n-n*) last coordinate axes.

Assume there exist finite numbers CL and CU such

that x and z(x) Mn*n( ): CL z(x) CU.

200

201

202

Theorem 1.: Let there be a sequence {zm} and an element z such that m1,…,+: zm Mn

n* (), z Lp(), p>1, ||zm-z||Lp() 0 as m + , where is an open bounded set. Then from the sequence {zm} a subsequence {zm

(k)} may be taken that converges to a function žMn

n* () at any point of and ž=z in Lp( ).

Corollary 1.: Mnn* ( ) is a compact set in Lp( ).

Corollary 2.:The sequence {zm(x)} considered in

Theorem 2.1 converges to the function ž(x) at any point of .

203

Theorem 2.: Let ||zm - z||Lp() 0 as m,

where zm , zMnn* (), p 1 and is an

open bounded set. Then the sequence {zm }

converges to z uniformly on any closed set

.

204

Let D=[a1,b1] [a2,b2] … [an, bn]. On each segment [ai ,bi], we define a grid Xi={xi

j}j=1ni such

that ai = xi1 < xi

2 <…< xini = bi. Let X=X1 X2 …

Xn. A vector of indices J=(j1, j2,…, jn) for a grid point with coordinates (x1

j1,x2j2,…,xn

jn). Then the point is written as xJ.

For any xD there is a set BJ=[x1j1, x1

j1+1]… [xn

jn, xnjn+1]: x BJ. As an approximation of a

function z(x) we use a function zN (x) that is linear on grid values of z(x) at vertices of BJ.

205

After finite dimensional approximation we

obtain a set ŽM which is a polytope.

If x1, x2, x3 are grid points that belong to a line

parallel to an i-th coordinate axis and there

is no another grid point between them, then

for a uniform grid Xi: -z1+ 2z2 - z3 0 (in*)

or z1 - 2z2 + z3 0 (otherwise). (zk = z(xk))

206

Error estimation

1) Find the minimum and the maximum values foreach coordinate of ŽM

. Denote them by zli and

zui, 1 i n. They form vectors žl, žu.

2) Secondly, using žl, žu we construct functions zl(x)and zu(x) close to ZM

such that zZM: zl(x)

z(x) zu(x).

Therefore, we should minimize a linear functionon a convex set. We may approximate the set bya convex polyhedron and solve a linearprogramming problem. The simplex-method orthe method to cut convex polyhedra may be used.We also may construct the sequence W0 W1… Wm of convex polyhedrons contained thepoint of minimum.

207

Let D=[0,d1 ] [0,d2 ], d1, d2<+, and for w(x,y,t) there are the heat conduction equation and zero boundary conditions:

Denote z(x,y)=w(x,y,0), u(x,y)=w(x,y,T), 0<T<+ . Therefore

u(x,y)=G(x, y,,, T) z(,) dd.

0),,(0),0,(

0),,(0),,0(

2

1

2

2

2

22

tdxwtxw

tydwtyw

y

w

x

wa

t

w

208

Assume the exact solution zM20 (D). We set n1 =

n2 = 11, d1 = d2 = 1.0, the grids are uniform, a = 1.0, T=0.001. As the exact solution the function z(x,y) = sin( x) · sin( y) is taken. The approximate right-hand side we take as u = ū. The error of finite dimensional approximation = 0.01· ||ū|| 0.005.

In the figure there is an upper function zU(x,y) that bounds all approximate solutions. To construct it we use additional grid values.

We find that ||zU-zL|| = 0.212 ( 0.424 · ||z||).

209

Linear ill-posed problems on sets

of convex functions on two-dimensional sets

This case is more complicated. It is possible to prove that

the set of bounded convex 2D functions is a compact in

Lp( ). A set Rn is closed bounded set. So we can

find a quasisolution and its error estimation.

The detailed description of the algorithm so as the

algorithm of the previous paragraph can be found in our

joint publications with Valery Titarenko.

210

211

1

3

54 h

0

z

2

Fig.1

Electron microscopy

212

z

z'

1

0

2

Fig.2

213

отδ

IAφ , (1)

where

Aφ= ρr

zρdρφγρK0

z)()z,( ,

>0 – error of assignment of the right part of the equation

(1), i.е. δII relrel

, rel

IφA , (0)III rel - relative

intensity.

214

2

2

AL

IφφFδ

, (2)

At that it is enough to find such an element δ

φ , that

2δφFδ .

γ

γ

ρzотjρziij

N

j

N

ihIhφKφf

1 1

2

)( (3)

At finite-difference approximation set Z transforms

into set

ρzi

ρziii

iii

Niφ

NNi

Ni

φφφ

φφφ

φZinfl

infl

,1,2,0,

1,1,

1,2,

0,2

0,2

:ˆ11

11

,

(4)

215

Let )( jT , mj ,0,1, ( ρzNm ) – apexes of a convex

limited polyhedron Z .

Lemma. Let Zφ . Then the unique representation is correct

m

j

jjTaφ

1

)( ,

at that m,,,j,aj

210 .

216

It is obvious, that mm TT

RZ and ZR 1 ,

Where mR - set of vectors mm RR , that have all non-

negative coordinates mξ R , if mj,ξj

,1,2,0 .

Let us examine function )()( ξTfξY , determined on

set mR .

We need to find such an element mδξ R , that

2)( δξYδ . The approximate solution of the original

problem is found then by the formula δδ

Tξφ .

Let us examine operator T from mR in mR ,

determined by the formula

m

j

mjj

Rξ,TξξT1

)(.

217

0,0 0,5 1,0

0,1

0,2

(

Z)

Z

218

0,00

0,05

0,10

0,15

0,20

0,25

0,0 0,5 0,9

exact

calculated

219

0,0 0,5 1,00,0

0,1

0,2

(

Z)

Z

220

INVERSE PROBLEM OF

CATHODOLUMINESCENCE

MICROTOMOGRAPHY

221

The Scheme of Installation

1. Focused electrical probe

2. Object under investigation

3. Region of generation of nonequilibrium carriers

4. Ellipsoidal mirror

5. Diaphragm with detector

222

Problem

Develop method for determination of optoelectrical local

properties of cathodoluminescence objects with resolution of

micrometer part, having at our disposal the set of measurements

of intensity values.

Describe the scheme of experiment, mathematical statement and

the method of solution of the problem, which is ill-posed.The interaction of focused electrical probe with

cathodoluminescence substance was modulated. An alternative

method of microtomography in cathodolumenscence mode is

presented. The solution is based on confocal ellipsoidal

mirror [Phang J.C.H, Chan D.C.H.].

The photon rays transport in luminescence volume of

specimen and ellipsoid are calculated.

223

We have to solve the next inverse problem:define the internal quantum yield of the material

from Fredholm integral equation of the first kind:

],0[),( 0Rss

mirrorthetorespectin

objecttheofdeflectionthexdsssxKxIR

0

01 ,)(),()(

where - intensity, measured in experiment, as function of deflection

of the object in vertical direction,

-the distance from the surface of the object,

-maximal depth of penetration of electrons into the object,

-some continuous function, which was calculated by numerical

methods (the physical sense of is that is the contribution

into the total intensity the layer with center on the depth s and thickness

ds).

)(xI

0R

),(1 sxK

dssxK ),(1

s

],0[)( 0Rs ],[)( maxmin2 xxLxI

224

A Priori Information

Let it is known that the solution of the problem is sourcewiserepresented with help of completely continuous integral operator:

0

0

002 ],0[,)(),()(

R

RsdsKs

otherwise

sssK

,0

,2,2/2/1)cos((),(2

where

We shall consider that:

)(0 s ],0[ 02 RL ],0[)( 020 RLs

For solving the problem under such a priori information the method of extending compacts, which was described above, is used.

225

Model Calculations Results

226

227

An Inverse Problem of Nuclear Physics

228

An inverse problem of nuclear physics

Experiment:

Fig.1: 1 - the target for producing bremsstrahlung beam, 2 - the sample under consideration, D –

detector.

Passing through the first target the accelerated electrons produce the bremsstrahlung

beam (γ-rays). The bremsstrahlung spectrum is continuous. The sample 2 is bombarded

by the γ-rays. The scattered γ-rays are detected.

γe−

γ

1 2

D

229

Experimental data processing

Nuclear reaction:

Constraints:

A priori :

A posteriori:

is a monotone nondecreasing function

is a convex upwards function

is a monotone nonincreasing function

63 62

29 29Cu Cu n

0 ( ) 90, [10,24.1]E E

( ), [10,16]E E

( ), [16,18]E E

( ), [18,24.1]E E

230

Fig.2: (• • •) – the approximate cross section

from the Center of Data of Photonuclear

experiments (http://depni.sinp.msu.ru/cdfe/);

(• • •) – the approximate solution found by

Tikhonov regularization;

( – ) – the functions

bounded the set of approximate solutions

from below and from above .

( ), ( )low upper

E E

231

Experimental data processing

Nuclear reaction:

Constraints:

A priori:

A posteriori:




is a convex downwards function




34 33

16 15S P p

0 ( ) 45, [12.3,25.3]E E

( ), [12.3,16]E E

( ), [16,17]E E

( ), [17,18.5]E E

( ), [18.5,20]E E

( ), [20,22]E E

( ), [22,23]E E

( ), [23,25.3]E E

232

Fig.3: (• • •) – the approximate cross section

from the Center of Data of Photonuclear

experiments;

(• • •) – the approximate cross section found

by Tikhonov regularization;

( ─ ) – the functions

bounded the set of approximate solutions

from below and from above .

( ), ( )low upper

E E

233

Image reconstruction for gravitational lens

The system QSO 2237+0305, known as the

“Einstein Cross”: 4 quasar images against

the background of the lensing galaxy.

Several observation were carried out using

the Huble Space Telescope and Nordic

Optical Telescope.

Model of Kernel

ResidualsStar Kernel

PSF (Kernel) profile

Approximation of the star from

the frame with 2-dimensional

Gauss profile

FWHM ~ 5 pixels

Tikhonov Regularization

• Ill-posed problem

• Smoothing function:

)(*][2

zzkzMU

u

• Regularization parameter from discrepancy principle:

0,*

Uuzk

}:][inf{][ ZzzMzM

• Solution : z

A priori information

True Image = Galaxy + Quasar Components

K

kkbykaxkIyxgyxz

1

),(),(),(

K=4 , number of quasar components

K=5 , number of quasar components + galaxy nuclear


• Galaxy model 2

model)(

Gggg

• Nonnegativity of the solution, zij 0

• Galaxy: assumption about smoothness

}exp{01

modeln

en r/r b)I((r)g

bn=2n-0.324 for 1n4

generalized de Vaucouleurs profile (Sersic’s model)

; 2

)(G

gg BVWLG ,21,2


Sourcewise representation:

'*]'[ zrzRz

rsk *

PSF * PSF PSF FinalSourceTotal

),(),(),(1

yxgcybxrayxzK

kkkk

Results: L2

2

2)(

Lsersicggg

Observed image Reconstructed image

Results: L2

Galaxy Error distribution

Results: W21

Observed image Deconvolved image

2

21)(

Wsersicggg

Results: W21

Galaxy Error distribution

Results: MCS


2

2)(

Lgrgg

rs *kernel

Results: MCS

Quasar components Galaxy

Results: MCS

Quasar components Error distribution

Results: TV

11

1

12

1,1,,11,1

)(

N

m

N

nnm

gnm

gnm

gnm

gg


Results: TV

Quasar components Galaxy

Using parallel computing for solving multidimensional ill-

posed problems

248

249

1. Introduction

J

jij

i

ij

ijijj

i

r

M

r

rrMB

135

0)(3

4

iB

?jM

The equation describing the magnetic field Bi induced by sources of magnetic fields

Mj, located at a distance rij from the sensor i, is defined as

The inverse problem is to identify the permanent magnetization M (both strengths

and directions) using measurements of the magnetic flux density B.

1.1. An example of a multi-dimensional ill-posed problem

250

1. Introduction

35

0 )()(

))(),((3

4)(

qr

qMqr

qr

qMqrrBq

The total field of the ship can be expressed by the

integral

In other words, the source of the magnetic field with

magnetic moment M, located at a point with radius

vector q, creates at point with radius vector q a

magnetic field with induction B:


1. General mathematical formulation of the problem

q

V

dVqr

qMqr

qr

qMqrrB

35

0 )()(

))(),((3

4)(

But this statement of the problem is computationally

very difficult

1. Introduction1.1. An example of a multi-dimensional ill-posed problem

1. Some different types of simplifications of the numerical model for the assigned problem

But all these simplifications can be applied only in specific cases (not in general)251

1D-problem: Dividing the ship into subdivisions with

constant values of magnetization

2D-problem: Approximation the hull of the ship

by an ellipsoid


by polyhedrons


by a plane

252

1. Introduction

But if we do not have any a priori information about the investigated object we can

not use mentioned simplifications. In this case we have to solve the problem “in

general” that is very difficult to perform on common PC.

The answer is only one: we have to use parallel

computing

For example, how can 67 500 parameters be inverted

efficiently?


3. Inverse problem without any simplifications

253

2. Using parallel computing

Parallel computing is a form of computation in which many calculations are carried

out simultaneously, operating on the principle that large problems can often be

divided into smaller ones, which are then solved concurrently (“in parallel”). Parallel

computation can be performs on multi-processor clusters or on multi-core

computers having multiple processing elements within a single machine.

But not every problem can be parallelized efficiently

Large problem

Sub problem 1

Su

b p

rob

lem

N

Solving of smaller

problems

Process 1

Process 2

Process 3

Process N

. . . .

. .

Result 1

Result 2

Result 3

Result N

. . . .

. .

Result of solving

of the large

problem

Result of intermediate

calculationsResult

2.1. The main idea of parallel computing

254

2. Using parallel computing

It will be shown that parallelizable fraction for multidimensional Fredholm integral

equation of the 1st kind is ~ 99,(9)% that gives us high effectiveness

2.2. Parallel computing limitations

The speed-up of a program as a result of

parallelization is observed as Amdahl’s

law

N

PP

S

)1(

1

S -- the speed-up of the program (as a factor

of its original sequential runtime)

N -- number of processors

P -- the fraction of the program that is

parallelizable.

255

3. Parallelization of multidimensional ill-posed problems

The total field of the ship expressed by the integral

can be replaced by an equivalent 3D integral equation

V

ssssss dvzyxMzyxzyxKzyxB ),,(),,,,,(),,(

3.1. Formulation of the problem

x

x

y

y

z

z

R

L

R

L

R

L

dxdydzzyxMzyxrtsKrtsB ),,(),,,,,(),,(

q

V

dVqr

qMqr

qr

qMqrrB

35

0 )()(

))(),((3

4)(

and then, after change of variables,

256

m

iii

m

iiiN

l

N

l

N

l

n

jjj

m

m

lll

nm

llljjjzyx

N

j

N

j

N

j

nm

iiijjj

n

rtszyx

m

iii

m

iiim

iii

M

MBMKhhhKhhhhhh

M

MFMFgrad

x y zs t r

321

321

1 2 3

321321321321

1 2 3

321321

321

321

321

1 1 1

3

11 1 1

3

1

2

)(

When we solve minimization problem by conjugate gradient method, it is necessary to

calculate values of the functional MF

MFgrad

and its gradient

Finite-difference approximation of the Tikhonov functional is

Finite-difference approximation of its gradient is

Structure of algorithm allows to divide the large problem into smaller ones which

are then solved “in parallel”.

s t r x y zN

j

N

j

N

j n

N

i

n

jjj

N

i

N

i

m

iii

nm

iiijjjzyx

m

rts MBMKhhhhhhMF1 1 1

3

1

2

1 1 13

3

11 2 3 1

321

2 3

32121321

3.2. Finite-difference approximation of the functional and its gradient

3. Parallelization of multidimensional ill-posed problems

257

Paralleling of the functional is consider on the example

rtszyx hhhhhh ,,,,, are skipped for simplicity.

2 rtszyx NNNNNN ,

steps and smoothing functional

22

1

32

1

2

13

3

222

3

1

22

1

32

1

2

13

3

212

3

1

22

1

22

1

2

13

2

111

3

1

22

1

12

1

2

13

1

111

3

1

2

1

2

1

2

1

3

1

22

1

2

1

2

13

3

1

1

321

2 3

32121

1

321

2 3

32121

1

321

2 3

32121

1

321

2 3

32121

1 2 3 1

321

2 3

32121321

i

jjj

i i

m

iii

m

iii

m

i

jjj

i i

m

iii

m

iii

m

i

jjj

i i

m

iii

m

iii

m

i

jjj

i i

m

iii

m

iii

m

j j j n i

n

jjj

i i

m

iii

nm

iiijjj

m

BMK

BMK

BMK

BMK

BMKMF

Process 1

Process 2

Process 18

Process 24

. . . .

Result 1

Result 2

Result 18

Result 24

Aggregate

result

. . . .

+

. . . .

++

. . . .

++

3. Parallelization of multidimensional ill-posed problems3.3. One possible scheme of paralleling I

258

The scheme of calculating the value of the functional for a) zero process, b) non-

zero processes.

rts NNNN

i = 1

s

i = i + 1

i = ”number of

processes”

s(i) = ”square

of sum”

i = i + ”quantity

of processes”- 1

a) b)

][][ MMF

Ni Ni

MM

M

sMFMF ][][

)(is

3. Parallelization of multidimensional ill-posed problems3.3. One possible scheme of paralleling II

259

zyx NNNN

The scheme of calculating value of the gradient of the functional for: a) zero

process; b) non-zero processes.

i = 1

grad = 0

s, k

grad(k) = s

i = i + 1

k = ”number of

process”

s(k) =

”gradient”

k =k + ”quantity

of processes”-1

M

a) b)

Ni Nk

)(ks

M

M

3. Parallelization of multidimensional ill-posed problems3.3. One possible scheme of paralleling III

260

Results of restoring 100x15x15x3=67500 magnetization parameters. Time of calculation is

~29 hours on 256 processors (Intel Xeon E5472 3.0 GHz)

3. Parallelization of multidimensional ill-posed problems3.4. Some examples of calculations

261

4. Conclusion

The proposed method can be efficiently applied for

solving multidimensional Fredholm integral equations

of the 1st kind in many areas of physics were it is

necessary to solve inverse problems such as:

• radiophysics

• optics

• acoustics

• spectroscopy

• geophysics

• tomography

• image processing

• etc.

The testing calculations were performed on the Computing Cluster of the Moscow State University.

The work was partially supported by RFBR grant 10-01-91150- NFSC.

Introduction to the theory, numerical methods and applications of ill-posed problems ... · 2016-05-06 · 1 Introduction to the theory, numerical methods and applications of ill-posed

Documents