MA5233: Computational Mathematics Weizhu Bao Department of Mathematics & Center for Computational Science and Engineering National University of Singapore.

MA5233: Computational Mathematics

Weizhu Bao

Department of Mathematics & Center for Computational Science and Engineering

National University of SingaporeEmail: [email protected]

URL: http://www.math.nus.edu.sg/~bao

Computational Science

Third paradigm for– Discovery in Science– Solving scientific &

engineering problems

Interdisciplinary– Problem-driven– Mathematical models– Numerical methods– Algorithmic aspects—

computer science– Programming– Software– Applications, ……

Dynamics of soliton in quantum physics

Wave interaction in plasma physics

Wave interaction in particle physics

Vortex-pair dynamics in superfluidity

Vortex-dipole dynamics in superfluidity

Vortex lattice dynamics in superfluidity

Vortex lattice dynamics in BEC

Computational Science

Computational Mathematics – Scientific computing/numerical analysis

Computational PhysicsComputational ChemistryComputational BiologyComputational Fluid DynamicsComputational EnginneringComputational Materials Sciences……...

Steps for solving a practical problems

Physical or engineering problems Mathematical model – physical laws

Analytical methods – existence, regularity, solution, …

Numerical methods – discretization

Programming -- coding

Results -- computing

Compare with experimental results

Computational Mathematics

Numerical analysis/Scientific computingA branch of mathematics interested in constructive methodsObtain numerically the solution of mathematical problemsTheory or foundation of computational science– Develop new numerical methods– Computational error analysis:

• Stability• Convergence• Convergence rate or order of accuracy,….

History

Numerical analysis can be traced back to a symposium with the title ``Problems for the Numerical Analysis of the Future, UCLA, July 29-31, 1948.Volume 15 in Applied Mathematics Series, National Bureau of StandardsBoom of research and applications: Fluid flow, weather prediction, semiconductor, physics, ……

Milestone Algorithms

1901: Runge-Kutta methods for ODEs1903: Cholesky decomposition1926: Aitken acceleration process

1946: Monte Carlo method1947: The simplex algorithm1955: Romberg method1956: The finite element method

1limif2

)( 1

12

21

aSS

SS

SSS

SSST

n

n

nnnn

nnnn

Milestone algorithms

1957: The Fortran optimizing compiler 1959: QR algorithm1960: Multigrid method1965: Fast Fourier transform (FFT)1969: Fast matrix manipulations1976: High Performance computing & packages: LAPACK, LINPACK – Matlab1982: Wavelets1982: Fast Multipole method

Top 10 Algorithms

1946: Monte Carlo method1947: Simplex method for linear programming1950: Krylov subspace iterative methods1951: Decompositional approach for matrix computation1957: Fortran optimizing compiler1959-61: QR algorithms1962: Quicksort1965: Fast Fourier Transform (FFT)1977: Integer relation detection algorithm1982: Fast multipole algorithm

http://amath.colorado.edu/resources/archive/topten.pdf

Contents

Basic numerical methods– Round-off error – Function approximation and interpolation– Numerical integration and differentiation

Numerical linear algebra– Linear system solvers– Eigenvalue probems

Numerical ODENonlinear equations solvers & optimization

Contents

Numerical PDE– Finite difference method (FDM)– Finite element method (FEM)– Finite volume method (FVM)– Spectral method

Problem driven research: – Computational Fluid dynamics (CFD)– Computational physics– Computational biology, ……

How to do it well

Three key factors– Master all kinds of different numerical methods– Know and aware the progress in the applied science– Know and aware the progress in PDE or ODE

Ability for a graduate student– Solve problem correctly– Write your results neatly– Speak your results well and clear – presentation– Find good problems to solve

Numerical error

Example 1:

Example 2:

Example 3:

Example 4:

!error! no321

!!error! offround14159.414159.311

!error! Truncation2/1)cos(995.02/1.01)1.0cos( 22 xx

!error! off-roundTruncation2/1)cos(

945.02/110.012/333.01)3/1cos(2

2

xx

Numerical error

Truncation error or error of the method

Round-off error: due to finite digits of numbers in computer

Numerical errors for practical problems– Truncation error– Round-off error– Model error & observation error & empirical error etc.

)O(|x||R(x)|xxPx 42 error Truncation2/1)()cos(

....0000026.014159.314159.3 R

Absolute error

Absolute error:

Absolute error bound (not unique!!):

....003333.033.03/1*33.03/1

....00159.014.3*14.3

***

e

e

xxexx

0.002or 0.0016*3.14

0.5mm* mm minimum ruler with withlength measure

*******|*||*|

xxxxxxxe

Relative error

An example:

Relative error:

Relative error bound:

better? is ionapproximat which

5510001110 ** yx yx

*

*

*

* practical in,

** **

x

xx

x

ee

x

xx

x

ee rr

|*|

**

xr

%5.01000

5

|*|

*51000

%1010

1

|*|

*110

*

*

yy

xx

r

r

Absolute error bounds for basic operations

Suppose Error bounds

** with*, with* yx yyxx

1||

1||||//*/

1||

1||||*

|*)('|*)()(

|*|

|*||*|*/*/

|*||*|**

**

*

***

/

*

***

**)(

2

***

/

***

***

a

aaaxax

a

aaxaxa

xfxfxf

x

yxxyxy

yxyxyx

yxyx

x

xxax

x

xxax

xxf

xyxy

xyxy

yxyx

Significant digits

An example

Definition: n significant digits

Method:– Write in the standard form – Count the number of digits after decimal

digitst significan 614159.3

digitst significan 314.3

....1415926.3

mnmn

mn

x|xaaaxx

aaaax

1

21

121

105.0|* that such 10.0*

0 with10.0

Error spreading: An example

Algorithm 1: – Use 4 significant digits for practical computation

– Results

11

0

10

1

1

0

11

0

11

0

1

1

3,2,1,10

edxeeI

nIndxexnxeedxexeI

x

nxnnxxn

n

,3,2,1,1,1 11

0 nInIeI nn

,3,2,1,~

1~

,6321.0~

10 nInII nn

552.77280.02160.01120.01480.0~

98765

1704.02074.02642.03679.06321.0~

43210

n

n

I

n

I

n

!!!2!8,!,105.0 *0

*8

*0

*1

*4*0

nn nn

Error spreading: An example

Algorithm 2

– Result

– Truncation error analysis

!!!!!9/,/ *9

*9

*0

**1 nnn

0,8,9),ˆ1(1ˆ,0684.0

1010

1

2

1ˆ

10

1

10

1

1

9

1

0

911

0

919

1

0

911

nIn

Ie

I

dxexedxexeIdxxee

nn

x

0684.01035.01121.01268.01455.0ˆ98765

1708.02073.02642.03679.06321.0ˆ43210

n

n

I

n

I

n

Convergence and its rate

Numerical integration

Exact solution

kxkxxfdxxfI integer somefor )]sin(2[)( with)(0

k

k

k

kI

)cos()sin(2

2

Numerical methods

Composite midpoint rule

Composite Simpson’s rule

Composite trapezoidal rule

Error estimate

NihixhixN

h ii ,2,1,0,)2/1(,,0

2/1

)()()()( 2/12/122/112/1 NhM xfxfxfxfhII

0 1/2 1 1 1/2 2 2 1/2 1/2

1 2 1 2 1 2 2 1( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

6 3 3 3 3 3 3 6hS N NI I h f x f x f x f x f x f x f x f x

)(||),(||),(|| 422 hOIIhOIIhOII hS

hT

hM

)(

2

1)()()()(

2

11210 NN

hT xfxfxfxfxfhII

Numerical results

Numerical errors

Observations

Before h0

– Truncation error is too large !!

After h1

– Round-off error is dominated!!

Between h0 and h1

– Clear order of accuracy is observed for the method

We can observe clear convergence rate for proper region of the mesh size!!!

Numerical Differentiation

Numerical differentiation

The total error 21 )(

~)(,)(

~)(

2

)(~

)(~

2

)()()('

hxfhxfhxfhxf

h

hxfhxf

h

hxfhxfxf

)(:

)('''62

)('''62

)('2

)()()('

2

)(~

)(~

|*|

)('''6

)(''2

)(')()(

2

212

212

21

32

hEhAh

xfh

hxf

h

h

xfh

hxfhxfxf

h

hxfhxfe

xfh

xfh

xfhxfhxf

Numerical Differentiation

Numerical Differentiation

Total error depends – Truncation error: – Round-off error:– Minimizer of E(h):

– Double precision:

– Clear region to observe truncation error:

)( 2hO

Matlabin 10 16

02*)(''

)2

(*02)(' 3/12

AhEA

hhAh

hE

516 10*)1(,10 hOA

)1(10* 05 Ohhh

How to determine order of accuracy

Numerical approximation or method

How to determine p and C??– By plot log E(h) vs log h

ph

b

a

phh hCIIhEhOIIIdxxfI ||:)()(||)(

How to determine order of accuracy

– By quotation p

p

p

h

h

hC

Ch

II

II2

)2/(||

||

2/