EEGN403 – COMPUTER METHODS IN NUMERICAL ANALYSIS CSUF
EEGN403 – COMPUTER METHODS IN NUMERICAL ANALYSIS
CSUF
TEXTBOOK Numerical method for engineers
2
OUTLINE Motivation What Are Numerical Methods Why You Need to Learn Numerical Methods Course Topics Tools You Should Know History
Milestone Algorithms Top 10 Algorithms
Mathematical PreliminariesDerivatives Partial Derivatives 3
MATHEMATICAL PRELIMINARIES
Tangents and Gradients Functions as curves Review of Functions The Class of Polynomials Taylor’s Series Mean-Value Theorem Rolles Theorem Caveat
4
PRE-COMPUTER ERA computer era
Motivation
5
What Are Numerical Methods?
Techniques by which mathematical problems are formulated so that they can be solved with arithmetic operations {+,-,*,/} that can then be performed by a computer.
Why You Need to Learn Numerical Methods?
1. Numerical methods are extremely powerful problem-solving tools.
2. During your career, you may often need to use commercial computer programs (canned programs) that involve numerical methods. You need to know the basic theory of numerical methods in order to be a better user.
3. You will often encounter problems that cannot be solved by existing canned programs; you must write your own program of numerical methods.
4. Numerical methods are an efficient vehicle for learning to use computers.
5. Numerical methods provide a good opportunity for you to reinforce your understanding of mathematics.
You need that in your life as an engineer or a scientist.
COURSE TOPICS Mathematical Preliminaries Computer Representation of real numbers Algorithms for finding the roots of a function,
i.e., where f(x,y,z) = 0 Functional approximation via interpolation Numerical differentiation of functions.
8
COURSE TOPICS Numerical integration of functions
also termed quadrature Monte-Carlo and other randomized
techniques Numerical solution of linear systems
Motivation for algorithms Examples from EE related problems
9
TOOLS YOU SHOULD KNOW One of computer programs Matlab – a system for performing many
numerical tasks.
10
HISTORY Numerical analysis can be traced back to a
symposium with the title ``Problems for the Numerical Analysis of the Future, UCLA, July 29-31, 1948.
Volume 15 in Applied Mathematics Series, National Bureau of Standards
Boom of research and applications: Fluid flow, weather prediction, semiconductor, physics, ……
11
MILESTONE ALGORITHMS 1901: Runge-Kutta methods for ODEs 1903: Cholesky decomposition 1926: Aitken acceleration process
1946: Monte Carlo method 1947: The simplex algorithm 1955: Romberg method 1956: The finite element method
1limif2
)( 1
12
21
aSSSS
SSSSSST
n
n
nnnn
nnnn
12
MILESTONE ALGORITHMS1957: The Fortran optimizing compiler 1959: QR algorithm1960: Multigrid method1965: Fast Fourier transform (FFT)1969: Fast matrix manipulations1976: High Performance computing &
packages: LAPACK, LINPACK – Matlab1982: Wavelets1982: Fast Multipole method
13
TOP 10 ALGORITHMS
1946: Monte Carlo method 1947: Simplex method for linear programming 1950: Krylov subspace iterative methods 1951: Decompositional approach for matrix computation 1957: Fortran optimizing compiler 1959-61: QR algorithms 1962: Quicksort 1965: Fast Fourier Transform (FFT) 1977: Integer relation detection algorithm 1982: Fast multipole algorithm http://amath.colorado.edu/resources/archive/topten.pdf 14
DERIVATIVES Recall the limit definition of the first
derivative.
0
( ) ( )'( ) limh
dy f x h f xf xdx h
15
PARTIAL DERIVATIVES Same as derivatives, keep each other
dimension constant.
0
( , , ) ( , , )limxh
f f x h y z f x y zfx h
16
TANGENTS AND GRADIENTS Recall that the slope of a curve (defined as a
1D function) at any point x, is the first derivative of the function.
That is, the linear approximation to the curve in the neighborhood of t is l(x) = b + f’(t)x
f(x)
t
(1,f’(t))
x
y
17
TANGENTS AND GRADIENTS Since we also want this linear approximation
to intersect the curve at the point t. l(t) = f(t) = b + f’(t)t
Or, b = f(t) - f’(t)t We say that the line l(x) interpolates the
curve f(x) at the point t.
18
FUNCTIONS AS CURVES We can think of the curve shown in the
previous slide as the set of all points (x,f(x)). Then, the tangent vector at any point along
the curve is
, ( ) 1, '( )d x f x f xdx
19
SIDE NOTE ON CURVES There are other ways to represent
curves, rather than explicitly.Functions are a subset of curves
(x,y(x)).Parametric equations represent the
curve by the distance walked along the curve (x(t),y(t)).
Circle: (cos, sin) Implicit representations define a
contour or level-set of the function: f(x,y) = c.
20
TANGENT PLANES AND GRADIENTS In higher-dimensions, we have the same
thing: A surface is a 2D function in 3D:
Surface = (x, y, f(x,y) ) A volume or hyper-surface is a 3D function
in 4D:Volume = (x, y, z, f(x,y,z) )
21
TANGENT PLANES AND GRADIENTSThe linear approximation to the higher-
dimensional function at a point (s,t), has the form: ax+by+cz+d=0, or z(x,y) = …
What is this plane?
22
TANGENT PLANES AND GRADIENTSThe formula for the plane is rather
simple:
z(s,t) = f(s,t) - interpolatesz(s+dx,t) = f(s,t) + fx(s,t)dx = b + adx
Linear in dxOf course, the plane does not stay
close to the surface as you move away from the point (s,t).
( , ) ( , ) ( , )( ) ( , )( )x yz x y f s t f s t x s f s t y t
23
TANGENT PLANES AND GRADIENTS The normal to the plane is thus:
The 2D vector:
is called the gradient of the function. It represents the direction of maximal
change.
( , ), ( , ),1x yN f s t f s t
( , ), ( , )T
x yf s t f s t
24
GRADIENTS The gradient thus indicates the direction to
walk to get down the hill the fastest.
Also used in graphics to determine illumination.
( , ) ( , ), ( , )T
f ff x y x y x yx y
25
REVIEW OF FUNCTIONS Extrema of a function occur where f’(x)=0. The second derivative determines whether
the point is a minimum or maximum. The second derivative also gives us an
indication of the curvature of the curve. That is, how fast it is oscillating or turning.
26
THE CLASS OF POLYNOMIALS Specific functions of the form:
2 3 4 50 1 2 3 4 5
0
0 1 2 3 4 5
( )
( ( ( ( ( ) )))))
ii
i
p x a a x a x a x a x a x
a x
a x a x a x a x a x a
27
THE CLASS OF POLYNOMIALS For many polynomials, the latter coefficients
are zero. For example:p(x) = 3+x2+5x3
28
TAYLOR’S SERIES For a function, f(x), about a point c.
I.E. A polynomial
2 3
( )
0
( ) ( )( ) ( ) ( )( ) ( ) ( )2! 3!
( ) ( )!
kk
k
f c f cf x f c f c x c x c x c
f x x ck
29
TAYLOR’S THEOREM Taylor’s Theorem allows us to truncate this
infinite series:
( )
10
( 1)1
1
( )( ) ( )!
( ) ( )( 1)!
( ) ( , )
knk
nk
nn
n
f cf x x c Ek
fE x cn
where x c x
30
TAYLOR’S THEOREM
Some things to note:
1. (x-c)(n+1) quickly approaches zero if |x-c|<<12. (x-c)(n+1) increases quickly if |x-c|>>13. Higher-order derivatives may get smaller (for
smooth functions).
31
HIGHER DERIVATIVES What is the 100th derivative of sin(x)?
What is the 100th derivative of sin(3x)? Compare 3100 to 100!
What is the 100th derivative of sin(1000x)?
32
TAYLOR’S THEOREM Hence, for points near c we can just drop the
error term and we have a good polynomial approximation to the function (again, for points near c).
Consider the case where (x-c)=0.5
For n=4, this leads to an error term around 2.6*10-4 f()
Do this for other values of n. Do this for the case (x-c) = 0.1
( 1)
1 1
( ) 1( 1)! 2
n
n n
fEn
33
SOME COMMON DERIVATIVES
1( )
(sin ) cos
(cos ) sin
( )
1(ln )
n n
x x
d ax naxdxd x xdxd x xdxd e edxd xdx x
1( )
(sin ) cos
(cos ) sin
( )
1(ln )
n n
u u
d duau naudx dxd duu udx dxd duu u chain ruledx dxd due edx dxd duudx u dx
34
SOME RESULTING SERIES About c=0
2 3 4
1 2 2 3 3
1 2 2 3 3
2 3 4
12! 3! 4!
( 1) ( 1)( 2)( )2! 3!
1 2 31 1 1 1
1
x
n n n n n
n n n n
x x xe x
n n n n na x a na x a x a x
n n na a x a x a x
x x x x xx
35
SOME RESULTING SERIES About c=0
3 5 7
2 4 6
2 3 4 5
3 5 7
sin3! 5! 7!
cos 12! 4! 6!
ln(1 ) 1 12 3 4 5
1ln 2 1 11 3 5 7
x x xx x
x x xx
x x x xx x x
x x x xx xx
36
BOOK’S INTRODUCTION EXAMPLE Eight terms for first series not even yielding a
single significant digit. Only four for
second serieswith foursignificantdigits.
3 5 7
ln 2 0.6931471801 1 1 1 1 1 1ln(1 1) 12 3 4 5 6 7 8
0.63452
1 1 111 1 3 3 33ln 21 3 3 5 713
0.69313
37
MEAN-VALUE THEOREM Special case of Taylor’s Theorem, where n=0,
x=b.
Assumes f(x) is continuous and its first derivative exists everywhere within (a,b).
1( ) ( )( ) ( ) ( ) ( , )
f b f a Ef a b a f a b
38
MEAN-VALUE THEOREMSo what! What does this mean?Function can not jump away from
current value faster than the derivative will allow.
f(x)
a b secant 39
ROLLES THEOREM If a and b are roots (f(a)=f(b)=0) of a
continuous function f(x), which is not everywhere equal to zero, then f’(t)=0 for some point t in (a,b).
I.e., What goes up, must come down.
f(x)
a b
f’(t)=0
t40
CAVEAT For Taylor’s Series and Taylor’s Theorem to
hold, the function and its derivatives must exist within the range you are trying to use it.
That is, the function does not go to infinity, or have a discontinuity (implies f’(x) does not exist), …
41
REFERENCE Thanks….
42