ECIV 301 Programming & Graphics Numerical Methods for Engineers REVIEW II
ECIV 301 Programming & Graphics Numerical Methods for Engineers REVIEW II.
Dec 21, 2015
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ECIV 301
Programming & Graphics
Numerical Methods for Engineers
REVIEW II
Topics
• Introduction to Matrix Algebra• Gauss Elimination• LU Decomposition• Matrix Inversion• Iterative Methods
• Function Interpolation & Approximation• Newton Polynomials• Lagrange Polynomials
Matrix Algebra
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
Rectangular Array of Elements Represented by a single symbol [A]
Matrix Algebra
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
Row 1
Row 3
Column 2 Column m
n x m Matrix
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
Matrix Algebra
32a
3rd Row
2nd Column
Matrix Algebra
m321 bbbbB
1 Row, m Columns
Row Vector
B
Matrix Algebra
n
3
2
1
c
c
c
c
C
n Rows, 1 Column
Column Vector
C
Matrix Algebra
5554535251
4544434241
3534333231
2524232221
1514131211
aaaaa
aaaaa
aaaaa
aaaaa
aaaaa
A
If n = m Square Matrix
e.g. n=m=5e.g. n=m=5Main Diagonal
Matrix Algebra
9264
2732
6381
4215
A
Special Types of Square Matrices
Symmetric: aSymmetric: aijij = a = ajiji
Matrix Algebra
9000
0700
0080
0005
A
Diagonal: aDiagonal: aijij = 0, i = 0, ijj
Special Types of Square Matrices
Matrix Algebra
1000
0100
0010
0001
I
Identity: aIdentity: aiiii=1.0 a=1.0 aijij = 0, i = 0, ijj
Special Types of Square Matrices
nm
m333
m22322
m1131211
a000
aa00
aaa0
aaaa
A
Matrix Algebra
Upper TriangularUpper Triangular
Special Types of Square Matrices
nm3n2n1n
333231
2221
11
aaaa
0aaa
00aa
000a
A
Matrix Algebra
Lower TriangularLower Triangular
Special Types of Square Matrices
nm
3332
232221
1211
a000
0aa0
0aaa
00aa
A
Matrix Algebra
BandedBanded
Special Types of Square Matrices
Matrix Operating Rules - Equality
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
pq3p2p1p
q3333231
q2232221
q1131211
bbbb
bbab
bbbb
bbbb
B
[A]mxn=[B]pxq
n=p m=q aij=bij
Matrix Operating Rules - Addition
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
pq3p2p1p
q3333231
q2232221
q1131211
bbbb
bbab
bbbb
bbbb
B
[C]mxn= [A]mxn+[B]pxq
n=p
m=qcij = aij+bij
Matrix Operating Rules - Addition
Properties
[A]+[B] = [B]+[A]
[A]+([B]+[C]) = ([A]+[B])+[C]
Multiplication by Scalar
nm3n2n1n
m3333231
m2232221
m1131211
gagagaga
gagagaga
gagagaga
gagagaga
AgD
Matrix Multiplication
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
pq3p2p1p
q3333231
q2232221
q1131211
bbbb
bbab
bbbb
bbbb
B
[A] n x m . [B] p x q = [C] n x q
m=p
n
1kkjikij bac
Matrix Multiplication
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
pq3p2p1p
q3333231
q2232221
q1131211
bbbb
bbab
bbbb
bbbb
B
1nn13113
2112111111
baba
babac
11c
C
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
Matrix Multiplication
pq3p2p1p
q3333231
q2232221
q1131211
bbbb
bbab
bbbb
bbbb
B
3nn23323
2322132123
baba
babac
23c
C
Matrix Multiplication - Properties
Associative: [A]([B][C]) = ([A][B])[C]
If dimensions suitable
Distributive: [A]([B]+[C]) = [A][B]+[A] [C]
Attention: [A][B] [B][A]
nmm3m2m1
3n332313
2n322212
1n312111
T
aaaa
aaaa
aaaa
aaaa
A
nm3n2n1n
m3333231
m2232221
m1131211
aaaa
aaaa
aaaa
aaaa
A
Operations - Transpose
Operations - Inverse
[A] [A]-1
[A] [A]-1=[I]
If [A]-1 does not exist[A] is singular
Operations - Trace
5554535251
4544434241
3534333231
2524232221
1514131211
aaaaa
aaaaa
aaaaa
aaaaa
aaaaa
A
Square Matrix
tr[A] = tr[A] = aaiiii
Linear Equations in Matrix Form
10z8y3x5
6z3yx12
24z23y6x10
23610
3112
835
z
y
x
24
6
10 6
z
y
x
3112
24
z
y
x
23610
10
z
y
x
835
Gauss EliminationConsider
1035 yx(Eq 1)
5810 yx(Eq 2)
Solution
5.7
5.6
y
x
20610 yx2*(Eq 1)
5810 yx(Eq 2)
Solution
5.7
5.6
y
x!!!!!!
Scaling Does Not Change the SolutionScaling Does Not Change the Solution
Gauss EliminationConsider
20610 yx(Eq 1)
152 y(Eq 2)-(Eq 1)
Solution
5.7
5.6
y
x!!!!!!
20610 yx(Eq 1)
5810 yx(Eq 2)
Solution
5.7
5.6
y
x
Operations Do Not Change the SolutionOperations Do Not Change the Solution
Gauss Elimination
10835 zyx
2423610 zyx
6312 zyx
Example
Forward Elimination
Gauss Elimination
10835 zyx
24z23y6x10
zyx 835
5
1210
5
12
6312 zyx
-
305
81
5
310 zyx 302.162.60 zyx
Gauss Elimination
10835 zyx
24z23y6x10
6312 zyx 302.162.60 zyx
Substitute 2nd eq with new
Gauss Elimination
10835 zyx
24z23y6x10
302.162.60 zyx
zyx 835
5
1010
5
10-
439120 zyx
Gauss Elimination
10835 zyx
24z23y6x10
302.162.60 zyx
Substitute 3rd eq with new
439120 zyx
Gauss Elimination
10835 zyx
302.162.60 zyx
439120 zyx
zy 2.162.6
2.6
12 30
2.6
12-
064.62645.700 zyx
Gauss Elimination
10835 zyx
30970 zyx
Substitute 3rd eq with new
439120 zyx 064.62645.700 zyx
Gauss Elimination
Forward Elimination
064.62
30
10
645.700
2.162.60
835
z
y
x
Gauss Elimination
Back Substitution
118.8645.7/064.62 z
0502.26
2.6
118.82.1630
y
6413.0
5
118.880502.26310
x
064.62
30
10
645.700
2.162.60
835
z
y
x
Gauss Elimination – Potential Problem
10830 zyx
2423610 zyx
6312 zyx
Pivoting
6312 zyx
10830 zyx
Partial Pivoting
nn
nnnnn
lll
n
n
n
b
b
b
b
x
x
x
x
aaaa
aaaa
aaaaaaaa
aaaa
3
2
1
3
2
1
321
ln321
3333231
2232221
1131211
a32>a22
al2>a22
NO
YES
Partial Pivoting
nn
nnnnn
n
n
lll
n
b
b
b
b
x
x
x
x
aaaa
aaaa
aaaaaaaa
aaaa
3
2
1
3
2
1
321
2232221
3333231
ln321
1131211
Full Pivoting
• In addition to row swaping
• Search columns for max elements
• Swap Columns
• Change the order of xi
• Most cases not necessary
LU Decomposition
4.71
3.19
85.7
102.03.0
3.071.0
2.01.03
3
2
1
x
x
x
LU DecompositionPIVOTS
Column 1PIVOTS
Column 2
03333.0
1.0 02713.0
LU Decomposition
As many as, and in the location of, zeros
UpperTriangular
MatrixU
01200.1000
29333.000333.70
2.01.03
LU DecompositionPIVOTS
Column 1
PIVOTSColumn 2
LowerTriangular
Matrix
1
1
1
0
0
0
L
03333.0
1.0 02713.0
LU Decomposition
102713.01.0
0103333.0
001
=
This is the original matrix!!!!!!!!!!
01200.1000
29333.000333.70
2.01.03
102.03.0
3.071.0
2.01.03
LU Decomposition
4.71
3.19
85.7
102713.01.0
0103333.0
001
3
2
1
y
y
y
4.71
3.19
85.7
102.03.0
3.071.0
2.01.03
3
2
1
x
x
x
L y b
LU Decomposition
4.71
3.19
85.7
102713.01.0
0103333.0
001
3
2
1
y
y
y
L y b
85.71 y
5617.190333.03.19 12 yy
0843.70)02713.0(1.04.71 213 yyy
LU Decomposition85.71 y
5617.190333.03.19 12 yy
0843.70)02713.0(1.04.71 213 yyy
0843.70
5617.19
85.7
01200.1000
29333.000333.70
2.01.03
LU Decomposition
• Ax=b
• A=LU - LU Decomposition
• Ly=b- Solve for y
• Ux=y - Solve for x
Matrix Inversion
4.71
3.19
85.7
102.03.0
3.071.0
2.01.03
3
2
1
x
x
x
bxA
Matrix Inversion
[A] [A]-1
[A] [A]-1=[I]
If [A]-1 does not exist[A] is singular
Matrix Inversion
b xA bxA 1A 1A
I
Matrix Inversion
bAx 1
Solution
Matrix Inversion
• To calculate the invert of a nxn matrix solve n times :
nj
2j
1j
nj
2j
1j
nnn2n1
2n2221
1n1211
a
a
a
aaa
aaa
aaa
nj ,,2,1
otherwise
ji if
0
1ij
Iterative Methods
Recall Techniques for Root finding of Single Equations
Initial Guess
New Estimate
Error Calculation
Repeat until Convergence
Gauss Seidel
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa
11
31321211 a
xaxabx
22
32312122 a
xaxabx
33
23213133 a
xaxabx
Gauss Seidel
11
1
11
1312111
00
a
b
a
aabx
22
23112121
2
0
a
axabx
33
1232
113131
3 a
xaxabx
First Iteration: 0,0,0 321 xxx
Better Estimate
Better Estimate
Better Estimate
Gauss Seidel
11
1313
121212
1 a
xaxabx
22
1323
212122
2 a
xaxabx
33
2232
213132
3 a
xaxabx
Second Iteration: 13
12
11 ,, xxx
Better Estimate
Better Estimate
Better Estimate
Gauss SeidelIteration Error:
%1001
, ji
ji
ji
ia x
xx
s
Convergence Criterion:
n
jij
ijii aa1
nnn2n1
2n2221
1n1211
aaa
aaa
aaa
Jacobi Iteration
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa
11
31321211 a
xaxabx
22
32312122 a
xaxabx
33
23213133 a
xaxabx
Jacobi Iteration
11
1
11
1312111
00
a
b
a
aabx
22
2321212
00
a
aabx
33
3231313
00
a
aabx
First Iteration: 0,0,0 321 xxx
Better Estimate
Better Estimate
Better Estimate
Jacobi Iteration
11
1313
121212
1 a
xaxabx
22
1323
112122
2 a
xaxabx
33
1232
113132
3 a
xaxabx
Second Iteration: 13
12
11 ,, xxx
Better Estimate
Better Estimate
Better Estimate
Jacobi Iteration
Iteration Error:
%1001
, ji
ji
ji
ia x
xx
s
Determinants
nnn2n1
2n2221
1n1211
aaa
aaa
aaa
A
nnn2n1
2n2221
1n1211
aaa
aaa
aaa
det
AA
Are composed of same elements
Completely Different Mathematical Concept
Determinants
2221
1211
aa
aaA
Defined in a recursive form
2x2 matrix
122122112221
1211det aaaaaa
aaA
DeterminantsDefined in a recursive form
3x3 matrix
3231
222113
3331
232112
3332
232211
det
aa
aaa
aa
aaa
aa
aaa
A
333231
232221
131211
aaa
aaa
aaa
333231
232221
131211
aaa
aaa
aaa
Determinants
3332
232211 aa
aaa
3231
222113
3331
232112 aa
aaa
aa
aaa
3332
2322
aa
aaMinor a11
333231
232221
131211
aaa
aaa
aaa
Determinants
3331
2321
aa
aaMinor a12
3332
232211 aa
aaa
3331
232112 aa
aaa
3231
222113 aa
aaa
333231
232221
131211
aaa
aaa
aaa
Determinants
3231
2221
aa
aaMinor a13
3332
232211 aa
aaa
3331
232112 aa
aaa
3231
222113 aa
aaa
Singular Matrices
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
aaa
aaa
aaa
If det[A]=0 solution does NOT exist
Determinants and LU Decomposition
nnaaaaD 332211det U
33
2322
131211
00
0
a
aa
aaa
)operations pivoting no (if detdet UA D
Curve Fitting
Often we are faced with the problem…
x y0.924 -0.003880.928 -0.00743
0.93283 0.005690.93875 0.00188
0.94 0.01278
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
what value of y corresponds to x=0.935?
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
Curve Fitting
Question 1: Is it possible to find a simple and convenient formula that reproduces the points exactly?
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
e.g. Straight Line ?
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
…or smooth line ?
…or some other representation?
Interpolation
Curve FittingQuestion 2: Is it possible to find a simple and convenient formula that represents data approximately ?
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
e.g. Best Fit ?
Approximation
Linear Interpolation
iii
iii xx
xx
xfxfxfxf
1
11
Slope of Line
1st DIVIDED DIFFERENCEf [xi+1,xi]
First order interpolating polynomial
ii
ii
i
ii
xx
xfxf
xx
xfxf
1
11
Function Interpolation
Quadratic Interpolation
Better Accuracy if
2nd Order Polynomial -0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
x
12102 iii xxxxbxxbbxf
General Form of Newton’s Interpolating Polynomials
011 , xxfb
00 xfb
0122 ,, xxxfb
110
212
01
0
nn
n
xxxxxxb
xxxxb
xxb
b
xf
110 ,, nn xxxfb
Lagrange Interpolating Polynomials
• Reformulation of Newton’s Polynomials
• Avoid Calculation of Divided Differences
n
iiin xfxLxf
0
)()(x f(x)xo f(xo )
x1 f(x1 )
x2 f(x2 )
… …
xn f(xn)
n
ijj ji
ji xx
xxxL
0
)(
Lagrange Interpolating PolynomialCardinal Functions: Product of n-1 linear factors
ni
n
ii
i
ii
i
iii xx
xx
xx
xx
xx
xx
xx
xx
xx
xxxL
1
1
1
1
2
2
1
1
Skip xi
Property:
ji if 1
ji if 0ijji xL
Errors in Polynomial Interpolation
n321
n321
y y y yy f(x)
x x x xx
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
It is expected that as number of nodes increases, error decreases, HOWEVER….
n
iiin xfxLxf
11
At all interpolation nodes xi Error=0At all intermediate points
Error: f(x)-fn-1(x)
f(x)
Errors in Polynomial Interpolation
Beware of Oscillations….
For Example:Consider f(x)=(1+x2)-1 evaluated at 9 points in [-5,5]And corresponding p8(x) Lagrange Interpolating Polynomial
P8(x)f(x)
Other Methods
nn xaxaxaaoxf 2
21)(
Direct Evaluation
n+1 coefficients
n321
n321
y y y yy f(x)
x x x xx
n+1 Data Points
Interpolating Polynomial should represent them exactly
Other Methods
nn xaxaxaaoxf 2
21)(
Direct Evaluation
n321
n321
y y y yy f(x)
x x x xx
nn xaxaxaaoy 1
212111
nn xaxaxaaoy 2
222212
nnnnnn xaxaxaaoy 2
21
Other Methods
n
1
0
2
1211
0200
n
1
0
a
a
a
y
y
y
nnnn
n
n
xxx
xxx
xxx
Solve Using any of the methods we have learned
Other Methods
•Not the most efficient method
•Ill-conditioned matrix (nearly singular)
•If n is large highly inaccurate coefficients
•Limit to lower order polynomials
Inverse Interpolation
n321
n321
y y y yy f(x)
x x x xx
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
X=?X=?
Inverse Interpolation
-0.01
-0.005
0
0.005
0.01
0.015
0.92 0.925 0.93 0.935 0.94 0.945
X=?X=?
Switch x and y and then interpolate?
Not a Good Idea!
Splines
Splines
Piecewise smooth polynomials
tscoefficien n3
E.G Quadratic Splines
• Function Values at adjacent polynomials are equal at interior nodes
11112
11 iiiiii xfcxbxa
112
1 iiiiii xfcxbxa
ni 2
conditions )1(2 n
E.G Quadratic Splines• First and Last Functions pass through end
points
011201 xfcxbxa i
nnnnnn xfcxbxa 2
conditions )1(2 n
conditions 2
conditions 2n
ni 2
E.G Quadratic Splines• First Derivatives at Interior nodes are equal
baxxf 20
ni 2
conditions )1(2 n
conditions 2
conditions 13 n
iii
iii
bxa
bxa
1
111
2
2
conditions 1-n
E.G Quadratic Splines• Assume Second Derivative @ First Point=0
02 10 axf
conditions )1(2 n
conditions 2
conditions 3n
conditions 1-nconditions 1
E.G Quadratic Splines• Assume Second Derivative @ First Point=0
conditions 3n
tscoefficien edundetermin 3n
Solve 3nx3n system of Equations
baC
ix on based )( and
)( on based
xf
xf
Spline Interpolation
Polynomial InterpolationPolynomial Interpolation
Spline InterpolationSpline InterpolationPolynomial InterpolationPolynomial Interpolation
Polynomial InterpolationPolynomial Interpolation
Related Documents
