1 Chapter 1 Chapter 1 Introductory Introductory Concepts and Concepts and Calculus Review Calculus Review
Dec 30, 2015
1
Chapter 1 Chapter 1 Introductory Introductory
Concepts and Concepts and Calculus ReviewCalculus Review
2
IntroductionIntroduction
The subjectsThe subjects The derivation of the algorithmsThe derivation of the algorithms The implementation of the algorithmsThe implementation of the algorithms Analyze the algorithms mathematicallyAnalyze the algorithms mathematically
Accuracy, efficiency, and stabilityAccuracy, efficiency, and stability
3
1.1 Basic Tools of Calculus1.1 Basic Tools of Calculus
1.1.1 Taylor’s Theorem1.1.1 Taylor’s Theorem
Integral mean value
theorem
4
Three particular expansions of Three particular expansions of Taylor’s TheoremTaylor’s Theorem
...!3
)0(
!2
)0(
!1
)0(
!0
)0( 03
02
01
00
ex
ex
ex
ex
ex
where x0= ?
5
Three particular expansions of Three particular expansions of Taylor’s TheoremTaylor’s Theorem
where x0= 0
where x0= 0
6
Example : eExample : exx
]1,1[x
Finally, n can be found! (here n = 9)
If we wantthen
Let x0= 0
7
Example : eExample : exx
p2 (x)
p9 (x)
exp(x)
8
Example : eExample : exx
9
Example : eExample : exx
The result tells usThe result tells us We can approximate the exponential function We can approximate the exponential function
to within to within 1010-6-6 accuracy using a specific accuracy using a specific polynomial, and this accuracy holds for all polynomial, and this accuracy holds for all xx in in a specified interval.a specified interval.
10
Example 1.1Example 1.1 Let Let f f ((xx) = () = (xx+1)+1)1/21/2,, then the second-order Taylor then the second-order Taylor
polynomial (computed about polynomial (computed about xx00= 0= 0) is computed ) is computed as follows:as follows:
22
11
Example 1.2: sinExample 1.2: sin Function:Function: Accuracy:Accuracy:
12
Example 1.3: arctanExample 1.3: arctan Function:Function:
http://zh.wikipedia.org/wiki/File:Atan_acot_plot.svg
13
Example 1.3: arctanExample 1.3: arctan Function:Function:
Error term
Let
and
14
Example 1.3 : arctanExample 1.3 : arctan Please determine the error in a Please determine the error in a ninthninth-degree -degree
Taylor approximation to the arctangent function.Taylor approximation to the arctangent function. Since Since 22n n +1 = 9+1 = 9 implies that implies that n n = 4= 4, we have, we have
15
Taylor’s Theorem ExpansionTaylor’s Theorem Expansion
Let x x + h and x0 x
16
1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems
http://en.wikipedia.org/wiki/Mean_value_theorem
17
1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems
W
18
1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems
M m
Critical point
Critical point
19
1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems
The integral mean value theorem (a corollary of the intermediate value theorem) states that a function continuous on an interval takes on its average value somewhere in the interval. More exactly, if is continuous on , then there exists in such that .
20
1.1.2 Mean Value and Extreme Value 1.1.2 Mean Value and Extreme Value TheoremsTheorems
21
1.2 Error, Approximate Equality, and 1.2 Error, Approximate Equality, and Asymptotic Order NotationAsymptotic Order Notation
1.2.1 Error1.2.1 Error AA : a quantity( : a quantity( 數量數量 ) we want to compute) we want to compute AAhh: an approximation(: an approximation( 近似值近似值 ) to that quantity) to that quantity
Relative error (Relative error ( 相對誤差相對誤差 ) is better.) is better. These errors are both computational errors.These errors are both computational errors.
22
1.2.2 Notation: Approximate 1.2.2 Notation: Approximate EqualityEquality
Approximate equalityApproximate equality
It is an equivalence relation, and satisfy the folIt is an equivalence relation, and satisfy the following properties:lowing properties: Transitive(Transitive( 遞移性遞移性 ):): Symmetric(Symmetric( 對稱性對稱性 ):): Reflexive(Reflexive( 反身性反身性 ):):
23
1.2.3 Notation: Asymptotic Order 1.2.3 Notation: Asymptotic Order (Big O)(Big O)
24
Example 1.4Example 1.4
LetLet
Simple calculus shows that Simple calculus shows that
so that we haveso that we have HereHere
25
1.2.3 Notation: 1.2.3 Notation: Asymptotic Order (Big O) Asymptotic Order (Big O)
26
Example 1.6Example 1.6
27
28
1.3 A Primer on Computer Arithmetic1.3 A Primer on Computer Arithmetic
Computer arithmetic is generally Computer arithmetic is generally inexact.inexact. While the errors are very small, they can While the errors are very small, they can
accumulate and dominate the calculation.accumulate and dominate the calculation. Example: floating-point arithmeticExample: floating-point arithmetic
Reference: An Introduction to Computer Science, Reference: An Introduction to Computer Science, Chapter 3, Excess System (Excess_127 or Chapter 3, Excess System (Excess_127 or Excess_1023)Excess_1023)
is the sign of the number, f is the fraction (0 <= f <= 1),is the base of the internal number system
IEEE standards for floating-point representation
(底數 尾數)(底數 尾數)
Example Example
Show the representation of the normalized number + 26 x 1.01000111001
SolutionSolution
The sign isThe sign is positivepositive. The Excess_127 representation of . The Excess_127 representation of the exponent is the exponent is 133133. You add extra 0s on the . You add extra 0s on the right right to to make it 23 bits. The number in memory is stored as:make it 23 bits. The number in memory is stored as:
00 1000010110000101 0100011100101000111001000000000000000000000000
31
ErrorsErrors
Rounding error v.s. chopping errorRounding error v.s. chopping errorRounding: Rounding: 四捨五入四捨五入Chopping: Chopping: 無條件捨去無條件捨去Discussion:Discussion:
Rounding is more accurate but chopping is fastRounding is more accurate but chopping is faster.er.
The chopping error is indeed lager than the rouThe chopping error is indeed lager than the rounding error.nding error.
32
ExampleExample
Rounding errorRounding error
Chopping errorChopping error
33
Subtractive CancellationSubtractive Cancellation
If If aa and and bb are accurate to 16 decimal digits. are accurate to 16 decimal digits. What about their difference What about their difference cc = = aa - - bb ? ?
Example: Example:
The result The result cc is accurate to 12 digits. is accurate to 12 digits. This is because we were subtracting two This is because we were subtracting two
nearly equalnearly equal numbers. numbers.
0050009999990000.02)1000/1( eb
9983339999000049.02)100/1( ea
34
ExampleExample
Function :Function :
We know that :We know that :
Taylor’s Theorem : Taylor’s Theorem :
35
…………
36
1.5 Simple Approximations1.5 Simple Approximations
Error function: Error function:
(probability theory)(probability theory) It isIt is notnot possible to evaluate this integral by possible to evaluate this integral by
means of the fundamental theorem of calculus.means of the fundamental theorem of calculus. Use Taylor’s Theorem to approximate.Use Taylor’s Theorem to approximate.
wherewhere
37
Substitution: Substitution:
DefineDefine
So that we haveSo that we have
SetSet
where where cc depends on depends on tt andand
38
Apply the Integral Mean Value Theorem:Apply the Integral Mean Value Theorem:
The structured form:The structured form:
wherewhere
39
Use the big Use the big OO notation: notation:
Use the approximate equality notation:Use the approximate equality notation:
Simplify: Simplify: if the values of if the values of xx between between 00 and and 22
if if k k >=1>=1
thusthus
40
Fundamental IdeaFundamental Idea
When confronted with a computation that When confronted with a computation that cannot be done exactly, we often replace cannot be done exactly, we often replace that relevant function with something that relevant function with something simpler that approximates it, and carry out simpler that approximates it, and carry out the computation exactly on the simple the computation exactly on the simple approximation.approximation.