Numerics and Scientific Computing Numerics and Scientific Computing Lecture 1, 26.02.2016 http://www.math.uzh.ch/binf2180 Tulin Kaman Institute of Mathematics, University of Zurich E-mail: [email protected]Lecture 1, 26.02.2016 T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 1 / 30
30
Embed
Numerics and Scienti c Computing - UZHuser.math.uzh.ch/vorlesungen/binf2180/fs16/lectureNotes/Lecture1.pdf · 10 3.6288e+06 3.5987e+06 30104 0.008296 The absolute errors grow as n
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Numerics and Scientific Computing
Numerics and Scientific ComputingLecture 1, 26.02.2016
http://www.math.uzh.ch/binf2180
Tulin KamanInstitute of Mathematics, University of Zurich
50% of the exercise must be solved correctlyto get hands-on experiencebased on Matlabwill be posted on the course website http://www.math.uzh.zh/binf2180
Examination
closed book on paperon 17.06.2016 from 8:00-10:002hour written examination
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 4 / 30
when |u| ≈ 1→ not much difference between absolute and relativeerror measureswhen |u| � 1→ the relative error is more meaningful
we expect the approximation in the last row to be similar in quality tothe one in the first row
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 10 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
Example
Stirling approximation is used to approximate u = n! = 1.2. · · · n for largen.
v = Sn =√
2πn(n
e
)nThe formula involves the constant e = exp(1) = 2.7182818...
The MATLAB script computes the values of u and v for n = 1, · · · , 10e=exp(1);
n=1:10; % array
Sn=sqrt(2*pi*n).*((n/e).^n); % the Stirling
fact_n=factorial(n);
abs_err=abs(Sn-fact_n); % absolute error
rel_err=abs_err./fact_n; % relative error
format short g
[n; fact_n; Sn; abs_err; rel_err]’ % print out values
1 1 0.92214 0.077863 0.077863
2 2 1.919 0.080996 0.040498
3 6 5.8362 0.16379 0.027298
4 24 23.506 0.49382 0.020576
5 120 118.02 1.9808 0.016507
6 720 710.08 9.9218 0.01378
7 5040 4980.4 59.604 0.011826
8 40320 39902 417.6 0.010357
9 3.6288e+05 3.5954e+05 3343.1 0.0092128
10 3.6288e+06 3.5987e+06 30104 0.008296
The absolute errors grow as n grows, but the relative errors stay wellbehaved and indicate that in fact the larger n is, the better the quality ofthe approximation is.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 11 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
Error types
The types of error that may limit the accuracy of a numerical calculation:1 Errors in the problem to be solved.
Errors in the mathematical modelErrors in the input data
2 Approximation errors
Discretization errorsConvergence errorsDiscretization and convergence errors may be assessed by an analysis ofthe method used
3 Roundoff errors
Roundoff errors arise in any computation with real numbers because ofthe finite precision representation of real numbers on any computer
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 12 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
Discretization errors in action I
Problem
Approximating the derivative f ′(x0) of a given smooth function f (x) at thepoint x = x0.
Example
Let f (x) = sin(x) be defined on the real line −∞ < x <∞ and setx0 = 1.2 Thus, f (x0) = sin(1.2) ≈ 0.932....
Consider a situation where f (x) may be evaluated at any point x nearx0, but
f ′(x0) may not be directly availableor is computationally expensive to evaluate
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 13 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
Discretization errors in action II
⇒ approximate f ′(x0) by evaluating f at x near x0.
Taylor’s series gives us an algorithm to approximate f ′(x0): For somesmall, positive value h
f (x0 + h) = f (x0) + hf ′(x0) +h2
2!f ′′(x0) +
h3
3!f ′′′(x0) +
h4
4!f ′′′′(x0) + · · ·
f ′(x0) =f (x0 + h)− f (x0)
h−(
h
2f ′′(x0) +
h2
6f ′′′(x0) +
h3
24f ′′′′(x0) + ...
)The algorithm for approximating f ′(x0) is to calculate
f (x0 + h)− f (x0)
h
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 14 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
Discretization errors in action III
The approximation has the discretization error∣∣∣∣f ′(x0)− f (x0 + h)− f (x0)
h
∣∣∣∣ =
∣∣∣∣h2 f ′′(x0) +h2
6f ′′′(x0) +
h3
24f ′′′′(x0) + ...
∣∣∣∣Geometrically, the tangent f ′(x0) isapproximated by the chord (f (x0 +h)− f (x0))/h.
∣∣∣∣f ′(x0)− f (x0 + h)− f (x0)
h
∣∣∣∣ =h
2|f ′′(ξ)| , where x0 < ξ < x0 + h
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 15 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
Discretization errors in action IV
We expect the discretization error to decrease when h decrease:∣∣∣∣f ′(x0)− f (x0 + h)− f (x0)
h
∣∣∣∣ ≈ h
2|f ′′(x0)|
Big-O notation: We consider various computational errors depending ondiscretization step size h > 0 and ask how they decrease as h decreases.
Definition
For an error e depending on h we denote e = O(hq) if there are two positiveconstants q and C such that |e| ≤ Chq for all h > 0 small enough.
Using the Big-O notation:∣∣∣f ′(x0)− f (x0+h)−f (x0)
h
∣∣∣ = O(h)
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 16 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
For instance: f (x) = sin(x),the exact value f ′(x0) = cos(1.2) = 0.362357754476674...For h = 0.1,the approximation f ′(x0) ≈ (sin(1.3)− sin(1.2)) /0.1 = 0.315...,which is not very accurate.
The error appears to decrease like h. Using the knowledge off ′′(x) = − sin(x)⇒ f ′′(x)/2 ≈ −0.466The quantity 0.466h provides an accurate estimate for the absolute errorvalues.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 17 / 30
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 18 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Numerical algorithms and errors
The combined effect of discretization and roundoff errors.
The discretization error decreases as h decreases, and it dominates theroundoff error when h is relatively large.
When h gets below approximately 10−8 the discretization error becomes verysmall and roundoff error starts to dominate.
The roundoff error increases as h decreases.
This is one reason why we want it always dominated by the discretizationerror when solving problems involving numerical differentiation.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 19 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Performance features that may be expected from a good numericalalgorithm:
Accuracy refers to the closeness of a measured value to a standard orknown value. Precision refers to the closeness of two or moremeasurements to each other.
Efficiency depends on both CPU time and storage space requirement
How fast can we solve a certain problem?How much memory space do we need?
Rate of convergence. Floating point operations (flops)
Robustness Numerical software should run under all circumstances.Should yield correct results to within an acceptable error or should failgracefully if not successful.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 20 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Efficiency
Machine-independent estimate of the number of elementaryoperations required, (+ - * /), gives an idea of the algorithmsefficiency.
Operation counts do not give the complete picture regardingexecution speed, since they do not take into account the price (speed)of memory access which may vary considerably.
In modern computers, cache access, blocking and vectorizationfeatures, and other parameters are crucial in the determination ofexecution time.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 21 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Efficiency
Big-O notation: When estimating the efficiency of a particular algorithm,we are interested in a bound on the work estimate as a parameter nincreases unboundedly (e.g., n = 1/h).
Definition
For w = w(n) the expression w = O(nlogn) means that there is aconstant C > 0 such that w ≤ Cn log n as n→∞.
operation description #mul #add#div #sub
inner product (x ∈ Rn, x ∈ Rn) 7→ xTy n n-1 O(n)outer product (x ∈ Rm, x ∈ Rn) 7→ xyT mn 0 O(mn)
matrix product (A ∈ Rmxn,B ∈ Rnxk) 7→ AB mnk mk(n-1) O(mnk)
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 22 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Problem conditioning and algorithm stability
The problem and the numerical algorithm both yield errors, question arises
regarding the appraisal of a given computed solution.
⇓
Problem sensitivity and algorithm stability play an important role.
The problem is ill-conditioned if a small perturbation in the data mayproduce a large difference in the result, then no algorithm may befound for that problem which would meet our requirement of solutionrobustness.
Otherwise the problem is well-conditioned.
The algorithm is stable if its output is the exact result of a slightlyperturbed input.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 23 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
An unstable algorithm
An ill-conditioned problem of computing output values y given in terms ofinput values x by y = g(x): when the input x is slightly perturbed to x ,the result y = g(x) is far from y .For instance, the problem of numerical differentiation turns out to be ill-conditioned when extreme accuracy (translating to very
small values of h) is required.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 24 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
A stable algorithm
An instance of a stable algorithm for computing y = g(x): the output y isthe exact result, y = g(x), for a slightly perturbed input, i.e. x which isclose to the input x.Thus, if the algorithm is stable and the problem is well-conditioned, thenthe computed result y is close to the exact y .
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 25 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Example
The problem of evaluating the square root function g(x) =√
1 + x for anargument near the value 1 is well-conditioned. Show?
Suppose we fix x so that |x | � 1, and consider x = 0 as a smallperturbation of x . Then
y = g(x) = 1⇒ y − y =√
1 + x − 1
If we approximate√
x + 1 by the first 2-terms of its Taylor seriesexpansion: g(x) ≈ 1 + x
2
⇒ y − y ≈(
1 +x
2
)− 1 =
x
2=
1
2(x − x)
The conditioning of this problem is determined by g ′(0) = 1/2, because
g ′ ≈ g(x)−g(0)x−0 .
The problem is well-conditioned because this number is not large.T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 26 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Example
The problem of evaluating the function g(x) = tan(x)
1 for x near zero is well-conditioned.
2 for x near π2 is ill-conditioned. Show?
2 Show that a small perturbation in the input data produces a largedifference in the result. Set x = π
2 − 0.001 and x = π2 − 0.002
⇒ |x − x | = 0.001 but | tan(x)− tan(x)| ≈ 500
The derivative g ′(x) = 1cos2(x)
for x near π2 explains why.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 27 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Error accumulation
Explicitly, if En measures the relative error at the nth operation of analgorithm, then
En ≈ c0nE0 for some constant c0 represents linear growth
En ≈ cn1 E0 for some constant c1 > 1 represents exponential growth.
An algorithm exhibiting relative exponential error growth is unstable. Suchalgorithms must be avoided!
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 28 / 30
Numerics and Scientific Computing
1. Numerical Algorithms
Algorithm properties
Unstable algorithm
Consider the integrals yn =∫ 1
0xn
x+10 dx n = 1, 2, · · · 30 Observe thatanalytically, for n > 0,
yn + 10yn−1 =
∫ 1
0
xn + 10xn−1
x + 10dx =
∫ 1
0xn−1dx =
1
n
Also
y0 =
∫ 1
0
1
x + 10dx = ln(11)− ln(10)
An algorithm is
1 Evaluate y0 = ln(11)− ln(10)
2 For n = 1, · · · , 30 evaluate yn = 1n − 10yn−1
Applying the above recursion formula would give exact values if roundoff errors were not present.
However, this algorithm is unstable, as the magnitude of roundoff errors gets multiplied by 10 each time the recursion is applied.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 29 / 30
Numerics and Scientific Computing
Roundoff Errors
Next week:
We have much to say about the floating point representation of realnumbers and the accumulation of roundoff errors during a calculation.
T. Kaman Numerics and Scientific Computing Lecture 1, 26.02.2016 30 / 30