Numerical Analysis, lecture 9: Integration (textbook sections 7.1,2,3,5) • Newton-Cotes formulas & error • composite methods: trapezoid, Simpson • how to tame difficult integrals a b
Numerical Analysis, lecture 9: Integration
(textbook sections 7.1,2,3,5)
• Newton-Cotes formulas & error
• composite methods: trapezoid, Simpson
• how to tame difficult integrals
a b
Numerical Analysis, lecture 9, slide ! 2
a b
a b
a b
a b
We want to compute definite integrals using values of f(x) at x = x0, x1, …, xn (p.163)
Integrals are needed for• areas, lengths, moments, …• special functions• probability (mean, variance, …)• integral equations (e.g. tomography)
Numerical integration (“quadrature”) is needed because• the function is only known at discrete points, or• there is no formula for the antiderivative, or• the formula is too messy
f (x) dxa
b!
Numerical Analysis, lecture 9, slide ! 3
Integrating an interpolation-polynomial gives a quadrature formula (p. 167)
Newton-Cotes formulas:
exercise1
1+ xdx = ?
0
1
!
xi = a + ih (i = 0,1,…,n; h = b ! an)
P(x) dx
a
b
! = an xn dxa
b
! +!+ a0 1 dxa
b
! =
weighted sum of function values
f (x) dxa
b
! " Ai f (xi )i=0
n
!
f (x) dx ! h2a
a+h" f0 + f1( ) trapezoidal rule
f (x) dx ! h3a
a+2h" f0 + 4 f1 + f2( ) Simpson's rule
f (x) dx ! 3h8a
a+3h" f0 + 3 f1 + 3 f2 + f3( )
and so on...
Numerical Analysis, lecture 9, slide ! 4
Trapezoid rule’s error can be derived using the interpolation error formula (p. 164)
error formulaf (x) dx ! h
2a
b" f (a) + f (b)( ) = !
h3
12##f ($) for some $ %[a,b]
proof LHS = f (x) ! P(x) dxa
b" =
f (x) ! P(x)(x ! a)(x ! b)
(x ! a)(x ! b) dxa
b"
=f (x̂) ! P(x̂)(x̂ ! a)(x̂ ! b)
(x ! a)(x ! b) dxa
b"
=12!
##f ($) %!h3
6
mean value theorem for integralsIf ! is continuous, and if w is continuous and does not change sign in [a,b],
then !(x)w(x) dxa
b
" = !(x̂) w(x) dxa
b
" for some x̂ #[a,b].
Numerical Analysis, lecture 9, slide ! 5
Simpson’s rule is more accurate than it “should” be (p. 168)
lemma Simpson's rule is exact for polynomials of degree ! 3
lemmaIf P is the degree ! 3 polynomial that interpolates f (x0 ), f (x1), "f (x1), f (x2 ),
then f (x) # P(x) = f(4)($)4!
(x # x0 )(x # x1)2 (x # x2 ).
error formula
f (x) dx ! h3a
a+2h" f (a) + 4 f (a + h) + f (a + 2h)( ) = !
h5
90f (4)(#)
Proof: LHS =f (x) ! P(x)
(x ! a)(x ! a ! h)2 (x ! a ! 2h)(x ! a)(x ! a ! h)2 (x ! a ! 2h) dx
a
a+2h"
=f (x̂) ! P(x̂)
(x̂ ! a)(x̂ ! a ! h)2 (x̂ ! a ! 2h)(x ! a)(x ! a ! h)2 (x ! a ! 2h) dx
a
a+2h" =
14!f (4)(#) $ !4h5
15
Proof: x jdx0
2h
! =(2h) j+1
j +1=h3
(0) j + 4 " (h) j + (2h) j( ) for j # 0,1,2,3{ }
Numerical Analysis, lecture 9, slide ! 6
The method of undetermined coefficients is another way to derive quadrature formulas (p. 167-168)
The coefficients of the interpolation-based quadrature rule AiP(xi )i=0
n
!
can be found by solving Aixik
i=0
n
! = xk dxa
b
" (k = 0,1,…,n)
theorem
!
"##
$##
A0 = A2 =h3, A1 =
4h3
example: Simpson’s rule
f (x) dx ! A0 f ("h) + A1 f (0) + A2 f (h)"h
h
#
A0 !1+ A1 !1+ A2 !1 = 1 dx"h
h
# = 2h
A0 ! ("h) + A1 !0 + A2 !h = x dx"h
h
# = 0
A0 ! ("h)2 + A1 !0
2 + A2 !h2 = x2 dx
"h
h
# =23h3
Find the coefficients for
f (x) dx ! A0 f (0) + A1 f (h) + A2 f (2h)0
h
"
exercise
0 2hh
Numerical Analysis, lecture 9, slide ! 7
Interpolation-based quadrature coefficients computed with Matlab
function A=quadcoeff(x,a,b)N = length(x);V = ones(N,N);for k=2:N V(k,:) = V(k-1,:).*x(:).';endp = (1:N)';f = (b.^p-a.^p)./p;A = V\f;
>> quadcoeff([0 1 2],0,1)
ans = 0.4167 0.6667 -0.0833
high-degree polynomials can give poor results (p.169)
>> x=-1:0.25:1;>> f=1./(1+25*x.^2);>> A=quadcoeff(x,-1,1);>> dot(f,A)
ans = 0.3001
for example:
11+ 25x2
dx!1
1
" =25tan!1(5) ! 0.5494
f (x) dxa
b
! " Ai f (xi )i=0
n
!
Numerical Analysis, lecture 9, slide ! 8
Composite quadrature rules use low-degree piecewise polynomials (p. 165-166, 169-170)
f (x) dx
a
b! = h 1
2 f (a) + f (a + h) +!+ f (b " h) + 12 f (b)( ) " b " a12 h2 ##f ($)
f (x) dx
a
b! =
h3f (a) + 4 f (a + h) + 2 f (a + 2h) +!+ 4 f (b " h) + f (b)( ) " b " a
180h4 f (4)(#)
composite Simpson’s rulea b
composite trapezoid rulea b
error formulas via mean value theorem for sums
If ! is continuous on [a,b] then 1m
!(xi )i=1
m
" = !(x̂) for some x̂ #[a,b].
Numerical Analysis, lecture 9, slide ! 9
Composite formulas computed using Matlab (p. 166-167, 170)
function T = traprule(f,a,b,m)x = linspace(a,b,m+1);T = f(a)/2 + f(b)/2;for i = 1 : m-1 T = T + f(x(i+1));endT = (b-a)/m * T;
function S = simpsonrule(f,a,b,m)x = linspace(a,b,m+1);S = f(a)/3 + f(b)/3;for i = 2:2:m S = S + f(x(i))*4/3;endfor i = 3:2:m-1 S = S + f(x(i))*2/3;endS = (b-a)/m * S;
>> for k = 0:2 S = simpsonrule(f,0,1,2^k); disp([1/2^k S S-log(2)]); end
1.0000 0.5000 -0.1931 0.5000 0.6944 0.0013 0.2500 0.6933 0.0001
!"#O(h2 )
!"#O(h4 )
>> f = @(x) 1./(1+x);>> for k = 0:3 T = traprule(f,0,1,2^k); disp([1/2^k T T-log(2)]); end
1.0000 0.7500 0.0569 0.5000 0.7083 0.0152 0.2500 0.6970 0.0039 0.1250 0.6941 0.0010
!h
!error
Numerical Analysis, lecture 9, slide ! 10
Trapezoid & Simpson can’t be usedif integrand is infinite at an endpoint
arctan xx3 2
dx0
0.64
! " 1.56129860 0.640
8
example (p. 179)
f (x) dx0
0.64! = h 1
2 f (0)"! + f (0 + h) +"+ f (0.64 # h) + 1
2 f (0.64)$
%&&
'
())
Numerical Analysis, lecture 9, slide ! 11
The convergence is slower when the integrand is not smooth enough (p. 176-178)
h Simpson error3/16 6.5104e-043/32 -8.1380e-053/64 1.0173e-05
x x dx = 73!1
2
"examplediscontinuity in f ′′ !
"#
only O(h3)
arctan xx
= x1 2 !13x3 2 +!
!"#
only O(h1.5 )
example (p.176)
singularity in f′ arctan xx
dx0
0.64
! " 0.3239463
0 0.640
0.8
h Simpson error
0.08 -1.84 !10-3
0.04 -6.49 !10-4
0.02 -2.30 !10-4
Numerical Analysis, lecture 9, slide ! 12
A change of variables can make the integrand smoother (p. 177-178)
arctan xx
dx0
0.64
! =t= x! 2arctan t2 dt
0
0.64
!
h Simpson error
0.1 -6.27 !10-6
0.05 -3.85 !10-7
0.025 -2.40 !10-8
!"#
O(h4 )
0 0.80
1
2arctan t2 = 2t2 + 2
3t6 +!
other tricks:• integration by parts (p. 178)
• subtracting the singularity (p. 178)
• series
Numerical Analysis, lecture 9, slide ! 13
Termwise integration of a seriesnear the singularity (p. 178-179)
1. split off the singularity
x!1 2 arctan xdx0
0.64
" = x!1 2 arctan xdx0
#
" + x!1 2 arctan xdx#
0.64
"regular
! "### $###
3. choose ε so that |E| ≤ tol; compute the regular part using e.g. Simpson
Three stages:
2. integrate a few terms of the series, find the error
x!1 2 arctan xdx
0
"
# = x1 2 !13x5 2 +!$
%&'()dx
0
"
# =23" 3 2 + E, E *
221
" 7 2
Numerical Analysis, lecture 9, slide ! 14
what happened, what’s next
• integration formulas & error estimates from interpolating polynomials
• undetermined coefficients
• composite formulas, convergence
• how to cope with singularities
Next lecture: more integration (§7.4, 7.6)
Numerical Analysis, lecture 10: Integration II
(textbook sections 7.4-6)
• Romberg integration
• adaptive methods
• how to tame difficult integrals
0 1
1
Numerical Analysis, lecture 10, slide ! 2
A numerical integration formula is a weighted sum of function values (p. 171-172)
f (x) dxa
b! " Ai f (xi )
i=0
n
#
composite trapezoidal rulea b
f (x) dxa
b! = h 1
2 f (a) + f (a + h) +!+ f (b " h) + 12 f (b)( )
T (h)" #$$$$$$$$ %$$$$$$$$
"b " a12
h2 ##f ($)
T (h) =12 T (2h) + h f (a + h) + f (a + 3h)!+ f (b ! h)( )
recursive panel-halving
Compute T (1), T (12 ), T ( 1
4 ), T (18 ) for 1
1+ xdx
0
1
!example
Numerical Analysis, lecture 10, slide ! 3
The composite trap rule’s error formulais a series of even powers of h (p. 171)
The composite trapezoidal rule's error is
f (x) dxa
b! " T =
h2
12#f (a) " #f (b)( ) + (b " a) h
4
720f (4)($)
theorem
more generally: (Euler-Maclaurin summation formula)
f (x) dxa
b! = T +
1(2k)!
B2kh2k f (2k"1)(a) " f (2k"1)(b)( ) "
k=1
m"1
# b " a(2m)!
B2mh2m f (2m)($)
…consequently, the composite trap rule is very accurate for periodic integrands
Numerical Analysis, lecture 10, slide ! 4
Extrapolation uses 2 low-order approximationsto compute a high-order approximation (p. 171-172)
note: • T (h) can be computed using recursive panel-halving formula
•T (h) ! T (2h)
3 is an error estimate for T2(h)
example (p. 172)
11+ x
dx0
1
!
h T1(h) ! 3 T2(h)1 0.750000
1 2 0.708333 "13889 #10"6 0.694444
1 4 0.697024 "3770 #10"6 0.693254
1 8 0.694122 "967 #10"6 0.693155
I = T (h) + h2
12!f (a) " !f (b)( ) +O(h4 )
I = T (2h) + (2h)2
12!f (a) " !f (b)( ) +O(h4 )
!"#
$#I = T (h) + T (h) % T (2h)
22 %1T2(h)
! "### $###+O(h4 )
Numerical Analysis, lecture 10, slide ! 5
Apply extrapolation recursively to obtain higher-order approximations (p. 171-172)
T2 = T (h) +T (h) ! T (2h)
22 !1
T3 = T2(h) +T2(h) ! T2(2h)
24 !1
T4 = T3(h) +T3(h) ! T3(2h)
26 !1!
h T1(h) ! 3 T2(h) ! 15 T3(h) ! 63 T4 (h)1 0.750000
1 2 0.708333 "13889 #10"6 0.694444
1 4 0.697024 "3770 #10"6 0.693254 "79 #10"6 0.693175
1 8 0.694122 "967 #10"6 0.693155 "7 #10"6 0.693148 0 0.693148
example (p.172)
O(h4 )
O(h6 )
O(h8 )
Numerical Analysis, lecture 10, slide ! 6
The Romberg table computed in Matlab (p. 173)
>> f = @(x) 1./(1+x);>> romberg(f,0,1)
ans = 0.75000000000000 NaN NaN NaN 0.70833333333333 0.69444444444444 NaN NaN 0.69702380952381 0.69325396825397 0.69317460317460 NaN 0.69412185037185 0.69315453065453 0.69314790148123 0.69314747764483
function R = romberg(f,a,b,q)if nargin<4, q = 2; end R = NaN(q+2,q+2);h = b-a; m = 1;R(1,1) = h*(f(a)+f(b))/2;for i = 2:q+2 h = h/2; m = 2*m; R(i,1) = R(i-1,1)/2 + h*sum(f(a+h*(1:2:m))); for j = 1:i-1 R(i,j+1) = R(i,j) + (R(i,j) - R(i-1,j))/(2^(2*j)-1); endend
Numerical Analysis, lecture 10, slide ! 7
Adaptive quadrature recursively subdivides intervals where error is largest (p. 180-181)
Divide & conquer
f (x) dxa
b
! " adpi(a,b, tol, S[a,b])
function adpi(a,b, tol, I1)I2 = S
[a, a+b2
] + S
[a+b2
, b]
E =I2 ! I1
15if E " tol return I2
else
return adpi(a, a + b2
, tol2
,S[a, a+b
2]) + adpi( a + b
2,b, tol
2,S
[a+b2
, b])
Simpson’s rule value of integral
Numerical Analysis, lecture 10, slide ! 8
An adaptive quadrature Matlab program (p. 181-182)
function I = adaptint(f,a,b,tol)ff = f([a,(a+b)/2,b]);I1 = (b-a)*[1 4 1]*ff(:)/6;I = adpi(f,a,b,tol,ff,I1); function I = adpi(f,a,b,tol,ff,I1)h = (b-a)/2; m = (a+b)/2;fm = f([(a+m)/2 (m+b)/2]);IL = h*[1 4 1]*[ff(1);fm(1);ff(2)]/6;IR = h*[1 4 1]*[ff(2);fm(2);ff(3)]/6;I2 = IL + IR;I = I2 + (I2-I1)/15;if abs(I-I2) > tol IL = adpi(f,a,m,tol/2,[ff(1) fm(1) ff(2)],IL); IR = adpi(f,m,b,tol/2,[ff(2) fm(2) ff(3)],IR); I = IL + IR;end
>> f = @(x) x+(0.2)./(1000*(x-0.3).^2+1);>> adaptint(f,0,1,1e-6)
ans = 0.51892
0 1
1
>> quad(f,0,1,1e-6)
ans = 0.51892
Numerical Analysis, lecture 9, slide ! 9
Trapezoidal & Simpson rules can’t be usedif integrand is infinite at an endpoint
arctan xx3 2
dx0
0.64
! " 1.56129860 0.640
8
example (p. 179)
f (x) dx0
0.64! = h 1
2 f (0)"! + f (0 + h) +"+ f (0.64 # h) + 1
2 f (0.64)$
%&&
'
())
arctan xx3/2
= x!1 2 !13x1 2 +!
Numerical Analysis, lecture 9, slide ! 10
The convergence is slower when the integrand is not smooth enough (p. 176-178)
arctan xx
= x1 2 !13x3 2 +!
!"#
only O(h1.5 )
example (p.176) singularity in f′arctan x
xdx
0
0.64
! " 0.3239463
0 0.640
0.8
h Simpson error
0.08 -1.84 !10-3
0.04 -6.49 !10-4
0.02 -2.30 !10-4
Numerical Analysis, lecture 9, slide ! 11
A change of variables can make the integrand smoother (p. 177-178)
arctan xx
dx0
0.64
! =t= x! 2arctan t2 dt
0
0.64
!
h Simpson error
0.1 -6.27 !10-6
0.05 -3.85 !10-7
0.025 -2.40 !10-8
!"#
O(h4 )
0 0.80
1
2arctan t2 = 2t2 + 2
3t6 +!
other tricks:• integration by parts (p. 178)
• subtracting the singularity (p. 178)
• series
Numerical Analysis, lecture 9, slide ! 12
Termwise integration of a seriesnear the singularity (p. 178-179)
1. split off the singularity
x!1 2 arctan xdx0
0.64
" = x!1 2 arctan xdx0
#
" + x!1 2 arctan xdx#
0.64
"regular
! "### $###
3. choose ε so that |E| ≤ tol; compute the regular part using e.g. Simpson
Three stages:
2. integrate a few terms of the series, find the error
x!1 2 arctan xdx
0
"
# = x1 2 !13x5 2 +!$
%&'()dx
0
"
# =23" 3 2 + E, E *
221
" 7 2
Numerical Analysis, lecture 10, slide ! 13
There are several ways of dealing with an infinite integration interval (p. 179)
cutting off the tail
e!x2
0
"
# dx = e!x2
0
a
# dx + E
E = e!x2
a
"
# dx $ 1a
xe!x2
a
"
# dx = 12a
e!a2
>> f = @(x) exp(-x.^2);>> adaptint(f,0,4,1e-6)
ans = 0.88623
>> f = @(x) exp(-(-log(t)).^2)./t;>> adaptint(f,eps,1,1e-6)
ans = 0.88623
change of variables
e!x2
0
"
# dx
0 10
1
0 40
1
0 10
1
0 40
1
=
t=e! x!
1te!(! log t )
2dt
0
1
"
Numerical Analysis, lecture 10, slide ! 14
what happened, what’s next
• Trapezoid rule’s error formula has even powers of h
• Romberg method recursively combines different h approximations to obtain a higher order
• Adaptive quadrature based on local error estimates
• coping with singularities: change of variable, series
• Two ways of handling infinite interval
Next lecture: approximation (§9.1-3)