Page 1
IVP for ODEsbased on
Scientific Computing: An Introductory Survey by Michael T. Heathhttp://www.cse.uiuc.edu/heath/scicomp/
andApplied Numerical Methods for Engineers and Scientists by Singiresu S. Rao
http://cwx.prenhall.com/bookbind/pubbooks/rao/
Małgorzata Stojek,PhD Dr. Es-sciences techniques EPFL
Cracow University of Technology
CUT January 2011
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 1 / 14
Page 2
Initial Value ProblemProblem Definition
Find the solution of the first order differential equation
y ′ = y + 2x − 1
over the interval0 ≤ x ≤ 1
with the initial condition
y (x)|x=x0=0 = 1
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 2 / 14
Page 3
Exact Solutiony’=y+2x-1, y(x=0)=1
y(x) = −1− 2x + 2ex
0.0 0.2 0.4 0.6 0.8 1.01.0
1.5
2.0
x
y
y(0.0) = 1.0000y(0.1) = 1.0103y(0.2) = 1.0428y(0.3) = 1. 0997y(0.4) = 1.1836y(0.5) = 1. 2974y(0.6) = 1. 4442y(0.7) = 1. 6275y(0.8) = 1. 8511y(0.9) = 2.1192y(1.0) = 2.4366
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 3 / 14
Page 4
Euler’s Method
ODE:
y ′ = f (x , y)
y ′ = y + 2x − 1
Euler’s method (equivalent to Taylor’s series method of order 1):
yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )
where h is the uniform step sizeforward (explicit) iteration:
f (x , y) = y + 2x − 1⇓
yi+1 = yi + h (yi + 2xi − 1)Recall: backward (implicit) iteration
yi+1 = yi + h (yi+1 + 2xi+1 − 1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14
Page 5
Euler’s Method
ODE:
y ′ = f (x , y)
y ′ = y + 2x − 1Euler’s method (equivalent to Taylor’s series method of order 1):
yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )
where h is the uniform step size
forward (explicit) iteration:
f (x , y) = y + 2x − 1⇓
yi+1 = yi + h (yi + 2xi − 1)Recall: backward (implicit) iteration
yi+1 = yi + h (yi+1 + 2xi+1 − 1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14
Page 6
Euler’s Method
ODE:
y ′ = f (x , y)
y ′ = y + 2x − 1Euler’s method (equivalent to Taylor’s series method of order 1):
yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )
where h is the uniform step sizeforward (explicit) iteration:
f (x , y) = y + 2x − 1⇓
yi+1 = yi + h (yi + 2xi − 1)
Recall: backward (implicit) iteration
yi+1 = yi + h (yi+1 + 2xi+1 − 1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14
Page 7
Euler’s Method
ODE:
y ′ = f (x , y)
y ′ = y + 2x − 1Euler’s method (equivalent to Taylor’s series method of order 1):
yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )
where h is the uniform step sizeforward (explicit) iteration:
f (x , y) = y + 2x − 1⇓
yi+1 = yi + h (yi + 2xi − 1)Recall: backward (implicit) iteration
yi+1 = yi + h (yi+1 + 2xi+1 − 1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14
Page 8
Euler’s Methody’=y+2x-1, y(x=0)=1
assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1
yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1
initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14
Page 9
Euler’s Methody’=y+2x-1, y(x=0)=1
assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1
yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1
initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1
(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14
Page 10
Euler’s Methody’=y+2x-1, y(x=0)=1
assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1
yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1
initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2
(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14
Page 11
Euler’s Methody’=y+2x-1, y(x=0)=1
assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1
yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1
initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3
. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14
Page 12
Euler’s Methody’=y+2x-1, y(x=0)=1
assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1
yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1
initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .
(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14
Page 13
Euler’s Methody’=y+2x-1, y(x=0)=1
assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1
yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1
initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14
Page 14
Euler’s Method, Errorsy’=y+2x-1, y(x=0)=1
xi yi Euler y (xi ) exact absolute error % relative error
0.0 1.00000.1 1.0 1.0103 0.0103 1.01950.2 1.02 1.0428 0.0228 2.186 40.3 1.062 1. 0997 0.0377 3.61530.4 1.128 2 1.1836 0.0554 4.68060.5 1.221 1. 2974 0.0764 5.88870.6 1.343 1 1. 4442 0.1011 7.00040.7 1.497 4 1. 6275 0.1301 7.99390.8 1.687 1 1. 8511 0.164 8.85960.9 1.915 8 2.1192 0.2034 9.59801.0 2.187 4 2.4366 0.2492 10.227
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 6 / 14
Page 15
Runge-Kutta Methods
do not require the derivatives of f (x , y)
yi+1 = yi + hα (xi , yi , h)
where α (xi , yi , h) is the increment function
α (xi , yi , h) - the average slope over interval xi ≤ x ≤ xi+1α (xi , yi , h) = c1k1 + c2k2 + . . .+ cnkn
where: n - the order of Runge-Kutta method; ci - constants;ki - recurrence relations given by
k1 = f (xi , yi )
k2 = f (xi + p2h, yi + a21hk1)
k3 = f (xi + p3h, yi + a31hk1 + a32hk2)...
kn = f (xi + pnh, yi + an1hk1 + an2hk2 + . . .+ an,n−1hkn−1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 7 / 14
Page 16
Runge-Kutta Methods
do not require the derivatives of f (x , y)
yi+1 = yi + hα (xi , yi , h)
where α (xi , yi , h) is the increment functionα (xi , yi , h) - the average slope over interval xi ≤ x ≤ xi+1
α (xi , yi , h) = c1k1 + c2k2 + . . .+ cnkn
where: n - the order of Runge-Kutta method; ci - constants;
ki - recurrence relations given by
k1 = f (xi , yi )
k2 = f (xi + p2h, yi + a21hk1)
k3 = f (xi + p3h, yi + a31hk1 + a32hk2)...
kn = f (xi + pnh, yi + an1hk1 + an2hk2 + . . .+ an,n−1hkn−1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 7 / 14
Page 17
Runge-Kutta Methods
do not require the derivatives of f (x , y)
yi+1 = yi + hα (xi , yi , h)
where α (xi , yi , h) is the increment functionα (xi , yi , h) - the average slope over interval xi ≤ x ≤ xi+1
α (xi , yi , h) = c1k1 + c2k2 + . . .+ cnkn
where: n - the order of Runge-Kutta method; ci - constants;ki - recurrence relations given by
k1 = f (xi , yi )
k2 = f (xi + p2h, yi + a21hk1)
k3 = f (xi + p3h, yi + a31hk1 + a32hk2)...
kn = f (xi + pnh, yi + an1hk1 + an2hk2 + . . .+ an,n−1hkn−1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 7 / 14
Page 18
Second-Order Runge-Kutta Methods
yi+1 = yi + h (c1k1 + c2k2)
Modified Euler’s Method
yi+1 = yi + hk2
k1 = f (xi , yi ) , k2 = f(xi +
h2, yi +
12hk1
)
Ralston’s Method
yi+1 = yi +13h (k1 + 2k2)
k1 = f (xi , yi ) , k2 = f(xi +
34h, yi +
34hk1
)Heun’s Method- the simplest one
yi+1 = yi +12h (k1 + k2)
k1 = f (xi , yi ) , k2 = f (xi + h, yi + hk1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 8 / 14
Page 19
Second-Order Runge-Kutta Methods
yi+1 = yi + h (c1k1 + c2k2)
Modified Euler’s Method
yi+1 = yi + hk2
k1 = f (xi , yi ) , k2 = f(xi +
h2, yi +
12hk1
)Ralston’s Method
yi+1 = yi +13h (k1 + 2k2)
k1 = f (xi , yi ) , k2 = f(xi +
34h, yi +
34hk1
)
Heun’s Method- the simplest one
yi+1 = yi +12h (k1 + k2)
k1 = f (xi , yi ) , k2 = f (xi + h, yi + hk1)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 8 / 14
Page 20
Second-Order Runge-Kutta Methods
yi+1 = yi + h (c1k1 + c2k2)
Modified Euler’s Method
yi+1 = yi + hk2
k1 = f (xi , yi ) , k2 = f(xi +
h2, yi +
12hk1
)Ralston’s Method
yi+1 = yi +13h (k1 + 2k2)
k1 = f (xi , yi ) , k2 = f(xi +
34h, yi +
34hk1
)Heun’s Method- the simplest one
yi+1 = yi +12h (k1 + k2)
k1 = f (xi , yi ) , k2 = f (xi + h, yi + hk1)SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 8 / 14
Page 21
Second-Order Runge-Kutta Method, Heun’s Methody’=y+2x-1, y(x=0)=1
for f (x , y) = y + 2x − 1 we have:
k1 = f (xi , yi ) = yi + 2xi − 1k2 = f (xi + h, yi + hk1) = (yi + hk1) + 2 (xi + h)− 1
= (yi + h (yi + 2xi − 1)) + 2 (xi + h)− 1
yi+1 = yi + 12h (k1 + k2)
yi+1 = yi + h(xi (2+ h) + yi
(1+
h2
)+
(h2− 1
))
for h = 0.1yi+1 = 0.21 · xi + 1.105 · yi − 0.095
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 9 / 14
Page 22
Second-Order Runge-Kutta Method, Heun’s Methody’=y+2x-1, y(x=0)=1
for f (x , y) = y + 2x − 1 we have:
k1 = f (xi , yi ) = yi + 2xi − 1k2 = f (xi + h, yi + hk1) = (yi + hk1) + 2 (xi + h)− 1
= (yi + h (yi + 2xi − 1)) + 2 (xi + h)− 1
yi+1 = yi + 12h (k1 + k2)
yi+1 = yi + h(xi (2+ h) + yi
(1+
h2
)+
(h2− 1
))
for h = 0.1yi+1 = 0.21 · xi + 1.105 · yi − 0.095
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 9 / 14
Page 23
Second-Order Runge-Kutta Method, Heun’s Methody’=y+2x-1, y(x=0)=1
for f (x , y) = y + 2x − 1 we have:
k1 = f (xi , yi ) = yi + 2xi − 1k2 = f (xi + h, yi + hk1) = (yi + hk1) + 2 (xi + h)− 1
= (yi + h (yi + 2xi − 1)) + 2 (xi + h)− 1
yi+1 = yi + 12h (k1 + k2)
yi+1 = yi + h(xi (2+ h) + yi
(1+
h2
)+
(h2− 1
))
for h = 0.1yi+1 = 0.21 · xi + 1.105 · yi − 0.095
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 9 / 14
Page 24
Heun’s Method, continuedy’=y+2x-1, y(x=0)=1
yi+1 = 0.21 · xi + 1.105 · yi − 0.095
initial value (0, 1) 0 → (0.1, 1. 01) 1
y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01
(0.1, 1.01) 1 → (0.1, 1.0421) 2
y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421
. . .(0.9, ) 9 → (1.0, 2.4282) 10 → STOP
y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14
Page 25
Heun’s Method, continuedy’=y+2x-1, y(x=0)=1
yi+1 = 0.21 · xi + 1.105 · yi − 0.095
initial value (0, 1) 0 → (0.1, 1. 01) 1
y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01
(0.1, 1.01) 1 → (0.1, 1.0421) 2
y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421
. . .(0.9, ) 9 → (1.0, 2.4282) 10 → STOP
y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14
Page 26
Heun’s Method, continuedy’=y+2x-1, y(x=0)=1
yi+1 = 0.21 · xi + 1.105 · yi − 0.095
initial value (0, 1) 0 → (0.1, 1. 01) 1
y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01
(0.1, 1.01) 1 → (0.1, 1.0421) 2
y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421
. . .
(0.9, ) 9 → (1.0, 2.4282) 10 → STOP
y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14
Page 27
Heun’s Method, continuedy’=y+2x-1, y(x=0)=1
yi+1 = 0.21 · xi + 1.105 · yi − 0.095
initial value (0, 1) 0 → (0.1, 1. 01) 1
y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01
(0.1, 1.01) 1 → (0.1, 1.0421) 2
y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421
. . .(0.9, ) 9 → (1.0, 2.4282) 10 → STOP
y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14
Page 28
Heun’s Method, Errorsy’=y+2x-1, y(x=0)=1
xi yi Heun y (xi ) exact absolute error % relative error
0.0 1.00000.1 1.01 1.0103 0.0003 2. 969 4× 10−20.2 1.0421 1.0428 0.0007 6. 712 7× 10−20.3 1.0985 1. 0997 0.0012 0.109120.4 1.1818 1.1836 0.0018 0.152080.5 1.2949 1. 2974 0.0025 0.192690.6 1. 4409 1. 4442 0.0033 0.22850.7 1. 6232 1. 6275 0.0043 0.264210.8 1. 8456 1. 8511 0.0055 0.297120.9 2. 1124 2.1192 0.0068 0.320881.0 2. 4282 2.4366 0.0084 0.34474
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 11 / 14
Page 29
Forth-Order Runge-Kutta Method
yi+1 = yi +h6(k1 + 2k2 + 2k3 + k4)
where
k1 = f (xi , yi )
k2 = f(xi +
12h, yi +
12hk1
)k3 = f
(xi +
12h, yi +
12hk2
)k4 = f (xi + h, yi + hk3)
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 12 / 14
Page 30
Forth-Order Runge-Kutta Method, Errorsy’=y+2x-1, y(x=0)=1
xi yi 4th R-K y (xi ) exact absolute error % relative error
0.0 1.00000.1 1.0103 1.0103 0 00.2 1.0428 1.0428 0 00.3 1.0997 1. 0997 0 00.4 1.1836 1.1836 0 00.5 1.2974 1. 2974 0 00.6 1.4442 1. 4442 0 00.7 1. 6275 1. 6275 0 00.8 1. 8511 1. 8511 0 00.9 2.1192 2.1192 0 01.0 2.4366 2.4366 0 0
Note: for five digits precisionSS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 13 / 14
Page 31
% Relative Errorsy’=y+2x-1, y(x=0)=1
xi Euler’s Method Heun’s Method 4th R-K
0.00.1 1.0195 2. 969 4× 10−2 00.2 2.186 4 6. 712 7× 10−2 00.3 3.6153 0.10912 00.4 4.6806 0.15208 00.5 5.8887 0.19269 00.6 7.0004 0.2285 00.7 7.9939 0.26421 00.8 8.8596 0.29712 00.9 9.5980 0.32088 01.0 10.227 0.34474 0
SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 14 / 14