COMPUTING RIGOROUS BOUNDS ON THE SOLUTION OF AN INITIAL VALUE PROBLEM FOR AN ORDINARY DIFFERENTIAL EQUATION Nedialko S toyanov Nedialkov A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Computer Science University of Toronto Copyright 1999 by Nedialko Stoyanov Nedialkov
163
Embed
COMPUTING RIGOROUS BOUNDS SOLUTION … · 1 must acknowledge Ole Stauning, Ron van Iwaarden, and Wayne Hayes. Ole and Ron provided two different interval automatic differentiation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
COMPUTING RIGOROUS BOUNDS ON THE SOLUTION OF AN INITIAL VALUE PROBLEM FOR AN ORDINARY DIFFERENTIAL EQUATION
Nedialko S toyanov Nedialkov
A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy
Graduate Department of Computer Science University of Toronto
Copyright 1999 by Nedialko Stoyanov Nedialkov
National Library 1+1 ,Canada Bibliothèque nationale du Canada
Acquisitions and Acquisitions et Bibliographie Services selvices bibliographiques
395 Wellington Street 395, rue Wellington Ottawa ON K1A ON4 Ottawa ON K1A ON4 Canada Canada
Your fi@ Votre reftirmce
Our fi& Norre fëference
The author has granted a non- exclusive Licence aiiowing the National Library of Canada to reproduce, loan, distribute or sell copies of this thesis in microform, paper or electronic formats.
The author retains ownership of the copyright in this thesis. Neither the thesis nor substantid extracts fiom it may be printed or othenvise reproduced without the author's permission.
L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la forme de microfiche/film, de reproduction sur papier ou sur format électronique.
L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation.
Abstract
Computing Rigorous Bounds on the Solution of an Initial Value Problem for an
Ordinary Differential Equation
Nediako Stoyanov Nedialkov
Doctor of Philosophy
Graduate Department of Computer Science
University of Toronto
1999
Compared to standard numerical methods for initia1 value problems (IVPs) for ordi-
nary differential equations (ODEs), validated (aiso called interval) methods for IVPs for
ODEs have two important advantages: if they return a solution to a problem, then (1)
the problem is guaranteed to have a unique solution, and (2) an enclosure of the true
solution is produced.
To date, the only effective approach for computing guaranteed enclosures of the solu-
tion of an IVP for an ODE has been interval methods based on Taylor series. This thesis
derives a new approach, an interval Hermite-Obreschkoff (IHO) method, for cornputing
such enclosures.
Compared to interval Taylor series (ITS) methods, for the same order and stepsize,
our IHO scheme has a smaller truncation error and better stability. As a result, the
IHO method allows Iarger stepsizes than the corresponding ITS methods, thus saving
computation time. In addition, since fewer Taylor coefficie~its are required by IHO than
ITS methods, the IHO method performs better than the ITS methods when the function
for cornputing the right side contains many terms.
The stability properties of the ITS and IHO methods are investigated. We show
as an important by-product of this analysis that the stability of an interval method is
determined not only by the stability funct ion of the underlying formula, as in a standard
method for an IVP for an ODE, but aIso by the associated formula for the truncation
error.
This thesis also proposes a Taylor series rnethod for validating existence and unique-
ness of the solution, a simple stepsize control, and a program structure appropriate for a
large class of validated ODE solvers.
Acknowledgement s
My special thanks to Professors Ken Jackson and George Corliss. Ken was my su-
pervisor during my Ph.D. studies. He had many valuable suggestions and was always
available to discuss my work. George helped us to get started in this area, read many of
my drnfts, and constantly encouraged me. Although he could not be officially a member
of my committee, 1 consider him as such.
1 want to thank rny committee members: Professors Christina Christara, Wayne
Enright, Rudi Matton, and Tom Fairgrieve for their helpful suggestions and prompt
reading of my proposais. Thanks to Professor Luis Seco for his participation as an
external committee memher during my senate exam.
1 am thankful to my external examiner Dr. John Pryce. His comments and questions
forced me to understand even better some of the issues in validated ODE solving.
1 must acknowledge Ole Stauning, Ron van Iwaarden, and Wayne Hayes. Ole and
Ron provided two different interval automatic differentiation packages, which helped me
to move on quickly in my software development and numerical experiments. Wayne had
many good comments on the material and the validated solver that 1 am developing.
The late Professor Tom Hull is an unfading presence in my life.
1 am grateful to my spouse Heidi for her belief in me, patience, and support.
My son Stoyan brought happiness to my work. He has already expressed interest in
my thesis, but realizes he must grow up before he understands it.
1 am grateful to my parents for encouraging my endless studies.
1 gratefully acknowledge the financid support from the Department of Computer
Science at the University of Toronto.
Contents
1 Introduction 1
. . . . . . . . . . . . . . . . . . . . . . . . . . . 1 I The Initial Value Problem 1
We also define the following quantities for intervals [50]:
a width w ([a]) = à - a;
a midpoint m ([a]) = (a + 412;
The interval-arit hmetic operations are inclusion monotone. That is, for real intervals
[a], [al], [b], and [bl] such that [a] C [al] and [b] C [bill we have
Although interval addition and multiplication are associative, the distributive law
does not hold in general [2, pp. 3-51. That is, we can easily find three intervals [a], [b],
and [cl, for which
However, for any three intervals [a]! [b], and [cl, the subdistributive iaw
does hold. Moreover, there are important cases in which the distributive law
[al (Pl + [cl = [al [bl + [al [cl
does hold. For example, it holds if [b] [cl 2 O, if [a] is a thin interval, or if [b] and [cl are
symmet ric.
Some other useful results for interval arithmetic follow. For [a] and [6] E IR,
[2, pp. 14-17]. If [a] is symmetric, then
From (2.1.4) and (2.1.6), if [a] is a symmetric interval, then
for any [a'] with w ([a']) = w ([a]).
2.2 Interval Vectors and Matrices
By an interval vector we mean a vector with interval components. By an intenta1 rnatrix
we mean a rnatrix with interval components. We denote the set of n-dimensional real
interval vectors by IRn and the set of n x m real interval matrices by IRnXm. The
arithmetic operations involving interval vectors and matrices are defined by using the
same formulas as in the scalar case, except that scalars are replaced by intervals. For
example, if [A] E IRnxn has components [aij], and [b] E IRn has components [bk], then
the components of [cl = [A] [b] are given by
An inclusion for interval matrices (and vectors) is defined component-wise by
The m a x i m u m n o m of an interval vector [a] E I[W is given by
and of a matrix [A] by
We also use the symbol 1 1 - 1 1 to denote the maximum norm of scalar vectors, scalar ma-
trices, and functions.
Let A and B c Rn be compact non-empty sets. Let q(A,B) denote the Hausdorff
distance between A and XI:
(A, B) = max rnax min llx - yll, max min 1 1 ~ - YI/} XEA yEB YECJ XEA
The distance between two intervals [a] and [b] is
and the distance between two interval vectors [u] and [v] E IRn is
Let [A] E IRnxm. We define the following quantities component-wise for interval
matrices (and vectors):
midpoint
Addition of interval matrices is associative, but multiplication of interval matrices is
not associative in general [53, pp. 50-811. Also, the distributive law does not hold in
general f ~ r interval matrices [53, p. 791. That is, we can easily find [A] E ItRnxm and [BI
and [Cl E IRmxP, for which
[Al ([BI + [CI) # [Al [BI + [Al [Cl
However, for any [A] E IRnXm and [BI and [Cl E IRmXP, the subdistributive law
does hold. Moreover, there are important cases in which
does hold. For example, the distributive law holds if [bij] [qj] 2 O (for al1 i, j ) , if [A] is a
point matrix, or if al1 cornponents of [BI and [Cl are symmetric intervals.
Some ot her useful results for interval matrices follow. Let [A] and [BI E IIRnx". Then
[2, pp. 125-1261. Let the components of [BI be symmetric intervais. Then
w ([Al [BI) = I[All ([BI) and
([Al [BI 5 ([Al P'1)
for any [B'] with w ([B']) = w ([BI).
Let [cl E ItRn be a symmetric vector (al1 components of [cl are symmetric intervals).
Then
Throughout this thesis, we assume exact real interval arithmetic, as described in this
subsection. In floating-point implementation, if one or both end-points of a real interval
are not representable (which is often the case), then they must be rounded outward to
the closest representable floating-point numbers. Interval arithmetic is often called a
machine, or rounded, interval arithmetic. A discussion of its properties can be found in
WI-
2.3 Interval-Valued Funct ions
Let f : Rn + W be a continuous function on Ir) C Rn. We consider functions whose rep-
resentations contain only a finite number of constants, variables, arithmetic operations,
and standard functions (sin, cos, log, exp, etc.).
We define the range of f over an interval vector [a] C 2) by
A fundamental problem in interval arithmetic is to compute an enclosure for R (f ; [a]).
We want this enclosure to be as tight as possible. For example, in our work, we are
interested in f being the right side of a differential equation. The naive interual-anthme tic
evaluation off on [a] , which we denote by f ([a]), is obtained by replacing each occurrence
of a real variable with a corresponding interval, by replacing the standard functions with
enclosures of their ranges, and by performing interval-arithrnetic operations instead of the
real operations. In practice, f ([a]) is not unique, because it depends on how f is evaluated
in interval arithmetic. For example, expressions that are rnathematically equivalent for
scalars, such as x(y+z) and xy+xz, may have different values if x, y, and z are intervals.
However, since we are interested in the interval-ôrithmetic evaluation of f on a cornputer,
we can assume that f ([a]) is uniquely defined by the code list, or computational graph,
of f . No rnatter how f ([a]) is evaluated, it follows from the inclusion monotone property
of the the interval operations that
R (f; [al) C f ([al).
If f satisfies a Lipschitz condition on V C Rn7 then for any [a] C D,
for some constant ci 2 O independent of [a], where q (-, -) is defined by (2.2.2), [50, p.
341, [2].
Mean-value form
If f : Rn + R is continuously differentiable on 2) 2 Rn and [a] 2 D, then for any y and
b E [al,
[50, p. 471. The expression f (b) + f'([a])([a] - b) is called the mean-value form of f.
Mathematically, fkI is not iiniquely defined, but it is uniquely determined by the code
list of f' and the choice of 6. If, in addition, f' satisfies a Lipschitz condition on V, then
for any [a] 2 D,
for some constant CL> 2 O independent of [a], [53, pp. 55-56]. Therefore, the mean-value
evaluation is quadratically convergent in the sense that the distance between R (f; [ a ] )
and fM([a] , 6) approaches zero as the square of I l w ([a]) 1 1 , as Ilw ([a]) 1 1 approaches zero.
Similar results apply to functions from Rn to Rn.
Integration
Let f : D + Rn be a continuous function on V C R and [a] C 2). Then,
2.4 Automat ic Generation of Taylor Coefficients
Moore [50, pp. 107-1301 presents a method for efficient generation of Tayior coefficients.
Ra11 [58] describes in detail algorithms for automatic differentiation and generation of
Taylor coefficients. He also considers applications of automatic differentiation, includ-
ing applications to ordinary differential equations. Two books containing papers and
extensive bibliographies on automatic differentiation are [9] and [23].
Since we need point and interval Taylor coefficients, we briefly describe the idea of
tiieir recursive generation. Denote the ith Taylor coefficient of u ( t ) evaluated at some
point t j by
where u(')( t) is the i th derivative of u(t) . Let ( u ~ ) ~ and ( ~ j ) ; be the ith Taylor coefficients
of u ( t ) and v ( t ) at tj. It can be shown that
(:)- v~ t = : { ( ~ j ) i - k ( v j ) ~ ( : ) V J r=l i-r } - Similar formulas can be derived for the generation of Taylor coefficients for the standard
functions [50, p. 1141.
Consider the autonomous differential system
We introduce the sequence of functions
Using (2.4.18-2.4.20), the Taylor coefficients of y ( t ) at t j satisfy
where (f ( y j ) ) i-, is the (i - 1)st coefficient of f evaluated at yj- By using (2.4.15-2-4-17),
similar formulas for the Taylor coefficients of the standard functions, and (2.4.23), we can
recursively evduate ( y j ) ; , for i 2 1. It can be shown that if the nurnber of the arithmetic
operations in the code list of f is N, then the number of arithmetic operations required
for the generation of k Taylor coefficients is between Nk and N k ( k - 1 ) /2 , depending
on the ratio of additions, multiplications, and divisions in the code list for f, [50, pp.
1 11-1 121 (see also Appendix A).
Let y ( t j ) = gj E [yj]. If we have a procedure to compute the point Taylor coefficients
of y ( t ) and perform the computations in interval arithmetic with [y j] instead of y j , we
obtain a procedure to cornpute the interval Taylor coefficients of y ( t ) . We denote the ith
interval Taylor coefficient of y ( t ) at t j by [y j]; = f [ " ( [ y j ] ) -
Chapter 3
Taylor Series Methods for IVPs for
ODEs
In most validated methods for IVPs for ODEs, each integration step consists of two
phases [52]:
ALGORITHM 1: Cornpute a stepsize hi and an a priori enclosure [cj] of the solution such
that y ( t ; tj, yj) is guaranteed to exist for al1 t E [tj7 tj+l] and all yj E [yj], and
ALGORITHM II: Using [ci], compute a tighter enclosure [ Y ~ + ~ ] of ~ ( t j + ~ ; to, [y,]).
Usually, the algorithm to validate the existence of a unique solution uses the Picard-
Lindelof operator and the Banach fixed-point theorem. In Taylor series met hods, the
computation of a tighter enclosure is based on Taylor series plus rernainder, the mean-
value theorem, and various interval transformations.
We discuss a constant enclosure method for implementing Algorithm 1 in 83.1. In
53.2, we present the basis of the ITS methods for implementing Algorithm II, illustrate
the wrapping effect, and explain Lohner7s method for reducing it. We also consider the
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES 18
wrapping effect in generating interval Taylor coefficients and the overestirnation in one
step of ITS methods.
Surveys of Taylor series and other interval methods can be found in [4], [14], [15];
[ j l ] , [54], [60], [70], and [71]. These papers give a "high-level" description of existing
methods. A more detailed discussion of Taylor series methods c m be found in [52].
3.1 Validating Existence and Uniqueness of the
Solution: The Constant Enclosure Method
Suppose that at t j we have an enclosure [y j ] of y ( t j ; t,, [y,]). In this section, we consider
how to find a stepsize h j > O and an a priori enclosure [Gj] S U C ~ that for any y j E [ y j ]
has a unique solution y ( f ; t j , y j ) E [C j ] for t E [ t j , t j+ l ]*
The constant enclosure method [19, pp. 59-67], [44, pp. 27-31] for validating exis-
tence and uniqueness of the solution is based on the application of the Picard-Lindelof
operator
to an appropriate set of functions and the Banach fixed-point theorem.
THEQREM 3.1 Banach fixed-point theorem. Let O : Y -+ Y 6e defined on a complete
non-empty metric space Y with a metrie d (-, -). Let y satisfy O 5 7 < 1, and let
for ail x and y E Y . Then Q has a unique fixed-point y* E Y .
Let h j = t j + ~ - t j and [Qj] be S U C ~ that
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
(yj E [gj]) Consider the set of continuous functions on [tj, tj+1] with ranges in [yj],
For aj > 0, the exponential norm of a function E Co[tj7 tj+l] is defined by
The set U is complete in the maximum norm and therefore in the exponential norm.
By applying the Picard-Lindelof operator (3.1.2) to u E U and using (3.1.4), we
obtain
T maps U into itself.
Let Lj = 118 f ([cj])/dyll . It can be shown that the Picard-Lindelof operator is a
contraction on U in the exponential norm with aj > Lj7 which implies y = L j / a j < 1,
[19, pp. 66-67] (see also [44, pp. 27-29]).
Therefore, if (3.1.4) holds, and we can compute d f ([ej])/By7 then T has a unique
fixed point in U. This fixed point, which we denote by ~ ( t ; tj, yj), satisfies (3.1.1) and
y(t; tj, yj) E [yj] for t E [t;, tj+l]. Note that to prove existence and uniqiieness of the
solution of (3.1.1), we do not have to compute y < 1 such that the operator T is a
contraction. Note also that in bounding the kth Taylor coefficient over [cj] in Algori t hm II
(see §3.2), we evaluate f ik1 ([cj]). Because of the relation (2.4.22), if we cannot evaluate
a f ([cj])/ay, then we are not able to evaluate f [ k I ( [ ~ j ] ) .
Let h j m d [ej] be S U C ~ that'
We use superscripts on vectors to indicate different vectors, not powers.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES 20
Then (3.1.4) holds for any yj E [ y j ] , and (3.1.1) has a unique solution y ( t ; t j , y j ) that
sat isfies
for al1 t E [tj, t j c i ] and al1 yj E [ y j ] . Furthermore, since f ( [ $ ] ) f ( [ C j ] ) , we have
for d t E [t j7t j+i] md a11 yj E [ y j ] -
In (3.1.6), we should require [y j ] C [Y j ] and [ y j ] # [cj]- If [ Y ~ ] = [ci], then (3.1.6)
becomes
which implies either hi = O or f ( [ y j ] ) = [O, O ] . If none of the corresponding endpoints of
[ y j ] and [ijj] are equal, the stepsize, h j , can always be taken small enougli such that the
inclusion in (3.1.6) holds. In some cases, such a stepsize can be taken when some of the
endpoints of [y j ] and [y j ] coincide.
The inclusion in (3.1.6) can be easily verified. However, a serious disadvantage of the
method is that the stepsize is restricted to Euler steps, even when high-order methods are
used in Algorithm II to tighten the a priori enclosure. One can obtain better methods by
using polynomial enclosures [45] or more terms in the Taylor series for validation [50, pp.
100-1031, [13], [52]. We do not discuss the polynomial enclosure method in this thesis,
but propose in Chapter 5 a Taylor series method for validating existence and uniqueness.
In 58.4, we show by numerical experiments that our Taylor series method for validation
enables larger stepsizes than the constant enclosure method.
3.2 Computing a Tight Enclosure
Suppose that at the (j + l )s t step we have computed an a priori enclosure [ y j ] such that
In this section, we show how to compute in Algorithm II a tighter enclosure [ y j c l ] E [III7
Consider the Taylor expansion
where yj E [ y j ] and f [Y(y; t j 7 t j + i ) denotes f ik] with its [th component evaluated at y ( t j i ) ,
for sorne Ej l E [ t j , t j + l ] . If (3.2.1) is evaluated in interval arithmetic with yj replaced by
[ y j ] , and f [ k l ( y ; t j , t j+1) replaced b y f 'k'([ i j j ] ) , we obtain
With (3.2.2), we can compute enclosures of the solution, but the width of [ y j ] always
increases with j , even if the true solution contracts. This follows from property (2.1.3)
applied to (3.2.2),
where an equality is possible only in the trivial cases h j = O or ~ ( f ' ' l ( [ ~ j ] ) ) = 0,
i = 1 , . . . k - 1, and w ( f W ( [ i j ] ) ) = O.
If we use the mean-value evaluation (2.3.13) for cornputing the enclosures of the ranges
R (f [q; [ I J ~ ] ) , i = 1 , . . . , k- 1, ins tead of the direct evaluation f [d ( [ y j ] ) , we can often obtain
enclosures with smaller widths than in (3.2.2) [60]. By applying the mean-value theorem
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES 33
where J (f['l; yj , ijj) is the Jacobian of f[d with its [th row evaluated at yj + B i r ( c j - ~ j )
for sorne Bii E [O, 11 ( l = 1, . . . , n). Then from (3.2.1) and (3.2.3),
This formula is the basis of the interval Taylor series methods of Moore [4S], [49], [50],
Eijgenraam [19], Lohner [l], [44], [46], and Rihm [61] (see also [52]). Before we explain
how (3.2.4) can be used, we consider in 53.2.1 a major difficulty in interval methods: the
wrapping effect.
3.2.1 The Wrapping Effect
The wrapping effect is clearly illustrated by Moore's example [50],
The solution of (3.2.5) with an initial condition y0 is given by y ( t ) = A ( t ) y o , where ( cos t sin t ) - sin t cos t
Let y0 E [yo]. The interval vector [yo] E 1W2 c m be viewed as a rectangle in the ( y l , y2)
plane. At t l > to , [yo] is mapped by A(t l ) into a rectangle of the same size, as shown in
Figure 3.1. If we want to enclose this rectangle in an interval vector, we have to wrap it by
another rectangle with sides parallel to the yi and y2 axes. This larger rectangle is rotated
on the next step, and so must be enclosed in a still larger rectangle. Thus, a t each step,
the enclosing rectangles become larger and larger, but the set {A( t ) yo 1 y0 E [yo] , t > t o }
remains a rectangle of the same size. Moore [50, p. 1341 showed that at t = 27r, the
interval inclusion is inflated by a factor of e2" x 535, as the stepsize approaches zero.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
Figure 3.1: Wrapping of a rectangle specified by the interval vector ([-1,1], [lO,ll])T.
The rotated rectangle is wrapped at t = qn, where n = 1, . . . ,4.
Jackson [32] gives a definition of wrapping.
DEFINITION 3.1 Let T E RnXn, [XI E IRn, and c E Rn. Shen the wrapping of the
parallele piped
is the tightest interval vector containing P .
It can be easily seen that the wrapping of the set {TI + c 1 x E [ X I ) is given by T [ X I + c,
3.2.2 The Direct Method
A straightforward method for computing a tight enclosure [yj+1] at tj+1 is based on the
evaluation of (3.2.4) in interval arithmetic. From (3.2.4), since
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
we have
Here, [y j ] is an a priori enclosure of y ( t ; t j , [ y j ] ) for al1 t E [ t j , t j+l] , [y j ] is a tight enclosure
of the solution at t j , and J (f[d; I Y j ] ) is the Jacobian of f ['l evaluated at [ y j ] . We choose
Co to be the midpoint (we explain later why) of the initial interval [yo]. Then, we choose
That is, ijj+l is the midpoint of the enclosure of the point solution at t j+l starting from
9,. For convenience, we introduce the notation ( j 2 0)
k-1
[vj+ï] = Gj + C h ; fril(,) + hr f W ( [ i j j ] ) and i= 1
Using (3.2.8-3.2.9), (3.2.6) can be written in the form
By a direct method we mean one using (3.2.6), or (3.2. IO), to compute a tight enclosure
of the solutiun. This method is summarized in Algorithm 3.1. Note that from (3.2.7-
3.2.8) and (3.2.10), ijj+l = m ( [ ~ j + ~ ] ) = m ( [ ~ j + ~ ] ) . This equality holds because the
interval vector [S j ] ( [y j ] - i j j ) is symmetric.
Algorithm 3.1 Direct ~ e t h o d
Computing [Sj]
We show how the matrices [Si] can be computed [l]. Consider the variational equation
It can be shown that
where ~ [ d is defined in (2.4.19-2.4.20), and J (f ['l; y) is the Jacobian of j[iI. Then, from
the Taylor series expansion of I( t ) and (3.2.11-3.2. 12), we have
Q(tj+i) = I + C hi J (f [d; y ( t j ) ) + (Remainder Term). i= 1
the interval matrices [Sj] can be computed by computing the interval Taylor series (3.2.14)
for the variational equation (3.2.11).
Alternatively, the Jacobians in (3.2.14) can be computed by differentiating the code
list of the corresponding fal, [5], [6].
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
Wrapping Efïect in the Direct Method
If we use the direct method to compute the enclosures [y,], we might obtain unacceptably
large interval vectors. This can be seen from the following considerations [60].
Using (3.2.10), we compute
[~ j+ l I = [vj+lI + IsjI ([YjI - G j )
= Ivj+lI + [SjI ([vjI - Yj)
+ [Sjl ([Sj-il (IV,-il - Yj-1))
+...
+ [Sj] ([Sj-11- - *.([Sl] ([SOI ([vo] - CO))) . ), where [vol = [yo]. Note that the interval vectors [vil - cl (1 = O , . . . , j ) are symmetric,
and denote them by [&] = [ut] - cl- Let us consider one of the summands in (3.2.16), for
To simplify our discussion, we assume that the matrices in (3.2.17) are point matrices
and denote them by Sj , Sj-17.. . , So. We wish to compute the tightest interval vector
that contains the set
This set is the same as
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPs FOR ODES
which is wrapped by the interval vector
(see 53.2.1). In practice, though, we compute
and we can have wrapping at each step. That is, we first compute So [JO], resulting in
one wrapping, then we compute Sl(So [JO]), resulting in another wrapping, and so on.
We can also see the result of the wrapping efFect if we express the widths of the interval
vectors in (32.18) and (3.3.19):
Frequent ly, w ((SjSj-l - - Si So) [do]) < w (Sj(Sj-1 - (Si (So [do])) - - - )) for j large, and
the direct method often produces enclosures with increasing widths.
By choosing the vectors Yi = rn ([vil), we provide symmetric intervals [vil - c l , and
by (2.2. IO), we should have smaller overestimations in the evaluations of the enclosures
than if we were to use nonsymmetric interval vectors.
Contracting Bounds
Here, we consider one of the best cases that can occur. If the diagonal elements of
J ( f I i J ; are negative, then, in many cases, we can choose h j such that
That is, [yj] - ijj propagates to a vector [Sj]([yj] - cj) at tj+1 with smaller norm of the
width than Il ~ ( [ y j ] ) 11.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
3.2.3 Wrapping Effect in Generat ing Int erval Taylor
Co efficients
Consider the constant coefficient problem
In practice, the relation (2.4.22) is used for generating interval Taylor coefficients. With
this relation, we compute interval Taylor coefficients for the problem (3.2.20) as follows:
Therefore, the computation of the i th Taylor coefficient may involve i wrappings. In
general, this irnplies that the cornputed enclosure of the kth Taylor coefficient, f['.l([~j]),
on [tj, tj+i] may be a significant overestimation of the range of ~ ~ ( " ( t ) / k ! on [tj, tj+t]. As
a result, a variable stepsize control that controls the width of h ; ~ [ ~ ] ( [ i j ~ ] may impose a
stepsize limitation much smaller than one would expect. In this example, it would be
preferable to compute the coefficients
which involves at most one wrapping.
directly by
In the constant coefficient case, we can easily avoid the evaluation (3.2.21) by using
(3.2.22)) but generally, we do not know how to reduce the overestimation due to the
wrapping effect in generating interval Taylor coefficients.
CHAPTER 3- TAYLOR SERIES METHODS FOR IVPS FOR ODES
3.2.4 Local Excess in Taylor Series Methods
We consider the overestimation in one step of a Taylor series method based on (3.2.4).
The Taylor coefficient f [kI(y; tj, tj+l) is enclosed by f [kI([fij])- If [Gj] i~ a good enclosure
of y(t; tj, [yj]) on [tj, tj+t], then Ilw ([Gj]) I I = O(hj), asuming I[w([yj])l[ = O(hS) for
some r 2 1. From (2.3.12)? the overestimation in f [kI([gj]) of the range of f tk] over [Qj] is
O(IIW ([fij]) 11) = O(hj). Therefore, the overestimation in hf f[kI([~j]) is 0(h tc ' ) .
The matrices J (f Id; y j , ijj) are enclosed by J (f [.l; That is, by evaluating the
Jacobian of f [q on the intervd [yj]- As a result, the overestimation frorn the second line
in (3.2.4) is of order 0(hjll~([~j])l12)7 [19) pp. 87-90]. This may be a major difficulty
for problems with interval initial conditions, but should be insignificant for point initial
conditions or interval initial conditions with srnall widths, provided that the widths of
the computed enclosures remain sufficiently srnall throughout the computation.
Hence, if f [ k I ( y ; t j , tj+i) and J (f[q; yj, ijj) are enclosed by f[kj([&]) and J ( f[']; [TJ~]),
respectively, the overestimation in one step of Taylor series methods is given by
We refer to this overestimation as local excess and define it more formally in 56.1. Ad-
vancing the solution in one step of Taylor series methods usually introduces such an
excess (see Figure 3.2).
We should point out that by computing h: f[kI([gj])7 we bound the local truncation
error in ITS methods for al1 solutions y(t;tj,yj) with yj E [yj]. Since this includes al1
solutions y ( t ; to, yo) with y, E [Y,], we are in effect bounding the global truncation error
too. Thus, the distinction between the local and global truncation errors is sornewhat
blurred. In this thesis, we cal1 h; f[kl([ijj]) the truncation error. A similar use of the
truncation error holds for the IHO method discussed later.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
Figure 3.2: If [ ~ j ] is an enclosure of the solution at t j ; then the enclosure [zJ~+,] at tj+l
contains y(tj+i; t j7 [ y j ] ) and the Local excess.
3.2.5 Lohner's Met hod
CVe derive Lohner's method from (3.2.4) in a d ifferent way t han in [l] , [44], and [46
show how [y,] and [y,] are computed and then give the algorithm for any [ y j ] .
Let
where [Sj] is defined in (3.2.9). Also let
Ao = 1, 90 = = ([yo]) , and r o = y, - Co E [ro] = [yo] - go,
where I is the identity matrix.
Using the notation (3.2.24-3.2.28), we obtain from (3.2.4)
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
where AI E RnXn is nonsingular and
We explain later how the matrices Aj ( j 2 1) can be chosen.
Similarly,
where A2 E RnXn is nonsingular and
Continuing in this fashion, we obtain Lohner's method.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES 32
Alaorit hm 3.2 Lohner's Method
The Parallelepiped Method
If Aj+1 = m ([Sj] -4j), then we have the parallelepiped method for reducing the wrapping h
effect. Let Sj = rn ([Sj]) and [S,] = + [Ei] Shen
Since
if IIg;l[~j] 11 is small and cond(Aj) is not too large, then AZ1 ([Sj] Aj) x I . AS a result,
there is no large overestimation in the evaluation of (A$~([S~] Aj)) [rj]. However, the
choice of Aj+l does not guarantee that it is well conditioned or even nonsingular. In fact,
may be il1 conditioned, and a large overestimation rnay arise in this evaluation.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPs FOR ODES
The QR-factorization Method
We describe Lohner's QR-factorization method, explain how it works, and illustrate i t
with a simple example. CI n
Let E [Sj] Aj, and let Aj+l = -Zj+ipj+i, where Pj+1 is a permutation rnatrix.
We explain Iater in this subsection how Pj+1 is chosen. We perform the QR-factorization h
Aj+l = Qj+lRj+17 where Qj+1 is an orthogonal rnatrix, and Rj+1 is an upper triangular
matrix. If Aj+l is chosen to be Qj+1 in Algorithm 3.2, we have the QR-factorization
method for computing a tight enclosure of the solution.
We now give an intuitive explanation of how this method works. At each step, we
want to compute an enclosure of the set
that is as tight as possible. Consider first the set
If IIA;iiII is not much larger than 1, then
will not be much larger than Ilw ([zj+i]) 1 1 . In this method, A;:, = Q$, = Q L l is
orthogonal, so I[AZl 1 1 5 fi In addition, w ([zj+1]) can be made small by reducing the
stepsize or changing the order of the Taylor series. Therefore, the set (3.2.30) can be
enclosed in the interval vector
whose width can be kept small.
Consider now the set
CHAPTER 3. TAYLOR SERIES ~ ~ E T H O D S FOR IVPS FOR ODES
in (3-2.29). If Aj+l E { S j ~ j 1 Sj E [Sj] ) [Sj] Aj md tü ([Sj]) is small, then
From (3.2.31) and (3.2.32), we have
Note that Aj+i [rj] iç the wrapping of the set
while ( ~ ~ ~ A j + ~ ) [ r ~ ] is the wrapping of the set
which is the set {r j E [rj]) mapped by Aj+t and then the result mapped by Qg, The vector corresponding to the first column of Qjti is parallel to the vector corre-
h
sponding to the first column of The matrix Q j+l induces an orthogonal coordinate
system, where the axis corresponding to the first column of Qj+1 is parâllel to those edges
of the parallelepiped (3.2.34) that are parallel to the first colurnn of Âjci- Intuitivelx
we can expect an enclosure with less overestimation in the coordinate system induced by
Qj+1 than in the original coordinate system. Furthermore, if the first column of Qjt1 is
parallel to the longest edge of the parallelepiped in (3.2.34), we can expect a better result
than if this column were parallel to a shorter edge. This is the reason for rearranging the -
columns of Aj+1 by the permutation matrix Lohner suggests that Pj+l be chosen
such that the first column of Âj+1 corresponds to the longest edge of (3.2.34), the second
column to the second longest and so on.
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES 35
N - If II - Il2 is the Euclidean n o m of a vector, Aj+lYi is the ith column of Aj+i7 and [rjli
is the i th component of Irj], then the lengths of the edges of (3.2.34) are given by
Let 1 = (11, 12r , ln)T. The matrix Pj+1 is such that the components of lTPj+l are
in non-increasing order (from 1 to n). As a result, the vector corresponding to the first n N
column of Aj+1 = Aj+1 Pj+1 is parallel to the longest edge of (3.2.34), and the first column
of Qj+l is parallel to that edge as well.
Example Let
A = (' 2 ') 1 and [r]= (;::::). The QR-factorization of A is
Consider the set
{ ~ r 1 r E [r]). (3.2.37)
The parallelepiped specified by [r] (see Figure 3.3(a)) is mapped by A into the paral-
lelepiped shown in Figure 3.3(b). The filled part in Figure 3.3(b) is the overestimation
of (3.2.37) by A[r] . However, if the set in (3.2.37) is wrapped in the coordinate sys-
tem induced by Q, we obtain a better enclosure (less overestimation) of this set (see
Figure 3.3(c)).
Consider now the set
{ Q - ' A ~ 1 r E [ r ] ) . (32.35)
The matrix Q-' maps (3.2.37) into a parallelepiped with its shorter edge parallel to the
original x a i s . As a result, the wrapping of (3.2.38) is (Q-lA)[r] (see Figure 3.3(d)).
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES
Figure 3.3: (a) The set {r ( r E [r]).
(b) { ~ r r E [r] ) enclosed by A [r] .
(c) { ~ r 1 r E [ r ] ) enclosed in the coordinate system induced by Q.
(d) {(Q-'A)T 1 r E [r]) enclosed by (Q-' A) [TI. (e ) { A T 1 r E [r]) enclosed in the coordinate systern induced by 0. (f) { ( @ ' ~ ) r 1 r E [r] ) enclosed by (@'A) [î].
CHAPTER 3. TAYLOR SERIES METHODS FOR IVPS FOR ODES 37
Now, interchange the columns of A, denote the new matrix by Â, and cornpute the
QR-factorization
If we wrap the set (3.2.37) in the coordinate systern induced by Q (see Figure 3.3(e)), rve
obtain a better enclosure than in the coordinate system induced by Q. In Figure 3.3(f),
the parallelepiped { ~ r 1 r E [ r ] ) is rotated by 0-1. The longest edge of the rotated
parallelepiped is parallel to the x axis, and the overestirnation in (0-'A)[T] is smaller
than in (Q-'A)[r] and A[r] .
To summarize, let A E IRnXn, [r] E ERn, and A = QR, where Q is an orthogonal
matrix and R is an upper triangular rnatrix. Normally, if we wrap t h e parallelepiped
{ A r 1 r E [ T I } in the coordinate system induced by Q, we obtain a better enclosure than in
the original coordinate system. Moreover, if we rearrange the columns of A, as described
in this subsection, before computing Q, we usually obtain a better enclosure than without
rearranging t hose columns.
Chapter 4
An Interval Hermit e-Obreschkoff
Method
In this chapter, we derive an interval Hermite-Obreschkoff (IHO) method and compare
it with the "standard" interval Taylor series methods.
Hermite-Obreschkoff methods are usually considered for computing an approximate
solution of a stiff problern [22], [24], ['El, [78]. Here, we are not interested in obtaining
a method that is targeted specifically to solving stiff problems-our purpose is to obtain
a general-purpose method that produces better enclosures at a smaller cost than the
explicit validated methods based on Taylor series.
Hermite-Obreschkoff methods have smaller truncation errors and better stability than
Taylor series methods with the same stepsize and order. AIso, for the same order, the
IHO method needs fewer Taylor coefficients for the solution to the IVP and its variational
equation than an ITS method. However, the former requires that we enclose the solution
of a generally nonlinear system, while the latter does not. The extra cost of enclosing
such a solution includes one matrix inversion and a few matrix-matrix multiplications.
The method that we propose consists of two phases, which can be considered as a
predictor and a corrector. The predictor cornputes an enclosure of the solution at
( 0 ) t j + ~ . Using (Y~+,], the corrector computes a tighter enclosure [yjil] C at tjiL.
In the next section, we derive the interval Hermite-Obreschkoff method; in 54.2, we
give an algorithmic description of it; and in 54.3, explain why the IHO method may
perform better than ITS methods.
4.1 Derivation of the Interval Hermite-Obreschkoff
Method
First, in 84.1.1, we show how the point Hermite-Obreschkoff method can be obtained.
Then in s4.1.2, we outline our new IHO method. Fioally, in 54.1.3, we derive it: we
describe how to improve the predicted enclosure and how to represent the improved
enclosure in a rnanner that reduces the wrapping effect in propagating the solution.
4.1.1 The Point Method
Let
4 . p = q! ( q + p - i ) ! , and ( P + Q ) ! ( 9 - i l !
where p 2 O, q 2 0 , O 5 i 5 q, and g ( t ) is any ( p + q + 1) times differentiable function.
If we integrate J,' ~,,~(s)~(p+q+~)(s) ds repeatedly by parts, we findL
If y ( t ) is the solution to the IVP
IThis derivation is sometimes attributed to Darboux [16] and Hermite [28].
where y j + ~ = y ( t j + h;), and the functions f are defined in (2.4.19-2.4.20)- Also,
where the lth component of y(p+q+l)(t; tj, tj+') i~ evaluated at some Cji E [tj, tj+l].
From (4.1.4) and (4.1.7-4.1.9),
For a given yj, if we solve the nonlinear (in general) system of equations
for yj+l, we obtain an approximation of local order O(hpCq+') to the solution of (4.1.5).
The system (4.1.11) defines the point (q, p) Hermite-Obreschkoff method [22], [24], [27,
1. If p > O and q = O, we obtain an explicit Taylor series formula:
2. If p = O and q > O, then (4.1.10) becomes an implicit Taylor series formula:
Therefore, we can consider the Hermite-Obreschkoff methods that we obtain from (-2.1.10)
as a generalization of Taylor series methods.
4.1.2 An Outline of the Interval Method
Suppose that we have computed an enclosure of the solution a t t j - The idea behind our
IHO method is to compute bounds on the solution at tj+l, for d l yj in the solution set at
t j , by enclosing the solution of the generally nonlinear system (4.1.10). We enclose this
solution in two phases, which we denote as a predictor and a corrector.
PREDICTOR: Cornpute an enclosure of the solution at tj+l using an interval Taylor
series method of order (p + 1).
CORRECTOR: Improve this enclosure by enclosing the solution of (4.1.10).
In the corrector, we perform a Newton-like step to tighten the bounds computed by
the predictor. From (4.1.10), we have to bound the (p + q + 1)st Taylor coefficient on
[tj7 tj+i]. We can enclose this coefficient by generating it with the a priori enclosure
computed in Algorithm 1. This computation is the same as enclosing the remainder term
in ITS methods (see 53.2).
4.1.3 The Interval Method
Suppose that we have computed [yj], Cj, Aj, and [rj] a t t j S U C ~ that
where ej = m ( [ y j ] ) , Ai E Rn'" is nonsingular, and [rj] E IRn. The interval vectors [y j ]
and ijj + Aj[rj] are not necessarily the same. We use the representation {ij, + Ajrj 1 rj E
[ r j ] ) to reduce the wrapping effect in propagating the solution and the representation
[ y j ] t o compute the coefficients for the solution to the variational equation (see $32.2) .
Suppose also that we have verified existence and uniqueness of the solution on [ t j7 t j+1]
(0) and have computed an a priori enclosure [i,] on [tj7 tj+i] and an enclosure 5 [Ij] a t
t j+1 . We show in 54.2.2 how to compute in the predictor. Here, we describe how
to constmct a corrector based on (4.1.10).
Our goal is to cornpute (at t j c1) a tighter enclosure [yj+,] of the solution than
and a representation of the enclosure set in the form
(0) for some y0 E [yo], and gzl = rn ( [y j+l] ) . Since
we can apply the mean-value theorem to the two sums in (4.1.10) to obtain
where J ( ~ ( 4 ; yj+i, Gol) is the Jacobian of f['l with its lth row evaluated a t yj+i + ~ i l ( $ : = ) ~ - yj+i) for some Bi, E [O, 11, and J (f ['l; yj7 cj) is the Jacobian of f fil with its Zth
row evduated at yj + vii(ej - y j ) for some E [O, 11, 1 = 1 , . . . , n.
Using (4.1.14), we show how to compute a tighter enclosure than at t j + l .
Let
With the notation (4.1.15-4.1.23), we write (4.1.14) as
there exists rj E [rj] S U C ~ that yj - cj = Ajr j . Therefore, we c m transform (4.1.24) into
h
For small h j , we can compute the inverse of Sj+1,-. Then from (4. L -25) and using (4.1 .l5-
4.1 .Z),
We compute an interval vector that is a tight enclosure of the solution at tj+l by
[ ~ j + l I = ($;Tl + [BjI [rj] + [cj] [vj] + 3~:':l,- [ ~ j + L I ) [el] 3 (4.1-27)
tvhere "n" denotes intersection of interval vectors. For the next step, we propagate
Cj+l = ( [ y j + ~ I ) 7 (4.1.28)
which is the Q-factor from the QR-factorization of m ([Bi]) , and
Remarks
1. Since we enclose the Taylor coefficient
the overestimation in the t e m h?+'+l f '+'+']([ijj]) is of ~(h;+'+'), provided t hat
II~([@j])ll = O(hj); see 53.2.4. Therefore, the order of the IHO method is ( p + q + 1).
Note that in the point case, the order of an Hermite-Obreschkoff method is (P + q).
In 58.2, we verify empirically that the order of an IBO method with p and q is
indeed (p + q + 1).
2. We have explicitly used the inverse of gj+l,- in our rnethod. This is due in part
to the software available to us. It may be useful to consider other ways to perform
this computation at a Iater date.
A
3. We could use the inverse of the interval matrix [Sj+1,-] instead of Sj;', -. However,
it is easier to compute the enclosure of the inverse of a point matrix than of an
interval matrix. In fact, computing a tight enclosure of the inverse of an interval
matrix is N P hard in general 1631.
h
4. In (4-1-27), we intersect ~2~ + [Bj][rj] + [Cj][vj] + S;:L,-[dj+i] and As a
result, the computed enclosure, [ ~ j + ~ ] , is always contained in
Therefore, we can never compute a wider enclosure than [y,!=)l].
( 0 ) 5. Once we obtain [ Y ~ + ~ ] , we can set [Yj+l] = [yj+1] and compute another enclosure,
hopefully tighter than [yj+l], by repeating the same procedure. Thus, we can
improve t his enclosure iteratively. The experiments that we have performed show,
however, that this iteration does not improve the results significantly, but increases
the cost.
6. If we intersect the cornputed enclosure as in (4.1.27), it is important to choose
A (0) + E [yj+i]. If we set Cj+l = yj+,, it rnight happen that cj+l = 62, $ [yj+1],
( 0 ) because is the midpoint of [yj,,], which is generally a wider enclosure than
l ~ j + i l -
7. The interval vectors [rj] ( j > 0) are not symmetric in general, but they are sym-
metric in Lohner's method (see 53.2.5).
4.2 Algorithmic Description of the Interval Hermite-
Obreschkoff Method
In this section, we show how to compute the coefficients and c:". Then, we describe
the predictor and corrector phases of the IHO method in a form suitable for implemen-
tation.
4.2.1 Computing the Coefficients < l q and c;lP
From (4.1.2)
Since c;lP = 1, we can compute the coefficients c;J for i = 1 , . . . , q by (4.2.1). In a similar
way, we compute Cvq for i = 1,. . . ,p.
4.2.2 Predict ing an Enclosure
( 0 ) We compute an enclosure [yj+,] for the solution at tj+l by Algorithm 4.2, which is part
of Lohner's method (see 83.2.5).
Algorithm 4.1 Compute the coefficients c $ ' ~ and
COMPUTE:
CQJ' .- (-, .- 1;
for i := 1 to q
cf" := - i f l ) / ( q + p - i + 1);
end
GTq := 1;
for i := 1 to p
4" := <21(p - i + l ) / ( q + p - i + 1);
end
OUTPUT:
for i = O,. .. , p ;
cOppl for i = O,. .. q.
Algorithm 4.2 Predictor: cornpute an enclosure with order q + 1.
4.2.3 Improving the Predicted Enclosure
Suppose that we have cornputed an enclosure of y ( t j+1 ; to, [%]) with Algorithm 4.2.
In Algorithm 4.3, we describe an algorithm based on the derivations in 54.1.3 for improv-
Remarks
1. We could use the a priori enclosure [g j ] from Algorithm 1 instead of computing
( 0 ) (0) [ Y ~ + ~ ] . We briefly explain the reasons for computing [yj+,].
( a ) The a priori enclosure [ij,] rnay be too wide and the corrector phase may not
produce a tight enough enclosure in one iteration. As a resuft, the corrector,
which is the expensive part, may need more than one iteration to obtain a
tight enough enclosure (see 58.3.2, p. 110).
(b) Predicting a reasonably tight enclosure is not expensive: we need to
generate the terrns and [Fjt i] , for i = 1, . . . , q. We need them in the cor-
rector, but for i = 1,. . . , p. Usually, a good choice for q is q € {P, p + 1, p + 3)
(see 54.3.1). Therefore, we do not create much extra work when generating
these terms in Algorithm 4.2.
2. Algorithm 4.3 describes a general method. If, for example, the problem being
solved does not exhibit exponential growth of the widths of the enclosures due to
the wrapping effect, we do not have to compute a QR-factorization and represent
the enclosure as in (4.1.13).
3. The matrix Aj+1 is a floating-point approximation to an orthogonal matrix. Since
is not necessarily equal to the transpose of Aj+1, A$, must be enclosed in
interval arithmetic.
Algorithm 4.3 Corrector: improve the enclosure and prepare for the next step.
INPUT:
p, q, < ' q f o r i = 0 , . . . , p 7 c:"fori=O ,..., q;
(0) hj, Y j , Aj) [rj]) [ ~ j + l I ;
f i i f o r i = 1 7 ..., g; [ Z j + l ] := h,P+qf l f [P+ q f 11 ( [ g j l ) --
4. It is convenient to cornpute the terms hi f [ ' J ( [ ~ j ] ) for i = 1 ,2 , . . . , ( p + q + 1) in
Algorithm 1 (see Chapter 7). Then, we do not have to recompute h j f l f [ q ' l l ( [ i j j ] )
in the predictor and hjpfq+l f h q f l l ( [ i j j ] ) in the corrector.
4.3 Cornparison with Interval Taylor Series Methods
We explain why the fHO method may perform better than the ITS rnethods. First, in
$4.3.1 and 54-32, we show that on constant coefficient problems, the IHO method is more
stable and produces smaller enclosures than an ITS method with the same stepsize and
order. Then, in 54.3.3, we study one step of these methods in the general case and show
again that the IHO should produce smaller enclosures than the ITS methods. Finally, in
$4.3.4, we consider the amount of work in one step of each of these rnethods.
In this section, we assume that both methods have the same order of the truncation
error. That is, if the order of the Taylor series is k, we consider an IHO method with p
and q such that p + q + I = k.
4.3.1 The One-Dimensional Constant Coefficient Case.
Instability Results
Consider the problem
where X E R and X <
DEFINITION 4.1 We say that an interval rnethod for enclosing the solution of &.KI)
with a constant stepsize is asymptotically unsta6le, if
'Since we have not defined complex interval arithmetic, we do not consider problems with X complex.
In this and in the next subsection, we consider methods with constant stepsize h for
simplicity of analysis.
The Interval Taylor series method
Suppose that at t j > O, we have computed a tight enclosure [y:TS] of the solution with
an ITS method, and [ij:TS] is an a priori enclosure of the solution on [tj7 t j+ l ]7 for al1
and let
Using (4.3.2-4.3.3), an interval Taylor series method for computing tight enclosures of
the solution to (4.3.1) can be written as
cf. (3.2.4). Since w([~j '~' ] ) 2 ~([~j'*~]), we derive from (4.3.2-4.3.4)
Therefore, the ITS met hod given by (4.3.4) is asymptotically unstable for stepsizes h
such that
This result implies that we have restrictions on the stepsize not only from the function
Tk-l(Xh), as in point methods for IVPs for ODES, but also from the factor IXhlk/b! in
the remainder term. Note also that the stepsize restriction arising from (4.3.5) is more
severe than the one that would arise from the standard Taylor series methods of order k
or k + 1 .
The Interval Hermite-Obreschkoff method
Let yj E [ y i H o ] , where rve assume that is computed with an IHO method and
r H 0 [y0 ] = [ y0 ] . Frorn (4.1.10), the true solution yj+l corresponding to the point yj satisfies
where [ E [ t j , t j + i ] Let
-.
i=O
where c;lp ( C P V q ) are defined in (4 .1 .2) . Also let
(k = p + q + l), where [$Ho] is an a priori enclosure of the solution on [ t j , t j+ l ] for any
yj E IyjlH01.
Let y,, = q ! p ! / ( p + q)!. From (4.3.6-4.3.9), we compute a tight enclosure [Yiff'] by
r H 0 I H O 7 ~ ~ 9 I H O bj+l I = %,q(Xh)[yj 1 + (-1)'
Qp,q(Xh) [%+L 1-
From (4.3.9-4.3.10),
Therefore, the IHO method is asymptoticalIy unstable for h such that
In (4.3.5) and (4.3.11),
are approximations to ez of the same order. However, &,,,(z) is a rational Padé approxi-
mation to ez (see for example [59]). If z is complex wit h Re(=) < 0, the following results
are linown:
(see also [42, pp. 236-2371). Consider (4.3.5) and (4.3.11). For the ITS method,
ITk-l(Ah)l < 1 when Ah is in the stability region of Tk-'(2). However, for the IHO
rnethod with X E R, X < O , IR,,,(Xh)l < 1 for any h > O when q 2 p, and &,,(Ah) + O
as Ah -t -00 when q > p. Roughly speaking, the stepsize in the ITS met hod is restricted
by both
l Ahlk ITk-l(Xh)l and - k! '
while in the IHO method, the stepsize is limited mainly by
Since yplq/ 1 QpA(Xh) 1 is usually much smaller than one, I x ~ l k / l c ! < 1 implies a more severe
restriction on the stepsize than (4.3.12). Thus, the stepsize limit for the IHO method is
usually much larger than for the ITS method.
An important point to note here is that an intervai version of a standard numerical
method that is suitable for stiff problems may still have a restriction on the stepsize. To
obtain an interval method without a stepsize restriction, we must find a stable formula
not only for advancing the step, but also for the associated truncation error.
Consider again (4.3.4) and (4.3.10). From (4.3.4), we can derive
The width of [y::f] is
We derive from (4.3.10),
The width of is
and if we assume that
r H 0 w ([yo!) = O and [zi ] x [ z ! ~ ' ] ,
then from (4.3.13) and (4.3.14),
for i = 1,2, . . . j + 1,
(4.3.15)
That is, for X < O and small h, the widths of the intervals in
proximately / 1 Qplq (Ah) 1 < 1 times t h e corresponding widths
by the ITS method. As the stepsize increases, I~k-l(r\h)l + 1
the IHO method are ap-
of the intervals produced
~ h l ~ l k ! becomes greater
than one. Then, the ITS method is asymptotically unstable and produces intervals with
increasing widths. For the same stepsizes, the IHO method may produce intervals with
decreasing widths when q > p.
In Table 4.1, we give approximate values for yplq = q ! p ! / ( p + q)!, for p = 3,4, , . . . ,13
and q E {p, p + 1, p + 2). As can be seen from this table, the error constant q ! p ! / ( p + q)!
becomes very small as p and q increase.
In 58.3.1, we show numerical results comparing the ITS and IHO rnethods on (4.3.1)
for X = -10.
Table 4.1: Approximate values for 79,,,, p = 3,4,. . . 13 , q E {p, p + 1, p + 7).
4.3.2 The n-Dimensional Constant Coefficient Case
Coasider the IVP
where B € Etnxn and n > 1.
We compare one step of an ITS method, which uses Lohner's technique for reducing
the wrapping effect, and one step of the IHO method, which uses a similar technique
for reducing the wrapping effect. Then, we compare the enclosures after several steps of
these methods. We assume that in addition to an enclosure [ y j ] of the solution at t j 7 we
also have a representation of the enclosure in the form
where cj E [yj], Aj E RnXn is nonsingular, and [rj] E IRn. We also assume that we have
an a priori enclosure [&] of the solution on [tj, tj+l]7 where h = tj+i - t j .
Enclosures &ter One Step
The Interval Taylor Series Method Using (4.3.3)' we can write an ITS method,
with Lohner's coordinate transformation, as
where
The Interval Hermite-Obreschkoff method Using (4.3.7-4.3.5) and (4.3.19), the
(Note that for h small, we can compute the inverse of the rnatrix Qp,q(hB).) The width
of [yip] is given by
w([yj:HPI) = IRp,q(h~)~jIw([r i I ) + %,q 1 ( Q P . ~ ( ~ B ) ) - ' I W ( [ ~ ~ + ~ I ) - (4.3.21)
Comparing (4.3.21) and (4.3.20), we see that in the IHO method we multiply the
width of the error terrn, Z U ( [ Z ~ + ~ ] ) , frorn the ITS method by y p , q l (Q, , , (~B))- ' 1. If, for
example, p = q = S1 then
y8.8 z 7.8 x 10-~
(see Table 4.1). Consider ( Q , , ~ ( ~ B ) ) - ' and suppose that q > O. For small h,
This implies that for small h, multiplying by the rnatrix 1 (QPpq(h B ) ) -' 1 does not sig-
nificantly increase ~ ( [ z j + ~ ] ) . Furthermore, it often happens that II (Q, , , (~B) ) -' Il < 1.
Hence, multiplying by this matrix may reduce w([zj+l]) still further.
In Lohner's method, we propagate ( T ~ - ' ( ~ B ) A ~ ) [ri], where Tk-1 ( h B ) is an approxi-
mation of the matrix exponential of order k:
In the IHO method, we propagate ( & , q ( h B ) ~ j ) [ri], where &,,,(hB) is a rational approx-
imation to the matrix exponential of order k:
If h B is small, then
Enclosures after Several Steps
Now, we study how the enclosures propagate after several steps in the ITS and the IHO
methods. For simplicity, we assume that the matrix B in (4.3.16) is diagonalizable and
can be represented in the form
where D = diag(Xl, XÎ, . . . , A,) and {A1, X2, . . . , A n ) are the eigenvalues of B.
DEFINITION 4.2 We Say that an interval method for enclosing the solution of (4.f.16)
with a constant stepsize is asymptotically unstable, if
The Interval Taylor series method We compute [yiTS] by
where
and is an a priori enclosure of the solution on [t;, ti+l] for al1 2/i E Then,
instead of representing the enclosure at t l in the form of a parallelepiped as in (4.3.17)
m d computing an enclosure a t t2 by the formula (3.3.18), we assume that we compute
where there may be wrappings in the evaluation of ( ~ k - 1 (h B ) ) ~ [yo] and Tk-1 ( h B ) [-.il.
Following this procedure, we assume that we compute [ y j r ; f ] by
We make this assumption to obtain a simple formula for in terms of [yo] and [z!~'] ,
i = 1, . . . , (j + 1). Otherwise, if we had used (4.3.18), we would have products involving
the transformation matrices Aj and a more complicated formula to analyze. The formula
(4.3.22) gives, in general, tighter enclosures than (4.3.18) (see 332.2).
The width of is given by
The Interval Hermite-Obreschkoff method Similar to the considerations in $4.3.1
and in the previous paragraph, we can derive for the IHO method
j+i I H O
b j + i I = (%q ( h B ) ) j f L [YOI + (-l)'yp,q C ( & l q ( h ~ ) ) j + l - i ((Q,,, ( h B ) ) - l [ z I ~ o ] )
where
Consider (4.3.23) and (4.3.24) and suppose that Re(Xi) < O for i = 1 , ..., n. The
matrices Tk-1 ( h D) and & , , q ( h D) are diagonal with diagonal elements Tk-1 (hX;) and
&,,(hXi), respectively, where X i is an eigenvalue of B. As h increases, IITk-l(hD)II will
eventually becorne greater than one, and then the ITS method is asymptotically unstable.
However, for any h > O , IIRp,q(hD)II < 1 for q = p, p + 1, or p + 2, and II&,q(hD)II -t O
as h t w for q = p + l or q = p + 2 (see 54.3.1). Therefore, if we ignore the wrapping
effect, the IHO method does not have stability restrictions from the associated stability
function l$,,q(z) when q = {p, p + 1, p + 2). However, it still has a restriction from the
formula for the truncation error.
We can show for the ITS method that
and for the IHO method that
These two inequalities suggest that the restriction on the stepsize in the IHO method
occurs at values significantly larger than in ITS methods.
As in the previous subsection, if w([yo]) = O, then for small stepsizes and small h B,
we should expect
Moreover, for larger stepsizes and eigenvalues satisfying Re(Xi) < O, i = 1, .... n, the IHO
with q = p, p + 1, or p + 2 is more stable than the ITS method.
In 58.3.1, we show numerical results comparing the two met hods on a two-dimensional
constant coefficient problem.
4.3.3 The General Case
Comparing an ITS method with the IHO rnethod in the nonlinear case is not as simple
as in the constant coefficient case. We can easily compare the corresponding remainder
terms on each step, but we cannot make precise conclusions, as in the constant coefficient
case, about the propagation of the set { ~ j r ~ 1 rj E [ r j ] ) . However, we show by numerical
experiments in 58.32 the advantages of the IHO method over ITS methods on some
nonlinear problems.
The interval Taylor Series Method
The Interval Hermite-Obreschkoff Method
From (4.1.26), we compute a tight enclosure by the formula
For simplicity in the discussion, we do not intersect with as in
an intersection produces an enclosure [yj+l] C [y:F]. Therefore, Our conclusions are
valid for [zJ~+~]. The width of is3
([YZYI) = I (321,- [s j ,+ l )~ j lw([ r j l ) + 1321.- I W ( [ J ~ + ~ I )
( 0 ) + Ir - g21,- [sj+l,-1 I W ( [ Y ~ + ~ I ) *
Let again k = p + q + 1 and consider the terms in (4.3.27).
h
The term ISzl,- 1 ~([6,+1]). Since Z U ( [ ~ ~ + ~ ] ) = w ( [ c ~ + ~ ] ) ~ (see (4.1.23)),
h h
Is,*ll- lw([6j+l]) = ISj*l ,-I~([i+ll) = ( ~ p , q l S j ; l l , - l ) h ~ ~ ( f ' ~ I ( [ ~ j l ) ) * (4-3-25)
Cornparing the terms involving hrw( f [kI([ijj])) in (4.3.26) and (1.3.28), we see that h
in (4.3.28) the reduction is roughly assuming that the components of 1 s$,,- 1 are not large (which is the case if h j is sufficiently small). This situation is similar
to the n-dimensional constant coefficient case.
31f [rj! is symmetric, then for an interval rnatrix [A], w([A][rj]) = I[A]lw([rj]) (cf. (2.2.10)). The interval vector [rj] ( j > O ) is symmetric in Lohner's method, but it is generally nonsymmetric in the IHO method. Assurning [yj] symmetric, ive obtain a simple formrila as in (4.3.27).
h
The term Ii - sZl,- [sj+l,-] l . ~ ( [ y ( i " ~ ] ) . Let
h
[Si+,,-] = sj+l,- + [-Ej+i Ej+t] 7
h
where Ej+1 is a point matrix, and Sj+,,- = rn ( [S j+ l , - ] ) ; cf. (4.1.15). Then
1 ^, 2
af (O) - h ~ l ~ i i + l ~ - l ~ ( G ( [ y j + l l ) )
If I I W ( [ & ! ~ ] ) ~ ~ = ~ ( h ; ' ~ ) (see -4lgorithm 4.2 and 53.2.4) then
If, for example, p = q = (k - 1)/2, then ~ ( h : ~ + ~ ) = ~ ( h : + ~ ) , which is two orders
higher than the order of the truncation error in the ITS method.
The term 1 (Sj:,,- [sj,+]) A ~ ~ w ( [ T ~ ] ) . In the IHO rnethod,
while in the ITS method,
Assuming that the Jacobian d f /dy does not change significantly from step to step,
we have from (4.3.30) and (4.3.31),
(cTpP + grq = 1). Therefore, we should expect
Comparing (4.3.26) and (4.3.27), and taking into account (4.3.28), (4.3.29), and (4.3.32),
we conclude that the propagation of the set {yj - ijj = Ajrj 1 rj E [ri]) is similar in the
IHO and Lohner7s methods, but the truncation error can be much smaller in the former
than in the latter.
4.3.4 Work per Step
We briefly discuss the rnost expensive parts of the ITS and IHO methods: generating
high-order Jacobians, matrix-matrix multiplications, and enclosing the inverse of a point
matrix. We measure the work by the number of floating-point operations. However, the
time spent on memory operations may not be insignificant for the following reasons.
0 The packages for automatic differentiation are often implemented through oper-
ator overloading [5], [6], [25], which may involve many memory allocations and
deallocat ions.
0 In generating Taylor coefficients, there may be a significant overhead caiised by
reading and storing the Taylor coefficients, f [d, and their Jacobians [Z].
Generating High-Order Jacobians
To obtain an approximate bound for the number of floating point operations to generate
(k - 1) Jacobians, 6' f[d/dy for i = 1,. . . , (k - 1), we assume that they are computed
by differentiating the code list of the corresponding f[d and using information from the
previously cornputed 3 f [']/ay, for Z = 1, . . . , (i - 1). The FADBADITADIFF [5], [6] and
IADOGC [31] packages compute a f ['7/ôy by differentiating the code list of f ri] (IADO L-
C is an interval version of ADOL-C [25]). We also assume that the cost of evaluating
d f [d/dy is roughly n tirnes the cost of evaluating f [d, 1221.
For simplicity, suppose that f contains only arithmetic operations. If N is the number
of operations, and cf 2 O is the ratio of multiplications and divisions to additions and
subtractions in t hese N operations, then to generate k coefficients f i l , i = 1, . . . , k, we
need c f N k Z + O ( N k ) operations [50, pp. 111-1121 (see Appendix A).
Let Ops (f [d) be the number of arithrnetic operations in the code list for evaluating
fEil from the already computed Taylor coefficients. In -4ppendix A, we show that
Ops ( f [ q ) = 2cf Ni + O(N) , for i > 0.
to generate k - 1 Jacobians in an ITS method, we use
arithmetic operations. Let p = q and k = p + q + 1. In the IHO method we generate
p = (k- 1 ) / 2 terms for the forward solution and q = p = (12- 1)/2 terms for the backward
one. The corresponding work is
That is, the IHO method requires about half as rnuch work as the ITS method of the
same order to generate high-order Jacobians.
Matrix Inverses and Matrix-Matrix Multiplications
In Lohner's method and in the IHO method with the QR-factorization technique, we
compute an enclosure of the inverse of a point matrix, which is a floating-point approx-
imation to an orthogonal matrix. However, in the IHO method, we also enclose the
inverse of a point matrix (see 54.1.3). In generd, enclosing the inverse of an arbitrary
point matrix is more expensive thon enclosing the inverse of a floating-point approxima-
tion to an orthogonal matrix. However, we can still enclose the inverse of an arbitrary
point matrix in 0 ( n 3 ) operatioos [2].
Lohner's method has 2 matrix-matrix multiplications, while the IHO method has 6
matrix-matrix multiplications.
To summarize, in the IHO method, we reduce the work for generating Jacobians, but
increase the nurnber of matrix operations. Suppose that N n2. This number can be
easily achieved if each component of f contains approximately n operations, as happens,
for example, in N-body problems. Then, (4.3.33) and (4.3.34) become
c f n 3 k 2 + 0 ( n 3 k ) and c f n 3 k 2 / 2 + O ( n 3 k ) .
Therefore, we should expect the IHO rnethod to outperform ITS methods in terms of
the amount of work per step when the right side of the problem contains many terms. If
the right side contains a few terms only, an ITS method may be less expensive for low
orders, but we expect that the IHO method will perform better for higher orders. Note
also that we expect the IHO method to allow larger stepsizes for methods of the same
order, thus saving computation time during the whole integration. In addition, the IHO
method (with p = q) needs half the memory for storing the point Taylor coefficients and
the high-order Jacobians.
In 58.3.2, we study empirically the amount of work per step on Van der Pol's equation.
Chapter 5
A Taylor Series Method for
Validation
We introduce a Taylor series method that is based on the validation test suggested by
Moore [50, pp. 100-1031 (see also [13] aud [52]) for proving existence and uniqueness
of the solution. Our goal is to obtain a method that validates existence and uniqueness
with the supplied stepsize, if possible, or a stepsize that is not much srnaller than the
supplied one. Furthermore, we want to avoid as many stepsize reductions in this method
as possible.
Usually, a Taylor series method for validation enables larger stepsizes than the con-
stant enclosure method, which has been used in the past [44], [69]. As we pointed out
in 53.1, the constant enclosure method restricts the stepsizes to Euler steps. We also
combine better algorithms for computing tight enclosures, such as Lohner's method and
the IHO method, with our algorithm for validating existence and uniqueness. As a result ,
we obtain a method that behaves similarly to the traditional numerical methods for IVPs
for ODES in the sense that the stepsize is controlled more by the accuracy requirements
of Algorithm II than by restrictions imposed by Algorithm 1.
Section 5.1 defines the validation problem; Section 5.2 describes how to compute an
CHAPTER 5 . A TAYLOR SERIES METHOD FOR VALIDATION 67
initial guess for the a priori enclosure; and Section 5.3 gives an algorithmic description
of the method we propose.
5.1 The Validation Problem
Let yj E [y,] and no component of yj is an endpoint of the corresponding component of
unique solution
i= 1
For an interval [ y j ] , the condition (5.1.1) translates to
k-1
k Ikl [ Y ~ I + C(t-tj)'f"'([~j]) + ( t - t j ) f ( [ G j I ) C [GjI- i= 1
TO find the Iargest tji1 > t j S U C ~ that (5.1.2) holds for al1 t E [ t j l t j+l] ,
(5.1.1)
yj has a
(5.1.2)
we have
to compute rigorous lower bounds for the positive real roots of 2n algebraic equations,
which are determined from (5.1.2). This task is not trivial to carry out.
However, since t - t j E [O , hj] for t E [ t j , t j+i] , if hj is S U C ~ that
holds, then (5.1.2) holds for al1 t E [ t j , t j+i]. Verifying (5.1.3) is not difficult, and Our
validation procedure is based on (5.1.3). Given [y j ] at t j and a stepsize h j 7 we want to
find [ y j ] such that (5.1.3) is satisfied. Usually, h j is predicted from the previous step. In
the validation step, we try to verify existence and uniqueness with this h j . If we cannot
verify with h j ) we try to verify with a smaller stepsize than h j .
CHAPTER 5. A TAYLOR SERIES METHOD FOR VALIDATION 6s
Before we consider how to implement a method based on (5.1.3), we illustrate this
approach with a few examples.
Consider
and let [Go] = [ l , 21. Then (5.1.3) on (5.1.4) with [go] = [ l , 21 gives
which is satisfied if
For k = 1 and 3, (5.1.5) holds for ho 2 0.5 and ho 5 0.63, respectively.
Now, let [go] = [l, 81. The inclusion (5.1.3) holds if
For k = 1 and 3, (5.1.6) holds for ho 5 0.875 and ho 5 1.48, respectively.
Here, we can compute larger stepsizes with wider a priori bounds. With a variable
stepsize control, we normally control the local excess per unit step (LEPUS), such that
LEPUS is less than some tolerance (see Chapter 6). Depending on the tolerance, we
can afford wider a priori bounds. For example, suppose that Algorithm II uses Taylor
series of order k = 15. Then, LEPUS is given by (hA4/15!)w([ijo]). With ho = 0.63
and [go] = [1,2], LEPUS E= 1.2 x 10-15, and with ho = 1.48 and [Co] = [ l , SI, LEPUS
x 1.3 x IO-'. If the tolerance is 1 0 - ~ , we can use ho = 1.48 and [GO] = [l, QI. Consider
and let [CO] = [0.5,1.5]. For k = 1, we obtain from (5.1.3) the constant enclosure method:
1 + [O, ho][-1.5, -0.51 C [0.5,1.5],
CHAPTER 5. A TAYLOR SERIES METHOD FOR SE LI DATION
In this exarnple, the maximum stepsize with E = 3 is srnaller than with k = 2. The
reason is t hat we ensure (5.1.1) by verifying (5.1.3). If we solve (5.1.1) direct ly, then we
are often able to verify existence and uniqueness on larger intervals. For example, (5.1.1)
for problem (5.1.7) and k = 3 reduces to
which holds for t satisfying
1 - t + t 2 / 2 - 1.5t3/6 2 0.5 and 1 - t + t 2 / 2 - 0.5t3/6 < 1.5. (5.1 .S )
These inequalities are true for t 5 0.63. Note that the inequalities in (5.1 .S) are more
sirnilar to stability conditions than Euler-type stepsize restrictions.
TO sumrnarize, by computing tj+i such that (5.1.2) holds for al1 t E [tj , t j+l], we
can often take larger stepsizes than with the constant enclosure method. The stepsize
restriction imposed by (5.1.2) is more a "stability-type" than an Euler-type restriction.
To implement (5.1.2) is more difficult than to implement (5.1.3). The latter often allows
larger stepsizes than the constant enclosure rnethod, although the stepsizes are generally
smaller t han the ones permitted by (5.1.2).
5.2 Gi-essing an Initial Enclosi-re
Suppose that we have computed pj] such that
where
Then, using (5.1.3) and (5.2.1),
Therefore, if [OI,] is such that (5.2.1) holds, then (5.1.2) is satisfied, and there exists a
unique solution
How t o compute an approximation for [Pj]
Let yj E l y j ] and t E [O, h j ] . Consider the nonlinear system of equations for P,,
Ideally, we want to find an enclosure of the set of values for pj such that (5.2.3) holds for
al1 yj E [ y j ] and t E [O, hjj. In practice, computing such a set may be expensive.
CHAPTER 5. A TAYLOR SERIES METHOD FOR VALIDATION 71
Here, we suggest a simple method for computing an approximation to this set. From
and therefore,
Since we are interested
can compute from (5.2.5),
in computing an approximation to the set containing ,Bi, we
Since [pi] is an approximation1, in the algorithm that we describe in the next section, we
inflate [pj] to reduce the iikelihood of failure in (5.2.1).
In (5.2.4), we could have used the approximation
which is perhaps a better approximation than (5.2.4). However, if we use (5.2.6), we
have to generate the coefficients a f [d/dy evaluated at Lyj] + [O, hj] f Ld([yj]), for i =
1, . . . , k, while in (5.2.4), we need f [kI/dy evaluated at [yj], which coefficients can be
reused in Algorithm II (see the next section).
=Note that [Pj] is a guess for the enclosure of the kth Taylor coefficient, not a rigorous enclosure.
CHAPTER 5 . A TAYLOR SERIES METHOD FOR VALIDATION
5.3 Algorit hmic Description of the Validation
Method
The method that we propose is described in Algorithm 5.1. Here, we explain some of the
decisions we have made in designing it.
Input part If we use an ITS method with order k. of the Taylor series, we have to
cornpute the coefficients f[il([yj]) and d f[iI([yj])/dy7 for i = 1, . . . , k - 1, in Algorithm II.
Therefore, we can use f['l([yj]), for i = 1, . . . , k - 1, in Algorithm 1 without doing addi-
tional work to generate them. However, we have to compute fLk]([yj]) and d f[k'([yj])/dy-
If we use a (p,q) IHO method, we have to generate, in addition to the coefficients
f[q([z~j]) for i = 1,. . . ,q, the coefficients f"l([yj]) for q = i + 1 , . . . , k and df[kl([yjj])/dy.
Compute part In line 7, we M a t e [p,]. Since it is already an approximation to the
enclosure of the kth Taylor coefficient on [tj7 tj+i], by inflating [/3j], we hope to enclose
this coefficient on [tj, tj+l]. We choose E = 1, but we can use other values instead. With
E = 1, we add [- 1 , Ioj l ] to [p;]. Since [pj] is multiplied by [O, hr] , adding [- IPj 1 , lfljl] to pj] does not contribute significantly to the widths of the components of [$jO'].
If the condition in line 11 is satisfied, then we have verified existence and uniqueness
with the computed [$:] in line 9. Otherwise, in line 15, we compute a new guess [Gy] for
the initial enclosure. Then, in the second while loop (line 18), we try to validate with
order s := 1 5 k. If we succeed, then in the third while loop (line 29), we try to improve
the enclosure with the order s, with which we have verified existence and uniqueness.
Otherwise, in line 38, we reduce the stepsize. If this is the second reduction, we start
the computations from the beginning (line 4); otherwise, we repeat the while loop at line
18 with a smaller stepsize. The reason for starting the computations al; line 4 after the
second stepsize reduction is to try with a new guess for the a priori enclosure, before
cont inui.ng wit h furt her stepsize reductions.
CHAPTER 5. A TAYLOR SERIES METHOD FOR VALIDATION
Algorithm 5.1 Validate existence and uniqueness with Taylor series.
INPUT:
1 [y j ] , h j , k, hmi,, a = 0.8, E = 1;
2 for i = 1, . . . , (k - 1) . ay
COMPUTE:
3 Verif i ed := false ;
4 while h j 2 hm;,., and not Verified do
14 end-if
15 [gjO'] := [ ~ j ] + [O, h;] f Lk1 ( [ g y ) ] ) ; 16 Generate f ([#y)]) , for i = 1, . . . , k; 17 Reduced := 0;
18 while not Verif i ed and Reduced < 2 do
19 for 1 = 1 to k do
break ;
end-if
26 end-for
CHAPTER 5 . A TAYLOR SERIES METHOD FOR VALID-ATION '74
Algorithm 5.1 Continued
if Verif i e d t hen
Improving := true ;
while Improving do
Generate f for i = 1, . . . , s;
fmproving := fdse ;
end
end- w hile
hj := ahj;
Reduced := Reduced + 1;
end-if
end-while
if h j < hm;, then
print "Stepsize too small: cannot verify existence and uniqueness";
exit ;
end-if
Cornpute hi f [d ( [ y j ] ) , for i = 1, . . . , k.
We do not halve the stepsize, but reduce it by rnultiplying by a, which we choose to
be 0.8. As with E , the value that we choose for a is somewhat arbitrary, but we want it
to be closer to 1 than to 0.5. We have not thoroughly studied the influence of the choice
for e and cr on the performance of Algorithm 5.1.
Chapter 6
Estimating and Controlling the
Excess
In 53.2.4, we considered the local excess in one step of the ITS methods discussed in this
thesis. The IHO method has the same sources of local excess as the ITS methods, but in
the IHO method, we also enclose the solution of the nonlinear system (4.1.10). Since the
excess that arises frorn enclosing the solution of this nonlinear system is usually small
(see 54.3.3), we do not disruss the local excess in the IHO method.
In 56.1, we define local and global excess and discuss controlling the global and
estirnating the local excess. In 56.2, we propose a simple stepsize control based on
controlling an approximation of the local excess.
6.1 Local and Global Excess
Let the set Uj be an enclosure of the solution at tj. In this thesis, Uj is represented by
an interval vector or a paralïelepiped. W e define local and global excess by
CHAPTER 6. ESTIMATING AND CONTROLLING THE EXCESS 76
respectively [19, p. 87, p. 1001, [71], where q(-, -) is the Hausdorff distance between two
sets given by (2.2.1).
6.1.1 Controlling the Global Excess
Similar to the standard numerical methods for IVPs for ODES, our approach in VNODE
is to allow the user to specify a tolerance Tol. Then the code tries to produce enclosures,
at points tj7 such that
yj CjTol for j 2 1, (6.1.3)
where C j is an unknown constant that depends on the problem and the length of the
interval of integra.tion, but not Tol. We try to achieve (6.1.3) by controlling the local
excess per unit step (LEPUS) [71]. That is, we require
on each step. Eijgenraarn shows [19, p. 1151 that
where cr is a constant depending on the problem. This constant may be negative since
the logarithmic norm is used in its definition [19, p. 461. Using (6.l.4), we obtain from
(6.1.5) that
ea( '~- ' l ) ( t j - to)Tol, if a > 0; T j L
( t j - to )To l , i f a 5 0 .
Therefore, by controlling LEPUS, we can obtain a bound for the global excess. In this
sense, by reducjng Tol, we should compute tighter bounds.
6.1.2 Estimating the Local Excess
From 53.2.4, the local excess in an ITS method is given by
(f) Input stepsize 0.2, IHO (e) Input stepsize 0.2, ITS
Figure 8.8: ITS(17) and IH0(8,8) on the two-body problem, constant enclosure method.
Example 3 Lorenz system
We integrated
Y: = 4 ~ 2 - Y1
Y: = Y ~ ( P - 93) - Y2
Y; = Y I Y ~ - B Y ~ ,
y(0) = (15,15,36)*, t E [O, 101:
where c = 10, p = 28, and ,û = 813, with the ITS(17) and IHO(8,S) methods and used
a constant enclosure method in Algorithm 1. The input stepsizes for Algorithm 1 are
0.01,0.05,0.1. The results are shown in Table 8.14, and the stepsizes versus step number
are shown in Figure 8.9. As in the two-body problem, the IHO method produces tighter
enclosures in less time, than the ITS method.
H Steps Excess Time
ITS IHO ITS IHO ITS IHO
Table 8.14: ITS(17) and IHO(8,S) on the Lorenz system, constant enclosure method.
We also tried the IHO method with the a priori bounds from Algorithm 1 as an input
to the corrector, instead of computing bounds with the predictor from 54.2.2. With
H = 0.01 and T = 0.8, the excess a t T = 0.8 was 26.8. Therefore, if we want to eliminate
the predictor step, we have to perform at least one more step of the corrector, which is
more expensive than the predictor.
In the next two examples, we compare the ITS and IHO methods with a variable step-
size control (see 56.2) and our version of a Taylor series method for validating existence
and uniqueness of the solution (see Chapter 5).
Step nurnber
(a) Input stepsize 0.01, ITS
Step nurnber
(c) Input stepsize 0.05, ITS
0.015 t I I I I
O 50 100 150 200 250
Step number
(e) Input stepsize 0.1, ITS
Step number
(b) Input stepsize 0.01, IHO
Step nurnber
(d) Input stepsize 0.05, IHO
Step number
(f) Input stepsize 0.1, IHO
Figure 8.9: ITS(17) and IH0(8,8) on the Lorenz system, constant enclosure method.
Example 4 Van der Pol's equation
We integrated Van der Pol's equation? written as a system,
with
for t E [0,20], where p = 5. We used the ITS(11) and IH0(5,5) methods and tolerances
1w8, . . . , 10-12.
From Table 8.15 and Figure 8.10, we see that, for approxirnately the same excess,
VNODE using the IHO method took fewer steps than it did using the ITS method, thus
saving cornputation time. In Figure 8.10, we plot the logarithms of the excess, tirne, and
tolerance. In Figure 8.10(d), the stepsize corresponding to the IHO method is not as
smooth as the one corresponding to the ITS method. In the regions where the stepsize
is not smooth, the Taylor series method for validation could not verify existence and
uniqueness wit h the supplied stepsizes, but verified with reduced stepsizes. Note also
that we control the local excess per unit step and report the global excess in Table 5.15.
Thus the global excess can be larger than the tolerance.
We also integrated (8.3.7-8.3.8) on [0,0.1] with an input stepsize of 0.01 to Algo-
rithm 1. We used orders k = 3,7,11,17,25,31,37,43, and 49 for the ITS method and
p = q = (k - 1)/2 for the IHO method. Algorithm I did not reduce the input stepsize. As
a result, the solver could take the same number of steps with the ITS and IHO methods.
In Figure 8.1 1, we plot the logarithm of the CPU time against the logarithm of the order
for these two methods. Although on this probiem, the IHO method is more expensive
for "low" orders, including k = 11, we still have savings in time (for the same excess)
due to the fewer steps taken.
-12 -11 -10 -9 -8 -7 -6
Excess
Excess
(4
ITS ---- - IHO - -
Step number
(d) T O ~ = 1 0 - ~
Figure 8.10: ITS(11) and IH0(5,5) on Van der Pol's equation, Taylor series for validation,
variable stepsize control with Tol = IO-', 10-~ , . . . , 10-12.
Figure 8.11: ITS and IHO with orders 3,7,11,17,25,31,37,43, and 49 on Van der Pol's
equation.
Tol Excess Steps Time
ITS IHO ITS IHO ITS IHO
Table 8.15: ITS(11) and IH0(5 ,5 ) on Van der Pol's equation, Taylor series for validation,
variable stepsize control.
Exarnple 5 Stiff DETEST Problem D l
We integrated the Stiff DETEST problem D l [21],
with
Y@) = (O, O, o ) ~ , for t E [O, 4001.
Here, we used the ITS(17) and IH0(8,8) methods, Taylor series for validation, and a
variable stepsize control with tolerances IO-^, 10-', . . . , 10-Io.
With the IHO method, we computed tighter bounds with fewer stepsizes, than with
the ITS method; see Table 8.16 and Figure 8.12. The reduction in the stepsize on the
last step for the IHO method seen in Figure 8.12(d) is a result of our program reducing
the stepsize to hit the endpoint exactly.
Tol Excess Steps Time
ITS IHO ITS IHO ETS IHO
Table 8-16: ITS(17) and IHO(8,a) on Stiff DETEST Dl , Taylor series for validation,
variable stepsize control.
1 . 8 5 ~ ~ ~ ~ ~ I l ~ I l ~
1.8 - =-. E S + - -_ -- IHO -e-
1.75 - -J--. -- -. -. e ------- - rn----__a-
- -
1.55 -
-10.5-10 -9.5 -9 -8.5 -8 -7.5 -7 -6.5 -6 -5.5
Excess
7500 i p i i i i i i i
7000 - % -. ITS *-- - 6500 - *-. IHO ++
Excess
Tol
(b)
Step number
(d) Toi = 10-Io
Figure 8.12: ITS(17) and IH0(8,8) on Stiff DETEST D 1, Taylor series for validation,
variable stepsize control with Tol = 10e6, IO-', . . . , IO-''.
8.4 Taylor Series versus Constant Enclosure Method
We integrated the following problems, which we denote by P l , Pz, P3, and P4.
Pl: (5.3.6) with y(0) = 1, for t E [O, 201;
PZ: y: = y*, = -YI, with y(0) = (O, I ) ~ , for t E [O, 1001;
P3: (S.3.2) with y(0) = (1, -1)=, for t E [0,50]; and
P4: (8.5.9) with y(0) = (0, O, o ) ~ , for t E [O, 501.
For al1 of these tests, we used order k = 17 for the ITS method and p = q = S for the
IHO method, and LEPUS error control with Tol = 10-lO.
Tables 8.17 and 8-18 show the number of steps taken by VNODE, when Algorithm 1
uses a constant enclosure (CE) method (see 53.1) and our Taylor series enclosure (TSE)
met hod (see Chapter 5), and the corresponding excess and times. The resolts in Ta-
ble 5.17 are ~roduced with the ITS rnethod, and the results in Table S.18 are produced
with the IHO method. In Figures 5.13 and 8.14, we plot the stepsizes against the step
number.
From Table 8.17, we see that if we use a TayIor series enclosure method, rve have
a significant reduction in the number of steps (with Tol = 1 0 - ' O ) . Furthermore, from
the obtained excess, we see that with a Taylor series enclosure method the stepsize Is
controlled from the accuracy requirements. In the constant enclosure method, we achieve
more accuracy than we have asked for, implying that the stepsize was controlled from
Algorithm 1 in that case. We should note, though, that the TSE method may still reduce
the stepsizes determined from the stepsize control mechanism.
In Table 8.18, we see a further reduction in the number of steps with the TSE method,
while the number of steps with the CE method rernains t h e same (except for a slight
difference for Pl) . Note also that with the IHO method, we generally cornpute smaller
enclosures in less time; cf. Tables 5-17 and 8.18-
Problem Steps Excess Time
TSE CE TSE CE TSE CE
Table 8.17: TSE and CE methods, ITS method, variable stepsize control with
Tol = 10-1°.
Problem Steps Excess Time
TSE CE TSE CE TSE CE
Table 8.18: TSE and CE rnethods, IWO method, variable stepsize control with
Tol = 10-'O.
Step number Step number
Step number Step number
1.2 1 I I r I r r 0.04 I I I 1 1 I
JU
Figure 8.13: TSE and CE methods, ITS method, variable stepsize control with
Tol = 10- 'O.
0.035 TSE - - CE -
0.03 - l - N
1
0.8 8 -
TSE - - CE ---- - - -
2 i 0.6 - $. 0.025 L - 0.4
G 0.02 - -
0.2 -------------------------------------------- 0.01 5 rd------------------- ----------------, - I I
O - ? 1 1 f t 1 1 . I r 1 I I t 0.01 O 50 100 150 200 250 300 350 400 O 500 1Ooo 1500 2000 2500 3000 3500
Step number Step number
Step number Step number
Figure 8.14: TSE and CE methods, IHO method, variable stepsize control with
Tol = 1 0 - = O .
Chapter 9
Conclusions and Directions for
Furt her Research
We have developed and studied an interval Hermite-Obreschkoff method for computing
rigorous bounds on the solution of an IVP for an ODE. Compared to interval Taylor
series methods with the same order and stepsize, our method has a smaIler truncation
error, better stability, and is usually less expensive for ODES for which the right side
contains many terms. Although Taylor series methods can be considered as a special
case of the more general Hermite-Obreschkoff methods, we have developed a different
approach (from Taylor series) to cornpute bounds for the solution of an IVP for an ODE.
While our study was not directed towards producing an interval method for stiff
problems, we have shown that an interval version of a scheme suitable for stiff problems
(in traditional numerical methods) may still have a restriction on the stepsize. To obtain
an interval method without stepsize limitations, we need to find a scheme with a stable
formula not only for advancing the step but also for the truncation error.
We proposed a Taylor series method for validating existence and uniqueness of the
solution. This method was designed to ameliorate the stepsize restriction imposed by
Algorithm 1, but we have not tried to produce an algorithm that always verifies existence
CHAPTER 9. CONCLUSIONS AND DIRECTIONS FOR FURTHER RESEARCH 121
and uniqueness (if possible) with the supplied stepsize. Further work is necessary to
produce a very good implementation of Algorithm 1. Such an implementation can be
considered as an optimization problem: rnaximize the step length, subject to a tolerance
restriction.
Our stepsize control mechanism is relatively simple. It worked well for our tests,
but we have not performed a thorough empirical investigation. Further studies rnay
be necessary. New developments on stepsize selection for standard and validated ODE
rnethods rnight be appropriate for considerations in a validated solver; see for example
[26] and [36].
There has not been a comprehensive study of order control heuristics. Eijgenraam [19,
pp. 129-1361 describes the only order selection scheme known to the author. Some in-
sights into the problem of order control are given in [50, pp. 100-1151 and [70]. To
develop an order control strategy based on the amount of work per step, we need to
estimate this work. Obtaining a theoretical bound for the number of arithmetic oper-
ations in generating Taylor coefficients for the solution is not difficult, but obtaining a
reasonably accurate formula for the number of arithmetic operations in generating their
Jacobians is more cornplex. These Jacobians can be computed by a forward (TADIFF)
or a reverse mode (IADOL-C) of automatic differentiation [5S], sparsity rnay or rnay not
be exploited, and different packages rnay implement the same method differently; for
example, with a tape in ADOL-C or using only the main memory as in TADIFF. In ad-
dition to estimating the number of floating-point operations, the time spent on memory
operations rnay be nonnegligible.
As the area of validated ODE solving develops, we will need a methodology for as-
sessing validated methods. A part of such a methodology should be an estimate of the
amount of work. It may be possible to express it as a number of function and Jaco-
bian evaluations. Then, we rnay compare validated methods in a framework simiiar to
DETEST [30] or Stiff DETEST [21].
Appendix A
Number of Operations for
Generating Taylor Coefficients
We obtain formulas for the number of arithrnetic operations for generating one Taylor
coefficient and k Taylor coefficients for the solution to y' = f (y), y(to) = yo. For simplic-
ity, we assume that the code list of f contains only arithmetic operations. Let N I , i&
and N3 be, respectively, the number of additions (we count subtractions as additions),
multiplications, and divisions in the code list of f. If we have computed the Taylor
coefficients (y), , (y),, . . . , (y);, we can compute the (a' + 1)-st coefficient from
where (f (y)), is the ith Taylor coefficient of f (y) (see 52.4). The number of arithmetic
operations required for cornputing (f(~))~, using (y),, (y),, . . . , (y),, are ca~culated in
Table A.1. If Ops (g) denotes the number of arithmetic operations for computing some
function g, then from Table A.1, the number of arithmetic operations to compute (f ( y ) ) i
is
APPENDIX A. OPERATIONS FOR GENERATING TAYLOR COEFFICIENTS 123
OP- # Formula Number of
Table A. 1: Number of additions, multiplications, and divisions for comput ing (f (y));.
where N = Ni + .V2 + f 3 and cf = (N2 + N3)/Ar. Because of (A-l) , (A.%) also gives the
number of operations for cornputing f = (y)i+, .l Therefore,
Ops (f[il(y)) = 2c fN( i - 1 ) + N = 2cfNi + ( 1 - 2 q ) N
= Zc Ni + O ( N ) .
The total number of arithmetic operations to compute k 2 1 Taylor coefficients, (y),,
(y127 . - , (y)k, can be obtained by summing the number of arithmetic operations to
compute (f (y))i for i = O , . . . , k - 1:
' We do not count the multiplication & x (f (y))(.
Appendix B
A Validated Ob ject-Oriented Solver
B.1 Objectives
Our primary goal is to provide a program environment that wiIl assist researchers in the
numerical study and cornparison of schemes and heuristics used in computing validated
solutions of IVPs for ODEs. The VNODE (Validated Numerical ODE) package that we
are developing is intended to be a u n i f o m implementation o f a generic validated solverjbr
IVPs /or ODEs. Uniform means that the design and implementation of VNODE follow
well-defined patterns. As a result, implernenting, modifying, and using methods can
be done systematically. Generic means that the user can construct solvers by choosing
appropriate methods from sets of methods. This property enables us to isolate and
compare methods implementing the same part of a solver. For example, we can assemble
two solvers that differ only in the module implementing Algorithm II. Then, the difference
in the numerical results obtained by executing the two solvers will indicate the difference
in the performance of Algorithm II. Since we would like to investigate algorithms, being
able to isolate them is an important feature of such an environment.
We list and briefly explain some of the goals we have tried to achieve with the design of
VNODE. Provided that a validated method for IVPs for ODEs is implemented correctly,
the reliability issue does not exist: if a validated solver returns an enclosure of the
solution, then the solution is guaranteed to exist within the computed bounds.
Modularity The solver should be organized as a set of modules with well-defined inter-
faces. The implementation of each module should be hidden, but if necessary, the
user should be able to modify the implementation.
Flexibility Since we require well-defined interfaces, we should be able to replace a
method, inside a solver, without affecting the rest of it. Furthermore, we should
be able to add methods following the established structure and wit hout rnodifying
the existing code.
Efficiency The methods incorporated in VNODE do not have theoretical limits. How-
ever, these methods require the computation of high-order Taylor coefficients and
Jacobians of Taylor coefficients. As a result, the efficiency of a validated solver
is determined mainly by the efficiency of the underlying automatic differentiation
package. Other factors that contribute to the performance are: the efficiency of
the interval-arithmetic package, the programming language, and the actual im-
plementation of the methods. To achieve flexibility, we may need to repeat the
same calculations in two parts of a solver. For example, to separate Algorithm I
and Algorithm II, we may need to generate the same Taylor coefficients in both
algorithms. However, the repetition of such computations should be avoided.
Since VNODE is to be used for comparing and assessing methods, it has to contain the
existing ones. Moreover, VNODE should support rapid prototyping.
The area of computing validated solutions of IVPs for ODEs is nl ot as devel ed as the
area of computing approximate solutions. Some of the difficulties that arise in interval
methods are discussed in Chapter 3 and [52]. With respect to the tools involved, a
validated solver is inherently more complex than a classical ODE solver. In addition to
an interval-arithmetic package, a major component of a validated solver is the module
for automat ic generation of interval Taylor coefficients (see SB .4).
Currently, there are three available packages for computing guaranteed bounds on the
solution of an IVP for an ODE: AWA [44], ADIODES [69] and COSY INFINITY [SI. We
briefly summarize each in turn.
AWA is an implementation of Lohner's method (53.2.5) and the constant enclosure
approach (83.1). This package is written in Pascal-XSC [37], an extension of Pascal for
scientific computing .
ADIODES is a C++ irnplementation of a solver using the constant enclosure method
in Algorithm 1 and Lohner7s method in Algorithm II. The stepsizes in both ADIODES
and AWA is restricted to Euler steps by Algorithm 1.
COSY INFINITY is a Fortran-based code for study and design of beam physics sys-
tems. The method used for verified integration of ODEs is based on high-order Taylor
polynornials with respect to time and the initial conditions. The wrapping effect is re-
duced by est ablishing functional dependency between initial and final conditions (see ['il ) .
For that purpose, the computations are carried out with Taylor polynomials with real
floating-point coefficients and a guaranteed error bound for the remainder term. Thus,
the arithmetic operations and standard functions are executed with such Taylor polyno-
mi& as operands. Although the approach described in [7] reduces the wrapping effect
substantially, working with polynomials is significantly more expensive than working with
i n t e rd s .
B.3 Object-Oriented Concepts
Since our goal is to build a flexible, easy-buse, and easy-to-extend package, we have
chosen an object-oriented approach in designing VNODE. This is not the first object-
oriented design of an ODE solver. The Godess project [57] offers a generic ODE solver
t hat implements traditional methods for N P s for O DES. Anot her successful package is
Diffpack [43], which is devised for solving partial differential equations. In [43], there is
also an example of how to construct an object-oriented ODE solver-
In this section, we review some object-oriented concepts supported in C++. A good
discussion of object-oriented concepts, analysis, and design can be found in [Il]. An
excellent book on advanced C++ styles and idioms is [12]. A study of nonprocedural
paradigms for numerical analysis, including ob ject-oriented ones, is presented in [72].
Data Abstraction
In the object model, a software system can be viewed as a collection of objects that
interact with each other to achieve a desired functionality. An object is an instance of
a class, which defines the structure and behavior of its objects. By grouping data and
methods inside a class and specifying its interface, we achieve encapsulation, separating
the interface from the implementation. Hence, the user can change the data represen-
tation and the implementation of a method' (or methods) of a class without modifying
the software that uses it. By encapsulating data, we can avoid function calls with long
parameter lists, which are intrinsic to procedural languages like Fortran 77. A ciass can
encapsulate data or algorithms, or both.
'We use method in two different contexts: to denote a rnember function of a class or a method in VNODE.
Inheritance and Polymorphism
Inheritance and polymorphism are powerful features of ob ject-oriented languages. In-
heritance aIlows code reuse: the derived class can use the data and functions of its base
class(es). Polyrnorphism serves to apply a given function to diKerent types of objects.
Often polymorphism and inheritance are used with abstract classes. An abstract class
defines abstract operations, which are implemented in its subclasses; it has no instances
and an object of such a class cannot be created.
Operat or Overloading
Operator overloading allows the operators of the language to b e overloaded for user de-
fined types. To program interval operations without cxplicit function calls, we have to
use a language that supports operator overloading. Without it, programming interval-
arithmetic expressions is cumbersome. Both Cf+ and Fortran 90 provide operator over-
loading. This feature is used to build interval-arithmetic libraïies like PROFIL/BIAS
[38] (C++) and INTLIB (Fortran 90) [34].
B.4 Choice of Langi-age: C++ versi-s Fortran 90
We have chosen C++ [20] over Fortran 90 [47] to implement VNODE. Procedural lan-
guages like C or Fortran 77 can be used to implement an object-oriented design [3].
However, using a Ianguage that supports object-oriented programming usually reduces
the effort for implementing object-oriented software. Our choice was determined by the
following considerations, listed in order of importance:
1. availability of software for automatic generation of interval Taylor coefficients;
2. performance and built-in functions of the available interval-arithmetic packages;
3. support of object-oriented concepts; and
APPENDIX B. A VALIDATED OBJECT-ORIENTED SOLVER
4. efficiency.
In this section, we discuss each in turn.
B A l Software for Automatic Generation of Interval Taylor
Co efficients
Although packages for automatic differentiation (AD) are available (see for example [33]
and [79]), to date, only two free packages for automatic generation of interval Taylor
coefficients for the solution of an ODE and the Jacobians of these coefficients are known
to the author. These are the FADBADITADIFF [5 ] , 161 and IADOL-C [31] packages.
They are written in C++ and implement AD through operator overloading.
TADIFF and FADBAD are two different packages. TADIFF can generate Taylor
coefficients with respect to time. Then, FADBAD can be used to compute Jacobians
of Taylor coefficients by applying the forward mode of AD [55] to these coefficients.
FADBAD and TADIFF are not optimized to handle large and sparse systems. Also,
they perform al1 the work in the main rnernory.
The IADOGC package is an extension of ADOL-C [25] that allows generic data
types. ADOL-C can compute Taylor coefficients by using the forward mode and their
Jacobians by applying the reverse mode [67] to these coefficients. The basic data type
of ADOL-C is double. To use a new data type in IADOL-C, the user has to overload
the ai thmetic and cornparison operations and the standard fuoctions for that data type.
Then, using IADOGC is essentially the same as using ADOL-C. Since I.4DOL-C replaces
only the double data type of ADOL-C, IADOL-C inherits al1 the functionality of A D O L
C. However, it was reported that the operator overloading, in IADOL-C, for a basic data
type incurs about a three times speed penalty over ADOL-C [31]. This appears to be a
phenomenon of the C++ compilers rather than the AD package [31].
The ADOL-C package records the computation graph on a so-callod tape. This tape
is stored in the main memory, but, when necessary, is paged to disk. When generating
Jacobians of Taylor coefficients, ADOL-C exploits the sparsity structure of the Jacobian
of the function for computing the right side. Since optimization techniques are used
in ADOL-C, we expect the interval version, IADOL-C, to perform better than FAD-
BADITADIFF on large and complex problems. But, still, FADBADITADIFF should
perform well on small to medium-sized problems.
Currently, VNODE is configured with FADBADITADIFF, but we have also used
IADOL-C. VNODE with these AD packages is based on the INTERVAL data type from
the PROFIL/BIAS package, which we discuss in sB.4.2 and sB.4.3.
B A.2 Int erval Arit hmet ic Packages
The most popular and free interval-arithmetic packages are PROFIL/BIAS [38], writteo
in Cf+, and INTLIB [35], written in Fortran 77 and available with a Fortran 90 interface
[34]. The Fortran 90 version of INTLIB uses operator overloading. For references and
comments on other available packages, see for example [34] or [38]. Recently, an interval
extension of the Gnu Fortran compiler was reported [65], where intervals are supported
as an intrinsic data type.
PROFIL/BIAS seerns to be the fastest interval package. In cornparison with other
such packages, including INTLIB, PROFIL/BIAS is about one order of magnitude faster
[38]. Also, PROFIL/BIAS is easy-to-use, and provides matrix and vector operations
and essential routines, for example, guaranteed linear equation solvers and optirnization
routines. For efficiency, it uses the rounding mode of the processor on the machines on
which it is installed. Portability is provided by isolating the machine dependent code in
small assembler files, which are distributed with the package.
B.4.3 Efficiency
Compared to Fortran, C++ has been criticized for its poor performance for scientific
computing. Here, we discuss an important performance problem: the pairwise evaluation
of arithmetic expression with arguments of array types (e.g., matrices and vectors). More
detailed treatment of this and other problems can be found in [62], [75], and [76].
In C++, executing overloaded arithmetic operations betweeii array data types creates
temporaries, which can int roduce a sipificant overhead, particularly for small ob jects.
For example, if A , B, C, and D are vectors, the evaluation of the expression
creates two temporaries: one to hold the result of A + B, and another to hold the result
of (A + B) + C. Furthermore, this execution introduces three loops. Clearly, it would be
better to compute this sum in one loop without temporaries. In Fortran 90, mathematical
arrays are represented as elementary types and optimization is possible at the compiler
level.
Because of better optimizing compilers and template techniques [74], [76], C f+ is
becoming more cornpetitive for scientific computing. A good technique for reducing the
overhead in the pairwise evaluation of expressions involving arrays is to use expression
templates [74]. The expression template technique is based on performing compile- t ime
transformation of the code using templates. Wi t h t his technique, expressions cont aining
vectors and matrices can be evaluated in a single pass without allocat ing temporaries. For
example, with expression templates, it is possible to achieve a loop fusion [74], allowing
the above surn to be evaluated in a single loop:
f o r ( i n t i = 1; i <= N; i + + )
D ( i ) = ~ ( i ) + B ( i ) + C ( i ) ;
However, executing this loop in interval arithmetic may not be the best solution for the
following reason. Each interval addition in this loop involves two changes of the rounding
mode. In modern RISC architectures, rounding mode switches cost nearly the sarne or
even more than fioating-point operations [3S], [65]. The approach of PROFIL/BiAS is
to minimize these switches. Suppose that we want to compute in PROFIL/BIAS
where A, B, and C are vectors of the same dimensions. If we denote the components of
A , B, and C by a;, bi7 and ci, respectively, PROFIL/BIAS changes the rounding mode
downwards and cornputes ci = gi + bi, for i = 1,2,. . . , n. Then, this package changes -
the rounding mode upwards and cornputes = ai + bi, for i = 1 ,2 , . . . n. Therefore, the
result of A + B is computed with two rounding mode switches. However, PROFIL/BIAS
still creates ternporaries.
B.4.4 Support of Object-Oriented Concepts
Cf+ is a fully object-oriented laquage, while Fortran 90 is not, because it does not
support inheritance and polymorphism. The features of C++ (e-g., data abstraction, op-
erator overloading, inheritance, and polymorphism) allow the goals in §B. 1 to be achieved
in a relatively simple way. Inheritance and polymorphism can be simulated in Fortran
90 [17], but this is cumbersome.
B.5.1 Structure
From an object-oriented perspective, it is useful to think of a numerical problem as an
object containing al1 the information necessary to compute its solution. Also, we can
think of a particular method, or a solver, as an object containing the necessary data and
functions to perform the integration of a given problem. Then, we can cornpute a solution
by "applying" a method object to a problem object. Most functions in VNODE have
Figure B. 1: Problem classes.
such objects as parameters. The description of the numerical problem and the rnethods
in VNODE are implemented as classes in C++.
The problem classes are shown in Figure B.1, and the method classes are shown in
Figure B.2. A box in Figures B.1 and B.2 denotes a class; the rounded, filled boxes
denote abstract classes. Each of them declares one or more virtual functions, which are
not defined in the corresponding abstract class, but must be defined in the derived classes.
The lines with A indicate an is-a relationship, which can be interpreted as a derived class
or as a specialization of a base class; the lines with O indicate a has-a relationship. It is
realized either by a complete containment of an object Y within another object X or by
a pointer from X to Y. The notation in these figures is similar to that suggested in [64].
In the next two subsections, we Iist the problem and method classes and provide brief
explsnations. Here, we do not discuss the classes for generating Taylor coefficients in
VNODE. A detailed description of VNODE will be given in the documentation of the
code at http://ww~r.cs.toronto.edu/NA.
Problem Classes Class ODESROBLEM specifies the mathematical problem, that is, to,
[yo], T, and a pointer to a function to compute the right side of the ODE. It also contains
a pointer to a class PROBLEMINFO. It indicates, for example, if the problem is constant
coefficient, scalar, h a , a closed form solution, or has a point initial condition. Such
information is useful since the solver can determine from it which part of the code to
execute.
ODENUMERIC specifies the numerical problern. This class contains data such as abso-
lute and relative2 error tolerances, and a pointer to a class O D E m E R I C representing a
solution. The user-defined problems, Pl, P2, and P3 in Figure B.1 are derived from this
class. New problems can be added by deriving them from ODENUMERIC.
ODESOLUTION contains the last obtained a priori and tight enclosures of the solution
and the value of t where the tight enclosure is computed. ODESOLUTION contains also a
pointer to a file that stores information from the preceding steps (e-g., enclosures of the
solution and stepsizes).
Method Classes Class ODESOLVER is a general description of a solver that "solves"
an ODENUMERIC problem. ODESOLVER declares the pure virtual function Int egrat e. Its
definition is not provided in this class. As a result, instances of ODESOLVER cannot be
created. This class also contains the class METHOD-CONTROL, which includes different flags
(encapsulated in FLAGS) and statistics collected during the integration (encapsulated in
STATISTICS).
Class VODESOLVER implements a general validated solver by defining the Int egrat e
function. We have divided this solver into four abstract rnethods: for selecting an order,
selecting a stepsize, and computing initial and tight enclosures of the solution. These
methods are realized by the abstract classes ORDER-CONTROL, STEP-CONTROL, INITXNCL,
and TIGHTmCL, respectively. 'ïheir purpose is to provide general interfaces to partic-
ular methods. A new method c m be added by deriving it from these abstract classes.
In tegrate performs the integrations by calling objects that are instances of classes de-
rived from ORDER-CONTROL, STEP-CONTROL, INITIMCL, and T I GHTXNCL.
ORDER-CONTROL has only one derived class, CONSTSRDER, whose role is to return a
constant order. Currently, VNODE does not implement variable-order methods.
- -
'Hout to specify and interpret relative error toierance will be discussed in the documentation of VNODE.
For selecting a stepsize, CONSTSTEP returns a constant stepsize on each step, and
VARSTEPCONTROL implements the stepsize selection scheme from 56.2.
There are two methods for validating existence and uniqueness of the solution in
VNODE: a constant enclosure method (CONSTINITXNCL) and a Taylor series method
(TAYLINIT~CL). The purpose of the FIXEDINITBJCL class is to compute a priori
enclosures of the solution from the formula for the true solution, if the problem has a
closed form solution. This class turns out to be helpful when we want to isolate the
influence of Algorithm 1, because this algorithm often reduces the input stepsize.
There are two methods for computing a tight enclosure of the solution: an in-
terval Hermite-O breschkoff method (OBRESCHKOFF-TIGHT_ENCL) and Lohner's met hod
(LOHNER-TIGHTXNCL). The VODESOLVER class has also a pointer to DATAREPR, which is
responsible for generating and storing Taylor coefficients and their Jacobians.
B.5.2 An Example Illustrating the Use of VNODE
Suppose that we want to compare two solvers that differ only in the method implementing
Algorithm II. In addition, we want to compare them with a constant enclosure method
and then with a Taylor series enclosure rnethod in Algorithm 1. Here, we show and discuss
oart of the VNODE code that can be employed for this study. As an example of an ODE,
Pol's equation, written as a system,
(B.5.1)
I
we use Van der
Y: = Y2
In a traditional ODE solver, we provide a function for comput ing the right side. In a
validated solver, we have to provide also functions for generating Taylor coefficients and
their Jacobians. Since we use an AD package for generating such coefficients, we have to
specify a function for computing the right side of (B.5.1) for this package. We write the
template function
template cclass YTYPE> void ~ ~ p t e r n ~ l a t e ( Y T Y P ~ *yp , c o n s t YTYPE *Y)
ypC01 = yC11;
yp Cl1 = Mu*(l-sqr(yC0I 1) *yCa - yC01;
which is used by FADBADITADIFF and IADOL-C to store the computation graph, and
by VNODE to create a function for computing the right side. Then we derive a class
VDP from ODE_NUMERIC. Since the details about the declaration of VDP are not essential
to understand our example, we omit this declaration.
Figure B.3 shows a typical use of VNODE classes. First, we create an ODEJUMERIC
object3, W P , and load the initial condition, the interval of integration, and tolerance by
calling the function LoadProblemParam (Part A). For testing, i t is convenient to have a
function that supplies different sets of data depending on the parameter to this function.
Then, we create methods and return pointers to them (Part B), as described below.
ITS and IHO are pointers to abjects for computing enclosures using Lohner7s and the
IHO methods, respectively. I n i t E n c l is a pointer to an object for validating existence
and uniqueness of the solution with the constant enciosure method; st e p c o n t r o i refers
to an object that implements a variable stepsize control; and OrderControl points to an
an object that provides a constant value for the order.
The purpose of class TAYLOREXPANSION is to generate and store Taylor coefficients
and their Jacobians. It is a template class, for which instances are created by specifying
a class for generating Taylor coefficients and a class for generating Jacobians of Tay-
lor coefficients. Here, we create such an instance with parameters VDPTaylGenODE and
VDPTaylGenVar, which are classes4 for generating Taylor coefficients and their Jacobians
for (B.5.1).
31n Figure B.3, P t r stands for pointer in PtrODENumeric, PtrTightEncl, etc. We do not describe these classes here.
In part C, we create two solvers, SolverITS and SolverIHO and integrate the problem
by calling the I n t e g r a t e function on these sol ver^.^ Note that they differ only in the
method for computing a tight enclosure of the solution. Thus, we can isolate and compare
the two methods implementing Algorithm II.
Now, in part D, we want to replace the constant enclosure method for validating
existence and uniqueness of the solution with a Taylor series rnethod and repeat the
same integrations. We create an instance of TAYLINITWCL by
I n i t E n c l =
new TAYL-INIT-ENCL(ODE->S~~~,~~~ VDPTaylGenODE,new VDPTaylGenVAR);
set i t by calling the S e t I n i t E n c l function, and integrate.
We explain how class I N I T X N C L works; the same idea is used in the other abstract
classes. I N I T Z N C L is an abstract class since it contains the pure virtual function
v i r t u a l void Val ida te( . . . ) = 0 ;
(for simplicity, we leave out the parameters). Each of the derived classes of INITBNCL
must declare a function with the same name and parameters and specify the body of
the function. In In t eg ra t e , there is a cal1 to Val idate . During execution, depending
on the object set, the appropriate Validate function will be calied. We use dynamic
or late binding: the function that is cailed is determined by the type of object during
the execution of the program. In our example, the method for validating existence and
uniqueness is replaced, but the integrator function is not changed. If the user wants to
implement his/her own Algorithm 1, he/she has to define a derived class of INITXNCL
and an associate Validat e function.
We omit the details about extracting data after an integration.
// * . *
/ / A . Crea te t h e ODE problem. PtrODENumeric ODE = new VDP; ODE->~oad~roblern~aram(l);
// B . Create t h e methods. i n t K , P, Q ; K = il; // order P = Q = (K-1)/2;
PtrTightEncl ITS = new LOHNER-TIGHT-ENCL(K); P t rTightEncl IHO = new OBRESCHKOFF-TIGHT-ENCL(P,Q);
P t r I n i t E n c l In i tEncl = new CONST-INIT-ENCL(ODE->S~~~, new V D P T ~ ~ ~ G ~ ~ V A R ) ; P t r S t e p C t r l S tepCtr l = new VAR-STEP-cONTROL(ODE->S~Z~); PtrOrderCtr l OrderCtrl = new CONST,ORDER(K); PtrDataRepr DataRepr = new TAYLOR-EXPANSION<VDPTaylGenODE, VDPTaylGenVAR> ;
// P a r t C . Create t h e s o l v e r s and i n t e g r a t e . PtrVODESolver SolverITS = new
VODE-SOLVER(ODE, DataRepr , OrderCtr l , S t e p C t r l , I n i t Encl, ITS) ;
PtrVODESolver SolverIHO = new VODE,SOLVER(ODE, Dat aRepr , OrderCtr l , S t e p C t r l , In i tEncl , IHO) ;
/ / Part D . Replace t h e method implementing Algorithm 1 and i n t e g r a t e . I n i t E n c l =
new TAYL-INIT-ENCL(0DE->Size, new VDPTaylGenODE , new V D P T ~ ~ ~ G ~ ~ V A R ) ;
~olver1~~->Set~nit~ncl(~nit~ncl); ~ o l v e r 1 ~ 0 - > S e t I n i t E n c l ( In i tEncl ) ;
Figure B.3: The test code.
Bibliography
[l] E. Adams, D. Cordes, and R. Lohner. Enclosure of solutions of ordinary initial value
problerns and applications. In E. Adams, R. Ansorge, Chr. GroDrnann, and H. G.
Roos, editors, Discretization in Diflerential Equations and Enclosures, pages 9-25.
Akademie-Verlag, Berlin, 1957.
[2] G. Alefeld and J. Herzberger. Introduction to Intenial Cornputations. Academic
Press, New York, 1983.
[3] S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith. Parallelism in object-
oriented numerical software libraries. In Erlend Arge, Are Magnus Bruaset, and
Hans Pet ter Langt angen, edi tors, Modern Software Tools in Scien tific Computing,
pages 163-303. Birkhauser, Boston, 1997. See http://www.mcs.anl.gov/petsc/.
[4] H. Bauch and W. Kimmel. Solving ordinary initial value problems with guaranteed