Top Banner
Abstract In- terpretation of Floating- Point Computa- tions Sylvie Putot and Eric Goubault Laboratory for ModElling and Analysis of Systems in Interaction, CEA- LIST/X/CNRS Abstract Interpretation of Floating-Point Computations Sylvie Putot and Eric Goubault Laboratory for ModElling and Analysis of Systems in Interaction, CEA-LIST/X/CNRS Presentation at the University of Verona February 20, 2007 1
51

Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Aug 26, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubault

Laboratoryfor

ModEllingand

Analysis ofSystems inInteraction,

CEA-LIST/X/CNRS

Abstract Interpretation of Floating-Point

Computations

Sylvie Putot and Eric GoubaultLaboratory for ModElling and Analysis of Systems in

Interaction, CEA-LIST/X/CNRS

Presentation at the University of Verona

February 20, 2007

1

Page 2: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Outline

Introduction

Floating-point computationsStatic analysis and abstract interpretation

An abstract interpretation for floating-point computations: a relational domain relying on affine arithmetic

Introduction to affine arithmeticRelational domain for real value computation

arithmetic operationsjoin, meet, order

From real to floating-point computation : relationaldomain for values and errors

Examples

References

2

Page 3: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Floating-point numbers (defined by the IEEE 754norm)

Normalized floating-point numbers

(−1)s1.x1x2 . . . xn × 2e (radix 2 in general)

implicit 1 convention (x0 = 1)n = 23 for simple precision, n = 52 for double precisionexponent e is an integer represented on k bits (k = 8 forsimple precision, k = 11 for double precision)

Denormalized numbers (gradual underflow),

(−1)s0.x1x2 . . . xn × 2emin

3

Page 4: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

ULP : Unit in the Last Place

ulp(x) = distance between two consecutive floating-pointnumbers around x = maximal rounding error of a numberaround x

A few figures for simple precision floating-point numbers :

largest normalized ∼ 3.40282347 ∗ 1038

smallest positive normalized ∼ 1.17549435 ∗ 10−38

largest positive denormalized ∼ 1.17549421 ∗ 10−38

smallest positive denormalized ∼ 1.40129846 ∗ 10−45

ulp(1) = 2−23 ∼ 1.19200928955 ∗ 10−

4

Page 5: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Some difficulties of floating-point computation

Representation error : transcendental numbers π, e, butalso

1

10= 0.00011001100110011001100 · · ·

Floating-point arithmetic :

absorption : 1 + 10−8 = 1 in simple precision floatassociative law not true :(−1 + 1) + 10−8 6= −1 + (1 + 10−8)cancellation : important loss of relative precision when twoclose numbers are subtracted

Some more trouble :

re-ordering of operations by the compilerstorage of intermediate computation either in register or inmemory, with different floating-point formats

5

Page 6: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example of cancellation : surface of a flattriangle

(a, b, c the lengths of the sides of the triangle, a close tob + c):

A =√

s(s − a)(s − b)(s − c) s =a + b + c

2

Then if a,b, or c is known with some imprecision, s − a is veryinaccurate. Example,

real number floating-point number

a = 1.9999999 a = 1.999999881...

b = c = 1 b = c = 1

s − a = 5e − 08 s − a = 1.19209e − 07

A = 3.16...e − 4 A = 4.88...e − 4

6

Page 7: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

In real world : a catastrophic example

25/02/91: a Patriot missile misses a Scud in Dharan andcrashes on an american building : 28 deads.

Cause :

the missile program had been running for 100 hours,incrementing an integer every 0.1 secondbut 0.1 not representable in a finite number of digits inbase 2

1

10= 0.00011001100110011001100 · · ·

Truncation error ∼ 0.000000095 (decimal)Drift, on 100 hours ∼ 0.34s

Location error on the scud ∼ 500m

7

Page 8: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

But also some other costly errors ...

Explosion of Ariane 5 in 1996 (conversion of a 64 bitsfloat into a 16 bits integer : overflow)

Vancouver stock exchange in 1982

index introduced with initial value 1000.000after each transaction, updated and truncated to the 3rdfractional digitwithin a few months : index=524.881, correct value1098.811explanation : biais. The errors all have same sign

Sinking of an offshore oil platform in 1992 : inaccuratefinite element approximation

Collection of Software Bugs at urlhttp://www5.in.tum.de/∼huckle/bugse.html

8

Page 9: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Validation of accuracy “by hand” ?

A popular way : try the algorithm with different precision(using matlab for example) and compare the results

Example (by Rump) : in FORTRAN on an IBM S/370,computing with x = 77617 and y = 33096 and x1 = 61.0

11,

f = 333.75y6+x2(11x2y2−y6−121y4−2)+5.5y8+x/(2y)

gives :

in single precision, f = 1.172603...in double precision, f = 1.1726039400531...in extended precision, f = 1.172603940053178...

We would deduce computation is correct ?

True value is f = −0.82739... !!!

9

Page 10: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

IEEE 754 norm : correct (or exact) rounding

The user chooses one among four rounding modes(rounding to the nearest which is the default mode,rounding towards +∞, rounding towards −∞, or roundingtowards 0)

The result of x ∗ y , ∗ being +,−,×, / or of√

x , is therounded value of the real result (thus the rounding error isless than the ulp of the result)

→ Allows to prove some properties on programs usingfloating-point numbers

10

Page 11: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Static Analysis

Analysis of the source source, for a set of inputs andparameters, without executing it

The program is considered as a discrete dynamical system

Find in an automatic, and guaranteed way :

invariant properties (true on all trajectories - for allpossible inputs or parameters).Example : bounds on values of variablesliveness properties (that become true at some moment onone trajectory).Examples : state reachability, termination

11

Page 12: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

But undecidable in general

Thus abstraction to compute over-approximations of sets ofvalues : Abstract Interpretation

var x

var y

intervalles

var x

var y

Octogonesvar x

var y

Polyèdres

The analysis must terminate, may return an over-approximatedinformation (“false alarm”), but never a false answer

12

Page 13: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Abstract Interpretation (Cousot & Cousot 77)

Theory of semantics approximation (operators, fixpointtransfers)

−2 −1 0 1 2

−2, −1 −2, 0 −1,0 −1,1 −1,2 −2,1 −2,2 0,1 0,2 1,2

−2,−1,0 0,1,2−2,−1,1 −2,−1,2 −2,0,1 −2,0,2 −2,1,2 −1,0,1 −1,0,2 −1,1,2

−2,−1,0,1 −2,−1,1,2 −2,−1,0,2 −2,0,1,2 −1,0,1,2

T

T T

T

0

− +

α

γ

13

Page 14: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Fixpoint computation

To automatically find local invariants :

Abstract domain (lattice) for sets of value

The semantic is given by a system of equations, of whichwe compute iteratively a fixpoint :

X =

X1

. . .Xn

= F

X1

. . .Xn

F is non-decreasing, least fixpoint is the limit of Kleeneiteration X 0 = ⊥, X 1 = F (X 0), . . . , X k+1 = X k ∪ F (X k),. . .

Iteration strategies, extrapolation (called widenings) toreach a fixpoint in finite time

14

Page 15: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example : lattice of intervals

Intervals [a, b] with bounds in R with −∞ and +∞Smallest element ⊥ identified with all [a, b] with a > b

Greatest element ⊤ identified with [−∞,+∞]

Partial order : [a, b] ⊆ [c , d ] ⇐⇒ a ≥ c and b ≤ d

Sup : [a, b] ∪ [c , d ] = [min(a, b),max(c , d)]

Inf : [a, b] ∩ [c , d ] = [max(a, b),min(c , d)]

15

Page 16: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example

int x=0; // 1

while (x<100) { // 2

x=x+1; // 3

} // 4

x1 = [0, 0]x2 = ] −∞, 99] ∩ (x1 ∪ x

x3 = x2 + [1, 1]x4 = [100,+∞[∩(x1 ∪ x3

- Iterate i + 1 (i < 100) [Kleene/Jacobi/Gauss-Seidl] :

x12 = [0, 0]

x13 = [1, 1]

x14 = ⊥

x i+12 = [0, i ]

x i+13 = [1, i + 1]

x i+14 = ⊥

- Fixpoint (after 101 Kleene iterates or widening/narrowing) :

x∞2 = [0, 99]; x∞

3 = [1, 100]; x∞4 = [100, 100]

16

Page 17: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Analysis of programs using floating-pointnumbers

What is a correct program when using floating-point numbers ?

No run-time error, such as division by 0

But also the program does compute what is expected withrespect to some tolerance (the programmer usually thinksin real numbers)

For that, we need :

Bounds of floating-point values (ASTREE, FLUCTUAT)

Bounds on the discrepancy error between the real andfloating-point computations (FLUCTUAT)

If possible, the main source of this error (FLUCTUAT)

17

Page 18: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Related work and tools

The ASTREE static analyzer (see references)

Detection of run-time error for large synchronousinstrumentation softwareUsing in particular octogons and domains specialized fororder 2 filters (ellipsoids)Taking floating-point arithmetic into account

http://www.astree.ens.fr/

CADNA : estimation of the roundoff propagation inscientific programs by stochastic testinghttp://www-anp.lip6.fr/cadna/

GAPPA : automatic proof generation of arithmeticproperties http://lipforge.ens-lyon.fr/www/gappa/

18

Page 19: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Analysis for the floating-point value

First natural idea : Interval Arithmetic (IA) withfloating-point bounds, where min bound computed withrounding to −∞ and max bound computed with roundingto +∞

[a, b] + [c , d ] = [a + c , b + d ][a, b] − [c , d ] = [a − d , b − c][a, b] × [c , d ] = [min(ac , ad , bc , bd), max(ac , ad , bc , bd)]

Defect : too conservative, non relational

extreme example : if X = [−1, 1], X − X computed ininterval arithmetic is not 0 but [−2, 2]

A solution : Affine Arithmetic, an extension of IA thattakes linear correlations into account

but correlations true only for computations on real numbers

19

Page 20: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Affine Arithmetic for real numbers

Proposed in 1993 by Comba, de Figueiredo and Stolfi as amore accurate extension of Interval Arithmetic

A variable x is represented by an affine form x :

x = x0 + x1ε1 + . . . + xnεn,

where xi ∈ R and εi are independent symbolic variableswith unknown value in [−1, 1].

x0 ∈ R is the central value of the affine formthe coefficients xi ∈ R are the partial deviations

the εi are the noise symbols

The sharing of noise symbols between variables expressesimplicit dependency

20

Page 21: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Concretization as a center-symmetric convexpolytope

Concretization (x,y) for the two affine forms

x = 20 − 4ε1 + 2ε3 + 3ε4

y = 10 − 2ε1 + ε2 − ε4

21

Page 22: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Affine arithmetic : arithmetic operations

Assignment of a of a variable x whose value is given in arange [a, b] introduces a noise symbol εi :

x =(a + b)

2+

(b − a)

2εi .

Addition is computed componentwise (no new noisesymbol):

x + y = (αx0 + αy

0) + (αx1 + αy

1)ε1 + . . . + (αxn + αy

n)εn

For example, with real (exact) coefficients , f − f = 0.

Multiplication : we select an approximate linear form, theapproximation error creates a new noise term :

x × y = αx0α

y0 +

n∑

i=1

(αxi α

y0 + αy

i αx0)εi + (

n∑

i=1

|αxi |.|

n∑

i=1

|αyi |)εn+1.

22

Page 23: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Affine forms define implicit relations : example

Consider, with a ∈ [−1, 1] and b ∈ [−1, 1], the expressions

x = 1 + a + 2 * b;

y = 2 - a;

z = x + y - 2 * b;

The representation as affine forms is x = 1 + ǫ1 + 2ǫ2,y = 2 − ǫ1, with noise symbols ǫ1, ǫ2 ∈ [−1, 1]

This implies x ∈ [−2, 4], y ∈ [1, 3]

It also contains implicit relations, such asx + y = 3 + 2ǫ2 ∈ [1, 5] or x + y − 2b = 3: we thus get

z = x + y − 2b = 3

Whereas we get with intervals

z = x + y − 2b ∈ [−3, 9]

23

Page 24: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Affine forms and existing relational domains

More expressive (less abstract) than zones or octogons [A.Mine]

Close to dynamic templates [Z. Manna]

Provides Sub-polyedric relations (there is a concretization

to center-symmetric bounded convex polyedra)

But by some aspects better than polyhedra [P. Cousot/N.Halbwachs]

for example, to interpret non-linear computations :

dynamic linearization of non-linear computations

much more efficient in computation time and memory

dynamic construction of relationsno static packing of variables needed

24

Page 25: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Comparative example

x = [0,2]y = x+[0,2]

z = xy;t = z-2*x-y;

Zones/polyhedra (with a simple semantics):

0 ≤ x ≤ 20 ≤ y − x ≤ 20 ≤ z ≤ 8−8 ≤ t ≤ 8

Affine forms:

x = 1 + ε1 ∈ [0, 2]y = 2 + ε1 + ε2 ∈ [0, 4]z = 2.5 + 3 ε1 + ε2 + 1.5 ε3 ∈ [−3, 8]t = −1.5 + 1.5 ε3 ∈ [−3, 0]

(in practice coupled with intervals, thus z ∈ [0, 8])

25

Page 26: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Concretisation of affine forms (x,y,z)

concretization of affine form with classical polyhedronfinds z − 2x − y ∈ [−3, 0]

26

Page 27: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Concretisation of affine forms (x,y,t)

concretization of affine form with classical polyhedron

27

Page 28: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Implementation using floating-point numbers

For the computation of the affine form for the real value,the analyzer also uses finite precision arithmetic :

Affine form with floating point coefficients (with higherprecision floating-point numbers, using the MPFR library)Uncertainty in the computation of coefficients is handledby creating new noise terms

28

Page 29: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Join (and meet) operations on affine forms

Let [αxi ∪ αy

i ] = [αxi , α

yi ] if αx

i ≤ αyi else [αy

i , αxi ]

A natural join between x and y is

x ∪ y = [αx0 ∪ αy

0 ] +∑

i∈L

[αxi ∪ αy

i ] εi

Result might be greater than the union of enclosingintervals, but may be more interesting to keep correlations

But with interval coefficients (x ∪ y) − (x ∪ y) 6= 0

we get back to the defects of intervals

29

Page 30: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Join (and meet) operations on affine forms

For an interval i, we note

mid(i) =i + i

2, dev(i) = i − mid(i)

the center and deviation of the interval.

A better join is then

x∪y = mid([αx0 , α

y0 ])+

i∈L

mid([αxi , α

yi ]) εi+

i∈L∪{0}

dev([αxi , α

yi ]) εu

k

Then we have affine forms with real coefficients again

Order on affine forms considers noise symbols due to joinoperations differently than noise symbols due to arithmeticoperations

30

Page 31: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example (join)

Let x = 1 + 2ε1 + ε2 and y = 2 − ε1.

Join on intervals : [x ] ∪ [y ] ∈ [−2, 4]

First join on affine forms :

x ∪ y = [1, 2] + [−1, 2]ε1 + [0, 1]ε2 ⊂ [−2, 5]larger enclosure than on intervals but it may still beinteresting for further computations to keep relations

Second join on affine forms :

x ∪ y = 1.5 + 0.5ε1 + 0.5ε2 + 2.5εu3 ⊂ [−2, 5]

same enclosure, but (x ∪ y ) − (x ∪ y) = 0

31

Page 32: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Order on affine forms with real coefficients

For variable x , let αxi , i ∈ L denote terms due to

“classical” noise symbols and βxk denote terms due to

“union” noise symbols :

x ≤ y iff∑

i∈L∪{0}

|αxi − αy

i | ≤∑

k

|βyk| −

k

|βxk |

Projection of “union” noise symbols on “classical” noisesymbols in arithmetic operations

Then we have a complete partial order (under somerestrictions)

32

Page 33: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Correctness of the semantics on affine forms

Affine forms define implicit relations

the concretization of an affine form representing a variablemust contain the concrete values of the variableand in whatever expression using the affine forms, theconcretization as interval of the expression must containthe concrete values it can take

we must not introduce non-existing relations by unduesharing of noise symbols

33

Page 34: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Control of the cost of the computation

The number of noise symbols must be controlled to avoida too costly analysis, for example :

relations introduced inside a loop are useful for an accurateresult at the end of the loopmay be no longer useful after the loop

34

Page 35: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

From real to floating-point computation

Affine arithmetic uses symbolic properties of real numbercomputation, such as associativity and distributivity of+,×These properties do not hold exactly for floating-pointnumbers, thus affine arithmetic can not be directly usedfor floating-point estimation

Example :

let x ∈ [0, 2] and y ∈ [0, 2], we consider ((x + y) − x) − y .with affine arithmetic : x = 1 + ε1, y = 1 + ε2

((x + y) − x) − y = ((2 + ε1 + ε2) − 1 − ε1) − 1 − ε2 = 0false in floating-point numbers : take x = 2 and y = 0.1,then in simple precision((x + y) − x) − y = −9.685755e − 08

35

Page 36: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Overview for floating-point computation

Affine arithmetic for real number estimation

Estimation of the loss of precision due to the use offloating-point numbers

using ideas from affine arithmeticdecomposition of errors on their provenance in the program

We deduce bounds for the floating-point value

36

Page 37: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Representation of values (concrete)

The set of floating-point values that a variable x can take isexpressed as:

f x = r x + ex1 + ex

ho

= r x +⊕

i∈I αxi + ex

ho

where:

r x is the real-number value that would have beencomputed if we had exact arithmetic available

αxi is the coefficient expressing the first-order error

introduced by the arithmetic operation labelled i in theprogram, propagated on x

exho is the higher-order error

37

Page 38: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example

float x = 0.1; // [1]

float y = 0.5; // [2]

float z = x+y; // [3]

float t = x*z; // [4]

x = 0.1 + 1.49011612e−9 [1]y = 0.5z = 0.6 + 1.49011612e−9 [1]+

2.23517418e−8 [3]t = 0.06 + 1.04308132e−9 [1]

+2.23517422e−9 [3]−8.94069707e−10 [4]−3.55271366e−17 [ho]

38

Page 39: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Abstraction

Affine Arithmetic for the real part r x as already presented

First natural idea: use interval arithmetic for coefficientsαx

i and exho

Rounding errors given by the IEEE 754 standard:

in general, an interval of width ulp(x) when x is not just asingleton

But of course, we run into dependency problems : affinearithmetic on errors also

39

Page 40: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

First-order errors

ex1 =

l∈L txl +

l∈L t ′xl ηl

txl : center of the first-order error associated to the

operation l

t ′xl ηl : deviation on the first-order error associated to

operation l

40

Page 41: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

First-order errors

ex1 =

l∈L txl +

l∈L t ′xl ηl+

txl : center of the first-order error associated to the

operation l

t ′xl ηl : deviation on the first-order error associated to

operation l

the other terms are useful for modelling the propagation ofthe first-order error terms after non-linear operations

41

Page 42: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

First-order errors

ex1 =

l∈L txl +

l∈L t ′xl ηl+

i∈I t ′′xi εi+

txl : center of the first-order error associated to the

operation l

t ′xl ηl : deviation on the first-order error associated to

operation l

the other terms are useful for modelling the propagation ofthe first-order error terms after non-linear operations

For instance, the term t ′′x×yi εi comes from the

multiplication of txl by αy

i εi , and represents the uncertaintyon the first-order error due to the uncertainty on the value,at label i

42

Page 43: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

First-order errors

ex1 =

l∈L txl +

l∈L t ′xl ηl+

i∈I t ′′xi εi+ βx

0 +⊕

p∈P βxp ϑp

txl : center of the first-order error associated to the

operation l

t ′xl ηl : deviation on the first-order error associated to

operation l

the other terms are useful for modelling the propagation ofthe first-order error terms after non-linear operations

For instance, the term t ′′x×yi εi comes from the

multiplication of txl by αy

i εi , and represents the uncertaintyon the first-order error due to the uncertainty on the value,at label i

The multiplications of noise symbols εiηl cannot berepresented in our linear forms: we use a new affine formϑp

43

Page 44: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

First example : an amazing scheme by Kahanand Muller

Compute, with x0 = 11/2.0 and x1 = 61/11.0, the sequence

xn+2 = 111 −(1130 − 3000

xn)

xn+1

If computed with real numbers, converges to 6. Ifcomputed with any approximation, converges to 100.

Results with Fluctuat :for x10 : finds the floating-point value of xn equal tof10 = 100, with an error e10 in [-94.1261,-94.1258], andthus a real value r10 in [5.8812,5.8815]for x100 :

with default precision of the analysis (fp numbers with 60bits mantissa), or even 400 mantissa bits numbers, findsf100 = 100, e100 = ⊤ and r100 = ⊤ : indicates highunstabilitywith 500 mantissa bits numbers, finds f100 = 100,e100 = −94 and r100 = 5.99...

44

Page 45: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example : a non linear Newton scheme

Computes the inverse of A, that can take any value in [20,30] :

double xi, xsi, A, temp;

signed int *PtrA, *Ptrxi, cond, exp, i;

A = __BUILTIN_DAED_DBETWEEN(20.0,30.0);

/* initial condition = inverse of nearest power of 2 */

PtrA = (signed int *) (&A);

Ptrxi = (signed int *) (&xi);

exp = (signed int) ((PtrA[0] & 0x7FF00000) >> 20) - 1023;

xi = 1; Ptrxi[0] = ((1023-exp) << 20);

temp = xsi-xi; i = 0;

while (abs(temp) > e-10) {

xsi = 2*xi-A*xi*xi;

temp = xsi-xi;

xi = xsi;

i++;

}

45

Page 46: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Analysis of the inverse computation

Symbolic execution

A = 20.0 : i = 5, xi = 5.0e-2 + [-2.82e-18,-2.76e-18]A = 30.0 : i = 9, xi = 3.33e-2 + [-5.28e-18,6.21e-18]

Static analysis for A in [20.0,30.0] :

Non relational : analysis does not prove termination of theNewton algorithmRelational (with 10000 subdivisions) : analysis finds

i in [5,9], xi in [3.33e-2,5.0e-2]+ [-4.21e-13,4.21e-13]

Study of this algorithm is not obvious (for example, execution ofthe same algorithm but with simple precision float variablesdoes not always terminate)

46

Page 47: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Example : second-order filter

A new independent input E at each iteration of the filter:

double S,S0,S1,E,E0,E1;

int i;

S=0.0; S0=0.0;

E=__BUILTIN_DAED_DBETWEEN(0,1.0);

E0=__BUILTIN_DAED_DBETWEEN(0,1.0);

for (i=1;i<=170;i++) {

E1 = E0;

E0 = E;

E = __BUILTIN_DAED_DBETWEEN(0,1.0);

S1 = S0;

S0 = S;

S = 0.7 * E - E0 * 1.3 + E1 * 1.1 + S0 * 1.4 - S1 * 0.7 ;

}

47

Page 48: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Second-order filter

Relational analysis on values and errors :

Values in [-1.09,2.76] Error in [-1.1e-14,1.1e-14]

48

Page 49: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

Second-order filter

Propagation of an error on the input:

Each input has now an error in [0,0.001]

Relational on errors : S in [-1.09,2.76], with a stabilizederror in [-0.00109,0.00276]

49

Page 50: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

References

What Every Computer Scientist Should Know AboutFloating-Point Arithmetic, by D. Goldberg, ACMComputing Surveys, 1991

http://cch.loria.fr/documentation/IEEE754/ACM/goldberg.pdf

An Introduction to Affine Arithmetic, by J. Stolfi and L.H. deFigueiredo, TEMA 2003

http://www.sbmac.org.br/tema/seletas/docs/v4 3/101 01summary.pdf

Abstract Interpretation: Achievements and Perspectives, by P.Cousot, SSGRR 2000

http://www.di.ens.fr/∼cousot/COUSOTpapers/SSGRRP-00-PC.sht

50

Page 51: Abstract Interpretation of Floating-Point Computationsprofs.sci.univr.it/~giaco/download/AVAS07/Lezione_Eric.pdf · 2007. 2. 26. · Abstract In-terpretation of Floating-Point Computa-tions

Abstract In-terpretationof Floating-

PointComputa-

tions

Sylvie Putotand EricGoubaultMEASI,CEA-

LIST/X/CNRS

References

A static analyzer for large safety-critical software, by B.Blanchet, P. and R. Cousot, J. Feret, L. Mauborgne, A.Min, D. Monniaux and X. Rival, PLDI 2003

http://www.di.ens.fr/∼cousot/COUSOTpapers/PLDI03.shtml

Static analysis-based validation of floating-point computations,by S. Putot, E. Goubault and M. Martel, Dagstuhl Seminar,LNCS 2991, Springer-Verlag, 2004.

http://www.di.ens.fr/∼goubault/papers/SPutot DagstuhlFinal.ps.

Static Analysis of Numerical Algorithms, by E. Goubault and S.Putot, SAS 2006

http://www-ist.cea.fr/publicea/exl-php/

200600004467-static-analysis-of-numerical-algorithms.html

51