Chair of Software Engineering 1 Doing more with contracts Bertrand Meyer, Ilinca Ciupa, Andreas Leitner, Lisa Liu SOFSEM, Harrachov, January 2007 ETH Zurich,

Chair of Software Engineering1

Doing more with contracts

Bertrand Meyer, Ilinca Ciupa, Andreas Leitner, Lisa LiuSOFSEM, Harrachov, January 2007

ETH Zurich, Chair of Software Engineering

2

The three questions

What does it expect?

What does it promise?

What does it maintain?

3

Progress in software quality

Tools, techniques, methods, languages,software quality in general…

Market forces work not for very high quality but for good enough software

Collective decision to turn IT into a service business

… have been improving steadily…

… but not enough!

4

Where best to apply our efforts?

End-user applications

Compilers & operating systems

General-purpose components

Specialized components

Relies on

5

Components should be contracted

Definition of what each element of the functionality:

Expects (precondition) Promises (postcondition) Maintains (invariant)

Does not have to be complete (but wait)

6

Design by Contract

Every software element is intended to satisfy a certain goal, for the benefit of other software elements (and ultimately of human users).

This goal is the element’s contract.

The contract of any software element should be Explicit Part of the software element itself

7

A class with contracts

classBANK_ACCOUNT

createmake

featuremake (n : STRING)

-- Set up with name n

requiren /= Void

doname := nbalance := 0

ensurename = n

end

name : STRING

balance : INTEGER

deposit ( v : INTEGER)

-- Add amount v

do

balance := balance + v

ensure

balance = old balance + v

end

invariant

name /= Void

balance >= 0

end

ensurename =

n

requiren /=

Void

ensure balance = old balance

+ v

invariantname /=

Voidbalance >= 0

8

The correctness of a class

For every creation procedure cp :

{precp} docp {INV and postcp}

For every exported routine r :

{INV and prer} dor {INV and postr}

a.h (…)

create a.make (…) S1

S2

S3

S4

a.g (…)

a.f (…)

9

What we do with contracts today

Write better software Analysis Design Reuse Implementation Bug avoidanceDocument software automaticallyHelp project managers do their job

Perform systematic testingGuide the debugging process

(with run-time monitoring)

10

deferred class VAT inherit

TANK

feature

in_valve, out_valve: VALVE

fill is-- Fill the vat.

require in_valve.open out_valve.closed

deferred ensure

in_valve.closed out_valve.closed is_full

end

empty, is_full, is_empty, gauge, maximum, ... [Other features] ...

invariant

is_full = (gauge >= 0.97 maximum)  and  (gauge <= 1.03 maximum)

end

Contracts for analysis, specification

11

Seamless development with Eiffel & contracts

Single notation, tools, concepts, principles

Continuous, incremental development

Keep model, implementation, documentation consistent

Reversibility

Example classes:PLANE, ACCOUNT, TRANSACTION…

STATE, COMMAND…

HASH_TABLE…

TEST_DRIVER…

TABLE…

Analysis

Design

Implemen-

tation

V&V

Generali-zation

12

Documentation: the contract view of a class

classBANK_ACCOUNT

createmake

featuremake (n : STRING)

-- Set up with name n

requiren /= Void

doname := nbalance := 0

ensurename = n

end

name : STRING

balance : INTEGER

deposit ( v : INTEGER)

-- Add amount v

do

balance := balance + v

ensure

balance = old balance + v

end

invariant

name /= Void

balance >= 0

end

ensurename =

n

requiren /=

Void

ensure balance = old balance

+ v

invariantname /=

Voidbalance >= 0

13

Lists with cursors

"Prague"

Cursor

index

count1

forth

afterbefore

14

Command “forth″

15

Where the cursor may go

count+1

Valid cursor positions

item

count1

afterbefore

0

16

From the invariant of class LIST

Valid cursor positions

17

Contracts and inheritance

r isrequire

ensure

r isrequire

ensure

a1: A

a1.r (…)…

Correct call in C:

if a1. then

a1.r (...) -- Here a1. holds end

r ++

C A

D B

ClientInheritance

++ Redefinition

18

Assertion redeclaration rule

When redeclaring a routine, we may only:

Keep or weaken the precondition

Keep or strengthen the postcondition

19

A simple language rule does the trick!

Redefined version may have nothing (assertions kept by default), or

require else new_preensure then new_post

Resulting assertions are: original_precondition or new_pre

original_postcondition and new_post

Assertion redeclaration rule in Eiffel

20

Contracts as a management tool

High-level view of modules for the manager:

Follow what’s going on without reading the code

Enforce strict rules of cooperation between units of the system

Control outsourcing

21

Contracts for testing and debugging

Contracts provide the right basis: Testing is there to find bugs A bug is a discrepancy between intent and reality Contracts describe intent

A contract violation always signals a bug: Precondition violation: bug in client Postcondition violation: bug in routine

In EiffelStudio: select compilation option for run-time contract monitoring at level of class, cluster or system.

22

Anecdotal & non-anecdotal evidence

HP 1: invariant

r = 2 ^ i

HP 2: Eiffel messes up our great system!

Axa Rosenberg: postcondition fails in deep_clone of TWO_WAY_LIST !

Patrice Chalin study (Concordia): Eiffel programmers do use contracts day in, day out.

23

Should we test software?

Some evidence of why we should: Limitations of proof technology Properties not formally specified

Tests as complement of proofs?

From a survey of 240 companies in N. America and Europe:

8% release software to beta sites without testing

83% of their developers don't like to test code.

53% don't like to test their own code because they find it tedious.

30% don't like to test because they find testing tools inadequate.

Testing is tedious

25

“Automated testing”

What can be automated:

Test execution

Robust execution

Regression testing

Test case generation (test suites)

Test result verification (test oracles)

Test scheduling

Test case minimization

Automated vs. manual unit tests

Automatically generated test cases complement manual ones:

Contracts may be “wrong” Implementation is unlikely to establish

postcondition on input satisfying A -> ¬B . Test cases will likely only be written for B (and not

for A -> ¬B ).

27

AutoTest: an automatic test framework

Input is a set of classes AutoTest generates instances and calls features

with automatically selected arguments Automatic tests rely on contracts:

Precondition violations: skip Postcondition/invariant violation: bingo!

Manual tests can be added explicitly:Relevant ones are determined automaticallyAny test (manual or automated) that fails

becomes part of the test suite

28

AutoTest demo

29

Automated testing and slicing

auto_test system.ace BANK_ACCOUNT STRING

create {STRING} v1v1.wipe_out

v1.append_character (’c’)

v1.append_double (2.45)create {STRING} v2v1.append_string (v2)

v2.fill (’g’, 254343)...create {BANK_ACCOUNT}

v3.make (v2)v3.deposit (15)

v3.deposit (100)

v3.deposit (-8901)...

classBANK_ACCOUNT

createmake

featuremake (n : STRING)

requiren /= Void

doname := nbalance := 0

ensurename = n

end

name : STRING

balance : INTEGER

deposit (v : INTEGER)

do

balance := balance + v

ensure

balance =

old balance + v

end

invariant

name /= Void

balance >= 0

end

Robust execution

31

Some AutoTest results (random strategy)

Library Total Failed Total Failed

EiffelBase 40,000 3% 2000 6%

Data Structures 21,000 5% 1400 6%

Gobo Math 1500 1% 140 6%

TESTS ROUTINES

32

Result analysis

(Raluca Borga, U. of Cluj-Napoca,diploma work at ETH)

Fault classification (manual): Specification error Implementation error Inheritance Don’t know

About 50% implementation, 50% specification Results polluted by “void” issue (Will no longer

exist thanks to attached types, see ECOOP 2005 paper)

AutoTest strategies

Random strategy

Use random input

Planning strategy

Employ information from postcondition to satisfy preconditions

...

34

Basic strategy

Object pool For each type, initialize through creation

procedures (constructors) This requires proper ordering of classes Diversify through procedures

Routine arguments

Basic values: heuristics for each type

Objects: get from pool Invoke selected routines AutoTest tests all routines, including inherited ones

(“Fragile base class” issue)

35

Basic idea: random testing may find bugs faster if inputs evenly spread over range of possible values

Example:r (m, n: INTEGER) do … end

m

n

Random inputs

m

n

ART

Adaptive Random Testing (ART, Chen et al.)

36

So far ART only applied to numeric inputs, on which a total order relation exists

Multi-field objects are not members of a totally ordered set

Need to define notion of distance between objects

Extending ART to objects

37

Object distance (Ilinca Ciupa)

p ↔ q

combination (

type_distance (p.type, q.type),

field_distance (p, q),recursive_distance (

{[p.r ↔ q.r] | r Reference_attributes } )

=Δ

38

The role of argument-less boolean queries

Externally, boolean queries often serve as preconditions and appear in invariants

Internally, boolean queries often serve as branch conditions

Lists: before, after, off, is_first, is_last, is_empty…Account: is_overdraft…

"Prague"

Cursor

index

count1

afterbefore

is_first

is_last

39

Boolean query conjecture

The argument-less boolean queries of a well-written class yield a partition of the object state space that helps the testing process.

40

Boolean query coverage (Lisa Liu)

A set of tests for a class satisfies boolean query coverage if and only if the execution of these tests can cover all the reachable abstract object states for that class.

Strategy:

Through constraint solving, generate states that satisfy boolean queries in invariant

(e.g. after implies off)

Through theorem proving, remove states that do not satisfy rest of invariant

(e.g. count < capacity)

Chair of Software Engineering41

"Prague"

Cursor

index

count1

afterbefore

is_first

is_last

42

Experimental results

Class Random Boolean Queries

RoutineCoverage

Bugs Found

Routine Coverage

Bugs Found

INT_STACK 33% 1 100% 2

LINKED_LIST 85% 1 99% 7

BINARY_TREE

88% 5 100% 11

ARRAYED_SET

84% 1 100% 6

FIXED_LIST 93% 12 99% 12

43

AutoTest developments

Large-scale extensive tests (cluster computing)

Comparison with manual efforts

Scaling up (memory, time) (mostly done)

Rigorous assessment of testing criteria & strategies

Complete integration with EiffelStudio environment

Background, unobtrusive, continuous testing

Automatic regression testing

Distributed cooperative testing (“Testi@home”)

44

Towards a real framework

Pluggable strategies Flexible execution mechanism Robust execution mechanism Unified interface Integrate tightly with manual testing Concise output

45

Proving classes

classLINKED_LIST [G]

feature…remove_front

-- Remove first itemrequire

not emptydo

first := first.rightensure

count = old count – 1first = old item (2)

end

…

end

first

right right right

46

Issues in proving classes

1. Providing full contracts

2. Devising models of the underlying world

3. Devising proof rules (i.e. semantics of the programming language)

4. Taking advantage of inheritance on the proof side

5. Doing the proofs, mechanically

47

The contract language

In Eiffel (and other formalisms based on Design by Contract): mostly boolean expressions of the language, plus some extensions such as old.

Seems not expressive enough — but is!

48

Beefing up expressive power

Eiffel Model Library

Components to prove(e.g. EiffelBase)

49

Eiffel Model Library (also known as MML)

(With Bernd Schoeller, Tobias Widmer)Classes correspond to mathematical concepts:

SET [G], FUNCTION [G, H ], TOTAL_FUNCTION [G, H ], RELATION [G, H ], SEQUENCE [G ], …

Completely applicative: no attributes (fields), no implemented routines (all completely deferred)

Specified with contracts (unproven) reflecting mathematical properties

Expressed entirely in Eiffel

50

Example MML class

class SEQUENCE [G] feature

count : NATURAL-- Number of items

last : G-- Last item

extended (x) : SEQUENCE [G]-- Identical sequence except x added at end

ensureResult.count = count + 1

Result.last = x

Result.sub (1, count) ~ Current

mirrored : SEQUENCE [G]-- Same items in reverse order

ensureResult.count = count…

…end

51

Specifying lists

classLINKED_LIST [G]

feature…remove_front

-- Remove first itemrequire

not emptydo

first := first.rightensure

end

…

end

first

right right right

count = old count – 1first = old item (2)

model = old

model.tail

52

Principles

Very simple mathematics only Logic Set theory

53

Proofs: current progress

Basic theory of O-O programming with pointers

Progress towards frame issue

Proof engine: Boogie (Microsoft Research)

Several publications in progress

54

Tests or proofs?

Tests and proofs! Failed proof: proof obligations remain –

candidates for tests Model checking counter-example: try to generate

test data that exercises it

TAPTests And Proofs

ETH Zurich, 11-13 Feb 2007

http://tap.ethz.ch

55

Concurrency

What do contracts become in a concurrent context?

56

put

57

store (b : [G ] ; v : G )

-- Store v into b. require

not b.is_full do

… ensure

not b.is_empty end

QUEUE BUFFER

my_queue : [T ]

…

if not my_queue.is_full then

store (my_queue, t )

end

BUFFER QUEUE

put

58

On separate target, precondition becomes wait condition

From preconditions to wait-conditions

store (buffer : separate BUFFER [INTEGER ] ; v : INTEGER) -- Store v into buffer.

requirenot buffer.is_fullv > 0

dobuffer.put (v)

ensurenot buffer.is_empty

end...store (my_buffer, 10 )

59

SCOOP in a nutshell

No intra-object-concurrency One keyword: separate, indicates thread of control

is “elsewhere” Reserve one or more objects through argument

passing Preconditions become wait conditions Automatic resynchronization on queries (lazy wait) Exception-based mechanism to break lock Asynchronous exceptions

60

Dining philosophers

class PHILOSOPHER inheritPROCESS

renamesetup as getup

redefine step end

feature {BUTLER}step do think ; eat (left, right)

end

eat (l, r : separate FORK) -- Eat, having grabbed l and r.

do … endend

61

Contract-Driven Development: demo

(Andreas Leitner)

62

Introductory programming course

Since 2003200-300 students

Uses object technology, Eiffel, Design by Contract

Outside-in approach (“inverted curriculum”): work from existing software

TRAFFIC library

Textbook in progress, Touch of Class, available onlinegoogle touch class meyer

63

Traffic GUI example

64

Traffic GUI example

65

The Traffic software (Michela Pedroni)

EiffelMedia- OpenGL (3D Drawing)- 2D Drawing Primitives- Sound- Collision detection

Gobo- Data structures- XML parsing- File handling

EWG- Library wrapping

Traffic- 2D and 3D display of a city- Transportation lines, roads, schedules- Random buildings- Vehicles, passengers

EiffelBase- Data structures- Kernel classes- Basic data types

66

The background technology: Eiffel

Method, language, IDE (EiffelStudio), libraries

Constantly refined since 1985

ECMA standard (2005); ISO standard (2006)

67

Some Eiffel projects

Axa RosenbergInvestment management: from $2 billion to >$40 billion 2 million lines of EiffelChicago Board of Trade Price reporting system Eiffel + CORBA + Solaris + Windows + …Boeing (Xontech) Large-scale Eiffel simulations of missile defense systemSwedish social security: accident reporting & management

Chair of Software Engineering68

Open-Source EiffelStudio

69

References

Object-Oriented Software Construction,2nd edition (Prentice Hall)

se.ethz.ch

eiffelstudio.origo.ethz.ch