Model Checking: An introduction & overview

Model Checking:An introduction & overview

Gordon J. PaceOctober 2005

History of Formal Methods Automata model of computation:

mathematical definition but intractable. Formal semantics: more abstract

models but proofs difficult, tedious and error prone.

Theorem proving: proofs rigorously checked but suffers from ‘only PhDs need apply’ syndrome.

The 1990s Radiation therapy machine

overdoses patients, Pentium FDIV bug, Ariane-V crash.

Industry willing to invest in algorithmic based, push-button

verification tools.

Model-Checking Identify an interesting computation

model, For which the verification question is

decidable, And tractable on interesting

problems. Write a program to answer

verification questions.

Formal Semantics Operational Semantics:

(P, (, ’)

(P;Q, (Q, ’)

(v:=n, ) (, [vn])

Formal Semantics Denotational Semantics of Timed

Systems:

v V’

[ delay (v’, v) ] =

v’(t+1)=v(t) /\ v’(0)=low

def [ ]

0

Transition Systems

Q = States = Transition relation ( Q x Q) I = Initial states ( Q)

Q, , I

Constructing TSs via OS

(v:=1; w:=v) || (v:=¬v)

v,w=1,0

pc=1,0

v,w=1,1

pc=1,0

v,w=0,0

pc=0,0

v,w=1,0

pc=0,0

v,w=0,1

pc=0,0

v,w=1,1

pc=0,0

v,w=1,0

pc=0,1

v,w=0,1

pc=0,1

Constructing TSs via TDSi

m

o

Q = Bool x Bool x Bool

I = {(i,m,o) | o = i /\ m }

= {((i,m,o),(i’,m’,o’)) | m’=o, o’=i’ /\ m’ }

Note: We will be ‘constructing’ TSs from a

symbolic (textual/graphical) description of the system. This is a step which explodes exponentially (linear increase in description may imply exponential increase in state-space size).

Properties of TSs Safety properties: ‘Bad things never

happen’. eg The green lights on a street will never be

on at the same time as the green lights on an intersecting street.

Liveness properties: ‘Good things eventually happen’.

eg A system will never request a service infinitely often without eventually getting it.

Safety Property Model

Are any of the red states reachable?

etc

Safety Property Model

Given a transition system M=Q,,I and a set of ‘bad’ states B, are there any states in B which are reachable in M?

A Reachability Algorithm

R0 = I

Rn+1 = Rn (Rn)

where: (P) = { s’ | sP: s s’ }

Reachable set is the fix-point of this sequence. Termination and correctness are easy to prove.

A Reachability Algorithm

R := I; Rprev := ;

while (R Rprev) do

Rprev := R; R := R (R); if (B R ) then BUG;

CORRECT;

State Space Representation Explicit representation

Keeping a list of traversed states. State-explosion problem. Looking at the recursion stack will give

counter-example (if one is found). Breath-first search guarantees a

shortest counter-example.

Typical Optimizations On-the-fly exploration: Explore

only the ‘interesting’ part of the tree (wrt property and graph).

Example: Construct graph only at verification time. Finding a bug would lead to only partial unfolding of the description into a transition system.

Typical Optimizations Partial order reduction: By

identifying commuting actions (ones which do not disable each other), we can ignore parts of the model.

Example: To check for deadlock in (a!; P b!; Q), we may just fire actions a and b in this order rather than take all interleavings.

Typical Optimizations Compositional verification: Build

TS bottom up, minimising the automata as one goes along.

Example: To construct (P Q), construct P and minimise to get P’, construct Q and minimise to get Q’, and then calculate (P’ Q’).

Typical Optimizations Interface-Based Verification: Use

information about future interfaces composands while constructing sub-components.

Example: Constructing the full rhs of (10c;P + 5c;Q + …) Huge (5c;Tea) gives a lot of useless branches which the last process never uses.

State Space Representation Symbolic state

representation: Use a symbolic formula to represent the set of states.

R := I; Rprev := ;

while (R Rprev) do

Rprev := R; R := R (R); if (B R ) then BUG;

CORRECT;

Requires: representation of empty set, union, intersection, relation application, and set equality test.

Symbolic Representation

Use boolean formulae

Let v1 to vn be the boolean variables in the state space. A boolean formula f(v1,…,vn) represents the set of all states (assignments of the variables) which satisfy the formula.

Symbolic Representation

Double the variables

To represent the transition relation, give a formula over variables v1,…,vn and v’1,…,v’n relating the values before and after the step.

Examplev1

v2

v3

Initial states:

I (v2=true) /\ (v3=v1 /\ v2)

Transition relation:

T (v3=v1 /\ v2) /\ (v’3=v’1 /\ v’2) /\ v’2=v3

1

Set Operators:

Empty set: = falseIntersection: P Q = P /\ Q Union: P Q = P \/ QTransition relation application:(P) = (vars: P /\ T)[vars’/vars]Testing set equality:

P=Q iff P Q

The Problem Calculating whether a boolean

formula is a tautology is an NP-complete problem.

In practice representations like Binary Decision Diagrams (BDDs) and algorithms used in SAT checkers perform quite well on typical problems.

Counter-Example Generation

I=R0Bad

Counter-Example Generation

I BadR1

Counter-Example Generation

I BadR1

R2

Counter-Example Generation

I BadR1

R2

Counter-Example Generation

I BadR1

R2

Counter-Example Generation

I BadR1

R2

Counter-Example Generation

I BadR1

R2

Set of all shortest counter-examples obtained

Abstract Interpretation Technique to reduce state space to

explore, transition relation to use. Collapse state space by

approximating wrt property being verified.

Can be used to verify infinite state systems.

Abstract Interpretation Example: Collapse states together

by throwing away variables, or simplifying wrt formula.

etc

Abstract Interpretation Example: Collapse states together

by throwing away variables, or simplifying wrt formula.

etc

Abstract Interpretation Example: Collapse states together

by throwing away variables, or simplifying wrt formula.

etc

Abstract Interpretation Concrete counter-example

generation not always easy. May yield ‘false negatives’.

etc

Other Techniques Backward Analysis

R0 = Bad

Rn+1 = Rn -1(Rn)

If R be the fix-point of this sequence, the system is correct iff R I = .

Other Techniques Induction (depth 1): If …

1. The initial states are good, and2. Any good state can only go to a

good state, then

The system is correct.

Other Techniques Induction (depth n): If …1. Any chain of length n starting from

an initial state yields only good states, and

2. Any chain of n good states can only be extended to reach a good state, then,

The system is correct.

Other Techniques Induction

By starting with n=1 and increasing, (plus adding some other constraints) we get a complete TS verification technique.

State-of-the-art Explicit state traversal: No more than

107 generated states. Works well for interleaving, asynchronous systems.

Symbolic state traversal: Can reach up to 10150 (overall) states. Works well for synchronous systems. Sometimes may work with thousands of

variables … With abstraction, 101500 states and above have

been reported!

State-of-the-art Combined with other techniques,

microprocessor producers are managing to ‘verify’ large chunks of their processors.

Application of model-checking techniques on real-life systems still requires expert users.

Tools Various commercial and academic tools

available. Symbolic:

BDD based: SMV, NuSMV, VIS, Lustre tools. Sat based: Prover tools, Chaff, Hugo, Bandera

toolset. Explicit state: CADP, Spin, CRL, Edinburgh

Workbench, FDR. Various high-level input languages: Verilog,

VHDL, LOTOS, CSP, CCS, C, JAVA.

Stating Properties Safety properties are easy to specify

Intuition: ‘no bad things happen’. If you can express a new output

variable ok which is false when something bad happens, then this your property is a safety property (observer based verification).

Not all properties are safety properties.

Observer Verification

Program

Observer

inputs outputs

ok

Advantage: Program and property can be expressed in the same language.

Safety Properties The system may only shutdown if the

mayday signal has been on and unattended for 4 consecutive time units.

shutdown

ok

mayday

Non-Safety Properties Bisimulation based verification Temporal logic based verification

Linear time logic (eg LTL) Globally (Finally bell) Branching time logic (eg CTL) AG (ding EF dong) Globally (Globally req Finally ack)

Beyond Finite Systems Example: Induction on structure:From:Prog(in,out) satisfies Prop(in,out)Prog(in,m) /\ Prop(m,out) satisfies

Prop(in,out)Conclude:Any chain of Prog’s satisfies Prop.

Philosophical Issues

So does this constitute a proof? Can I now claim my product to be

correct? Would a proof that P=NP change

verification as we now know it?

What I would have also liked to talk about … Other techniques (STE, BMC,…), More about infinite systems, Testing and combining testing with

verification, Interaction between theorem-provers and

model-checkers, Model-checking other types of systems

(hybrid systems, Petri-Nets, etc).

What now? Potential projects … Verification of Kevin & co’s

synchronisation algorithms, Use grammar induction to improve

interface based verification, SPeeDI and hybrid system

verification, Structural induction to model-check

compiler properties.