Verification of architectural memory models by model checking Shaz Qadeer Compaq Systems Research Center [email protected].

Verification of architectural memory models

by model checking

Shaz QadeerCompaq Systems Research Center

[email protected]

Outline

• Introduction to shared-memory systems and models

• Model checking method for verifying models on systems

Compiler

Multiprocessor

Multithreaded program

Executable code

Languagememory model(Java, Modula-3, C with threads)

Multiprocessor

P P P

P P P

INTERCONNECT NETWORK

Architecturalmemory model(SC, Alpha, Sun)

Multiprocessor

Compiler

Multiprocessor


Executable code


Compiler


Executable code



Verification Problem


Multiprocessor ?

Uniprocessor

Memory

PA := 1;if (B = 0) { ...}

Initially A = B = 0

Shared-memory multiprocessor

Memory

PA := 1;if (B = 0) { ...}

Initially A = B = 0

PB := 1;if (A = 0) { ...}

Memory

PW(A, 1)

R(B, ?)

Initially A = B = 0

PW(B, 1)

R(A, ?)

Shared-memory model

Sequential consistency

Memory

PW(A, 1)

R(B, 1)

Initially A = B = 0

PW(B, 1)

R(A, 1)


Memory

PW(A, 1)

R(B, 0)

Initially A = B = 0

PW(B, 1)

R(A, 1)


Memory

PW(A, 1)

R(B, 1)

Initially A = B = 0

PW(B, 1)

R(A, 0)


Memory

PW(A, 1)

R(B, 0)

Initially A = B = 0

PW(B, 1)

R(A, 0)

Dekker’s algorithm

Memory

PA := 1;if (B = 0) { CS}

Initially A = B = 0

PB := 1;if (A = 0) { CS}

Interconnect network

P1

C1

Memory +Directory

P2

C2

state[A] = INV state[A] = EXC

WR_REQ(A)

RDEX(A) FWD_RDEX(A, P1)


P1

C1

Memory +Directory

P2

C2

state[A] = INV state[A] = INV

WR_REQ(A)


RDEX_ACK(A)


P1

C1

Memory +Directory

P2

C2

state[A] = EXC state[A] = INV

WR_REQ(A)


RDEX_ACK(A)

WR_ACK(A)

• Programmers program according to a memory model

• System must satisfy memory model for software correctness

• Shared-memory systems are very complex

Parameterized shared-memory systems

Parameters: processors n, addresses mMemory actions: {R,W} {1,..,n} {1,..,m} ValInternal actions: I {1,..,n} {1,..,m}

State transition system: State variablesInitial predicateGuarded command for each action

State transition systemcache: array [1..n] of array [1..m] of {s: State, d: Val}queue: array [1..n] of Queue[m: Msg, a: [1..m]]…

(R,i,j,v) [] cache[i][j].s INV cache[i][j].d = v

(W,i,j,v) [] cache[i][j].s = EXC cache[i][j].d := v

(RRQ,i,j) [] cache[i][j].s = INV queue[i].enqueue(RD_REQ, j)…

(EventId, Proc, Addr, Data)

(WRQ, 2, 1)(WRP, 2, 1)(W, 2, 1, 1)(R, 1, 1, 0)(R, 2, 1, 1)(WRQ, 1, 1)(WRP, 1, 1)(R, 1, 1, 1)(W, 1, 1, 2)(RRQ, 2, 1)(RRP, 2, 1)(R, 2, 1, 2)

Run: finite action sequence

executable from initial state

Verification problem

Impl: state transition system with actions

Spec:1. Invariants, e.g., 1 i, j n. cache[i].s = EXC i j cache[j].s = INV2. Memory models, e.g., sequential consistency, Alpha memory model

Does Impl satisfy Spec?





Memory model

(R,1,1,0)

(R,1,1,1)

(W,1,1,2)


(W,2,1,1)

(R,2,1,1)

(R,2,1,2)

Processor 1 Processor 2

• n partial orders, one for each processor• i th partial order on memory actions at processor i


(R,1,1,0)

(R,1,1,1)

(W,1,1,2)


(W,2,1,1)

(R,2,1,1)

(R,2,1,2)

Processor 1 Processor 2

Sequential consistency(EventId, Proc, Addr, Data)

(R,1,1,0)

(W,2,1,1)

(R,2,1,1)

(R,1,1,1)

(W,1,1,2)

(R,2,1,2)

0

1

1

1

2

2

Addr 1(W,2,1,1)

(R,1,1,0)

(R,2,1,1)

(R,1,1,1)

(W,1,1,2)

(R,2,1,2)

swap!

Sequential consistency(EventId, Proc, Addr, Data)

(R,1,1,0)

(W,2,1,1)

(R,2,1,1)

(R,1,1,1)

(W,1,1,2)

(R,2,1,2)

Witness order

System S satisfies Model M iff

there is a witness order for every run

Debugging vs. Verification

McMillan, Schwalbe 91 Clarke et al. 93

Eiriksson, McMillan 95 Ip, Dill 96

Katz, Peled 92Alur et al. 96

Nalumasu et al. 98Henzinger et al. 99

Loewenstein, Dill 92Pong, Dubois 95

Park, Dill 96 Delzanno 00

Graf 94Henzinger et al. 99

TLA Plakal et al. 98

Invariants

ImplSpec

Memory models

Fixed parameters

Arbitraryparameters

needed in practice

Verifying Memory Models is Hard

Alur, McMillan, Peled 96 :

Checking sequential consistency for finiteparameter values is undecidable.

Contribution

Model checking algorithm to verify

a number of shared-memory models on

a useful class of shared-memory systemsfor

finite number of processors and addressesby

reduction to invariant verification.

Outline

• Introduction to shared-memory systems and models

• Model checking method for verifying models on systems

State transition systemcache: array [1..n] of array [1..m] of {s: State, d: Val}queue: array [1..n] of Queue[m: Msg, a: [1..m]]…

(R,i,j,v) [] cache[i][j].s INV cache[i][j].d = v

(W,i,j,v) [] cache[i][j].s = EXC cache[i][j].d := v

(RRQ,i,j) [] cache[i][j].s = INV queue[i].enqueue(RD_REQ, j)…





Data independence• Memory systems do not conjure up data values• Data values copied but not examined by actions

(except for read and write actions)• Every run can be generated from an unambiguous

run by suitably renaming data values.

(R,1,1,0)

(R,1,1,1)

(W,1,1,2)

(W,2,1,1)

(R,2,1,1)

(R,2,1,2)

Unambiguous run:

Suffices to analyze unambiguous runs!


(R,1,1,0)

(R,1,1,1)

(W,1,1,2)

(W,2,1,1)

(R,2,1,1)

(R,2,1,2)

Unambiguous run:

Witness write order for address 1(W,2,1,1) (W,1,1,2)

System S satisfies Model M ifffor every run there are witness write orders for all addresses

acyclic graph

witness order


Recipe for verification

For every unambiguous run,1. guess write order for each address2. generate graph and check for cycles


P1

C1

Memory +Directory

P2

C2

state[A] = EXC state[A] = INV

WR_REQ(A)


RDEX_ACK(A)

WR_ACK(A)

Simple write order

For each location, order write events according to

actual occurrence order !!

Examples

• Piranha chip multiprocessor (Compaq)

• Wildfire challenge problem (Compaq)

• DASH multiprocessor (Stanford)


For every unambiguous run,1. guess write order for each address

• simple write order2. generate graph and check for cycles

. . .

(*,i,z,*)

. . .

(*,i,x,*)

. . .

. . .

(*,j,x,*)

. . .

(*,j,y,*)

. . .

. . .

(*,k,y,*)

. . .

(*,k,z,*)

. . .

Nice cycles

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

3-nice cycle:•3 processors i, j, k•3 addresses x, y, z

k-nice cycle involves k processors and k addresses


Nice cycles

S has a cycle iff

S has a k-nice cycle for 1 k min(n,m)

S: memory system with n processors, m addresses

Lemma:


For every unambiguous run,1. guess write orders for each address

• simple write order2. generate graph and check for cycles

• reduce to nice cycles• detecting nice cycles by model checking

Detecting nice cycles

min(n,m) model checking lemmas: kth lemma checks for k-nice cycles

S has a cycle iff

S has a k-nice cycle for 1 k min(n,m)

S: memory system with n processors, m addresses

Lemma:

•Supj supplies write values for address j.

•Moni monitors memory events at processor i.

Memorysystem

Sup1 Mon1Model checker

Property = i. Moni@err1-nice cycle


Memorysystem

Sup1

Sup2

Mon1

Mon2

Model checker

Property = i. Moni@err


2-nice cycle



Memorysystem

Sup1

Sup2

Sup3

Mon1

Mon2

Mon3

Model checker



3-nice cycle



. . .

(*,i,z,*)

. . .

(*,i,x,*)

. . .

. . .

(*,j,x,*)

. . .

(*,j,y,*)

. . .

. . .

(*,k,y,*)

. . .

(*,k,z,*)

. . .

Nice cycles

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .



Unambiguous run:

(R,1,1,0)

(R,1,1,1)

(W,1,1,2)

(W,2,1,1)

(R,2,1,1)

(R,2,1,2)


Witness write order for address 1(W,2,1,1) (W,1,1,2)

Causal edges

Anti-causal edges

Supplier automata

Supplies 0 upto some nondeterministicallychosen write and then supplies 1 forever.

0

1

1

Supx (supplier for address x):

. . .

(*,i,z,1)

. . .

(*,i,x,0)

. . .

. . .

(*,j,x,1)

. . .

(*,j,y,0)

. . .

. . .

(*,k,y,1)

. . .

(*,k,z,0)

. . .

Nice cycles

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .

. . .



Monitor automata

Monj:

(*,j,x,1) (*,j,y,0)err

Analysis for sequential consistency

Number of processors = nNumber of addresses = mNumber of model checking runs = min(n,m)

Number of Sup automata = kNumber of Mon automata = kFor all j, number of states in Supj = 3

For all i, number of states in Moni = 3States in model checked system = |S|3k3k

kth model checking run (1 k min(n,m)):

Model checker


Other memory models?Alpha memory model, partial store order,

release consistency, weak ordering

Memorysystem

Sup1

Sup2

Sup3

Mon1

Mon2

Mon3

Model checker


Other write orders?

Can be generalized !!

Memorysystem

Sup1

Sup2

Sup3

Mon1

Mon2

Mon3

Wildfire challenge problem

• 2 processor, 2 location system– Supplier and monitor automata constructed– Seeded bug was found by invariant

verification on composed system by model checker TLC

Summary

Model checking algorithm to verify

a number of shared-memory models on

a useful class of shared-memory systemsfor

finite number of processors and addressesby

reduction to invariant verification.

Verification of architectural memory models by model checking Shaz Qadeer Compaq Systems Research Center [email protected].

Documents

memory model slide

memory actions

p1 slide

multiprocessor slide

uniprocessor memory

memory model system

acka slide

complex slide