Modular Verification of Multithreaded Software Shaz Qadeer Compaq Systems Research Center Shaz Qadeer Compaq Systems Research Center Joint work with Cormac.
Post on 20-Dec-2015
220 Views
Preview:
Transcript
Modular Verification of Multithreaded SoftwareModular Verification of Multithreaded Software
Shaz QadeerCompaq Systems Research
Center
Shaz QadeerCompaq Systems Research
CenterJoint work with
Cormac Flanagan, Stephen Freund, Sanjit Seshia
Multithreaded softwareMultithreaded software
Examples: operating systems, databases, web servers, Java libraries
Verification problem– for all inputs and interleavings, program
satisfies specification Verifying multithreaded programs is
difficult– large and complex
Modular verification of multithreaded programsModular verification of multithreaded programs
thread: verify each thread separately using specifications of other threads
procedure: in each thread, verify each procedure separately using specifications of called procedures
MethodMethod
Reduce verification of multithreaded programs to verification of sequential programs
Analysis of sequential programs well-studied
– dataflow analysis– abstract interpretation– theorem proving with verification
conditions
ActionAction
predicate on unprimed and primed variables
– (x’ = y+1 y’ = y) x := y+1
– (p x’ = x) py
– true skip
Atomic operationAtomic operation
actions p and q pq : all transitions allowed by p must also
be allowed by q (x’ = y+1 y’ = y) x’ y’ succeeds (x’ = y+1 y’ = y) x’ = y’ fails skipp assert p p abbreviates ptrue
OutlineOutline
modular verification – without procedure calls– with procedure calls
verification of Mercator
Simple multithreaded programSimple multithreaded program
Thread1
acq;x := x+1;assert x > 0;rel;
Thread2
acq;x := 0;rel;
m = i, if held by thread i0, if unheld
Init: x = 0
Simple multithreaded programSimple multithreaded program
Thread1
(m=0m’=1)m;x := x+1;assert x > 0;(m’=0)m ;
Thread2
(m=0m’=2)m;x := 0;(m’=0)m ;
Init: x = 0
Owicki-Gries-Lamport methodOwicki-Gries-Lamport method
Thread1
L1: m=0 x0 (m=0m’=1)m;L2: m=1 x0 x := x+1;L3: m=1 x1 assert x > 0;L4: m=1 x1 (m’=0)m ; L5: m=0 x0
Thread2
M1: m=0 x0 (m=0m’=2)m;M2: m=2 x0 x := 0;M3: m=2 x0 (m’=0)m ;M4: m=0 x0
Sequential correctness Non-interference
Why is Thread1 correct?Why is Thread1 correct?
Thread1 view
(m=0m’=1)m;*
*x := x+1;
*assert x > 0;
*(m’=0)m;
1-abstractionA1
;
(m=0m’=1)mA2 ;
A1;
x := x+1 A2 ;
A1;
assert x > 0 A2 ;
A1;
(m’=0)m A2 ;
(m’=0 x’ 0) (m = i m’ = i) (m = i x’ = x)
Ai Jones 83
Why is Thread1 correct?Why is Thread1 correct?
Thread1 view
(m=0m’=1)m;*
*x := x+1;
*assert x > 0;
*(m’=0)m;
1-abstraction
A1 reflexive-transitive !
Can replace A1
by A1 !
A1 ;(m=0m’=1)mA2 ;A1
;x := x+1 A2 ;A1 ;assert x > 0 A2 ;A1 ;(m’=0)m A2 ;
(m’=0 x’ 0) (m = i m’ = i) (m = i x’ = x)
Ai Jones 83
Sequentialchecker
Summary of methodSummary of method
Thread1
Thread2
Threadn
1-abs
2-abs
n-abs
Sequentialchecker
Sequentialchecker... ... ...
yes
yes
yes
Assume-guarantee decompositionAssume-guarantee decomposition
Misra-Chandy 81 – processes communicating over channels
Jones 83– parallel shared-memory programs
Abadi-Lamport 85– temporal logic
Alur-Henzinger 96, McMillan 97– Mealy machines (hardware)
. . .
OutlineOutline
modular verification – without procedure calls– with procedure calls
verification of Mercator
Simple multithreaded programSimple multithreaded program
Thread1
acq( );x := x+1;assert x > 0;rel( );
Thread2
acq( );x := 0;rel( );
Init: x = 0
Mutex implementation is not atomic !Mutex implementation is not atomic !
acq( ) { t := i; while (t 0) ACS(m,0,t);}
ACS(m,v,t) if (m=v) then m,t := t,m
rel( ) { m := 0;}
What is the specification of acq( ) and rel( )?
Modular verification of sequential programsModular verification of sequential programs
verify each procedure separately using specifications of called procedures
requires x > 0modifies xensures x’ = x+1inc( ) { x := x+1;}
. .assert x > 0;(x’ = x+1)<x>; . .
assume x > 0;x_pre := x;y_pre := y;x := x+1;assert x = x_pre+1;assert y = y_pre;
Pre/post-conditions not enough !Pre/post-conditions not enough !Thread1
acq( );x := x+1;assert x > 0;rel( );
acq( ) { t := 1; while (t 0) ACS(m,0,t);}
modifies x, mensures m=1 x0
abstraction boundary
A1 ;t := 1A2 ;while (t 0) { A1 ; ACS(m,0,t)A2 ;}A1 ;assert m=1 x0 ;
(m’=0 x’ 0) (m = i m’ = i) (m = i x’ = x)
Ai
Specification of acq() for Thread iSpecification of acq() for Thread i
Impl:acq( ) { t := i; while (t 0) ACS(m,0,t); }
Spec:skip* ; (m=0 m’=i)m
skip
(m=0 m’=i)m
m0
m=0
Often, witness can be guessed automatically !
Need witness to synchronize spec actions with impl actions !
Verification of acq() for Thread 1Verification of acq() for Thread 1
true ;t := 1 skip;true ;while (t 0) { true ; ACS(m,0,t) }
Spec: skip* ; (m=0 m’=1)m
m0 skip m=0 (m=0 m’=1)m
;
In general, assumption could involve variables in scope.
Specification of rel() for Thread iSpecification of rel() for Thread i
Impl:rel( ) { m := 0;}
Spec:(m’=0)m
(m’=0)m
Generalization of pre/post-conditionsGeneralization of pre/post-conditions
requires pmodifies Uensures q f( ) { . . .}
assert p ;
trueU
; assume q
f( ) { . . .}
Why is Thread1 correct?Why is Thread1 correct?
Thread1
skip* ; (m=0m’=1)m;x := x+1;assert x > 0;(m’=0)m;
Thread1
acq( );x := x+1;assert x > 0;rel( );
A1 ;{ skipA2 ; A1 ;};(m=0m’=1)mA2 ;A1 ;x := x+1A2 ;A1 ;assert x > 0A2 ;A1 ;(m’=0)mA2 ;
(m’=0 x’ 0) (m = i m’ = i) (m = i x’ = x)
Ai
SummarySummary
for each thread i– write assumption– write specifications for procedures– check each specification by inlining
specifications of called procedures– check assumptions of other threads
OutlineOutline
modular verification – without procedure calls– with procedure calls
verification of Mercator
ImplementationImplementation
Extended Static Checker for Java (Detlefs-Leino-Nelson-Saxe 98)
verification conditions proved by Simplify (Nelson 81)
More detailsMore details
atomic operation is read/write of a heap variable
optimizations for reducing assumptions between atomic operations
assume sequential consistency witness guessed automatically
Mercator (Heydon-Najork 99)Mercator (Heydon-Najork 99)
multithreaded web crawler written in Java in production use at Altavista synchronization based on Java monitors
and reader-writer locks
Mercator architectureMercator architecture
RAM DISK
. . .TC T1 TNUM
CheckpointingThread
Worker Threads
URL setURL queue
Reader-writer locksReader-writer locks
lock L1 :
– checkpointing thread acquires in write mode
– worker threads acquire in read mode
lock L2 :
– protects data structure on disk – worker thread acquires in write mode to
write and in read mode to read
Race detection Race detection
checkpointing thread – must hold L1 in write mode
worker thread– must hold L1 in read mode
– reading: must hold L2 in read mode
– writing: must hold L2 in write mode
VerificationVerification
analyzed ~1500 LOC Mercator’s reader-writer locks implement
specification– acq_read( ), acq_write( ), rel_read(),
rel_write( ) absence of races
– threads obey protocol for accessing shared data
found bug introduced by us
Annotation costAnnotation cost
55 annotations Scales with complexity of synchronization
patterns, not size of program
Verification of java.util.vectorVerification of java.util.vector
synchronization with Java monitors ~450 LOC 4 annotations found known race (Flanagan-Freund 00)
Annotation overheadAnnotation overhead
Category Fraction of Total
Read/Write Constraints & Data Invariants
6/55
Specification of Readers-Writer Locks 21/55
Other Interface Specifications 16/55
Environment Assumptions 12/55
File system
Web server
Disk
Buffer cache
Mutex
module Mutex { /* Data */ boolean b; int m;
/* Code */ acq() { } rel() { }}
create() { . acq(); .}
module Mutex { /* Data */ boolean b; int m;
invariant b m = 0;
/* Code */ spec …… acq() { . assert b; . }
spec …… rel() { }}
ProgramsPrograms
S SeqProg ::= atomic operation | S1 ; S2 composition | S1 S2 choice
| S1 iteration
| call p procedure call
P ParProg ::= S1 || . . . || Sn
Modular verification of multithreaded programsModular verification of multithreaded programs
one procedure active for each thread interleaving of atomic actions of active
procedures atomic action is unit of induction pre/post-conditions not enough need abstraction of each atomic action of
a procedure specification is an abstract program
top related