CS-E3220 Declarative Progamming Model-Checking and Validation Jussi Rintanen Department of Computer Science Aalto University November 6, 2019
CS-E3220 Declarative Progamming
Model-Checking and Validation
Jussi Rintanen
Department of Computer ScienceAalto University
November 6, 2019
This Lecture
Model-checking in Temporal logics
SAT-Based Bounded Model-Checking (BMC)
Abstraction
Counterexample-guided abstraction refinement (CEGAR)
Model-CheckingTest if a CTL formula is true in a transition system
AFfinalstateAGEFprogress
Test if an LTL formula is true in a transition system
GF(green1)G(¬(green1 ∧ red1) ∧ ¬(green1 ∧ yellow1) ∧ ¬(yellow1 ∧ red1))G(green1 → F (red1))G(red1 → (red1Uyellow1))G¬(green1 ∧ green2)
Model-Checking
Model-checking: test M |= φ for given model M and formula φApplications:
Computer-aided verification (LTL, CTL, CTL∗)Advanced databases: query evaluation (description logics)
Easier than testing logical consequence, but can be hard
logic logical consequence model-checkingpropositional logic co-NP-complete P-completePDL EXP-complete P-completeCTL EXP-complete P-completeCTL∗ 2-EXP-complete PSPACE-completeLTL PSPACE-complete PSPACE-complete
Why Model-Checking?
Deductive verification: system models as formulas, theorem-proving
Poor scalability of theorem-provers −→ model-checking preferable
Practical handling of transition system models with hundreds ofthousands of statesSince 1990ies, model-checking without explicitly represented system−→ symbolic model-checking with OBDD & SAT:
OBDD-based model-checking for CTLSAT-based model-checking for LTL
Model-Checking for Propositional Logic
fun MC(And(p,q),M) = boolAnd(MC (p,M),MC (q,M))
| MC(Or(p,q),M) = boolOr(MC (p,M),MC (q,M))
| MC(Neg p,M) = boolNot (p,M)
| MC(Atom a,M) = M(a)
fun boolAnd(true,true) = true
| boolAnd _ = false
fun boolOr(false,false) = false
| boolOr _ = true
fun boolNot false = true
| boolNot true = false
M is the model: mapping from propositional variables to { true, false }
This algorithm runs in linear time if M is constant time (e.g. arrayindexed by propositional variables)
Model-Checking a Modal Logic with One Modality
1 Label each world w with L(w) = {x ∈ X |M |=w x}2 Consider all subformulas φ′ of φ in the order of increasing length.
For every world w , do L(w) := L(w) ∪ {φ′} ifφ′ = ψ1 ∧ ψ2 and ψ1 ∈ L(w) and ψ2 ∈ L(w), orφ′ = ψ1 ∨ ψ2 and ψ1 ∈ L(w) or ψ2 ∈ L(w), orφ′ = ¬ψ1 and ψ1 6∈ L(w), orφ′ = �ψ and ψ ∈ L(w ′) for all w ′ such that wRw ′, orφ′ = ♦ψ and ψ ∈ L(w ′) for some w ′ such that wRw ′.
The runtime of this procedure is O(|φ| × |W |)
Model-Checking for Temporal Logics
LTL, CTL, CTL∗ talk about infinite paths, even when the transitionsystem is finiteP-time model-checking for PDL and CTL:
Label the computation graph with all subformulas φ0 of φTruth in a node a function of truth of subformulas in the node and itssuccessors
PSPACE-hard model-checking for LTL and CTL∗:Need to consider exponentially many exponentially long paths
CTL Model-Checking
For every w ∈ W , L(w) := {x ∈ X |M |=w x}Consider all subformulas φ′ of φ in the order of increasing length.For every world w , do L(w) := L(w) ∪ {φ′} if
φ′ = ψ1 ∧ ψ2 and ψ1 ∈ L(w) and ψ2 ∈ L(w), orφ′ = ψ1 ∨ ψ2 and ψ1 ∈ L(w) or ψ2 ∈ L(w), orφ′ = ¬ψ1 and ψ1 6∈ L(w).
Formulas E (φUψ) and EGφ handled in the following slides.All other operators can be reduced to the above:
EFφ ≡ E (>Uφ)AFϕ ≡ ¬EG¬ϕAGϕ ≡ ¬E (>U¬ϕ)AXϕ ≡ ¬EX¬ϕ
A(ϕUψ) ≡ ¬E (¬ψU(¬ϕ ∧ ¬ψ)) ∧ ¬EG¬ψ
CTL Model-Checking: Labeling for E (φUψ)
For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if
ψ ∈ L(w) or
φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′
until no more updates possible.
Example: E (aUb)
w1 w2
w3
w4
w5
b
a
aa
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for E (φUψ)
For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if
ψ ∈ L(w) or
φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′
until no more updates possible.
Example: E (aUb)
w1 w2
w3
w4
w5
bE (aUb)
a
aa
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for E (φUψ)
For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if
ψ ∈ L(w) or
φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′
until no more updates possible.
Example: E (aUb)
w1 w2
w3
w4
w5
bE (aUb)
aE (aUb)
aa
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for E (φUψ)
For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if
ψ ∈ L(w) or
φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′
until no more updates possible.
Example: E (aUb)
w1 w2
w3
w4
w5
bE (aUb)
aE (aUb)
aE (aUb)
aE (aUb)
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for E (φUψ)
Procedure EU(α,β)T := {w ∈ W |β ∈ L(w)};For Each w ∈ T Do L(w) := L(w) ∪ {E (αUβ)};While T 6= ∅ Do
take any w ∈ T ;T := T\{w};For Each t such that tRw DoIf α ∈ L(t) and E (αUβ) 6∈ L(t) Then
L(t) := L(t) ∪ {E (αUβ)};T := T ∪ {t};
End IfEnd For
End While
CTL Model-Checking: Labeling for EGφ
Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G
For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C
For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}
w1 w2
w3
w4
w5
w6
w7
w8 w9
w10 w11
a a
a
a
a
a
a
a a
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for EGφ
Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G
For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C
For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}
w1 w2
w3
w4
w5
w6
w7
w8 w9
w10 w11
a a
a
a
a
a
a
a a
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for EGφ
Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G
For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C
For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}
w1 w2
w3
w4
w5
w6
w7
w8 w9
w10 w11
a a
a
a
a
a
a
a a
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for EGφ
Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G
For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C
For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}
w1 w2
w3
w4
w5
w6
w7
w8 w9
w10 w11
a a
a
a
a
a
a
a a
EGa
EGa
EGa
EGa
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for EGφ
Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G
For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C
For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}
w1 w2
w3
w4
w5
w6
w7
w8 w9
w10 w11
a a
a
a
a
a
a
a a
EGa
EGa
EGa
EGa
EGa
EGa
Systematic algorithm given on the next slide...
CTL Model-Checking: Labeling for EGφProcedure EG(α)S ′ := {w ∈ W |α ∈ L(w)};SCC := {C |C is a SCC of 〈S ′,R ∩ (S ′ × S ′)〉, |C | ≥ 1 or there is w ∈ C with wRw};T := {w ∈ C |C ∈ SCC};For Each w ∈ T Do L(w) := L(w) ∪ {EGα};While T 6= ∅ Do
take any w ∈ T ;T := T\{w};For Each t ∈ S ′ such that tRw DoIf EGα 6∈ L(t) Then
L(t) := L(t) ∪ {EGα};T := T ∪ {t};
End IfEnd For
End While
Bounded Model-Checking for LTL
SAT-based LTL model-checking solves two exponential problems atthe same time:
Find a path in an exponential-size transition graphTest that the LTL formula is satisfied on that path
Large size of transition systems masks the PSPACE-hardness of LTLmodel-checking
LTL model-checking popular since 1999, due to very goodscalability of SAT solvers
Use of CTL model-checking decreasing, due to transition systemsbeing very large
Bounded Model-Checking for LTL
s0 s1 s2 s3 s4 s5
Model-checking limited by size of computation graphs (1M, 1B)
Symbolic methods (OBDD, SAT) can relax this limitation
Biere et al. (1999) reduced LTL model-checking to SAT
Consider paths of the form s0, s1, . . . ,︷ ︸︸ ︷sl , . . . , sk ,
︷ ︸︸ ︷sl , . . . , sk , sl , . . .
The segment sl , . . . , sk repeats an infinite number of times
Encode this path with a loop as a propositional formula
Bounded Model-Checking for LTL
s0 s1 sl−1 sl sl+1 sk−1 sk
[[x ]]l ,ki = x@i [[Xφ]]l ,ki = [[φ]]l ,ksucc(i) Here:succ(i) = i + 1 if i < ksucc(k) = l
[[¬x ]]l ,ki =¬x@i [[Gφ]]l ,ki =∧k
j=min(l ,i)[[φ]]l ,kj[[φ1 ∨ φ2]]l ,ki = [[φ1]]l ,ki ∨ [[φ2]]l ,ki [[Fφ]]l ,ki =
∨kj=min(l ,i)[[φ]]l ,kj
[[φ1 ∧ φ2]]l ,ki = [[φ1]]l ,ki ∧ [[φ2]]l ,ki [[φUψ]]l ,ki =∨ ∨k
j=i([[ψ]]l ,kj ∧∧j−1
z=i [[φ]]l ,kz )∨i−1j=l ([[ψ]]l ,kj ∧
∧j−1z=l [[φ]]l ,kz ∧
∧kz=i [[φ]]l ,kz )
[[φRψ]]l ,ki = similar
φ is true on some path (representable as a (k , l)-loop) iff[[φ]]l ,k0 ∧ I ∧ T [X @1/X ′] ∧ · · · ∧ T [X @(k − 1)/X ,X @k/X ′] ∧ T [X @k/X ,X @l/X ′] issatisfiable
Bounded Model-Checking for LTL
s0 s1 s2 s3 s4
[[XGa]]2,40 = a@1 ∧ a@2 ∧ a@3 ∧ a@4
[[aUb]]2,41 = b@1∨(b@2 ∧ a@1)∨(b@3 ∧ a@1 ∧ a@2)∨(b@4 ∧ a@1 ∧ a@2 ∧ a@3)
Bounded Model-Checking for LTL
2-D search: Test formulas for (k , l)-loops with increasing k , l
1-D search: Encode choice of l in formulas for fixed k , increase kStrengths:
Very effective in finding faulty behaviors (for “low” values of k)Applicable to far bigger systems than OBDDs
Weaknesses:No general method for deciding when to stop increasing kHence: Often not a practical method for proving correctness
Abstraction in Verification
Correctness of a system can sometimes be determined from anabstraction of a system
Ignore some aspects of the system modelAbstracted system has more possible behaviors than the original systemIf all executions of abstracted systems have given property, then so do thoseof the original system
Reasoning about the abstract system can be far easierFewer state variables −→ number of abstract states smaller
Applications:Software verification (conventional program code)sequential circuits (CPUs)others
How to Abstract a System Model
Abstracting a state = distinctions between states eliminatedAbstracting state variables = distinctions between values eliminated
Examples of abstracting state variablesintegers x , y by fx ,y : Z× Z→ {0, 1} such that
fx ,y(x , y) =
{1 if x > y0 otherwise
Boolean b by fb(v) = 0 (constant function: variable eliminated)
weekday d by fd(Sat) = 0, fd(Sun) = 0, fd(d) = 1 otherwise
How to Abstract a System ModelAbstracting states: s mapped to s ′ = f (s) by mapping values of eachvariable or subset of variables to its abstraction:
state variables X partitioned to X1, . . . ,Xn
abstraction functions are f1, . . . , fnabstract state variables are y1, . . . , ynvalue s ′(yi) in the abstracted state is obtained by fi :
s ′(yi) = fi(x i1, . . . , x
iji)
where Xi = {x i1, . . . , x
iji}
Equivalence relation ≡f induced by f : s1 ≡f s2 iff f (s1) = f (s2)
A Simple Special Class of Abstractions
We next only consider abstracting Boolean variables one by oneBoolean state variables x are abstracted either
by fx(v) = 0 (eliminating the variable), orfx(v) = v (retaining the variable)
A subset of state variables is eliminated, others remain intact
How to Abstract a System Model
Simple abstraction: eliminate some state variable(s)
Eliminate x from formula φ: generate ∃x .φ↔ φ[>/x ] ∨ φ[⊥/x ]Eliminate x from effect e:
Replace all assignments x := B by εReplace eITE(φ, e1, e2) by eITE(∃x .φ, e ′1, e
′2) where e ′i is ei with x eliminated
This allows eliminating x from any transition rule (φ, e)
Example
Load object in truck at A:(TatA∧OatA, (OatA := 0; OinT := 1))
Unload object from truck at B:(TatB∧OinT, (OinT := 0; OatB := 1))
Move truck from A to B:(TatA, (TatA := 0;TatB := 1))
Possible states AA, BA, AT, BT, BB AA
ATBT
BB
BA
load
A to B
unlo
ad
A to B
Example, AbstractedAbstract away both TatA and TatB:
Load object in truck at A:(OatA, (OatA := 0; OinT := 1))
Unload object from truck at B:(OinT, (OinT := 0; OatB := 1))
Move truck from A to B: (>, ε)
Possible states:
abstract concrete-B BB-T AT, BT-A AA, BA
-A
-T
-B
load
unlo
ad
Arrows in the abstraction are a superset of thearrows in the original system
Properties of Abstractions
Let T be a transition system and T ′ its abstraction (w.r.t vars Z ).Let φ be a formula and φ′ = ∃Z .φ.
Theorems:
If the path in T has length n, then the path in T ′ has length ≤ n
If there is path to φ in T , then there is a path to φ′ in T ′
Corollary:
If there is no path to φ′ in T ′, then there is no path to φ in T
Properties of Abstractions: Spurious Paths
If there is path p to φ′ in T ′, then there might be no path to φ in T .Here p is called a spurious path.Core question in model-checking with abstractions: What to dowith spurious paths?
We want to test if LTL formula φ can be true on a pathThere is such a path in the abstracted systemDoes this path have a counterpart in the concrete system?
Reachability Checking with Abstraction
Test if any state satisfying φ is reachable:
1 Build initial abstraction of transition system (not abstracting φ)2 Test if φ is reachable in the abstracted system.3 If φ not reachable in abstraction, also not reachable in unabstracted
system. Stop.4 Let t1, . . . , tn be the transition sequence reaching φ5 If t1, . . . , tn is a transition sequence also in the original system, stop.6 Generate a new (less abstract) abstraction.7 Go to step 2.
Counterexample-Guided Abstraction Refinement
A spurious path has been found
How is the abstraction refined?
1 Given t1, . . . , tn, identify the maximal prefix t1, . . . , ti that isexecutable in the original transition system
2 ti+1 is not possible in the original system, only in the abstraction3 Consider the corresponding state sequences s ′0, . . . , s
′i and s0, . . . , si
4 Transition ti+1 is possible in s ′i , but not in si5 This difference suggests how the abstraction must be changed!
Counterexample-Guided Abstraction RefinementThe (spurious) plan to transport object from A to B:
1 load object in vehicle in location A2 unload object from vehicle in location B
The state sequences:
action state original system abstract systemTatA TatB OatA OinT OatB TatA TatB OatA OinT OatB
load s0 1 0 1 0 0 - - 1 0 0unload s1 1 0 0 1 0 - - 0 1 0
s2 NA NA NA NA NA - - 0 0 1
In the original system, unload’s precondition TatB∧ OinT is false!Refine the abstraction: include the state variable TatB.
Example, Abstraction Refined
Load object in truck at A:(OatA, (OatA := 0; OinT := 1))
Unload object from truck at B:(TatB∧OinT, (OinT := 0; OatB := 1))
Move truck from A to B:(>, (TatB := 1))
Possible states -A, BA, -T, BT, BB -A
-TBT
BB
BA
load
A to B
unlo
ad
A to B
Counterexample-Guided Abstraction Refinement
CEGAR applicable to all leading methodsExplicit state reachability analysis (e.g. breadth-first search)Explicit state model-checkingOBDD-based reachability & model-checkingSAT-based reachability & model-checking
Critical in SW model-checking, where abstraction is unavoidable(due to the typically infinite state space)