Solving Quantified Verification Conditions using Satisfiability Modulo Theories Yeting Ge, Clark Barrett, Cesare Tinelli Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.1/28
Solving Quantified Verification Conditionsusing Satisfiability Modulo Theories
Yeting Ge, Clark Barrett, Cesare Tinelli
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.1/28
Motivation
First order logic provides a convenient formalism forspecifying verification conditions
Verification conditions often involve arithmetic and otherwell-established theories
Approaches to checking verification conditions in first orderlogic
Pure first order Automated Theorem Proving (ATP)Good at reasoning about quantified formulasNot so good at theory reasoning· Some useful theories are not finitely axiomatizable
Add ’ad-hoc’ axioms (Denney et al. IJCAR 2004)
Are there any alternatives?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.2/28
Motivation
First order logic provides a convenient formalism forspecifying verification conditions
Verification conditions often involve arithmetic and otherwell-established theories
Approaches to checking verification conditions in first orderlogic
Pure first order Automated Theorem Proving (ATP)
Good at reasoning about quantified formulasNot so good at theory reasoning· Some useful theories are not finitely axiomatizable
Add ’ad-hoc’ axioms (Denney et al. IJCAR 2004)
Are there any alternatives?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.2/28
Motivation
First order logic provides a convenient formalism forspecifying verification conditions
Verification conditions often involve arithmetic and otherwell-established theories
Approaches to checking verification conditions in first orderlogic
Pure first order Automated Theorem Proving (ATP)Good at reasoning about quantified formulasNot so good at theory reasoning
· Some useful theories are not finitely axiomatizableAdd ’ad-hoc’ axioms (Denney et al. IJCAR 2004)
Are there any alternatives?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.2/28
Motivation
First order logic provides a convenient formalism forspecifying verification conditions
Verification conditions often involve arithmetic and otherwell-established theories
Approaches to checking verification conditions in first orderlogic
Pure first order Automated Theorem Proving (ATP)Good at reasoning about quantified formulasNot so good at theory reasoning· Some useful theories are not finitely axiomatizable
Add ’ad-hoc’ axioms (Denney et al. IJCAR 2004)
Are there any alternatives?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.2/28
Motivation
First order logic provides a convenient formalism forspecifying verification conditions
Verification conditions often involve arithmetic and otherwell-established theories
Approaches to checking verification conditions in first orderlogic
Pure first order Automated Theorem Proving (ATP)Good at reasoning about quantified formulasNot so good at theory reasoning· Some useful theories are not finitely axiomatizable
Add ’ad-hoc’ axioms (Denney et al. IJCAR 2004)
Are there any alternatives?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.2/28
Motivation
First order logic provides a convenient formalism forspecifying verification conditions
Verification conditions often involve arithmetic and otherwell-established theories
Approaches to checking verification conditions in first orderlogic
Pure first order Automated Theorem Proving (ATP)Good at reasoning about quantified formulasNot so good at theory reasoning· Some useful theories are not finitely axiomatizable
Add ’ad-hoc’ axioms (Denney et al. IJCAR 2004)
Are there any alternatives?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.2/28
Motivation
Approaches to checking verification conditions in first orderlogic
Automated theorem proving based on Satisfiability Modulo Theories(SMT)
A SMT problem is to determine the satisfiability of some formulaϕ with respect to some fixed background theory T
· Is Select(Store(arr, i, a), i) 6= a satisfiable?Many useful background (combined) theories T can be decided byefficient procedures
Good at theory reasoningTraditionally for quantifier-free formulas onlyException: Simplify
Instantiation based and incompleteShown to work for practical problemsSuccessful, but no longer supported
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.3/28
Motivation
Approaches to checking verification conditions in first orderlogic
Automated theorem proving based on Satisfiability Modulo Theories(SMT)
A SMT problem is to determine the satisfiability of some formulaϕ with respect to some fixed background theory T· Is Select(Store(arr, i, a), i) 6= a satisfiable?
Many useful background (combined) theories T can be decided byefficient procedures
Good at theory reasoningTraditionally for quantifier-free formulas onlyException: Simplify
Instantiation based and incompleteShown to work for practical problemsSuccessful, but no longer supported
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.3/28
Motivation
Approaches to checking verification conditions in first orderlogic
Automated theorem proving based on Satisfiability Modulo Theories(SMT)
A SMT problem is to determine the satisfiability of some formulaϕ with respect to some fixed background theory T· Is Select(Store(arr, i, a), i) 6= a satisfiable?
Many useful background (combined) theories T can be decided byefficient procedures
Good at theory reasoningTraditionally for quantifier-free formulas onlyException: Simplify
Instantiation based and incompleteShown to work for practical problemsSuccessful, but no longer supported
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.3/28
Motivation
Approaches to checking verification conditions in first orderlogic
Automated theorem proving based on Satisfiability Modulo Theories(SMT)
A SMT problem is to determine the satisfiability of some formulaϕ with respect to some fixed background theory T· Is Select(Store(arr, i, a), i) 6= a satisfiable?
Many useful background (combined) theories T can be decided byefficient procedures
Good at theory reasoningTraditionally for quantifier-free formulas only
Exception: SimplifyInstantiation based and incompleteShown to work for practical problemsSuccessful, but no longer supported
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.3/28
Motivation
Approaches to checking verification conditions in first orderlogic
Automated theorem proving based on Satisfiability Modulo Theories(SMT)
A SMT problem is to determine the satisfiability of some formulaϕ with respect to some fixed background theory T· Is Select(Store(arr, i, a), i) 6= a satisfiable?
Many useful background (combined) theories T can be decided byefficient procedures
Good at theory reasoningTraditionally for quantifier-free formulas onlyException: Simplify
Instantiation based and incompleteShown to work for practical problemsSuccessful, but no longer supported
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.3/28
Outline
Quantifier reasoning in SMTSMT solvers and Abstract DPLL Modulo Theories frameworkTriggers, matching and instantiation
ChallengesTrigger selectionInstantiation loopsEager and lazy instantiationIrrelevant axioms
Experimental resultsComparison of different heuristics (in CVC3)Comparison of SMT solversComparison of ATP and SMT solvers
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.4/28
Outline
Quantifier reasoning in SMTSMT solvers and Abstract DPLL Modulo Theories frameworkTriggers, matching and instantiation
ChallengesTrigger selectionInstantiation loopsEager and lazy instantiationIrrelevant axioms
Experimental resultsComparison of different heuristics (in CVC3)Comparison of SMT solversComparison of ATP and SMT solvers
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.4/28
Outline
Quantifier reasoning in SMTSMT solvers and Abstract DPLL Modulo Theories frameworkTriggers, matching and instantiation
ChallengesTrigger selectionInstantiation loopsEager and lazy instantiationIrrelevant axioms
Experimental resultsComparison of different heuristics (in CVC3)Comparison of SMT solversComparison of ATP and SMT solvers
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.4/28
Solver for SMT
Modern SMT solvers (lazy) integrate a SAT solver and one ormore theory solvers
UF
Abstraction
SATTheorySolver
Arithmetic
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.5/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b))
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b)) b1 : f(a) = f(b)b2 : a = b
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b))
b2 ∧ ¬b1
b1 : f(a) = f(b)b2 : a = b
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b))
{b1 = F b2 = T}b2 ∧ ¬b1
b1 : f(a) = f(b)b2 : a = b
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b))
{b1 = F b2 = T}b2 ∧ ¬b1
a = b
b1 : f(a) = f(b)b2 : a = b
f(a) 6= f(b)
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b))
{b1 = F b2 = T}b2 ∧ ¬b1
a = bT-unsat
b1 : f(a) = f(b)b2 : a = b
f(a) 6= f(b)
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
(a = b) ∧ ¬(f(a) = f(b))
{b1 = F b2 = T}b2 ∧ ¬b1
No more a = bT-unsat
b1 : f(a) = f(b)b2 : a = b
f(a) 6= f(b)
Abstraction
TheorySolver
SAT
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
SMT Example
To prove (a = b) ∧ ¬(f(a) = f(b)) is unsatisfiable
Solver
T-unsat
(a = b) ∧ ¬(f(a) = f(b))
{b1 = F b2 = T}b2 ∧ ¬b1
a = b
b1 : f(a) = f(b)b2 : a = b
f(a) 6= f(b)
Abstraction
TheorySolver
SAT
Unsat
No more
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.6/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
¬P (3) ∧ ∀x.(x > 1 → P (x))
TheorySolver
SAT
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
∀x.(x > 1 → P (x))
TheorySolver
SAT
b2 ∧ ¬b1
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)¬P (3) ∧ ∀x.(x > 1 → P (x))
{b1 = F b2 = T} ¬P (3)
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
Instantiate x with 3
TheorySolver
SAT
b2 ∧ ¬b1
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)¬P (3) ∧ ∀x.(x > 1 → P (x))
{b1 = F b2 = T} ¬P (3)∀x.(x > 1 → P (x))
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
3 > 1 → P (3)
TheorySolver
SAT
b2 ∧ ¬b1
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)¬P (3) ∧ ∀x.(x > 1 → P (x))
{b1 = F b2 = T} ¬P (3)∀x.(x > 1 → P (x))Instantiate x with 3
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
3 > 1 → P (3)
TheorySolver
SAT
b2 ∧ ¬b1
T-unsat
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)¬P (3) ∧ ∀x.(x > 1 → P (x))
{b1 = F b2 = T} ¬P (3)∀x.(x > 1 → P (x))Instantiate x with 3
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
3 > 1 → P (3)
TheorySolver
SAT
b2 ∧ ¬b1
No moreT-unsat
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)¬P (3) ∧ ∀x.(x > 1 → P (x))
{b1 = F b2 = T} ¬P (3)∀x.(x > 1 → P (x))Instantiate x with 3
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Quantifier Example
To prove ¬P (3) ∧ ∀x.(x > 1→ P (x)) is unsatisfiable.
Solver
3 > 1 → P (3)
TheorySolver
SAT
b2 ∧ ¬b1
No moreT-unsat
Abstraction
b2 : ∀x.(x > 1 → P (x))b1 : P (3)¬P (3) ∧ ∀x.(x > 1 → P (x))
Unsat{b1 = F b2 = T} ¬P (3)
∀x.(x > 1 → P (x))Instantiate x with 3
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.7/28
Abstract DPLL Modulo Theories
Abstract DPLL Modulo Theories is a formalism for DPLL-based smtsolvers
Describes SMT solvers as transition systems (a set of states and transitionrules)
States:
Fail
M || F (M is a set of literals assumed so far, F is a set of CNFclauses)
Final state:Fail
M || F (M is T satisfiable and M |= F )
The goal:From initial state ∅ || F0, derive a final state
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.8/28
Abstract DPLL Modulo Theories
Abstract DPLL Modulo Theories is a formalism for DPLL-based smtsolvers
Describes SMT solvers as transition systems (a set of states and transitionrules)
States:
Fail
M || F (M is a set of literals assumed so far, F is a set of CNFclauses)
Final state:Fail
M || F (M is T satisfiable and M |= F )
The goal:From initial state ∅ || F0, derive a final state
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.8/28
Abstract DPLL Modulo Theories
Abstract DPLL Modulo Theories is a formalism for DPLL-based smtsolvers
Describes SMT solvers as transition systems (a set of states and transitionrules)
States:
Fail
M || F (M is a set of literals assumed so far, F is a set of CNFclauses)
Final state:Fail
M || F (M is T satisfiable and M |= F )
The goal:From initial state ∅ || F0, derive a final state
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.8/28
Abstract DPLL Modulo Theories
Abstract DPLL Modulo Theories is a formalism for DPLL-based smtsolvers
Describes SMT solvers as transition systems (a set of states and transitionrules)
States:
Fail
M || F (M is a set of literals assumed so far, F is a set of CNFclauses)
Final state:Fail
M || F (M is T satisfiable and M |= F )
The goal:From initial state ∅ || F0, derive a final state
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.8/28
Example of Transition Rules
Unit propagation in SATUnitPropagate :
M || F, C ∨ l =⇒ M l || F, C ∨ l if
M |= ¬Cl is undefined in M
Theory propagationT-Propagate :
M || F =⇒ M l || F if
M |=T l
l or ¬l occurs in Fl is undefined in M
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.9/28
Rules for Quantifier Instantiation
An abstract literal is either a literal or a quantified formulaϕ[x/t] denotes the result of substituting t for all freeoccurrences of x in ϕ
Inst_∃ :
M || F =⇒ M || F, (¬(∃x. P ) ∨ P [x/sk]) if
8
<
:
∃ x. P is in M
sk is a fresh constant.
Inst_∀ :
M || F =⇒ M || F, (¬(∀x. P ) ∨ P [x/t]) if
8
<
:
∀ x. P is in M
t is a ground term.
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.10/28
What to instantiate
Suppose φ = ∀x.P (f(x)) is asserted to be true
Instantiate x with every ground term (naive instantiation)Too many instantiations
Instantiate x with terms relevant to φIf some subterm of φ[x/t] appears in ground formulas inM , t is relevant to φSimilar to resolution
How to find relevant terms?1. Select a subterm of φ that contains x, say f(x)
2. If f(x) matches with a ground term that appears inM , say f(a), a is relevantf(x) is called a triggerE − unification
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.11/28
What to instantiate
Suppose φ = ∀x.P (f(x)) is asserted to be trueInstantiate x with every ground term (naive instantiation)
Too many instantiations
Instantiate x with terms relevant to φIf some subterm of φ[x/t] appears in ground formulas inM , t is relevant to φSimilar to resolution
How to find relevant terms?1. Select a subterm of φ that contains x, say f(x)
2. If f(x) matches with a ground term that appears inM , say f(a), a is relevantf(x) is called a triggerE − unification
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.11/28
What to instantiate
Suppose φ = ∀x.P (f(x)) is asserted to be trueInstantiate x with every ground term (naive instantiation)
Too many instantiationsInstantiate x with terms relevant to φ
If some subterm of φ[x/t] appears in ground formulas inM , t is relevant to φSimilar to resolution
How to find relevant terms?1. Select a subterm of φ that contains x, say f(x)
2. If f(x) matches with a ground term that appears inM , say f(a), a is relevantf(x) is called a triggerE − unification
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.11/28
What to instantiate
Suppose φ = ∀x.P (f(x)) is asserted to be trueInstantiate x with every ground term (naive instantiation)
Too many instantiationsInstantiate x with terms relevant to φ
If some subterm of φ[x/t] appears in ground formulas inM , t is relevant to φSimilar to resolution
How to find relevant terms?1. Select a subterm of φ that contains x, say f(x)
2. If f(x) matches with a ground term that appears inM , say f(a), a is relevantf(x) is called a triggerE − unification
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.11/28
Challenges
Given a set of quantified formulas and ground formulas1. Select some subterms of a quantified formula as triggers2. Match triggers with ground terms3. Instantiate quantified formulas
ChallengesTriggersMatching (equalities, fast matching algorithm)Instantiation
Instantiation loopsEager and lazy instantiationIrrelevant axioms
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.12/28
Challenges
Given a set of quantified formulas and ground formulas1. Select some subterms of a quantified formula as triggers2. Match triggers with ground terms3. Instantiate quantified formulas
ChallengesTriggersMatching (equalities, fast matching algorithm)Instantiation
Instantiation loopsEager and lazy instantiationIrrelevant axioms
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.12/28
Triggers
Trigger selectionTriggers should contain all bound variables
Triggers can have more bound variables than thosequantified by outermost quantifiers· ∀x : (P (x)→ ∀y : Q(x, y))· (Simplify does not allow this)
Sometimes no single subterm contains all boundvariables
Multi-triggers (as in Simplify)Special trigger heuristics
TransitivityAnti-symmetryArray index
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.13/28
Triggers
Trigger selectionTriggers should contain all bound variables
Triggers can have more bound variables than thosequantified by outermost quantifiers· ∀x : (P (x)→ ∀y : Q(x, y))· (Simplify does not allow this)
Sometimes no single subterm contains all boundvariables
Multi-triggers (as in Simplify)
Special trigger heuristicsTransitivityAnti-symmetryArray index
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.13/28
Triggers
Trigger selectionTriggers should contain all bound variables
Triggers can have more bound variables than thosequantified by outermost quantifiers· ∀x : (P (x)→ ∀y : Q(x, y))· (Simplify does not allow this)
Sometimes no single subterm contains all boundvariables
Multi-triggers (as in Simplify)Special trigger heuristics
TransitivityAnti-symmetryArray index
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.13/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)
∃y.f(3)− f(y) = 2f(3)− f(sk1) = 2f(x) matches f(sk1)f(sk2)......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)∃y.f(3)− f(y) = 2
f(3)− f(sk1) = 2f(x) matches f(sk1)f(sk2)......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)∃y.f(3)− f(y) = 2f(3)− f(sk1) = 2
f(x) matches f(sk1)f(sk2)......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)∃y.f(3)− f(y) = 2f(3)− f(sk1) = 2f(x) matches f(sk1)
f(sk2)......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)∃y.f(3)− f(y) = 2f(3)− f(sk1) = 2f(x) matches f(sk1)f(sk2)
......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)∃y.f(3)− f(y) = 2f(3)− f(sk1) = 2f(x) matches f(sk1)f(sk2)......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)f(x) is selected as trigger. Suppose f(3) appearssomewhere.f(x) matches f(3)∃y.f(3)− f(y) = 2f(3)− f(sk1) = 2f(x) matches f(sk1)f(sk2)......
3. Loops due to several formulas
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.14/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)
3. Loops due to several formulas
Loops are not always badWe experimented two kinds of loop prevention mechanism
Static loop test (as in Simplify)Dynamic loop detectionBoth are abandoned
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.15/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)
3. Loops due to several formulasLoops are not always bad
We experimented two kinds of loop prevention mechanismStatic loop test (as in Simplify)Dynamic loop detectionBoth are abandoned
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.15/28
Instantiation Loops
Instantiation could introduce loops1. ∀x.P (f(x), f(g(x))) (Simplify)2. ∀x.(∃y.f(x)− f(y) = 2)
3. Loops due to several formulasLoops are not always badWe experimented two kinds of loop prevention mechanism
Static loop test (as in Simplify)Dynamic loop detectionBoth are abandoned
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.15/28
Eager and lazy instantiation
Eager instantiationInstantiate when unit propagation rule does not apply
May find contradictions earlierMay introduce useless clauses
Lazy instantiationInstantiate when no other transition rule appliesInstantiate only when necessaryMay be too late
Is there a way to have a balance between lazy and eagerinstantiation?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.16/28
Eager and lazy instantiation
Eager instantiationInstantiate when unit propagation rule does not applyMay find contradictions earlierMay introduce useless clauses
Lazy instantiationInstantiate when no other transition rule appliesInstantiate only when necessaryMay be too late
Is there a way to have a balance between lazy and eagerinstantiation?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.16/28
Eager and lazy instantiation
Eager instantiationInstantiate when unit propagation rule does not applyMay find contradictions earlierMay introduce useless clauses
Lazy instantiationInstantiate when no other transition rule applies
Instantiate only when necessaryMay be too late
Is there a way to have a balance between lazy and eagerinstantiation?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.16/28
Eager and lazy instantiation
Eager instantiationInstantiate when unit propagation rule does not applyMay find contradictions earlierMay introduce useless clauses
Lazy instantiationInstantiate when no other transition rule appliesInstantiate only when necessaryMay be too late
Is there a way to have a balance between lazy and eagerinstantiation?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.16/28
Eager and lazy instantiation
Eager instantiationInstantiate when unit propagation rule does not applyMay find contradictions earlierMay introduce useless clauses
Lazy instantiationInstantiate when no other transition rule appliesInstantiate only when necessaryMay be too late
Is there a way to have a balance between lazy and eagerinstantiation?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.16/28
Irrelevant axioms
Verification conditions are often of the form Γ ∧ ¬ϕ where ϕis a formula and Γ is a large fixed T -satisfiable collection of(quantified) axioms
Many formulas and ground terms in Γ are irrelevant to the proof ofunsatisfiability of Γ ∧ ¬ϕ
The solver may spend a lot of resources on irrelevant axiomsIt is not always easy to determine whether axioms are relevant or not
How to prevent the solver from spending too many resourceson irrelevant axioms?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.17/28
Irrelevant axioms
Verification conditions are often of the form Γ ∧ ¬ϕ where ϕis a formula and Γ is a large fixed T -satisfiable collection of(quantified) axioms
Many formulas and ground terms in Γ are irrelevant to the proof ofunsatisfiability of Γ ∧ ¬ϕThe solver may spend a lot of resources on irrelevant axioms
It is not always easy to determine whether axioms are relevant or not
How to prevent the solver from spending too many resourceson irrelevant axioms?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.17/28
Irrelevant axioms
Verification conditions are often of the form Γ ∧ ¬ϕ where ϕis a formula and Γ is a large fixed T -satisfiable collection of(quantified) axioms
Many formulas and ground terms in Γ are irrelevant to the proof ofunsatisfiability of Γ ∧ ¬ϕThe solver may spend a lot of resources on irrelevant axiomsIt is not always easy to determine whether axioms are relevant or not
How to prevent the solver from spending too many resourceson irrelevant axioms?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.17/28
Irrelevant axioms
Verification conditions are often of the form Γ ∧ ¬ϕ where ϕis a formula and Γ is a large fixed T -satisfiable collection of(quantified) axioms
Many formulas and ground terms in Γ are irrelevant to the proof ofunsatisfiability of Γ ∧ ¬ϕThe solver may spend a lot of resources on irrelevant axiomsIt is not always easy to determine whether axioms are relevant or not
How to prevent the solver from spending too many resourceson irrelevant axioms?
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.17/28
Instantiation level: three birds one stone
Definition of instantiation level IL(g) of ground term g
All terms appearing in original problem have an instantiation level of 0If ground term g matches some trigger of ∀x.P and g has aninstantiation level IL(g), then all new terms in P [x/t] (as well as newterms derived from them) have instantiation level of IL(g) + 1.
Instantiation level and matchingOnly ground terms with an instantiation level less than an upperbound are used in matching
Instantiation strategyEager instantiationThe upper bound is increased if
CVC3 runs out of ground termsNo other transition rule applies
Iterative deepening on instantiation level
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.18/28
Instantiation level: three birds one stone
Definition of instantiation level IL(g) of ground term g
All terms appearing in original problem have an instantiation level of 0If ground term g matches some trigger of ∀x.P and g has aninstantiation level IL(g), then all new terms in P [x/t] (as well as newterms derived from them) have instantiation level of IL(g) + 1.
Instantiation level and matchingOnly ground terms with an instantiation level less than an upperbound are used in matching
Instantiation strategyEager instantiationThe upper bound is increased if
CVC3 runs out of ground termsNo other transition rule applies
Iterative deepening on instantiation level
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.18/28
Instantiation level: three birds one stone
Definition of instantiation level IL(g) of ground term g
All terms appearing in original problem have an instantiation level of 0If ground term g matches some trigger of ∀x.P and g has aninstantiation level IL(g), then all new terms in P [x/t] (as well as newterms derived from them) have instantiation level of IL(g) + 1.
Instantiation level and matchingOnly ground terms with an instantiation level less than an upperbound are used in matching
Instantiation strategyEager instantiationThe upper bound is increased if
CVC3 runs out of ground termsNo other transition rule applies
Iterative deepening on instantiation levelSolving Quantified Verification Conditions using Satisfiability Modulo Theories – p.18/28
Advantages of instantiation level
Neutralizes the harmful effect of instantiation loopsNew ground terms from instantiation will not beconsidered until the upper bound is increased
Balances the eagerness of instantiationsEagerly instantiateLazily increase the upper bound
Avoids spending too many resources on irrelevant axiomsNo ground term will have more attention than others
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.19/28
Advantages of instantiation level
Neutralizes the harmful effect of instantiation loopsNew ground terms from instantiation will not beconsidered until the upper bound is increased
Balances the eagerness of instantiationsEagerly instantiateLazily increase the upper bound
Avoids spending too many resources on irrelevant axiomsNo ground term will have more attention than others
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.19/28
Advantages of instantiation level
Neutralizes the harmful effect of instantiation loopsNew ground terms from instantiation will not beconsidered until the upper bound is increased
Balances the eagerness of instantiationsEagerly instantiateLazily increase the upper bound
Avoids spending too many resources on irrelevant axiomsNo ground term will have more attention than others
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.19/28
Experimental Results
Test cases are from SMT LIB.AUFLIA/Burns 14AUFLIA/misc 29AUFLIA/piVC 42AUFLIA/RicartAgrawala 14AUFLIA/simplify 833AUFLIRA/nasa 26504AUFNIRA/nasa 1561
Only hard cases (5599 out of 29004) are selectedAMD Opteron (64 bit), 1G memory, time limit 5 minutes
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.20/28
CVC3 with Different Heuristics
Category B-E B-L IL-EAUFLIA/Burns 12 12 12 12AUFLIA/misc 14 12 14 14AUFLIA/piVC 29 29 29 29AUFLIA/RicAgla 14 14 14 14AUFLIA/simplify 769 497 762 768AUFLIRA/nasa 4619 4527 4131 4526AUFNIRA/nasa 142 72 46 72
B-E : No instantiation level heuristic, with eager instantiationB-L : No instantiation level heuristic, with lazy instantiationIL-E: Instantiation level with eager instantiation
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.21/28
CVC3 with Different Heuristics
Category B-E B-L IL-EAUFLIA/Burns 12 12 12 12AUFLIA/misc 14 12 14 14AUFLIA/piVC 29 29 29 29AUFLIA/RicAgla 14 14 14 14AUFLIA/simplify 769 497 762 768AUFLIRA/nasa 4619 4527 4131 4526AUFNIRA/nasa 142 72 46 72
B-E : No instantiation level heuristic, with eager instantiationB-L : No instantiation level heuristic, with lazy instantiationIL-E: Instantiation level with eager instantiation
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.21/28
CVC3 with Different Heuristics
Category B-E B-L IL-EAUFLIA/Burns 12 12 12 12AUFLIA/misc 14 12 14 14AUFLIA/piVC 29 29 29 29AUFLIA/RicAgla 14 14 14 14AUFLIA/simplify 769 497 762 768AUFLIRA/nasa 4619 4527 4131 4526AUFNIRA/nasa 142 72 46 72
B-E : No instantiation level heuristic, with eager instantiationB-L : No instantiation level heuristic, with lazy instantiationIL-E: Instantiation level with eager instantiation
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.21/28
CVC3 with Different Heuristics
Category B-E B-L IL-EAUFLIA/Burns 12 12 12 12AUFLIA/misc 14 12 14 14AUFLIA/piVC 29 29 29 29AUFLIA/RicAgla 14 14 14 14AUFLIA/simplify 769 497 762 768AUFLIRA/nasa 4619 4527 4131 4526AUFNIRA/nasa 142 72 46 72
B-E : No instantiation level heuristic, with eager instantiationB-L : No instantiation level heuristic, with lazy instantiationIL-E: Instantiation level with eager instantiation
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.21/28
CVC3, Yices and Fx7
Yices, version of SMT competition 2006Fx7, as of Nov 15, 2006CVC3, version 1.1
fx7 yices CVC3
category #case #valid time #valid time #valid time
AUFLIA/Burns 12 12 0.429 12 0.011 12 0.020
AUFLIA/misc 14 12 0.682 14 0.050 14 0.048
AUFLIA/piVC 29 15 0.517 29 0.030 29 0.106
AUFLIA/RicAgla 14 14 0.640 14 0.026 14 0.041
AUFLIA/simplify 769 760 3.218 740 1.424 768 0.739
AUFLIRA/nasa 4619 4187 0.452 4520 0.082 4526 0.014
AUFNIRA/nasa 142 48 0.410 N/A N/A 72 0.012
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.22/28
CVC3, Yices and Fx7
Yices, version of SMT competition 2006Fx7, as of Nov 15, 2006CVC3, version 1.1
fx7 yices CVC3
category #case #valid time #valid time #valid time
AUFLIA/Burns 12 12 0.429 12 0.011 12 0.020
AUFLIA/misc 14 12 0.682 14 0.050 14 0.048
AUFLIA/piVC 29 15 0.517 29 0.030 29 0.106
AUFLIA/RicAgla 14 14 0.640 14 0.026 14 0.041
AUFLIA/simplify 769 760 3.218 740 1.424 768 0.739
AUFLIRA/nasa 4619 4187 0.452 4520 0.082 4526 0.014
AUFNIRA/nasa 142 48 0.410 N/A N/A 72 0.012
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.22/28
SMT and ATP
NASA casesVerification conditions of some NASA softwareIntroduced by Denney et al. at IJCAR 2004Claim: Modern ATPs are powerful enough for “practicalapplication in program certification”
T∅ The first set generated, the hardestT∀,→
Tprop (e.g. true ∨ P ===> true)Teval (e.g. succ(pred(x)) ===> x)Tarray
Tpolicy (ad-hoc simplification, the easiest)Tarray∗ simplification of T∀,→
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.23/28
CVC3, Simplify, SPASS, Vampire
SimplifyThe only SMT solvers for quantifier reasoning that ispublicly available in 2004
Vampire, version 8.1One of the best ATPs, won two categories of CASCcompetition in recent years
SPASS, version 2.2The best ATP in the IJCAR 2004 paper
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.24/28
CVC3, Simplify, SPASS, Vampire
category #cases Vampire SPASS Simplify CVC3T∅ 365 266 302 207 343T∀,→ 6198 6080 6063 5957 6174Tprop 1468 1349 1343 1370 1444Teval 1076 959 948 979 1052Tarray 2026 2005 2000 1943 2005Tpolicy 1987 1979 1974 1917 1979Tarray∗ 14931 14903 14892 14699 14905total 28051 27541 27522 27072 27902
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.25/28
CVC3, Simplify, SPASS, Vampire
category Vampire SPASS Simplify CVC3T∅ 9.277 1.765 0.068 0.017T∀,→ 2.154 0.673 0.017 0.004Tprop 4.322 1.066 0.339 0.006Teval 5.603 0.760 0.042 0.008Tarray 1.444 0.270 0.011 0.005Tpolicy 1.494 0.272 0.010 0.004Tarray∗ 0.695 0.232 0.010 0.005total 1.560 0.411 0.015 0.004
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.26/28
Related works
SMT solvers[1] D. Detlefs, G. Nelson, and J. B. Saxe. Simplify: a theorem prover for program checking.J. ACM, 52(3):365–473, 2005.[2] B. Dutertre and L. Moura. Yices. yices.csl.sri.com/
[3] M. Moskal. Fx7. nemerle.org/ malekith/smt/en.html
SMT benchmarks[4] S. Ranise and C. Tinelli. The satisfiability modulo theories library (SMT-LIB).www.SMT-LIB.org, 2006.
[5] E. Denney, B. Fischer, and J. Schumann. Using automated theorem provers to certify
auto-generated aerospace software. In D. A. Basin and M. Rusinowitch, editors, IJCAR,
volume 3097 of Lecture Notes in Computer Science, pages 198–212. Springer, 2004.
DPLL(T)[6] H. Ganzinger, G. Hagen, R. Knowings, A. Oliveras, and C. Tinelli. DPLL(T): Fast
decision procedures. In R. Alur and D. Peled, editors, Proceedings of the 16th International
Conference on Computer Aided Verification, CAV’04 (Boston, Massachusetts), volume 3114
of LUCKS, pages 175–188. Springer, 2004.Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.27/28
Summary
Instantiation level heuristic meets several challenges inquantifier reasoningFor certain kinds of verification conditions, SMT solversmay be a better choiceFuture work
Efficient multi-trigger matching with equalitiesTechniques from ATP
Solving Quantified Verification Conditions using Satisfiability Modulo Theories – p.28/28