Search Strategies M. Rueher Basics The CP Framework CPBPV DPVS Papers Constraint-Based Search Strategies For Bounded Program Verification Michel RUEHER University of Nice Sophia-Antipolis / I3S – CNRS, France (joined work with Hélène COLLAVIZZA, Nguyen Le VINH and Pascal Van HENTENRYCK) January 25, 2011 NII – Tokyo This work was partially supported by the ANR-07-SESUR-003 project CAVERN and the ANR-07 TLOG 022 project TESTEC 1
76
Embed
Constraint-Based Search Strategies For Bounded Program ...rueher/Publis/nii_11.pdf · CP: Overview CP Solving Global & Search Basics on BMC BMC: overview Algorithm The CP Framework
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVS
Papers
Constraint-Based Search StrategiesFor Bounded Program Verification
Michel RUEHER
University of Nice Sophia-Antipolis / I3S – CNRS, France
(joined work with Hélène COLLAVIZZA, Nguyen Le VINHand Pascal Van HENTENRYCK)
January 25, 2011
N I I – Tokyo
This work was partially supported by the ANR-07-SESUR-003 project CAVERN and the ANR-07TLOG 022 project TESTEC
1
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVS
Papers
Outline
Basics on Constraint Programming (CP) and on BoundedModel Checking (BMC)
A CP framework for Bounded Program Verification
CPBPV, a Depth First Dynamic Exploration of the CFG
DPVS, a Non Sequential Exploration Strategy of the CFG
Papers
2
SearchStrategies
M. Rueher
BasicsBasics on CP
CP: Overview
CP Solving
Global & Search
Basics on BMC
BMC: overview
Algorithm
The CPFramework
CPBPV
DPVS
Papers
Basics
I Basics on Constraint Programming
3
SearchStrategies
M. Rueher
BasicsBasics on CP
CP: Overview
CP Solving
Global & Search
Basics on BMC
BMC: overview
Algorithm
The CPFramework
CPBPV
DPVS
Papers
Constraint Programming: Overall view
Constraint Programming is a way of modeling andsolving combinatorial optimization problems
I CP combines techniques from artificial intelligence,logic programming, and operations research
I There exist several industrial solvers (e.g., ILOG/IBM,Eclipse, Xpress-Kalis, Comet), and academic solvers(e.g., Gecode, Choco, Minion)
I Many industrial applications, e.g., timetabling (Dutchrailway), hardware verification (Intel), scheduling,planning, ...
4
SearchStrategies
M. Rueher
BasicsBasics on CP
CP: Overview
CP Solving
Global & Search
Basics on BMC
BMC: overview
Algorithm
The CPFramework
CPBPV
DPVS
Papers
CP: key features
I Domain filtering→ Consider each constraint separately and
remove values that are triviallyinconsistent
I Searching strategies→ Try to exploit the structure of the problem
... and Global Constraints→ Use (efficient) specific algorithms for some
1 static int binary_search(int[] t, int v)2 int l = 0;3 int u = t.length-1;4 while (l <= u)5 int m = (l + u) / 2;6 if (t[m]==v) return m;7 if (t[m] > v)8 u = m - 1;9 else10 l = m + 1; // ERROR else u = m - 1;11 return -1;
I solve the CSPThere is No solution so the program is correct alongthis execution path
Go back to conditional if (t[m]==v) to explorethe else part
46
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Implementation
I Dedicated solvers• ad-hoc simplifier : trivial simplifications and calculus
on constants• linear solver (LP algorithm) + MIP solver• Boolean solver (SAT solver)
(Boolean relaxation of the non linear constraints)• CSP solver : used if none of the other solver did find
an inconsistency
I Prototype• Solvers : Ilog CPLEX11 and JSolver4verif• Written in Java using JDT (eclipse) for parsing Java
programs
!! CPLEX is unsafe but Neumaier & Shcherbina→ method for computing a certificate of infeasibility
47
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Current prototype – On the fly validation : ifc then ... else ...
I If c can be simplified into constant value “true” or“false”, select the branch which corresponds to c
I If c is linear1. add decision c in linear_CSP2. solve linear_CSP
I if linear_CSP has no solution, condition c is notfeasible for the current path choose another path
I if linear_CSP has a solution, we can’t concludeanything on complete_CSP investigate both branches c and ¬c
48
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Current prototype – On the fly validation : ifc then ... else ...
I If c is NOT linear :1. abstract decision c and add it in boolean_CSP2. solve boolean_CSP
I boolean_CSP has no solution choose anotherpath
I if boolean_CSP has a solution investigate bothbranches c and ¬c
Boolean abstraction• hash-table of decisions : keys are decisions, values
are Boolean variables• sub-expressions are shared→ rewriting
49
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Current prototype – On the fly validation :loops
Let c be the entrance condition
• if c is trivially simplified to “true” or “false” enter or exit the loop• if {c + linear_CSP } is inconsistent add ¬c to the CSPs and exit the loop
In other cases, unfold loop max times:
• If max is reached add ¬c to the CSPs and exit the loop• Else investigate both paths
50
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Experiments
We compared CPBVP with the following frameworks:
I ESC/Java, an Extended Static Checker for Java run-time errors in JML-annotated Java programs (staticanalysis of the code and its annotations)
I CBMC, a Bounded Model Checker for ANSI-C and C++programs verification of array bounds (buffer overflows), pointersafety, exceptions, and user-specified assertions
I BLAST, a software model checker for C program(Berkeley Lazy Abstraction Software Verification Tool)
I EUREKA, a C bounded model checker which uses an SMTsolver instead of an SAT solver
I Why, a verification platform which integrates provers (proofassistants such as Coq, PVS, HOL 4,...) and decisionprocedures (Simplify, Yices, ...)
51
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Binary search
length 8 16 32 64 128CPBPV time 1.08s 1.69s 4.04s 17.01s 136.80sCBMC time 1.37s 1.43s KOWhy inv 11.18s
– KOESC/Java Error
BLAST KO
• EUREKA tool : cannot handle because of expression m = (u + l)/2• CP execution paths explored given by the recurrence relation:
P(2) = P(4); P(2n) = 2P(n) + log(n)
length CPBPV ESC/Java CBMC WHY inv BLAST8 0.027s 1.21 s 1.38s KO KO
16 0.037s 1.347 s 1.69s KO KO32 0.064s 1.792 s 7.62s KO KO64 0.115s 1.886 s 27.05s KO KO128 0.241s 1.964 s 189.20s KO KO
Table: Experimental Results for an Incorrect Binary Search
• CBMC and ESC/Java only show the decisions taken along the faultypath (they do not provide any value for the array nor the searcheddata) 52
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Binary search
length 8 16 32 64 128CPBPV time 1.08s 1.69s 4.04s 17.01s 136.80sCBMC time 1.37s 1.43s KOWhy inv 11.18s
– KOESC/Java Error
BLAST KO
• EUREKA tool : cannot handle because of expression m = (u + l)/2• CP execution paths explored given by the recurrence relation:
P(2) = P(4); P(2n) = 2P(n) + log(n)
length CPBPV ESC/Java CBMC WHY inv BLAST8 0.027s 1.21 s 1.38s KO KO
16 0.037s 1.347 s 1.69s KO KO32 0.064s 1.792 s 7.62s KO KO64 0.115s 1.886 s 27.05s KO KO128 0.241s 1.964 s 189.20s KO KO
Table: Experimental Results for an Incorrect Binary Search
• CBMC and ESC/Java only show the decisions taken along the faultypath (they do not provide any value for the array nor the searcheddata) 53
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Tritype
Takes 3 integers (triangle sides) and returns the typeof triangle
I CP :10 paths explored among 57 – correspond toactual inputs because of complex conditionals
I CP and Why : time does not depend on the size of theintegers
I earlier approach (Boolean abstraction, TACAS’06):8.52s for integers coded on 16 bits, 92 spurious paths
CPBPV ESC/Java CBMC Why BLASTtime 0.287s 1.828s 0.82s 8.85s KO
1 int sum(int[] t, int n)2 int s = 0;3 int i = 0;4 while (i!=t.length)5 s=s+t[i]*t[i]6 i =i+1;7 return s;
• Using global constraint alldiff• Solving non linear problems• 66.179s for n = 10
55
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Role of the different solvers
I CPLEX, the MIP solver, plays a key role in all thesebenchmarks:
• Tritype: the CP solver is never called
• Binary search: there are only length calls to the CPsolver (and much more calls to CPLEX) but almost75% of the CPU time is spent in the CP solver
• Sum of squares: 80% of the CPU time is spent in theCP solver
56
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPVOverall view
Example
Implementation &Experiments
Implementation
Experiments
DPVS
Papers
Critical issues
I We do not need the Boolean abstraction to capture thecontrol structure of the program
→ Use the CFG and constraints to prune thesearch space
I Depth first dynamic exploration of the CFG
• Efficient if the variables are instantiated early
• Blind searching: post-condition becomes active verylate
57
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
DPVS
I A Non Sequential Exploration Strategy of the CFG
58
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
DPVS,Overall view
I A new search strategy for verifying a restricted classof Java or C programs:→ Non sequential dynamic exploration of the CFG
I Goal: generating counterexamples for real timeapplications
59
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
Non sequential dynamic constraint basedexploration strategy
• Essential observation
When the program is in an SSA-like form, a path canbe built in a non-sequential dynamic way
CFG does not have to be explored in a top down(or bottom up) way: compatible blocks can just be
collected in a non-deterministic way
• Constraint solving is integrated with stateexploration to prune the state space as early aspossible
60
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
Non sequential dynamic constraint basedexploration strategy
• DPVS starts from the post-condition anddynamically collects program blocks which involvevariables of the post-condition
• Collecting as much information as possible on a givenvariable
→ enforces the constraints on its domain andreduces the search space
61
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
A small example
void foo(int a, int b)int c, d, e, f ;if(a >= 0) {
if(a < 10) {f = b − 1;}else {f = b − a; }c = a;if(b >= 0) {d = a; e = b;}else {d = a; e = −b;} }
else {c = b; d = 1; e = −a;if(a > b) {f = b + e + a;}else {f = e ∗ a− b;} }
c = c + d + e;assert(c >= d + e); // property p1assert(f >= −b ∗ e); // property p2
62
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
A small example(continued)
To prove property p1, select node (4)→ the condition in node (0) must be trueS = {c1 < d0 + e0 ∧ c1 = c0 + d0 + e0 ∧ c0 = a0 ∧ a0 ≥ 0}
= {a0 < 0 ∧ a0 ≥ 0} ... inconsistent
63
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
A small example(continued)
Select node (8)→ condition in node (0) must be falseS = {c1 < d0 + e0 ∧ c1 = c0 + d0 + e0 ∧ c0 = b0
∧d0 = 1 ∧ e0 = −a0 ∧ a0 < 0}= {a0 < 0 ∧ b0 < 0}
Solution {a0 = −1,b0 = −1}
64
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
DPVS, Algorithm (scheme)
S ← negation of prop % constraint storeQ ← variables in prop % queue of variables
• IF Q 6= ∅, v ← POP(Q)• Search for a program block PB(v) where v is
definedPUSH(Q,new_var ), new_var = new variables (6=input variables) of PB(v)S ← S ∪ {definition of v and conditions required toreach definition of v }
• IF S is inconsistent, backtrack & search anotherdefinition (otherwise the dual condition is cut off)
• IF Q = ∅ search for an instantiation of the inputvariables (= counterexample)If no solution exists, DPVS backtracks.
65
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
FM Application: Description of the module
• A real time industrial application from a carmanufacturer (provided by Geensoft)
• Flasher Manager (FM): controller that drives severalfunctions related to the flashing lights
Purpose:
• to indicate a direction change• to lock and unlock the car from the distance• to activate the warning lights
100 65.190 9.750 9.750200 395.46 21.65 21.65400 TO 50.90 50.90
72
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
Discussion
Experiments on the binary search
Length CBMC DPVS CPBPV*4 5.732 0.529 0.1078 110.081 35.074 0.29816 TO TO 1.14964 TO TO 27.714128 TO TO 153.646
• DPVS and CBMC waste a lot of time in exploringthe different paths
• CPBPV* incrementally adds the decisions takenalong a path→ well adapted for the Binary Search program
73
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVSOverall view
A small example
Algorithm
FM Application
Description
Simulink model
Program
Experiments
Discussion
Binary search
Future work
Papers
Discussion (continued)
Future work
• Experiments on other applications
• Extension of our prototype→ handling pointers→ interfacing with a floating point number solver
• Combining strategies
74
SearchStrategies
M. Rueher
Basics
The CPFramework
CPBPV
DPVS
Papers
Papers
• A Constraint-Programming Framework forBounded Program VerificationHélène Collavizza, Michel Rueher, and Pascal VanHentenryckConstraints Journal, Springer Verlag, vol.15(2):238-264, 2010.
• Efficient Constraint-Based Dynamic Strategies ForGenerating CounterexamplesHélène Collavizza, Nguyen Le Vinh, Michel Rueher,Samuel Devulder, Thierry Gueguen26th ACM Symposium On Applied Computing,Software Verification and Testing Track,Taiwan,March 2011.