Interpolants as Classifiers Rahul Sharma Joint work with Aditya Nori (MSR India) and Alex Aiken (Stanford)
Interpolants as ClassifiersRahul SharmaJoint work with Aditya Nori (MSR India) and Alex Aiken (Stanford)
Interpolants
If then an interpolant satisfies:
contains only the variables common to and
An interpolant is a simple proof
Quantifier free formulas in linear arithmetic have quantifier free interpolants.
Example
x
y
Binary Classification
Input: labeled examples Positive examples: Negative examples:
Find true,false For all positives, true For all negatives, false
Verification and ML
Interpolant: separates from
Classifier: separates positive examples from negative examples
Is there a connection?
Yes!
Consider the common variables of and :
Interpolant if If :
separates ’s from ’s
Our Contributions
Main result: view interpolants as classifiers as positive example as negative example
Use state-of-the-art classification algorithms (SVMs) for computing interpolants
Encouraging empirical results
Parallels b/w Verification and ML
Unroll the loops Find interpolants Get general proofs (loop invariants)
Get positive and negative examples Find a classifier Get a predicate which generalizes
Support Vector Machine (SVM)
positive examples
negative examples
+
+
+
+
+
Support Vector Machine (SVM)
Separators
Separators
Separators
Separators
All separators are good candidates for interpolants
+
++
+
+
Support Vector Machine (SVM)
Optimal Margin Classifier+
+
+
+
Example
x = y = 0;while(*) x++; y++;while(x != 0) x--; y--;assert (y == 0);
x = y = 0;if(*) x++; y++;p:if(x != 0) x--; y--;if(x == 0) assert (y == 0);
Interpolants by SVM
x
y
(0,0)+
+ (1,1)
Basic Algorithm
Let be the common variables of and
Generate from satisfying assignments of
Generate from satisfying assignments of
Call SVM with and
Return predicate containing
Two Problems
Data is not linearly separable
The candidate interpolant is not an interpolant
x
y
+
+
No separating inequality
For each SVM( )
return
Cannot generate linear separators with both and .
x
y
(0,0)+
+ (1,1)(0,1)
(1,0)
Candidate is not an Interpolant
while(true) { Find candidate interpolant if ( ) Add to and continue;
if ( ) Add to and continue;
break; Exit if interpolant found } return ;
Theorem: terminates iff output is an interpolant between and
Example
x
y
(0,0)+
+ (1,1)
Interpolant!
Evaluation
1000 lines of C++ LIBSVM for SVM queries Z3 theorem prover
Experiments
Related Work
Interpolants used in tools BLAST, IMPACT …
Interpolants from proofs Krajícek[97], Pudlák[97], McMillan[05],
…
Interpolants from constraint solving ARMC, Rybalchenko et al. [07]
Conclusion
Connect interpolants and classifiers
A sound interpolation procedure
Future work: non-linear interpolants
Integrate with a verification tool EUF, arrays, bit-vectors, etc.