Program Verification using Probabilistic Techniques Sumit Gulwani Microsoft Research Invited Talk: VSTTE Workshop August 2006 Joint work with George Necula and Nebojsa Jojic
Mar 26, 2015
Program Verification using Probabilistic Techniques
Sumit GulwaniMicrosoft Research
Invited Talk: VSTTE WorkshopAugust 2006
Joint work with George Necula and Nebojsa Jojic
2
Probabilistic Techniques
• Used successfully in several areas of computer science.• Yields more efficient, precise, even simpler algorithms.• Technique 1: Random Interpretation
– Discovers program invariants– Monte Carlo Algorithm: May generate invalid invariants
with a small probability. Running time is bounded.– “Random Testing” + “Abstract Interpretation”
• Technique 2: Simulated Annealing– Discovers proof of validity/invalidity of a Hoare triple.– Las Vegas Algorithm: Generates a correct proof. Running
time is probabilistic.– “Forward Analysis” + “Backward Analysis”
3
Random Interpretation
= Random Testing + Abstract Interpretation
Random Testing:• Test program on random inputs• Simple, efficient but unsound (can’t prove absence of bugs)
Abstract Interpretation:• Class of deterministic program analyses• Interpret (analyze) an abstraction (approximation) of program
• Sound but usually complicated, expensive
Random Interpretation:• Class of randomized program analyses• Almost as simple, efficient as random testing• Almost as sound as abstract interpretation
4
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert(c+d = 0); assert(c = a+i)
c := 2a + b; d := b – 2i;
True False
FalseTrue
*
*
Example 1
5
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert(c+d = 0); assert(c = a+i)
c := 2a + b; d := b – 2i;
True False
FalseTrue
*
*
Example 1: Random Testing
• Need to test blue path to falsify second assertion.
• Chances of choosing blue path from set of all 4 paths are small.
• Hence, random testing is unsound.
6
a+b=i
a+b=i, c=-d
a=i-2, b=2
a+b=i c=2a+b, d=b-2ia+b=i
c=b-a, d=i-2b
a=0, b=i
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert(c+d = 0); assert(c = a+i)
c := 2a + b; d := b – 2i;
True False
FalseTrue
*
*
Example 1: Abstract Interpretation
• Computes invariant at each program point.
• Operations are usually complicated and expensive.
7
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert(c+d = 0); assert(c = a+i)
c := 2a + b; d := b – 2i;
True False
FalseTrue
*
*
Example 1: Random Interpretation
• Choose random values for input variables.
• Execute both branches of a conditional.
• Combine values of variables at join points.
• Test the assertion.
8
Random Interpretation: Outline
• Random Interpretation
Linear arithmetic (POPL 2003)
– Uninterpreted functions (POPL 2004)
– Inter-procedural analysis (POPL 2005)
9
Linear relationships in programs with linear assignments
• Linear relationships (e.g., x=2y+5) are useful for– Program correctness (e.g. buffer overflows)– Compiler optimizations (e.g., constant and copy
propagation, CSE, Induction variable elimination etc.)
• “programs with linear assignments” does not mean inapplicability to “real” programs– “abstract” other program stmts as non-
deterministic assignments (standard practice in program analysis)
10
Basic idea in random interpretation
Generic algorithm:
• Choose random values for input variables.
• Execute both branches of a conditional.
• Combine the values of variables at join points.
• Test the assertion.
11
Idea #1: The Affine Join operation
w = 7
a = 2b = 3
a = 4b = 1
a = 7(2,4) = -10b = 7(3,1) = 15
• Affine join of v1 and v2 w.r.t. weight w
w(v1,v2) ´ w v1 + (1-w) v2
• Affine join preserves common linear relationships (a+b=5)
• It does not introduce false relationships w.h.p.
12
Idea #1: The Affine Join operation
• Affine join of v1 and v2 w.r.t. weight w
w(v1,v2) ´ w v1 + (1-w) v2
• Affine join preserves common linear relationships (a+b=5)• It does not introduce false relationships w.h.p.• Unfortunately, non-linear relationships are not preserved
(e.g. a £ (1+b) = 8)
w = 5
a = 5(2,4) = -6b = 5(3,1) = 11
w = 7
a = 2b = 3
a = 4b = 1
a = 7(2,4) = -10b = 7(3,1) = 15
13
Geometric Interpretation of Affine Join
a
ba + b =
5
b = 2
(a = 2, b = 3)
(a = 4, b = 1)
: State before the join
: State after the join
satisfies all the affine relationships that are satisfied by both (e.g. a + b = 5)
Given any relationship that is not satisfied by any of (e.g. b=2), also does not satisfy it with high probability
i=3, a=0, b=3
i=3
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert (c+d = 0); assert (c = a+i)
i=3, a=-4, b=7
i=3, a=-4, b=7c=23, d=-23
c := 2a + b; d := b – 2i;
i=3, a=1, b=2
i=3, a=-4, b=7c=-1, d=1
i=3, a=-4, b=7 c=11, d=-11
False
False
w1 = 5
w2 = 2
True
True*
*
Example 1
• Choose a random weight for each join independently.
• All choices of random weights verify first assertion
• Almost all choices contradict second assertion
15
Correctness of Random Interpreter R
• Completeness: If e1=e2, then R ) e1=e2
– assuming non-det conditionals
• Soundness: If e1e2, then R e1 = e2
– error prob. ·
• j : number of joins• d: size of set from which random values are
chosen• k: number of points in the sample
– If j = 10, k = 4, d ¼ 232, then error ·
16
Proof Methodology
Proving correctness was the most complicated part in this work. We used the following methodology.
• Design an appropriate deterministic algorithm (need not be efficient)
• Prove (by induction) that the randomized algorithm simulates each step of the deterministic algorithm with high probability.
17
Random Interpretation: Outline
• Random Interpretation
– Linear Arithmetic (POPL 2003)
Uninterpreted functions (POPL 2004)
– Inter-procedural analysis (POPL 2005)
18
Problem: Global value numbering
a := 5;x := a*b;y := 5*b;z := b*a;
a := 5;x := F(a,b);y := F(5,b);z := F(b,a);
Abstraction
• x=y and x=z• Reasoning about multiplication is undecidable
• only x=y• Reasoning is decidable but tricky in presence of joins
• Axiom: If x1=y1 and x2=y2, then F(x1,x2)=F(y1,y2)
• Goal: Detect expression equivalence when program operators are abstracted using “uninterpreted functions”
• Application: Compiler optimizations, Translation validation
19
Random Interpretation: Outline
• Random Interpretation
– Linear arithmetic (POPL 2003)
– Uninterpreted functions (POPL 2004)
Inter-procedural analysis (POPL 2005)
20
Example 1
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert (c + d = 0); assert (c = a + i)
c := 2a + b; d := b – 2i;
True False
False
•The second assertion is true in the context i=2.
•Interprocedural Analysis requires computing procedure summaries.
True
*
*
i=2
a=0, b=i
a := 0; b := i;
a := i-2; b := 2;
c := b – a; d := i – 2b;
assert (c+d = 0); assert (c = a+i)
a=8-4i, b=5i-8
a=8-4i, b=5i-8c=21i-40, d=40-21i
c := 2a + b; d := b – 2i;
a=i-2, b=2
a=8-4i, b=5i-8c=8-3i, d=3i-8
a=8-4i, b=5i-8 c=9i-16, d=16-9i
False
False
w1 = 5
w2 = 2
Idea: Keep input variables symbolic
•Do not choose random values for input variables (to later instantiate by any context).
• Resulting program state at the end is a random procedure summary.
a=0, b=2c=2, d=-2
True
True
*
*
22
Experimental measure of error
The % of incorrect relationships decreases with increase in • S = size of set from which random values are chosen.• N = # of random summaries used.
2 95.5 95.5 95.5
3 64.3 3.2 0
4 0.2 0 0
5 0 0 0
6 0 0 0
S
N
The experimental results are better than what is predicted by theory.
210 216 231
23
Simulated Annealing
Problem: Given a program with a pre/post conditions, discover proof of validity/invalidity.
• Proof is in the form of an invariant at each program point that can be locally verified.
• Key Idea:– Initialize invariants at all program points to anything.– Pick a random program point whose invariant is not
locally consistent and update it to make it less inconsistent.
24
Simulated Annealing: Outline
• Simulated Annealing
Inconsistency Measure & Penalty Function
– Algorithm
– Experiments
25
Inconsistency Measure for an Abstract Domain
• Let A be an abstract domain with ) as the partial order and as the concretization function.
• An inconsistency measure IM: A £ A ![0,1] satisfies:– IM(1,2) = 0 iff 1 ) 2
– IM is monotonically decreasing in its first argument– IM is monotonically increasing in its second argument
• IM is a monotonic (increasing) measure of (1) - (2) [set of states that violate 1 ) 2]. The more strictly monotonic IM is, the more smooth it is.
26
Example of a Smooth Inconsistency Measure
Let A be the abstract domain of Boolean formulas (with the usual implication as the partial order).
Let 1 ´ a1 Ç … Ç an in DNF
and 2 ´ b1 Æ … Æ bm in CNF
IM(1, 2) = IM(ai,bj)
where IM(ai,bj) = 0, if ai ) bj
= 1, otherwise
27
Penalty Function
Penalty(I,) is a measure of how much inconsistent is I with respect to the invariants at neighbors of .
Penalty(I,) = IM(Post(), I) + IM(I,Pre()) • Post() is the strongest postcondition of the
invariants at the predecessors of at .• Pre() is the weakest precondition of the invariants
at the successors of at .
28
Example of Penalty Function
• Penalty(I, 2) = IM(Post(2), I) + IM(I, Pre(2))
I
Q
P
R
c
1
• Post(2) = StrongestPost(P,s)
• Pre(2) = (c ) Q) Æ (: c ) R)
s
Since Post() and Pre() may not belong to A, we define:• IM(Post(), I) = Min {IM(I1,I) | I12A, I1 overapproximates Post()}
• IM(I, Pre()) = Min {IM(I,I2) | I22A, I2 underapproximates Pre()}
29
Simulated Annealing: Outline
• Simulated Annealing
– Inconsistency Measure & Penalty Function
Algorithm
– Experiments
30
Algorithm
• Search for proof of validity and invalidity in parallel.
• Same algorithm with different boundary conditions.
• Proof of Validity
– Ientry = Pre
– Iexit = Post
• Proof of Invalidity
– Ientry Æ Pre is satisfiable
– Iexit = : Post
– This assumes that program terminates on all inputs.
31
Algorithm (Continued)
• Initialize invariant Ij at program point j to anything.
• While penalty at some program point is not 0:
– Choose j randomly s.t. Penalty(Ij, j) 0.
– Update Ij s.t. Penalty(Ij,j) is minimized.
• More precisely, Ij is chosen randomly with probability inversely proportional to Penalty(Ij,j).
32
Interesting Aspects of the Algorithm
• Combination of Forward & Backward Analysis
• No distinction between forward & backward information
• Random Choices– Program point to update– Invariant choice
33
Simulated Annealing: Outline
• Simulated Annealing
– Inconsistency Measure & Penalty Function
– Algorithm
Experiments
34
Example 2
y := 50;
y = 100
False
x := x +1;
x := x +1;y := y +1;
x < 50
x <100
True
True False
1
2
3
4
5
6
7
8
x = 0
Prog. Point
Invariant
1 x=0 Æ y=50
2 x·50 )y=50 Æ 50·x )x=y Æ x·100
3 x·50 )y=50 Æ 50·x )x=y Æ x<100
4 x<50 Æ y=50
5 x·50 Æ y=50
6 50·x<100 Æ x=y
7 50<x·100 Æ x=y
8 x·50 )y=50 Æ 50·x )x=y Æ x·100
Proof of Validity
35
Stats: Proof vs Incremental Proof of Validity
• Black: Proof of Validity• Grey: Incremental Proof of Validity• Incremental proof requires fewer updates
36
Stats: Different Sizes of Boolean Formulas
• Grey: 5*3, Black: 4*3, White: 3*2• n*m denotes n conjuncts & m disjuncts• Larger size requires fewer updates
37
Example 3
x := 0; m := 0;
n· 0 Ç 0· m < n
False
m := x;
x := x +1;
*
x < n
True
1
2
3
4
6
5
7
8
true
Prog. Point
Invariant
1 x=0 Æ m=0
2 n· 0 Ç (0·x Æ 0·m<n)
3 n· 0 Ç (0·x<n Æ 0·m<n)
4 n· 0 Ç (0·x<n Æ 0·m<n)
5 n· 0 Ç (0·x<n Æ 0·m<n)
6 n· 0 Ç (0·x<n Æ 0·m<n)
7 n· 0 Ç (0·x<n Æ 0·m<n)
8 n· 0 Ç (0·x·n Æ 0·m<n)
Proof of Validity
38
Stats: Proof of Validity
• Example 2 is “easier” than Example 1.• Easier example requires fewer updates.
39
Example 2: Precondition Modified
Prog. Point
Invariant
0 x¸100
1 x¸100 Æ y=50
2 x¸100 Æ y=50
3 false
4 false
5 false
6 false
7 false
8 false
Proof of Invalidity
y := 50;
y = 100
False
x := x +1;
x := x +1;y := y +1;
x < 50
x <100
True
True False
1
2
3
4
5
6
7
8
true
40
Stats: Proof of Invalidity
Conclusion
Lessons Learned
• Randomization buys efficiency and simplicity.
• Randomization suggests ideas for deterministic algorithms.
• Combining randomized and symbolic techniques is powerful.
Summary
• Random Interpretation:
•Linear Arithmetic: Affine Joins
•Uninterpreted Functions: Random Linear Interpretations
•Interprocedural Analysis: Symbolic Input Variables
• Simulated Annealing:
• Smooth Inconsistency Measure for an abstract domain