Heuristics for Efficient SAT Solving

Heuristics forEfficient SAT Solving

As implemented in GRASP, Chaff and GSAT.

Given a property p: (e.g. “always signal_a = signal_b”)

Is there a state reachable within k cycles, which satisfies p ?

. . .s0 s1 s2 sk-1 sk

p p p p p

Formulation of famous problems as SAT:Bounded Model Checking (1/4)

The reachable states in k steps are captured by:

I s s s s s s sk k( ) ( , ) ( , ) ... ( , )0 0 1 1 2 1

The property p fails in one of the cycles 1..k:

p p pk1 2 ...


( ): ( , )k I s s pi

k

i ii

k

i00

1

10

=

The safety property p is valid up to cycle k iff k is unsatisfiable:

. . .s0 s1 s2 sk-1 sk

p p p p p


Example: a two bit counter

l l rr r' ( )'

Property: always (l r).

00

01 10

11

: ( )( )( )

( )( )( )

FHG

IKJ

FHGG

IKJJl r

l l r r rl l r r r

l rl rl r

0 01 0 0 1 0

2 1 1 2 1

0 0

1 1

2 2

I l r0:

For k = 2, k is unsatisfiable. For k = 4 k is satisfiable

:

Initial state:

Transition:

For k = 2:


What is SAT?

SATisfying assignment!

Given a propositional formula in CNF, find an assignment to Boolean variables that makes the formula true:

1 = (x2 x3)

2 = (x1 x4)

3 = (x2 x4)

A = {x1=0, x2=1, x3=0, x4=1}

1 = (x2 x3)

2 = (x1 x4)

3 = (x2 x4)

A = {x1=0, x2=1, x3=0, x4=1}

Why SAT?

Fundamental problem from theoretical point of view Numerous applications:

– CAD, VLSI– Optimization– Bounded Model Checking and other type of formal verification– AI, planning, automated deduction

Given in CNF: (x,y,z),(-x,y),(-y,z),(-x,-y,-z)

Decide()

Deduce()

Resolve_Conflict()

-xx

-zz-yy

z -z y -y

() ()

(z ),(-z ) ()

(y),(-y,z ),(-y,-z )

()

() ()

(y),(-y)

(y,z ),(-y,z )

X

X X X X

A Basic SAT algorithm

While (true){

if (!Decide()) return (SAT);

while (!Deduce())

if (!Resolve_Conflict()) return

(UNSAT);}

Choose the next variable and value.Return False if all

variables are assigned

Apply unit clause rule.Return False if reached

a conflict

Backtrack until no conflict.

Return False if impossible

A Basic SAT algorithm

Basic Backtracking Search

Organize the search in the form of a decision tree

– Each node corresponds to a decision

– Depth of the node in the decision tree decision level

– Notation: x=v@d x 2 {0,1} is assigned to v at decision level d

Backtracking Search in Action

1 = (x2 x3)

2 = (x1 x4)

3 = (x2 x4)

1 = (x2 x3)

2 = (x1 x4)

3 = (x2 x4)

x1

x1 = 0@1

{(x1,0), (x2,0), (x3,1)}

x2 x2 = 0@2

{(x1,1), (x2,0), (x3,1) , (x4,0)}

x1 = 1@1

x3 = 1@2

x4 = 0@1 x2 = 0@1

x3 = 1@1

No backtrack in this example!

Backtracking Search in Action

1 = (x2 x3)

2 = (x1 x4)

3 = (x2 x4)

4 = (x1 x2 x3)

1 = (x2 x3)

2 = (x1 x4)

3 = (x2 x4)

4 = (x1 x2 x3)

Add a clause

x4 = 0@1

x2 = 0@1

x3 = 1@1

conflict

{(x1,0), (x2,0), (x3,1)}

x2

x2 = 0@2 x3 = 1@2

x1 = 0@1

x1

x1 = 1@1

DLIS (Dynamic Largest Individual Sum)For a given variable x:

– Cx,p – # unresolved clauses in which x appears positively

– Cx,n - # unresolved clauses in which x appears negatively

– Let x be the literal for which Cx,p is maximal

– Let y be the literal for which Cy,n is maximal

– If Cx,p > Cy,n choose x and assign it TRUE

– Otherwise choose y and assign it FALSE

Requires l (#literals) queries for each decision. (Implemented in e.g. Grasp)

Decision heuristics

Compute for every clause and every variable l (in each phase):

J(l) :=

Choose a variable l that maximizes J(l).

This gives an exponentially higher weight to literals in shorter clauses.

Jeroslow-Wang method

Decision heuristics

,

||2l

Let f*(x) be the # of unresolved smallest clauses containing x. Choose x that maximizes:

((f*(x) + f*(!x)) * 2k + f*(x) * f*(!x)

k is chosen heuristically.

The idea:

– Give preference to satisfying small clauses.

– Among those, give preference to balanced variables (e.g. f*(x) = 3, f*(!x) = 3 is better than f*(x) = 1, f*(!x) = 5).

MOM (Maximum Occurrence of clauses of Minimum size).

Decision heuristics

VSIDS (Variable State Independent Decaying Sum)

4. Periodically, all the counters are divided by a constant.

3. The unassigned variable with the highest counter is chosen.

2. When a clause is added, the counters are updated.

1. Each variable in each polarity has a counter initialized to 0.

(Implemented in Chaff)

Decision heuristics

VSIDS (cont’d)

• Chaff holds a list of unassigned variables sorted by the counter value.

• Updates are needed only when adding conflict clauses.

• Thus - decision is made in constant time.

Decision heuristics

VSIDS is a ‘quasi-static’ strategy:

- static because it doesn’t depend on current assignment

- dynamic because it gradually changes. Variables that appear in recent conflicts have higher priority.

Decision heuristics

This strategy is a conflict-driven decision strategy.

“..employing this strategy dramatically (i.e. an orderof magnitude) improved performance ... “

A (CNF) dependency graph D (V,E):

A partitioning C1..Cn:

An abstract dependency graph D’(V’, E’):

Variable ordering (Abstract dependency graphs)

For (k) there exists a partition C1..Cn s.t. the abstract dependency graph is linear

C0 C1 C2 CkC3 Ck-1

V0 V1 V2 VkV3 Vk-1

...

Variable ordering (The natural order of (k))

I0PkRiding on unreachable states...

k should satisfy I0

I0Riding on legal executions...

(k) should satisfy Pk

Pk

Variable ordering (simple static ordering)

Implication graphs and learning

1 = (x1 x2)

2 = (x1 x3 x9)

3 = (x2 x3 x4)

4 = (x4 x5 x10)

5 = (x4 x6 x11)

6 = (x5 x6)

7 = (x1 x7 x12)

8 = (x1 x8)

9 = (x7 x8 x13)

1 = (x1 x2)

2 = (x1 x3 x9)

3 = (x2 x3 x4)

4 = (x4 x5 x10)

5 = (x4 x6 x11)

6 = (x5 x6)

7 = (x1 x7 x12)

8 = (x1 x8)

9 = (x7 x8 x13)

Current truth assignment: {x9=0@1 ,x10=0@3, x11=0@3, x12=1@2, x13=1@2}

Current decision assignment: {x1=1@6}

6

6

conflict

x9=0@1

x1=1@6

x10=0@3

x11=0@3

x5=1@64

4

5

5 x6=1@62

2

x3=1@6

1

x2=1@6

3

3

x4=1@6

We learn the conflict clause 10 : (: x1 Ç x9 Ç x11 Ç x10)

Implication graph, flipped assignment

x1=0@6

x11=0@3

x10=0@3

x9=0@1

x7=1@6

x12=1@2

7

7

x8=1@6

8

10

10

10 9

9

’

x13=1@2

9

Due to the conflict clause

1 = (x1 x2)

2 = (x1 x3 x9)

3 = (x2 x3 x4)

4 = (x4 x5 x10)

5 = (x4 x6 x11)

6 = (x5 x6)

7 = (x1 x7 x12)

8 = (x1 x8)

9 = (x7 x8 x13)

10 : (: x1 Ç x9 Ç x11 Ç x10)

1 = (x1 x2)

2 = (x1 x3 x9)

3 = (x2 x3 x4)

4 = (x4 x5 x10)

5 = (x4 x6 x11)

6 = (x5 x6)

7 = (x1 x7 x12)

8 = (x1 x8)

9 = (x7 x8 x13)

10 : (: x1 Ç x9 Ç x11 Ç x10)

Non-chronological backtracking

Non-chronological backtracking

x1

4

5

6

’

Decision level

Which assignments caused the conflicts ? x9= 0@1

x10= 0@3

x11= 0@3

x12= 1@2

x13= 1@2

Backtrack to decision level 3

3

These assignmentsAre sufficient forCausing a conflict.

Deduction() allocates new implied variables and conflicts. How can this be done efficiently ?

Observation: More than 90% of the time SAT solvers perform Deduction().

More engineering aspects of SAT solvers

Hold 2 counters for each clause :

val1( ) - # of negative literals assigned 0 in +

# of positive literals assigned 1 in .

val0() - # of negative literals assigned 1 in +

# of positive literals assigned 0 in .

Grasp implements Deduction() with counters

Every assignment to a variable x results in updating the counters for all the clauses that contain x.

Grasp implements Deduction() with counters

is satisfied iff val1() > 0

is unsatisfied iff val0() = ||

is unit iff val1() = 0 val0() = || - 1

is unresolved iff val1() = 0 val0() < || - 1..

Backtracking: Same complexity.

Chaff implements Deduction() with a pair of observers

Observation: during Deduction(), we are only interested in newly implied variables and conflicts.

These occur only when the number of literals in with value ‘false’ is greater than || - 2

Conclusion: no need to visit a clause unless (val0() > || - 2)

How can this be implemented ?


Define two ‘observers’: O1(), O2().

O1() and O2() point to two distinct literals which are not ‘false’.

becomes unit if updating one observer leads to O1() = O2().

Visit clause only if O1() or O2() become ‘false’.

Both observers of an implied clause are on the highest decision levelpresent in the clause. Therefore, backtracking will un-assign them first.Conclusion: when backtracking, observers stay in place.


1 2 3 4 5

V[1]=0

1 2 3 4 5

V[5]=0, v[4]= 0

1 2 3 4 5

V[2]=0

O1 O2

1 2 3 4 5 Unit clause

Backtrackv[4] = v[5]= Xv[1] = 1

1 2 3 4 5

Backtracking: No updating. Complexity = constant.


The choice of observing literals is important. Best strategy is - the least frequently updated variables.

The observers method has a learning curve in this respect:

1. The initial observers are chosen arbitrarily.

2. The process shifts the observers away from variables that were recently updated (these variables will most probably be reassigned in a short time).

In our example: the next time v[5] is updated, it will point to a significantly smaller set of clauses.

GSAT: A different approach to SAT solving

for i = 1 to max_tries {

T := randomly generated truth assignment

for j = 1 to max_flips {

if T satisfies return TRUE

choose v s.t. flipping v’s value gives largest increase in

the # of satisfied clauses (break ties randomly).

T := T with v’s assignment flipped. } }

Given a CNF formula , choose max_tries and max_flips

Improvement # 1: clause weights

Initial weight of each clause: 1

Increase by k the weight of unsatisfied clauses.

Choose v according to max increase in weight

Clause weights is another example of conflict-driven decision strategy.

Improvement # 2: Averaging-in

Q: Can we reuse information gathered in previous tries in order to speed up the search ?

A: Yes! Rather than choosing T randomly each time, repeat ‘good assignments’ and choose randomly the rest.

Let X1, X2 and X3 be equally wide bit vectors.

Define a function bit_average : X1 X2 X3 as follows:

b1i b1

i = b2i

random otherwise

(where bji is the i-th bit in Xj, j {1,2,3})

Improvement # 2: Averaging-in (cont’d)

b3i :=

Improvement # 2: Averaging-in (cont’d)

Let Tiinit

be the initial assignment (T) in cycle i.

Let Tibest

be the assignment with highest # of satisfied clauses in

cycle i.

T1init := random assignment.

T2init := random assignment.

i > 2, Tiinit := bit_average(Ti-1

best, Ti-2best)

Heuristics for Efficient SAT Solving

Documents