Arc consistency
ac3, ac4, ac6/7/8
Di = {1,2,3,4,5}
V1
V3
V2
V4V4 V3
V1 V4 1V4 + V2 = 5
V1 V2
V2 V3 6
AR33 figure 18, page 35What can you infer?
Di = {1,2,3,4,5}
V1
V3
V2
V4 V4 V3
V1 V4 1
V4 + V2 = 5
V1 V2
V2 V3 6
D1 = {1,2}D2 = {2,3}D3 = {4,5}D4 = {2,3}
Do you agree?Was that easy?
V1 < V2 D1 = {1,2,3,4,5} D2 = {1,2,3,4,5} D1 := {1,2,3,4}V2 > V1 D1 = {1,2,3,4} D2 = {1,2,3,4,5} D2 := {2,3,4,5}V4 ≥ V1 + 1 D4 = {1,2,3,4,5} D1 = {1,2,3,4} D4 := {2,3,4,5}V1 ≤ V4 - 1 D1 = {1,2,3,4} D4 = {2,3,4,5} no changeV2 + V3 > 6 D2 = {2,3,4,5} D3 = {1,2,3,4,5} no changeV3 + V2 > 6 D3 = {1,2,3,4,5} D2 = {2,3,4,5} D3 := {2,3,4,5}V2 + V4 = 5 D2 = {2,3,4,5} V4 = {2,3,4,5} D2 = {2,3}V1 < V2 D1 = {1,2,3,4} D2 = {2,3} D1 = {1,2}V2 > V1 D2 = {2,3} D1 = {1,2} no changeV4 ≥ V1 + 1 D4 = {2,3,4,5} D1 = {1,2} no changeV3 + V2 > 6 D3 = {2,3,4,5} D2 = {2,3} D3 = {4,5}V2 + V3 > 6 D2 = {2,3} D3 = {4,5} no changeV4 + V2 = 5 D4 = {2,3,4,5} D2 = {2,3} D4 = {2,3}V1 ≤ V4 - 1 D1 = {1,2} D4 = {2,3} no changeV2 + V4 = 5 D2 = {2,3} D4 = {2,3} no changeV4 < V3 D4 = {2,3} D3 = {4,5} no changeV3 > V4 D3 = {4,5} D4 = {2,3} no change
Here’s the reasoning
V4 V3
V1 V4 1
V4 + V2 = 5
V1 V2
V2 V3 6
Di = {1,2,3,4,5}
Arc consistency: so what’s that then?
A constraint Cij is arc consistent if
• for every value x in Di there exists a value y in Dj that supports x• i.e. if v[i] = x and v[j] = y then Cij holds• note: we are assuming Cij is a binary constraint
A csp (V,D.C) is arc consistent if
• every constraint is arc consistent This is also called 2-consistency
If (V,D,C) is arc consistent then• I can choose any variable v[i] • assign it a value x from its domain Di• I can now choose any other variable v[j]• I can find a consistent instantiation for v[j] from Dj
NOTE: this is in isolation, where I have only 2 variables that I instantiate
A constraint Cij is arc consistent if
• for every value x in Di there exists a value y in Dj that supports x• i.e. if v[i] = x and v[j] = y then Cij holds• note: we are assuming Cij is a binary constraint
A csp (V,D.C) is arc consistent if
• every constraint is arc consistent
)],([)( ,, yxCDyDxCAC jijiji
)]([),,( ,, jiji CACCCCDVAC
Just because a problem (V,D,C) is arc consistent doesnot mean that it has a solution!
Arc-consistency processes a problem and removesfrom the domains of variables values that CANNOToccur in any solution
Arguably, it makes the resultant problem easier.Why?
The arc-consistent problem has the same set of solutions asthe original problem
32
31
21
}2,1{
VV
VV
VV
Di
Note: is constraint graph is a tree, AC is a decision procedure
So, is there 1-consistency?
Yip• when we have unary constraints• example odd(V[i])• 1-consistency, we weed out all odd values from Di• also called node-consistency (NC)
3-consistency? Given constraints Cij and Cjk, disallow all pairs (x,z) in the constraintCik where there is no value y in Dj such that Cij(x,y) and Cjk(y,z)
This adds nogood tuples to an existing constraints, or createsa new constraint!
sometimes called path-consistency (PC)
(Given 2 variables in isolation, we can instantiate those consistently, and pick any third variable and …)
Path-consistency (aka 3-consistency)
Vi
Vj
Vk
There might be no constraint CikTherefore 3-consistency may create it!
)],(),(),([:),,(3 ,,, zyCzxCyxCDyDzDxkjiCon kjkijijki
It may create nogood tuples {(i/x,k/z),…}Therefore increases size of model/problem.May result in more constraints to check!
)( 23dnO
M. Singh, TAI-95
ac3: Mackworth 1977
Alan Mackworth presented ac1, ac2, and ac3in 1977. ac1 and ac2 were “straw men”
ac3: revise a constraint (claire3 code)
[revise(d:array<set>,c:tuple,i:integer,j:integer) : boolean ->let revised := false in (for x in copy(d[i]) (let supported := false in (for y in d[j] (if check(c,x,y) (supported := true, return())), if not(supported) (delete(d[i],x), revised := true))), CONSISTENT := d[i] != {}, revised)]
ac3: revise a constraint (pseudo code)
revise(i,j) revised := false for x in d[i] // iterate over all values in d[i] do supported := false for y in d[j] while ¬supported // find first support in d[j] for x do supported := check(i,x,j,y) // is v[i]=x && v[j]=y consistent? if ¬supported // if no support, delete x from d[i] then d[i] := d[i] \ {x} revised := true // and set revised to true return revised // delivers true or false
Given constraint C_ij remove from the domain d_i all values that have no support in d_j
5
4
2
1
3
5
4
2
1
3
6323,2 VVC
support
The micro-structure of C23
2V 3V
revise searches for 1st support for a value Is it a bijection? What are implications of this?
[ac3(d:array<set>,constraints:set) : boolean -> let Q := copy(constraints) in (while (Q != {} & CONSISTENT) let c := Q[1], i := c[2], j := c[3] in (delete(Q,c), if revise(d,c,i,j) Q := Q U {z in constraints | z[3] = i})), CONSISTENT]
Ac3 (claire3 code)
Ac3 (pseudo code)
ac(v,d,c) consistent := true; q := c // enqueue all constraints while q ≠ {} & consistent do (i,j) := dequeue(q) // get a constraint if revise(i,j) // if d_i has values removed then q := q U {(k,i) | Cik in C} // need to revise all constraints C_ki consistent := d[i] ≠ {} // stop if domain wipe out return consistent
if revise(d,c,i,j) Q := Q U {z in constraints | z[3] = i}
ac3
What?
if revise(i,j) then q := q U {(k,i) | (k,i) in c}
Note: ac3 has a queue of constraints that need revision because some values in the domains of the variables may be unsupported.
Remember: ac3 processes a queue of constraints
But forgive me, the queue might be treated as just a set
ac1 and ac2 (the straw men) essentially revisedconstraints over and over again,
until no change … until reaching a fixed point
[ac3(d:array<set>,constraints:set) : boolean -> let Q := copy(constraints) in (while (Q != {} & CONSISTENT) let c := Q[1], i := c[2], j := c[3] in (delete(Q,c), if revise(d,c,i,j) Q := Q U {z in constraints | z[3] = i})), CONSISTENT]
Complexity of ac3 proved in 1985 by Mackworth & Freuder (AIJ 25)
).( 3deO
e is number of constraintsd is domain size
[ac3(d:array<set>,constraints:set) : boolean -> let Q := copy(constraints) in (while (Q != {} & CONSISTENT) let c := Q[1], i := c[2], j := c[3] in (delete(Q,c), if revise(d,c,i,j) Q := Q U {z in constraints | z[3] = i})), CONSISTENT]
Prove it!Also look at paper by Zhang & Yap
).( 3deO
e is number of constraintsd is domain size
[ac3(d:array<set>,constraints:set) : boolean -> let Q := copy(constraints) in (while (Q != {} & CONSISTENT) let c := Q[1], i := c[2], j := c[3] in (delete(Q,c), if revise(d,c,i,j) Q := Q U {z in constraints | z[3] = i})), CONSISTENT]
The order that we revise the constraints make no difference to the outcomeIt reaches the same fixed point, the same set of arc-consistent domains
The order that we revise the constraints may make a difference to run time.
… constraint ordering heuristic, anyone?
Examples/Demos
Run ac3 with the following
• csp6 Barbara’s instance in AR33• csp7 the crystal maze
Revise ignored any semantics of the constraint
Is that dumb, or what?
Could we get round this?• Use OOP?• A class of constraint?• revise as a specialised method?
AC4, AC6, AC7, ….
“Optimal” support counting algorithms
AC4, AC6, AC7, ...
5
4
2
1
3
5
4
2
1
3 6323,2 VVC
2V 3V
Associate with each value in Di • a counter supportCount[x,i,j]
• the number of values in Dj that support x• a boolean supports[x,y,j]
• true if x supports y in Dj
1st stage of the algorithm builds up the supportCount and support flags
2nd stage• if supportCount[x,i,j] = 0 (x has no support in Dj over constraint Cij)
• delete(Di,x)• decrement supportCount[y,k,i] (where supports[x,y,k] is true)
• continue this till no change• i.e. propagate
If x supports y in Dk and x is deleted from DiThen support count for y in Dk over constraint Cki is decremented
Best case and worst case performance of ac4 is the same
).( 2deO
Ac6, 7, and 8 exploit symmetries, and lazy evaluation• if x supports y over constraint Cij then y supports x over Cji• find the 1st support for x, and only look for more when support is lost
Ac3 worst case performance rarely occurs(experimental evidence due to Rick Wallace)
Why is best case and worst case performance of ac4
).( 2deO
?
• ac1/2/3 due to Alan Mackworth 1977• ac4 Mohr & Henderson AIJ28 1986• ac6 , 7, 8 due to Freuder, Bessiere, Regin, and others
• in AIJ, IJCAI, etc
Downside of ac4, ac6, ac7, and ac8 algorithms is “hard to code”
ac3 is easy!
History Lesson
AC5
A generic arc-consistency algorithm and its specializationsAIJ 57 (2-3) October 1992
P. Van Hentenryck, Y. Deville, and C.M. Teng
ac5
Ac5 is a “generic” ac algorithm and can be specialised forspecial constraints (i.e. made more efficient when we knowsomething about the constraints)
Ac5 is at the heart of constraint programming
It can be “tweaked” to become ac3, or “tweaked” to become ac4But, it can exploit the semantics of constraints, and there’s the win
ac5
ac5 processes a queue of tuples (i,j,w)• w has been removed from Dj• we now need to reconsider constraint Cij• to determine the unsupported values in Di
A crucial difference between ac3 and ac5• ac3
• revise(i,j) because Dj has lost some values• seek support for each and every value in Di
• ac5• process(i,j,w) because Dj has lost the value w (very specific)• seek support, possibly, for select few (1?) value in Di
Therefore, by having a queue of (i,j,w) we might be able to do things much more efficiently
ac5, the intuition
• take an OOP approach• constraint is a class that can then be specialised• have a method to revise a constraint object• allow specialisation• have basic methods such as
• revise when lwb increases• revise when upb decreases• revise when a value is lost• revise when variable instantiated• revise initially
• methods take as arguments• the variable in the constraint that has changed• possibly, what values have been lost
ac5, the algorithm
[arcCons(d:array<set>,c:tuple) : set -> let s := {}, i := c[2], j := c[3] in (for x in d[i] let supported := false in (for y in d[j] (supported := check(c,x,y), if supported return()), if not(supported) s := s U set(x)), s)]
This is just like revise, but delivers the set s ofunsupported values in d[i] over the constraint C_i,j
s is the set of disallowed values in d[i]remember c is a tuple <r,i,j>, the relation r between v[i] and v[j]
ac5, the algorithm
[localArcCons(d:array<set>,c:tuple,w:integer) : set -> arcCons(d,c)]//// opportunity to specialize this with respect to c//
We can specialise this method if we know something about the semantics of c
We might then do much better that arcCons in termsof complexity
What we have above is the dummy’s version, or when we know nothing about the constraint (and we get AC3!)
localArcCons(d,c,w) - c is the tuple <r,i,j> - d[j] lost the value w - return the set of values unsupported in d[i]
[enqueue(i:integer,delta:set,q:set,constraints:set) : set -> for c in {z in constraints | z[3] = i} for w in delta q := q U set(tuple(c,w)), q]//// d[i] has lost the values delta// add to the q tuples (c,w) where// - the constraint c is the constraint C_k,i// - w is in delta//
ac5, the algorithm
The domain d[i] has lost values deltaAdd to the queue of “things to do” the tuple (c,w) where - c is the constraint C_k,i and - w is in delta
This is similar to the step in AC3
Q := Q U {z in constraints | z[3] = i}
[ac5(d:array<set>,constraints:set) : boolean -> let Q := {} in (for c in constraints let delta := arcCons(d,c), i := c[2] in (d[i] := difference(d[i],delta), Q := enqueue(i,delta,Q,constraints)), while (Q != {}) (let t := Q[1], c := t[1], w := t[2], i := c[2], delta := localArcCons(d,c,w) in (delete(Q,t), Q := enqueue(i,delta,Q,constraints), d[i] := difference(d[i],delta)))), CONSISTENT]
ac5, the algorithm
ac5 has 2 phases- revise all the constraints and build up the Q, using arcCons- process the queue and propagate using localArcCons, exploiting knowledge on deleted values and the semantics of constraints
Complexity of ac5
If arcCons is O(d2) and localArcCons is O(d) then ac5 is O(e.d2)
If arcCons is O(d) and localArcCons is O() then ac5 is O(e.d)
See AIJ 57 (2-3) page 299
Remember - ac3 is O(e.d3) - ac4 is O(e.d2)
Functional constraints
If c is functional
We can specialise arcCons and localArcCons for functional constraints
A constraint C_i,j is functional, with respect to a domain D, if and only if
• for all x in D[i] • there exists at most one value y in D[j] such that• C_i,j(x,y) holds
and
• for all y in D[j]• there exists at most one value in D[i] such that• C_j,i(y,x) holds
The relation is a bijection
5
4
2
1
3
5
4
2
1
3
iV jV
An example: 3.x = y
If c is functional
Let f(x) = y and f’(y) = x (f’ is the inverse of f)
[arcCons(d:array<set>,c:tuple) : set -> let s := {}, f := c[1], i := c[2], j := c[3] in (for x in d[i] if not(f(x) % d[j]) s := s U set(x), s)]//// functional arcCons// the relation is a bijection//
5
4
2
1
3
5
4
2
1
3
iV jV
The % operator is
Claire manual, section 4.5, inverse of a relation!
If c is functional
Let inverse(f) deliver the f’, the inverse of function f
[localArcCons(d:array<set>,c:tuple,w:integer) : set -> let f := c[1], i := c[2], j := c[3], g := inverse(f) in if (g(w) % d[i]) set(g(w)) else {}]//// assume inverse delivers the inverse of a function//
5
4
2
1
3
5
4
2
1
3
iV jV
In our picture,• assume d[j] looses the value 2 • g(2) = 3• localArcCons(d,c12,2) = {3}
localArcCons(d,c,w) - c is the tuple <r,i,j> - d[j] lost the value w - return the set of values unsupported in d[i]
Anti-functional constraints
If c is anti-functional
The value w in d[j] conflicts with only one value in d[i]
an example: x + 4 ≠ y
[localArcCons(d:array<set>,c:tuple,w:integer) : set -> let f := c[1], i := c[2], j := c[3], g := inverse(f) in if (size(d[i]) = 1 & g(w) = d[i][1]) set(d[i][1]) else {}]//// assume inverse delivers the inverse of a function//
5
4
2
1
3
5
4
2
1
3
iV jV
If D[i] has more than one value, no problem!
Other kinds of constraints that can be specialised
• monotonic• • localArcCons tops and tails domains
• but domains must be ordered• piecewise functional• piecewise anti-functional• piecewise monotonic
See: AIJ 57 & AR33 section 7
The AIJ57 paper by van Hentenryck, Deville, and Teng is a remarkable paper
• they tell us how to build a constraint programming language• they tell us all the data structures we need
• for representing variables, their domains, etc• they introduce efficient algorithms for arc-consistency• they show us how to recognise special constraints and how we should process them• they show us how to incorporate ac5 into the search process• they explain all of this in easy to understand, well written English
It is a classic paper, in my book!
AIJ57
AC2001
Taken from IJCAI01 “Making AC-3 an Optimal Algorithm”
by Y Zhang and R. Yap
and
“Refining the basic constraint propagation algorithm”by
Christian Bessiere & Jean-Charles Regin
The complexity of ac3 A beautiful proof
• A constraint C_i,j is revised iff it enters the Q• C_i,j enters the Q iff some value in d[j] is deleted• C_i,j can enter Q at most d times (the size of domain d[j])• A constraint can be revised at most d times• There are e constraints in C (the set of constraints)• revise is therefore executed at most e.d times• the complexity of revise is O(d2)• the complexity of ac3 is then O(e.d3)
[revise(d:array<set>,c:tuple,i:integer,j:integer) : boolean ->let revised := false in (for x in copy(d[i]) (let supported := false in (for y in d[j] // this is naive (if check(c,x,y) (supported := true, return())), if not(supported) (delete(d[i],x), revised := true))), CONSISTENT := d[i] != {}, revised)]
When searching for a support for x in d[j] we always start from scratchThis is naïve.
• Assume domains are totally ordered• Let resumePoint[i,j,x] = y
• where y is the first value in d[j] that supports x in d[i]• i.e. C_i,j is satisfied by the pair (x,y)
• Assume succ(y,d) delivers the successor of y in domain d • (and nil if no successor exists)
When we revise constraint C_i,j, we look for support for x in d[j]• get the resumePoint for x in domain d[j] over the constraint• call this value y• if y is in d[j] then x is supported! Do nothing!• If y is not in d[j] then
• we search through the remainder of d[j] looking for support• we use a successor function to get next y in d[j]• if we find a y that satisfies the constraint set resumePoint[i,j,x] = y
[succ(x:integer,l:list) : integer -> if (l = nil) -1 else if (l[1] > x) l[1] else succ(x,cdr(l))]//// find the successor of x in ordered list l// NOTE: in the ac2001 algorithms we assume // we can do this in O(1)//
[existsY(d:array<set>,c:tuple,i:integer,j:integer,x:integer) : boolean -> let y := RP[i][j][x] in (if (y % d[j]) true else let supported := false in (y := succ(y,list!(d[j])), while (y > 0 & not(supported)) (if check(c,x,y) (RP[i][j][x] := y, supported := true) else y := succ(y,list!(d[j]))), supported))]//// find the first value y in d[j] that supports x in d[i]// if found return true otherwise false//
[revise(d:array<set>,c:tuple,i:integer,j:integer) : boolean -> let revised := false in (for x in copy(d[i]) (if not(existsY(d,c,i,j,x)) (delete(d[i],x), revised := true)), revised)]
In the two papers, the algorithm is shown to be optimal,for arbitrary binary csp’s
).( 2deO
• arc-consistency is at the heart of constraint programming• it is the inferencing step used inside search• it has to be efficient• data structures and algorithms are crucial to success
• ac is established possibly millions of times when solving• it has to be efficient
• we have had an optimal algorithm many times• ac4, ac6, ac7, ac2001
• ease of implementation is an issue• we like simple things
• but we might still resort to empirical study!
MAC
What’s that then
Maintain arc-consistency
• Instantiate a variable v[i] := x• impose unary constraint d[i] = {x}• make future problem ac• if domain wipe out
• backtrack and impose constraint d[i] x• make future ac
• and so on
Maintain arc-consistency
• why use instantiation?• Domain splitting?• resolve disjunctions first
• for example (V1 < V2 OR V2 < V1)
You now know enough to go out andbuild a reasonably efficient and useful CP toolkit
Now its time to start using a CP toolkit