Non-binary constraints Toby Walsh Dept of CS University of York England.

Non-binary constraints

Toby WalshDept of CSUniversity of YorkEngland


• BibliographyBacchus & van Beek, AAAI-98Chen, PhD thesis, UAlberta, 2000Dechter, AAAI-90Mohr & Masini, ECAI-88Regin, AAAI-94Stergiou & Walsh, AAAI-99 & IJCAI-99….


• Major contribution to competitiveness of CP– despite academic focus on binary

CSPs!

• Some classic examples: all-different, cardinality, ...


• Compact & declarative problem specifications– that often run with minimal “tweaking”

• Efficient algorithms for enforcing high levels of consistency– Regin’s specialized algorithm for a k-ary

all-different constraint– O(k^2d^2) compared compared to

O(d^k) for a generic algorithm


• Comparison with binary constraints:– binary models without additional vars

(binary decompositions)– binary models with additional hidden

vars (binary encodings)

• Theoretical and empirical results

Binary decompositions

• Certain non-binary constraints decompose into binary constraints on same vars

• Sometimes called “network decomposable”


• Two examples:– all-different(x1,x2,x3) = x1<> x2,

x1<>x3, x2<>x3– monotone(x1,x2,x3) = x1 < x2 < x3

• One non-example:– even(x1+x2+x3)


• Theoretical comparison direct– compare pruning of vars in binary

decomposition with that in non-binary

• Empirical experiments reinforce theory– decomposing non-binary constraints can

add orders of magnitude to solution cost

Binary consistencies

• Arc-consistency (AC)– any assignment can be extended to a

second

• Path-consistency (PC)– any pair of assignments can be extended to

a third

• Strong path-consistency (strong PC)– AC and PC


• Path-consistency rarely used in practice– time complexity

– adds new (binary) constraints

• Path-consistency can be achieved with a (generalized) arc-consistency algorithm– see later


• Between AC and PC– inverse and singleton consistencies– just prune values (no additional

binary constraints)

• Experiments suggest can be useful– good pruning/time


• Path inverse consistency (PIC)– any assignment can be extended to

two more vars

• Neighbourhood inverse consistency (NIC)– any assignment can be extended to

all vars in immediate neighbourhood


• Singleton arc-consistency (SAC)– after any assignment, problem can be made

AC

• Restricted path-consistency (RPC)– problem is AC, and if any assignment is

consistent with just a single value for a 2nd var then any 3rd var has a consistent value


• Simple relationships between them

strongPC > SAC > PIC > RPC > AC

NIC > PIC

NIC ~ SAC

NIC ~ strongPC

Non-binary consistencies

• Generalized arc-consistency (GAC)– any assignment can be extended to

all other vars in same constraint

• Efficient algorithms exist for GAC on certain non-binary constraints– e.g. Regin’s all-different algorithm


• GAC v AC

{1,2}

{1,2} {1,2}

x1

x2 x3

x1<>x2, x1<>x3, x2<>x3 ACall-different(x1,x2,x3) not GAC


• GAC can be used to enforce path-consistency– given a binary CSP– construct a ternary CSP’– CSP is PC iff CSP’ is GAC


• Given CSP has:– vars xi with values a,b,c,...– binary constraints R(a,b)

• Construct CSP’ with:– vars xij with values <a,b>,...

– ternary constraints R’(<a,b>,<b,c>,<c,a>) iff R(a,b) & R (b,a) & R(c,a)


• Forward checking – forward checking (FC) on binary

constraints enforces limited form of AC– most recently instantiated var and all

futures vars are made AC

• Choices about how to generalize this to non-binary constraints


• nFC0– any constraint with one un-

instantianted var made AC

• nFC1– apply one pass of AC to each

constraint and projection involving current var and one future var

• … nFC5


• Upper and lower bound on FCnFC1 on non-binary > FC on

decomposition > nFC0 on non-binary

• Gaps can be exponentialConsider n-ary all-different with n-1 valuesnFC1 takes (n-1) branchesFC on decomposition takes (n-1)! branches


• GAC lower boundGAC on non-binary > AC on decompositionGap again can be exponentialBut if we decompose too much, GAC=AC!

• GAC upper boundIn general, GAC ~ NIC, GAC ~ PIC ..BUT if decomposition to clique, NIC > GAC


• Tighter results provable for stricter classes

• Tree decomposable constraints– constraint graph is tree

• Triangle preserving constraints– non-binary constraints on all triangles


• Tree decomposable constraints– e.g. monotone(x1,x2,x3)– GAC=AC– not surprising as AC enough to solve!

• Decomposition here doesn’t lose us anything– but even one cycle is enough for GAC>AC


• Triangle preserving decompositione.g. all-different(x1,x2,x3), quasigroups, ...GAC > PIC, gap can again be exponentialGAC ~ SAC, strongPC

• PIC is very strong consistency to be achieving at each node– GAC can do even better than this!– decomposition carries a very large price


• Experimental results– quasigroup completion– quasigroup existence

• Quasigroup is a Latin square– completion is completing partially

filled square– existence is finding one with

additional properties


• Modelling the quasigroup problem– n^2 vars, each with domain of size n

• Non-binary model– 2n all-different constraints (one for

each row and column

• Binary decomposition– 2n cliques of not-equals constraints


• Quasigroup completion– Gomes & Selman report “heavy-tailed”

distributions

• Maintaining AC on binary decomposition– problems often take long time to solve

• Maintaining GAC on all-different– almost all problems trivial


• Quasigroup existence– best paper at IJCAI-93– of interest to design theory

• Open results first proved by computer– in some cases, only ever proved by

computer

• Maintaining GAC very competitive– compared to specialized model finders like

FINDER, SEM

Intermission

• Mid-point of talk

• Short commercial breakCSPLib: a benchmark library for

constraintshttp://csplib.cs.strath.ac.uk

CSPLib: a benchmark library for constraints

http://csplib.cs.strath.ac.uk

A Why build it?

A What it contains?

A How to add to it?

Why build CSPLib?

• Benchmarking– shift field from random to more

realistic problems

• Modelling• Show-case• Challenges

Why not build CSPLib?

• Over-fitting– CSPLib needs to be large & extensible

• Unrepresentative– CSPLib needs your “realistic” problems

• No standard modelling language– CSPLib uses natural language

• Hard work– CSPLib needs to be a “community” effort

What does CSPLib contain?

• Over 20 problem types in 5 categories– bin packing (e.g. vessel loading)– scheduling (e.g. basketball tournament)– frequency assignment (e.g. Golomb rulers)– combinatorics (e.g. perfect square)– puzzles (e.g. nonagrams)

• Other useful links– solvers, other benchmark libraries, ...

What does CSPLib contain?

• Example entry:– NL specification– results– references– (data) for

instances– code– anything else of

use

“A Golomb ruler is a set of m integers,0=a1<a2<..<am such that the m(m-1)/2differences, aj-ai distinct. Such a ruleris of length am. The objective is to find the smallest length ruler…”

How to add to CSPLib?

• Send us your entries. NOW!– it takes just 1/2 hour

• Everyone has at least one problem to contribute– get others to solve your open problems– a problem shared is a problem solved

• Email problems to [email protected]

End of intermission

Now back to ...

Binary encodings

Binary encodings

• Not all non-binary constraint decompose into binary constraints– on the same set of variables

• Consider again– even(x1+x2+x3)

• But binary CSPs NP-complete

Binary encodings

• Every non-binary constraint can be encoded into binary constraints– using polynomial number of

additional (hidden) variables

• Two popular encodings– hidden variable encoding– dual encoding

Binary encodings

• Hidden variable encoding– add hidden var for each non-binary

constraint

• Dual encoding– add hidden var for each non-binary

constraint– throw away original variables

Binary encodings

• Dual encoding– consider c1:even(x1+x2),

c2:odd(x2+x3){00,11}

{01,10}

c1

c2

R21 R21=<00,01> or <11,10>

Binary encodings

• Hidden variable encoding– consider c1:even(x1+x2),

c2:odd(x2+x3)

{0,1}

{00,11}

{01,10}

{0,1} {0,1}x1 x2 x3

c1

c2

r1

r1

r2

r2r1=<0*,0> or <1*,1>r2=<*0,0> or <*1,1>

Binary encodings

• Hidden var encoding -> dual encoding– compose relations, discard original

vars

{0,1}

{00,11}

{01,10}

{0,1}x1 x3

c1

c2

R21

R21 = r2 + r1

Binary encodings

• Double encoding– dual + hidden var encoding– original vars + hidden vars– all constraints of dual and of hidden

• Also called “combined encoding”

Binary encodings

• Theoretical analysis complicated by:– encoding “builds” in GAC for hidden

vars– must translate between (original and

hidden) vars– pruning in dual can infer large arity

nogoods in original

Binary encodings

• Hidden var encoding

FC on hidden ~ nFC0 on original

each can be exponentially better than the other

• FC+ propogates through hidden vars

FC+ on hidden = nFC1 on original

Binary encodings

• Hidden var encoding

AC on hidden = GAC on original

• Before looking for efficient (specialized) GAC algorithm– try AC on hidden var encoding

Binary encodings

• No point doing NIC on hidden var encodingNIC on hidden = AC on hiddendue to star shaped topology of constraint

graph

• Higher consistencies remain distinctstrongPC on hidden > SAC on hidden > NIC, AC on hidden

Binary encodings

• Dual encodingFC on dual ~ nFC0 on originaleach can be exponentially better than

the other

• Dual better for tight constraints– domains for hidden vars then small

Binary encodings

• Dual encodingAC on dual > GAC on original

• BUT domains of hidden vars very large when non-binary constraints loose– AC on dual prohibitively expensive

Binary encodings

• Dual v hidden encodingBT on dual = FC on hiddenAC on dual > AC on hiddenSAC on dual > SAC on hiddenstrongPC on dual = strongPC on hidden

• Path consistency is enough to get through the hidden star!

Binary encodings

• Experimental results– crossword puzzles– Golomb rulers– random CSPs– random 3-SAT

Binary encodings

• Crossword puzzles• “Original” model

– vars for letters, domains {A-Z}

• Dual model– vars for words, domains = dictionary

• Dual at least 1,000 times faster on larger problems

Binary encodings

• Golomb ruler– set of ticks, xi on a ruler– all inter-tick distances, dij different

• Ternary constraints, dij=xi-xj– encoded using hidden var/double

encoding

• Competitive with non-binary model– runs on any binary CSP solver!

Binary encodings

• Random CSPs and SAT– Bacchus and van Beek plot contours to

show constraint tightness where encoding pays

– gives good predictions for performance on structured problems like crosswords, …

– number of satisfying tuples in constraint is most important factor in such predictions

Conclusions

• Non-binary v binary decompositions– GAC on non-binary can be stronger

than PIC on decomposition

• Non-binary v binary encodings– GAC on non-binary = AC on hidden– AC on dual > GAC on non-binary

Conclusions

• Non-binary v binary decompositions– decomposition can add significantly

to search cost

• Non-binary v binary encodings– encoding pays in practice on tight

constraints

Future directions

• Modelling with non-binary constraints– many more models possible– role of auxiliary vars and implied

constraints

• Different levels of consistency within a non-binay model– e.g. GAC on all-different, but bounds

consistency on arithmetic constraints

Future directions

• Mixed models– e.g. dual encoding for only some of

the (non-binary) constraints

• Optimization– objective function typically non-binary

Non-binary constraints Toby Walsh Dept of CS University of York England.

Documents

binary csps

gac slide

strongpc slide

path consistency

later slide

pathconsistency pc

evenx1 x2 x3 slide

generic algorithm slide