Constraint Programming: modelling Toby Walsh NICTA and UNSW
Feb 23, 2016
Constraint Programming: modelling
Toby WalshNICTA and UNSW
Golomb rulers
• Mark ticks on a ruler Distance between any two ticks (not just
neighbouring ticks) is distinct
• Applications in radio-astronomy, cystallography, … http://www.csplib.org/prob/prob006
Golomb rulers
• Simple solution Exponentially long ruler Ticks at 0,1,3,7,15,31,63,…
• Goal is to find minimal length rulers turn optimization problem into sequence of satisfaction
problemsIs there a ruler of length m?Is there a ruler of length m-1?….
Optimal Golomb rulers
• Known for up to 23 ticks• Distributed internet project to find large rulers
0,10,1,3
0,1,4,60,1,4,9,11
0,1,4,10,12,170,1,4,10,18,23,25
Solutions grow as approximately O(n^2)
Modelling the Golomb ruler
• Variable, Xi for each tick• Value is position on ruler
• Naïve model with quaternary constraints For all i>j,k>l>j |Xi-Xj| \= |Xk-Xl|
Problems with naïve model
• Large number of quaternary constraints O(n^4) constraints
• Looseness of quaternary constraints Many values satisfy |Xi-Xj| \= |Xk-Xl| Limited pruning
A better non-binary model
• Introduce auxiliary variables for inter-tick distances Dij = |Xi-Xj| O(n^2) ternary constraints
• Post single large non-binary constraint alldifferent([D11,D12,…]). Tighter constraints and denser constraint graph
Other modeling issues
• Symmetry A ruler can always be reversed! Break this symmetry by adding constraint:
D12 < Dn-1,n Also break symmetry on Xi
X1 < X2 < … Xn Such tricks important in many problems
Other modelling issues
• Additional (implied) constraints Don’t change set of solutions But may reduce search significantly
E.g. D12 < D13, D23 < D24, …E.g. D1k at least sum of first k integers
• Pure declarative specifications are not enough!
Solving issues
• Labeling strategies often very important Smallest domain often good idea Focuses on “hardest” part of problem
• Best strategy for Golomb ruler is instantiate variables in strict order Heuristics like fail-first (smallest domain) not
effective on this problem!
Experimental results
Runtime/sec Naïve model Alldifferent model
8-Find 2.0 0.1
8-Prove 12.0 10.2
9-Find 31.7 1.6
9-Prove 168 9.7
10-Find 657 24.3
10-Prove > 10^5 68.3
Something to try at home?
• Circular (or modular) Golomb rulers Inter-tick distance
variables more central, removing rotational symmetry?
• 2-d Golomb rulers
All examples of “graceful” graphs
Summary
• Modelling decisions: Auxiliary variables Implied constraints Symmetry breaking constraints
• More to constraints than just declarative problem specifications!
Case study 2: all interval series
All interval series
• Prob007 at www.csplib.org• Comes from musical composition
Traced back to Alban Berg Extensively used by Ernst Krenek
Op.170 “Quaestio temporis”
All interval series
• Take the 12 standard pitch classes c, c#, d, .. Represent them by numbers 0, .., 11
• Find a sequence so each occurs once Each difference occurs once
All interval series
• Can generalize to any n (not just 12)
Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct
• Finding one solution is easy
All interval series
• Can generalize to any n (not just 12)
Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct
• Finding one solution is easy[n,1,n-1,2,n-2,.., floor(n/2)+2,floor(n/2)-1,floor(n/2)+1,floor(n/2)]Giving the differences [n-1,n-2,..,2,1]
Challenge is to find all solutions!
Basic methodology
• Devise basic CSP model What are the variables? What are the
constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints
Basic CSP model
• What are the variables?
Basic CSP model
• What are the variables?Si = j if the ith note is j
• What are the constraints?
Basic CSP model
• What are the variables?Si = j if the ith note is j
• What are the constraints? Si in [0,n)
All-different([S1,S2,… Sn]) Forall i<i’ |Si+1 - Si| =/ |Si’+1 - Si’|
Will this model be any good? If so, why?If not, why not?
Basic methodology
• Devise basic CSP model What are the variables? What are the
constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints
Improving basic CSP model
• Is it worth introducing any auxiliary variables? Are there any loose or messy constraints we
could better (more compactly?) express via some auxiliary variables?
Improving basic CSP model
• Is it worth introducing any auxiliary variables? Yes, variables for the pairwise differences
Di = |Si+1 - Si|
• Now post single large all-different constraintDi in [1,n-1]All-different([D1,D2,…Dn-1])
Basic methodology
• Devise basic CSP model What are the variables? What are the
constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints
Break symmetry
• Does the problem have any symmetry?
Break symmetry
• Does the problem have any symmetry? Yes, we can reverse any sequence
S1, S2, … Sn is an all-inverse seriesSn, …, S2, S1 is also
• How do we eliminate this symmetry?
Break symmetry
• Does the problem have any symmetry? Yes, we can reverse any sequence
S1, S2, …, Sn is an all-inverse seriesSn, …, S2, S1 is also
• How do we eliminate this symmetry?• As with Golomb ruler!
D1 < Dn-1
Break symmetry
• Does the problem have any other symmetry?
Break symmetry
• Does the problem have any other symmetry? Yes, we can invert the numbers in any sequence
0, n-1, 1, n-2, … map x onto n-1-xn-1, 0, n-2, 1, …
• How do we eliminate this symmetry?
Break symmetry
• Does the problem have any other symmetry? Yes, we can invert the numbers in any sequence
0, n-1, 1, n-2, … map x onto n-1-xn-1, 0, n-2, 1, …
• How do we eliminate this symmetry?S1 < S2
Basic methodology
• Devise basic CSP model What are the variables? What are the
constraints?• Introduce auxiliary variables if needed• Consider dual or combined models• Break symmetry• Introduce implied constraints
Implied constraints
• Are there useful implied constraints to add?
Implied constraints
• Are there useful implied constraints to add? Hmm, unlike Golomb ruler, we only have
neighbouring differences So, no need to consider transitive closure
Implied constraints
• Are there useful implied constraints to add? Hmm, unlike Golomb ruler, we are not
optimizing So, no need to improve propagation for
optimization variable
Performance
• Basic model is poor• Refined model able to compute all solutions
up to n=14 or so GAC on all-different constraints very beneficial As is enforcing GAC on Di = |Si+1-Si|
This becomes too expensive for large nSo use just bounds consistency (BC) for larger n
Case study 3: orthogonal Latin squares
Modelling decisions
• Many different ways to model even simple problems
• Combining models can be effective Channel between models
• Need additional constraints Symmetry breaking Implied (but logically) redundant
Latin square
• Each colour appears once on each row
• Each colour appears once on each column
• Used in experimental design Six people Six one-week drug
trials
Orthogonal Latin squares
• Find a pair of Latin squares Every cell has a
different pair of elements
• Generalized form: Find a set of m Latin
squares Each possible pair is
orthogonal
Orthogonal Latin squares
1 2 3 4 1 2 3 42 1 4 3 3 4 1 23 4 1 2 4 3 2 14 3 2 1 2 1 4 3 11 22 33 44 23 14 41 32 34 43 12 21 42 31 24 13
• Two 4 by 4 Latin squares
• No pair is repeated
History of (orthogonal) Latin squares
• Introduced by Euler in 1783 Also called Graeco-Latin or Euler squares
• No orthogonal Latin square of order 2 There are only 2 (non)-isomorphic Latin
squares of order 2 and they are not orthogonal
History of (orthogonal) Latin squares
• Euler conjectured in 1783 that there are no orthogonal Latin squares of order 4n+2 Constructions exist for 4n and for 2n+1 Took till 1900 to show conjecture for n=1 Took till 1960 to show false for all n>1
• 6 by 6 problem also known as the 36 officer problem“… Can a delegation of six regiments, each of which sends a colonel,
a lieutenant-colonel, a major, a captain, a lieutenant, and a sub-lieutenant be arranged in a regular 6 by 6 array such that no row or column duplicates a rank or a regiment?”
More background
• Lam’s problem Existence of finite projective plane of order 10 Equivalent to set of 9 mutually orthogonal Latin
squares of order 10 In 1989, this was shown not to be possible after 2000
hours on a Cray (and some major maths)• Orthogonal Latin squares also used in
experimental design
A simple 0/1 model
• Suitable for integer programming Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise Avoiding advice never to use more than 3 subscripts!
• Constraints Each row contains one number in each square
Sum_jl Xijkl = 1 Sum_il Xijkl = 1 Each col contains one number in each square
Sum_jk Xijkl = 1 Sum_ik Xijkl = 1
A simple 0/1 model
• Additional constraints Every pair of numbers occurs exactly once
Sum_kl Xijkl = 1 Every cell contains exactly one pair of numbers
Sum_ij Xijkl = 1
Is there any symmetry?
Symmetry removal
• Important for solving CSPs Especially for proofs of optimality?
• Orthogonal Latin square has lots of symmetry Permute the rows Permute the cols Permute the numbers 1 to n in each square
• How can we eliminate such symmetry?
Symmetry removal
• Fix first row 11 22 33 …
• Fix first column112332..
• Eliminates all symmetry?
What about a CSP model?
• Exploit large finite domains possible in CSPs Reduce number of variables O(n^4) -> ?
• Exploit non-binary constraints Problem states that squares contain pairs that
are all different All-different is a non-binary constraint our
solvers can reason with efficiently
CSP model
• 2 sets of variables Skl = i if the 1st element in row k col l is i Tkl = j if the 2nd element in row k col l is j
• How do we specify all pairs are different? All distinct (k,l), (k’,l’) if Skl = i and Tkl = j then Sk’l’=/ i or Tk’l’ =/ j
O(n^4) loose constraints, little constraint propagation!
What can we do?
CSP model
• Introduce auxiliary variables Fewer constraints, O(n^2) Tightens constraint graph => more propagation Pkl = i*n + j if row k col l contains the pair i,j
• Constraints 2n all-different constraints on Skl, and on Tkl All-different constraint on Pkl Channelling constraint to link Pkl to Skl and
Tkl
CSP model v O/1 model
• CSP model 3n^2 variables Domains of size n, n
and n^2+n O(n^2) constraints Large and tight non-
binary constraints
• 0/1 model n^4 variables Domains of size 2 O(n^4) constraints Loose but linear
constraints• Use IP solver!
Solving choices for CSP model
• Variables to assign Skl and Tkl, or Pkl?
• Variable and value ordering• How to treat all-different constraint
GAC using Regin’s algorithm O(n^4) AC using the binary decomposition
Good choices for the CSP model
• Experience and small instances suggest: Assign the Skl and Tkl variables Choose variable to assign with Fail First
(smallest domain) heuristic• Break ties by alternating between Skl and Tkl
Use GAC on all-different constraints for Skl and Tkl
Use AC on binary decomposition of large all-different constraint on Pkl
Performance
n 0-1 modelFails t/sec
CSP model ACFails t/sec
CSP model GACFails t/sec
4 4 0.11 2 0.18 2 0.38
5 1950 4.05 295 1.39 190 1.55
6 ? ? 640235 657 442059 773
7* 20083 59.8 91687 51.1 57495 66.1
Case study 4: Langford’s problem
Langford’s problem
• Prob024 @ www.csplib.org
• Find a sequence of 8 numbers Each number [1,4]
occurs twice Two occurrences of i
are i numbers apart• Unique solution
41312432
Langford’s problem
• L(k,n) problem To find a sequence of k*n
numbers [1,n] Each of the k successive
occrrences of i are i apart We just saw L(2,4)
• Due to the mathematician Dudley Langford Watched his son build a
tower which solved L(2,3)
Langford’s problem
• L(2,3) and L(2,4) have unique solutions• L(2,4n) and L(2,4n-1) have solutions
L(2,4n-2) and L(2,4n-3) do not Computing all solutions of L(2,19) took 2.5 years!
• L(3,n) No solutions: 0<n<8, 10<n<17, 20, .. Solutions: 9,10,17,18,19, ..
A014552Sequence: 0,0,1,1,0,0,26,150,0,0,17792,108144,0,0,39809640,326721800,
0,0,256814891280,2636337861200
Basic model
• What are the variables?
Basic model
• What are the variables?Variable for each occurrence of a number
X11 is 1st occurrence of 1X21 is 1st occurrence of 2..X12 is 2nd occurrence of 1X22 is 2nd occurrence of 2..
• Value is position in the sequence
Basic model
• What are the constraints? Xij in [1,n*k] Xij+1 = i+Xij Alldifferent([X11,..Xn1,X12,..Xn2,..,X1k,..Xnk
])
Recipe
• Create a basic model Decide on the variables
• Introduce auxiliary variables For messy/loose constraints
• Consider dual, combined or 0/1 models
• Break symmetry• Add implied constraints• Customize solver
Variable, value ordering
Break symmetry
• Does the problem have any symmetry?
Break symmetry
• Does the problem have any symmetry? Of course, we can invert any sequence!
Break symmetry
• How do we break this symmetry?
Break symmetry
• How do we break this symmetry? Many possible ways For example, for L(3,9)
• Either X92 < 14 (2nd occurrence of 9 is in 1st half)• Or X92=14 and X82<14 (2nd occurrence of 8 is in
1st half)
Recipe
• Create a basic model Decide on the variables
• Introduce auxiliary variables For messy/loose constraints
• Consider dual, combined or 0/1 models
• Break symmetry• Add implied constraints• Customize solver
Variable, value ordering
What about dual model?
• Can we take a dual view?
What about dual model?
• Can we take a dual view?• Of course we can, it’s a permutation!
Dual model
• What are the variables? Variable for each position i
• What are the values?
Dual model
• What are the variables? Variable for each position i
• What are the values? If use the number at that position, we cannot
use an all-different constraint Each number occurs not once but k times
Dual model
• What are the variables? Variable for each position i
• What are the values? Solution 1: use values from [1,n*k] with the
value i*n+j standing for the ith occurrence of j Now want to find a permutation of these
numbers subject to the distance constraint
Dual model
• What are the variables? Variable for each position i
• What are the values? Solution 2: use as values the numbers [1,n] Each number occurs exactly k times Fortunately, there is a generalization of all-different
called the global cardinality constraint (gcc) for this
Global cardinality constraint
• Gcc([X1,..Xn],l,u) enforces values used by Xi to occur between l and u times All-different([X1,..Xn]) = Gcc([X1,..Xn],1,1)
• Regin’s algorithm enforces GAC on Gcc in O(n^2.d) Regin’s papers are tough to follow but this
seems to beat his algorithm for all-different!?
Dual model
• What are the constraints? Gcc([D1,…Dk*n],k,k) Distance constraints?
Dual model
• What are the constraints? Gcc([D1,…Dk*n],k,k) Distance constraints:
• Di=j then Di+j+1=j
Combined model
• Primal and dual variables• Channelling to link them
What do the channelling constraints look like?
Combined model
• Primal and dual variables• Channelling to link them
Xij=k implies Dk=i
Solving choices?
• Which variables to assign? Xij or Di
Solving choices?
• Which variables to assign? Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic? Fail First or Lex?
Solving choices?
• Which variables to assign? Xij or Di, doesn’t seem to matter
• Which variable ordering heuristic? Fail First very marginally better than Lex
Recipe
• Create a basic model Decide on the variables
• Introduce auxiliary variables For messy/loose constraints
• Consider dual, combined or 0/1 models
• Break symmetry• Add implied constraints• Customize solver
Variable, value ordering