Path Consistency for 1- General CSPs 2- STPs
Peter SchletteWesley BothamCSCE990 Advanced CP, Fall 2009
Outline
04/22/23
2
General CSPs Review of Path Consistency & PC
Algorithms Path Consistency Algorithms
PC-1, PC-2, DPC, PPC, PC-8, PC-2001 STPs
Review of Triangulated Graphs Path Consistency on STPs
Floyd-Warshall, Bellman-Ford, STP, P3C, Prop-STP
Path Consistency: Properties
04/22/23
3
A CSP is path consistent iff it is strongly 3-consistent [van Beek & Dechter, JACM95] Domains are filtered by arc consistency Consistent solutions over 2 variables can be
extended to every 3rd variable PC algorithms typically iterate over triplets of
variables End variables in triplets need not be distinct In STP, variables domains are not relevant, thus PC
algorithms on STPs enforce only 3-consistency A given algorithm
Must determine that a CSP is path consistent or not May or may not filter the constraints as much as
possible (e.g., DPC)
List of Algorithms Discussed
PC-1 [Mackworth 77, , Dechter Fig 3.10]
PC-2 [Mackworth 77, Dechter Fig 3.11]
DPC [Dechter & Pearl 89, Dechter Fig. 4.9]
PPC [Bliek & Sam-Haroud 99]
PC-8 [Chmeiss & Jégou 98]
PC-2001 [Bessière et al. 05]
04/22/23
4
Path Consistency: Algorithms
04/22/23
5
They all stop when a relation/domain is empty; omitted for clarity
Queue Does is it have one? Edges (e.g., PPC) Triplets of variables (e.g., PC-2) Tuples of ‘vv-pair, variable’ (e.g., PC-8 & PC-2001)
Properties Determines strong 3-consistency? What is the time and space complexity?
Requires additional data structures to remember supports? Graph: Complete? Chordal?
What is the practical performance (i.e., phase transition)?
PC-1 [Mackworth 77]
04/22/23
6
1 Repeat until quiescence2 For i,j,k variables3 Rij Rij Rik Rkj
Has 4 nested loops, iterates over vertices, needs no queue
Updates every edge & every domain (i=j) Uses composition and intersection Determines strong 3-consistency (when i=j) Time complexity is O(n5d5)
One sweep costs O(n3d3) Number of sweeps O(n2d2)
Space: no queue, no additional data structure, complete graph
PC-2 [Mackworth 77]
04/22/23
7
Has 1 loop over a queue of triplets of variables When an edge (domain) is updated, only triplets with an ‘external’ 3rd
node are added to queue Allows i=j, thus determines strong 3-consistency.
Dechter Fig 3.11 specifies i<j in which case domains are not updated Theoretically & practically faster than PC-1 (queue) Time complexity is O(n3d5) Space: Queue size is O(n3), no additional data structures,
complete graph
1 Q (i,j,k) | i j, i k, j k 2 While Q is not empty3 For (i,j,k) from Q4 Rij Rij Rik Rjk
5 If Rij changed, Q Q U (m,i,j), (m,j,i) | m i, m j
mm jik
n=16, a=16, d=30%
PC-1 vs. PC-2[Botham & Schlette]
04/22/23
8
PC-2
PC-1
PPC [Bliek & Sam-Haroud 99]
04/22/23
9
Q ← E
Until Q is empty do
edge ← DEQUEUE(Q)
for every triplet i,j,k related to edge
Rij ← Rij ∩ (Rik Rkj)
if Rij was changed then EnQueue((i,j), Q)
First triangulates the graph Keeps Q, a queue of edges For an edge in Q
Pops edge from Q, retrieves all triplets where edge appears In each triplet, updates each edge Each updated edge is added to Q Does not specify whether or not domains are filtered (some
should be for soundness)
PPC [Bliek & Sam-Haroud 99]
04/22/23
10
Triangulated graph is usually sparser than complete graph
For triplet (i,j,k) allows i=j, algorithm is sound (not clear in paper)
Enforces Strong path-consistency Weaker filtering than PC-2
Time: O(ed2), degree of graph Space: queue O(e) for storing triplets, no additional data
structure, chordal graph Weakness: if 2 or more edges of a given triplet are in Q
All three edges are updated once for each edge May do redundant work (fixed in STP)
i
vn
j
v2
v1
1 For k=n downto 1do2 For i =1 to k REVISE(i, Rik)3 For i,jk i,k connected & j,k connected4 Rij Rij Rik Rjk
k-2k
k-4
k-6
DPC [Dechter Fig 4.9, Dechter & Pearl 89]
Given an ordering for the variables From bottom to top, enforces directional arc-consistency (DAC) From bottom to top, for every variable, updates the edge
between every two of its parents Properties
Moralizes the graph, determines strong directional path consistency relative to ordering
Time O(min(t.d3,n3d3)) Space: No queue, no additional data structures, chordal graph
04/22/23
11
DPC: Constraint revision [Dechter Fig 3.9]
1 For each (a,b) Rij
2 If no c Dk is s.t. (a,c)Rik & (a,b)Rjk
3 Remove (a,b) from Rij
04/22/23
12
k
j
i
Does not operate on matrices in reality (,)
Iterates over Tuples in constraints (i.e., (a,b) Rij)
and Values in domain (i.e., c Dk)
Not yet tested against PC-1, PC-2, PPC
PC-8 [Chmeiss & Jégou 98]
PC-8
INITIALIZE
While Q do
POP((i,a),k), Q)
PROPAGATE((i,a),k)UPDATE
if WITHOUTSUPPORT((i,a),(j,b),k)
REMOVE (a,b) from Rij, (b,a) from Rij
Q ← Q U ((i,a,)j),((j,b),i)
04/22/23
13
INITIALIZEQ ← for i,j,k=1 to n (i<j,k≠i,k≠j) for (a,b)Rij
UPDATE((i,a),(j,b),k)
PROPAGATE for j=1 to n (j≠i,j≠k) for bDj and (a,b)Rij
UPDATE((i,a),(j,b),k)
PC-8 Analysis[Chmeiss & Jégou 98]
Queue: a list of (vvp, var) = ((var,val),var) Determines strong PC-property (if you allow i=j) Achieves ‘full’ filtering Time complexity
INITIALIZATION: O(n3d3) PROPAGATE is called O(n2d2) times, each call costs
O(nd2) PC-8: INITIALIZATION + (n2d2) PROPAGATE = O(n3d4)
Space Queue O(n2d), data structure Status-PC: O(n2d) Graph is complete
04/22/23
14
PC-8 vs. PC-2[Botham & Schlette]
04/22/23
15
PC-2
PC-8
n=16, a=16, d=30%
04/22/23
16
PC-2001 [Bessière+ 05]
PC-20011 INITIALIZE(Q)2 While Q 3 POP((i,a),k) from Q4 REVISEPATH((i,a),j)
INITIALIZE(Q)1 For i,j,k variables2 For each (a,b) Rij 3 If (a,b) has no support c in Dk
4 Remove (a,b) from Rij
5 Q Q U ((i,a),j), ((j,b),i)6 Else7 Last((i,a),(j,b),k) the first support of (a,b) Dk
04/22/23
17
PC-2001 [Bessière+ 05]
REVISEPATH((i,a),k,Q)1 For each j k2 For each b Dj | (a,b) Rij
3 support Last((i,a),(j,b),k)4 While support is nil or was deleted5 support next value in Dk
6 If no supports exist7 Remove (a,b) from Rij
8 Q Q U ((i,a),j), ((j,b),i) 9 Else10 Last((i,a),(j,b),k) support
Records supporting values to improve time complexity (at the cost of space overhead)
PC-2001 Analysis [Bessière+ 05] Queue: same as PC-8, list of (vvp,var) Achieves the same properties as PC-1, PC-2, PC-8 Time: O(n3d3) Space
Queue: O(n2d) Data structure: Last structure dominates O(n3d2) Graph complete
Compared to PC-8, PC-2001 Is easier to understand and implement Has lower time complexity Is faster in general in experiments Has worse space complexity
04/22/23
18
PC-2001 vs. PC-2, PC-8[Botham & Schlette]
04/22/23
19
PC-8
PC-2001
PC-2
n=16, a=16, d=30%
04/22/23
20
QueueDataS
trTime
Graph
PC-p?
Filtering Empirically
PC-1 None None O(n5d5)Comple
teYes Full
PC-2TripletsO(n3)
None O(n3d5)Comple
te Yes FullBetter than PC-1 in all cases
PPCEdgesO(e)
None O(ed2)Chorda
lYes Partial
Advantageous on sparse graphs
DPC None NoneO(min(td3,n3d3)
Chordal
No‘weak’ partial
Not evaluated yet, but likely best
PC-8(vvp,var
)* O(n2d)
StatusO(n2d)
O(n3d4)Comple
teYes Full
Significantly better than PC-2 around phase transition, ambiguous otherwise
PC-2001
(vvp,var)*
O(n2d)
LastO(n3d3)
O(n3d3)Comple
teYes Full
Better than PC-8 in nearly all cases
Summary of PC Algorithms for General CSPs
Comparisons: CPU time, #CC for preprocessing
Outline
04/22/23
21
General CSPs Review of Path Consistency & PC
Algorithms Path Consistency Algorithms
PC-1, PC-2, DPC, PPC, PC-8, PC-2001 STPs
Review of Triangulated Graphs Path Consistency on STPs
Floyd-Warshall, Bellman-Ford, STP, P3C, Prop-STP
Triangulated Graphs: Motivation [Bliek & Sam-Haroud 99] showed that PPC
Operates on triangulated graphs Determines the property of strong path consistency When constraints are convex, PPC also yields minimal CSP
[Xu & Choueiry 03] studied STP In STP constraints are convex Proposed STP, which adapts PPC to STPs w/o updating
domains (‘weak’ path consistency) May do fewer updates than PPC: queue of edges versus queue
of triangles
[Planken et al. 08] studied STP Showed that STP is O(t2), t is the number of triangles Proposed P3C, for STP, that is O(t)
04/22/23
22
Vertex elimination
Vertex elimination operation: When removing a vertex, connect all neighbors if they are not already connected
Fill edges: are the edges added when eliminating a vertex
Simplicial vertex Vertex whose neighbors are all connected (form a
clique) Eliminating a simplicial vertex does not add any edges
Perfect elimination ordering: There is always a simplicial vertex to be eliminated. All nodes can be eliminated w/o adding any fill edges
04/22/23
23
A graph is triangulated iff it has a perfect elimination order
The width of the triangulated graph is equal to the size of its largest clique -1. Why?
Finding the width is tractable, thus max. clique on triangulated graph is tractable (usually, NP-hard)
Using the reverse of the perfect elimination ordering of a triangulated graph yields A moralized graph The induced width of this ordering is equal to the
width of the triangulated graph, why? Moralizing an arbitrary ordering of a graph yields a
triangulated graph. Why?
Triangulated Graphs
04/22/23
24
Outline
04/22/23
25
General CSPs Review of Path Consistency & PC Algorithms Path Consistency Algorithms
PC-1, PC-2, DPC, PPC, PC-8, PC-2001 STPs
Review of Triangulated Graphs Path Consistency on STPs
Floyd-Warshall, Bellman-Ford, STP, Prop-STP, P3C
(Prop-STP is not discussed for lack of time)
Floyd-Warshall for STP [CLR]
Basic STP solver, three nested loops Initialization: builds the distance graph
Rij = [a,b] gives eijb and eji-a
When edge does not exist, add infinite distance (complete graph) Time (n3), Space: No queue but O(n2) new edges
04/22/23
26
FLOYDWARSHALLFor k 1 to n For i 1 to n For j 1 to n w(eij) MIN(w(eij),w(eik)+w(ekj))
d[s] 0for each vertex i other than source (i s) d[i]
Repeat n-1 times for each edge eij if d[i] + w(eij) < d[j] then d[j] d[i] + w(eij)
for each edge eij if d[i] + w(eij) < d[j] then return inconsistent
04/22/23
27
Bellman-Ford for STP [CLR]
Time: O(en), Space: No queue but O(n) new edges
Detects path consistency Edges are not guaranteed minimal
PPC Operates on triangulated graphs ‘Fully’ filters convex constraints
∆STP adapts & refines PPC to STP Keeps a queue of triangles (vs. a queue of edges) Pops a triangle from queue & updates all 3 edges Implicitly separate graph in biconnected
components Enqueues triangles adjacent to only the updated
edge Best performance when queue is FIFO
04/22/23
28
∆STP [Xu & Choueiry 03]
04/22/23
29
Performance on STPs: F-W, PPC, STP [Xu & Choueiry 03]
04/22/23
30
Constraint ChecksGenSTP with 50 nodes
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Density
Co
ns
tra
ints
Ch
ec
ks
BF+APDPC+AP STP
BF+AP
DPC+AP
STP
Performance on STPs: BF, DPC, STP [Shi+ 05]
04/22/23
31
Designers of ∆STP became aware of relevance of simplicial ordering in ∆STP in 2005 (ref. Nic Wilson)
Designers of Prop-STP exploited the idea The authors of P3C formalize the flaw of ∆STP Identified a pathological case where ∆STP does
unnecessary work (not useful filtering) Characterized it as set of problems where ∆STP runs in
Ω(t2), where t is the number of triangles P3C addresses flaw by using a simplicial ordering Proves that propagation can be achieved in (t)
P3C: The Idea [Planken+ 08]
P3C: Pathological Case[Planken+ 08]
ci→i+1 | 0 ≤ i ≤ t+1 with zero weight
ci→j | (1 ≤ i ≤ j−2 < t) ∧ i+j−t ∈ 1,2 with weight j−i−1
cj→i | (1 ≤ i ≤ j−2 < t) ∧ i+j−t ∈ 1,2 with weight t−(j−i−1)
04/22/23
32
Path. Case: Empirically[Planken+ 08]
04/22/23
33
cubi
cquadratic
linear
P3C: The Algorithm[Planken+ 08]
Given: A triangulated graph & a perfect elimination order
04/22/23
34
k
j
i
k
j
i
The algorithm has two steps Bottom up: For every node
Considers every pair of parents Updates the edge between parents
(ref. DPC) Top down: For every node
Considers every pair of parents Updates edges adjacent to node
1
2
P3C: Bottom up [Planken+ 08]
DPC: For each node update the edges between all parents given the edges with the node
04/22/23
35
P3C: Top Down [Planken+ 08]
Update the edges between every node and its parent
04/22/23
36
P3C: Sound & complete[Planken+ 08]
Claim: on iteration k of P3C’s second half, all edges in the subgraph consisting of Vi | i ≤ k are minimal
Base case: k = 2 We know that c1,2 (in
orange) must be minimal if it exists, due to DPC; the subgraph will always be minimal for k = 2
04/22/23
37
k=2
1
2
1
k=3
Induction hypothesis: Gk-1=Vi≤k-1 forms a minimal subgraph
Induction step: Gk-1 minimalGk minimal
In the kth iteration, wik=min(wik, wij+ wjk) Any part of a theoretical shorter path that extends
out of the Gk-1can be replaced by its two endpoints within Gk-1, due to the filtering from DPC
The only way wik could be non-minimal is if there were a shorter path wij+wjk, but this path was checked in the kth iteration
We have a base step and an inductive step, so the proof is complete! 04/22/23
38
P3C: Sound & complete[Planken+ 08]
k
i
j
Gk
k-1
Gk-1
1
Time Complexity [Planken+ 08]
P3C runs in Θ(t), t is the number of triangles
O(t) ⊆ O(nw*2), w* is the min. induced width
O(nw*2) ⊆ O(nδ2), where δ is max. degree
O(nδ2) ⊆ O(n3)
DPC visits each triangle exactly once
The second half of P3C visits each triangle exactly once
Thus we have a constant number of visits to each triangle and a linear time complexity in the number of triangles
04/22/23
39
Jobshop (enforced consistency) [Planken+ 08]
04/22/23
40
In spite of theoretical bounds, the two algorithms are quite close
Summary of STP Algorithms
GraphPC-p?
Minimality?
TimeQueu
eExperiments
FWComple
teYes Yes Θ(n3) None Worst
BF Partial Yes No O(en) None Better than FW
PPCChorda
lYes Yes O(ed2) O(e)
DPCChorda
lYes No (t) None
Low density: less good than STP, high density same as STP
STP
Chordal
Yes YesO(min(t2, ed2)) O(e)
Faster than PPC, better than DPC on low densities
P3CChorda
lYes Yes (t) None Best reported,
04/22/23
41
Future Work [Planken+ 08]
Extend P3C to general CSPs
Investigate efficiency of triangulation algorithms vs. P3C
Incremental P3C solver for STPs
04/22/23
42
References
CSP PC-1, PC-2: see [Mackworth, AIJ 1977] PPC: see [Bliek & Sam-Haroud, IJCAI 1999] DPC: see [Dechter 4.2.2, Dechter & Pearl, AIJ 1987] PC-8: see [Chmeiss & Jégou, IJAITools 1998] PC-2001: see [Bessière et. Al, AIJ 2005]
STP Floyd-Warshall, Bellman-Ford: see CLR textbook DPC: see [Dechter et al., AIJ 1991] ∆STP: see [Xu and Choueiry, TIME 2003] Prop-STP: see [Bui, Tyson, and Yorke-Smith, AAAI 07,
Workshop] P3C: see [Planken et al., ICAPS 2008]
04/22/23
43