Backtracking Technique
Eg. Big Castle – Large Rooms & “Sleeping Beauty”
• Systematic search - BFS, DFS• Many paths led to nothing but “dead-ends”
• Can we avoid these unnecessary labor? (bounding function or promising function)
• DFS with bounding function (promising function)• Worst-Case : Exhaustive Search
Backtracking Technique
• Useful because it is efficient for “many” large instances
• NP-Complete problems
Eg. 0-1 knapsack problem D.P. : O(min(2n, nW)) Backtracking : can be very efficient if good bounding function(promising function)
• Recursion
Outline of Backtracking Approach
• Backtracking : DFS of a tree except that nodes are visited if promising
• DFS(Depth First Search)
• Procedure d_f_s_tree(v: node)var u : node{ visit v; // some action // for each child u of v do d_f_s_tree(u);}
1
2
3 5 7 114
6
8 9 10
N-Queens Problem
N-Queens Problem
• nⅹn Chess board• n-Queens Eg. 8-Queens problem
, ,)(2
2
n
nn n ,! , nnn
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
N-Queens Problem: DFS
• State Space Tree for 4-Queens Problem
N-Queens Problem: DFS (Same Col/Row x)
• State Space Tree for 4-Queens Problem
X1=1
X2=2 3 4
43
4 3 4 2 3 2 3 2
2 4 32 3 4 1 4 1 3 2 4
1 3 4 1 2 4 321
2 3 4
(x1, x2, x3, x4)=(2, 4, 1, 3)
• #leaf nodes : n!=4!• DFS : “Backtrack” at dead ends – 4! leaf nodes (n!)Cf. Backtracking : “Backtrack” if non-promising
(pruning)
N-Queens Problem: Backtracking
• Promising Function(Bounding Function) (i) same row ⅹ (ii) same column ⅹ (col(i) ≠ col(k)) (iii) same diagonal ⅹ (|col(i)-col(k)|≠|i-k|)
Q (i,
col(i)) Q (k,
col(k))
Q (i,col(i))
Q (k,col(k))
N-Queens Problem: Backtracking
• State Space Tree for 4-Queens Problem
N-Queens Problem
• Better (faster) AlgorithmMonte Carlo Algorithm“place almost all queens randomly, then do remaining queens using backtracking.”
#Nodes CheckedDFS (nn)
#Nodes CheckedDFS
(same col/row X)(n!)
#Nodes CheckedBacktracking
34119,173,9619.73ⅹ1012
1.20ⅹ1016
2440,320
4.79ⅹ108
8.72ⅹ1010
6115,721
1.01ⅹ107
3.78ⅹ108
48
1214
n
Graph Coloring
• M-coloring problem:Color undirected graph with ≤m colors2 adjacent vertices : different colors
(Eg)V1 V2
V4 V3
2-coloring X
3-coloring O
Graph Coloring
• State-Space Tree : mn leaves• Promising function : check adjacent vertices for the same color
m
start
1 2 3
21 3
21 3
21 3
X
X
X
X X
Hamiltonian Circuits Problem
• TSP problem in chapter 3.– Brute-force: n!, (n-1)!– D.P. : – When n=20, B.F. 3,800 years D.P. 45 sec.– More efficient solution? See chapter 6
• Given an undirected or directed graph, find a tour(HC). (need not be S.P.)
32)2)(1( nnn
Hamiltonian Circuits Problem
• State-Space Tree(n-1)! Leaves : worst-case
• Promising function:1. i-th node (i+1)-th node2.(n-1)-th node 0-th node3. i-th node ≠ 0 ~ (i-1)-th node
V1 V2
V6V5
V3
V7
V4
V8
V1 V2
V5
V3
V4
HC : X
(Eg)
Sum-of-Subsets Problem
• A special case of 0/1-knapsack• S = {item1, item2,…, itemn}, w[1..n] = (w1, w2, …, wn) weights & W Find a subset A S s.t. ∑w⊆ i=W
(Eg) n=3, w[1..3]=(2,4,5), W=6 Solution : {w1,w2}
• NP-Complete• State Space Tree : 2n leaves
A
w1
w2
w3
w2
w3 w3 w3
O
O O
OOOO
{w1,w2}
Sum-of-Subsets Problem
• Assumption : weights are in sorted order• Promising function
ⅹ weight(up_to_that_node) + w i+1 > W (at i-th level) ⅹ weight(up_to_that_node) + total(remaining) < W(Eg) n=4, (w1,w2,w3,w4)=(3,4,5,6), W=13
ⅹ
12
7
3 97 8
13
4
0437
0
03
3 0
4
5
0 04
55
6
0 00
0ⅹ
ⅹ:
0+5+6=11<13
ⅹⅹ:
9+6=15>13
ⅹⅹ
w1=3
w2=4
w3=5
w4=6
0-1 Knapsack Problem
• w[1..n] = (w1, w2,…, wn) p[1..n] = (p1, p2,…, pn) W: Knapsack size• Determine A S maximizing⊆
• State-Space Tree
• 2n leaves
x1=1x2=
1x3=1
x4=1
00
0
00000
0000
001
11 1 1 0
11 1
11
x2=1
WwtspA
iA
i ..
Branch-and-Bound Backtracking
Non-optimization P. n-Queens, m-coloring…
Optimization Prob. 0/1 Knapsack
Compute promising fcn.
Branch & Bound Optimization Prob.- compute promising fcn.
Maximization Prob. → upper bound in each node
Minimization Prob. → lower bound in each node
BFS with B&B
Best-First-Search with B&B
Breadth First Search
procedure BFS(T : tree);{ initialize(Q);
v = root of T;visit v;enqueue(Q,v);while not empty(Q) do {
dequeue(Q,v);for each child u of v do { visit u;
enqueue(Q,u); } }}
1
2 3
4 5 6 7
8 9
1
2
5 6 7 8 9
3 4
10 11
12 13 14 15
BFS of a graph
BFS of a tree
0-1 Knapsack Problem
• BFS with B&B pruning (Eg) n=4, p[1…4]=(40,30,50,10), w[1…4]=(2,5,10,5),
W=1600
115
402
115
00
82
707
115
402
98
120170
707
80
801280
707
70
901298
402
50
100170
901290
305
82
801582
305
40
00
60
p1=40, w1=2
p2=30, w2=5
p3=50, w3=10
p4=10, w4=5
0-1 Knapsack Problem
– Bound = current profit + profit of remaining “fractional” K.S.
– Non-promising if bound ≤ max profit (or weight ≥W)(Eg) n=4, p[1…4]=(40,30,50,10), w[1…4]=(2,5,10,5), W=160
0115
402
115
00
82
707
115
402
98
120170
707
80
801280
707
70
901298
402
50
100170
901290
305
82
801582
305
40
00
60
p1=40, w1=2
p2=30, w2=5
p3=50, w3=10
p4=10, w4=5
0-1 Knapsack Problem
• Best First Search with B&B pruning
00
115
402
115
00
82
707
115
402
98
120170
707
80
901298
402
50
100170
p1=40, w1=2
p2=30, w2=5
p3=50, w3=10
p4=10, w4=5 901290