Exponential time algorithms Algorithms and networks
Dec 21, 2015
Exponential time algorithms
Algorithms and networks
Exponential time algorithms2
Today
• Exponential time algorithms: introduction
• Techniques
• 3-coloring
• 4-coloring
• Coloring
• Maximum Independent Set
• TSP
Exponential time algorithms3
What to do if a problem is NP-complete?
• Solve on special cases
• Heuristics and approximations
• Algorithms that are fast on average
• Good exponential time algorithms
• …
Exponential time algorithms4
Exponential time algorithms5
Good exponential time algorithms
• Algorithms with a running time of cn.p(n)– c a constant– p() a polynomial– Notation: O*(cn)
• Smaller c helps a lot!
Exponential time algorithms6
Important techniques
• Dynamic programming
• Branch and reduce– Measure and conquer (and design)
• Divide and conquer
• Clever enumeration
• Local search
• Inclusion-exclusion
Exponential time algorithms7
Running example
• Graph coloring– Several applications: scheduling, frequency
assignment (usually more complex variants)– Different algorithms for small fixed number of
colors (3-coloring, 4-coloring, …) and arbitrary number of colors
– 2-coloring is easy in O(n+m) time
Exponential time algorithms8
3-coloring
• O*(3n) is trivial
• Can we do this faster?
Exponential time algorithms9
3-coloring in O*(2n) time
• G is 3-colorable, if and only if there is a set of vertices S with– S is independent– G[V-S] is 2-colorable
• Algorithm: enumerate all sets, and test these properties (2n tests of O(n+m) time each)
Exponential time algorithms10
3-coloring
• Lawler, 1976:– We may assume S is a maximal independent set
– Enumerating all maximal independent sets in O*(3n/3) = O*(1.4423n) time
• There are O*(3n/3) maximal independent sets (will be proved later.)
– Thus O*(1.4423n) time algorithm for 3-coloring
• Schiermeyer, 1994; O*(1.398n) time• Beigel, Eppstein, 1995: O*(1.3446n) time
Exponential time algorithms11
4-coloring in O*(2n) time
• Lawler, 1976
• G is 4-colorable, if and only if we can partition the vertices in two sets X and Y such that G[X] and G[Y] are both 2-colorable
• Enumerate all partitions– For each, check both halves in O(n+m) time
Exponential time algorithms12
4-coloring
• Using 3-coloring– Enumerate all maximal independent sets S– For each, check 3-colorability of G[V-S]– 1.4423n * 1.3446n = 1.939n
• Better: there is always a color with at least n/4 vertices – Enumerate all m.i.s. S with at least n/4 vertices– For each, check 3-colorability of G[V-S]– 1.4423n * 1.34463n/4 = 1.8009n
• Byskov, 2004: O*(1.7504n) time
Exponential time algorithms13
Coloring
• Arbitrary number of colors• With polynomial memory: Björklund, Husfeld,
2005• Divide and conquer approach• Set some constant r, 0 < r < 1• There is a color with at least rn vertices, or we can
divide the colors in two groups, such that for each group, there are between (1- r)/2 n and (1+r)/2 n vertices with a color in the group
Exponential time algorithms14
Algorithm
• Enumerate all sets S– If S is a m.i.s. with at least rn vertices, compute
chromatic number of G[V-S] recursively; add 1– If (1-r)/2 n |S| (1+r)/2 n then recursively
compute chromatic number of G[S] and recursively compute chromatic number of G[V-S], and add these numbers
• Output the best over all |S|.
Exponential time algorithms15
Setting r
• Using Maple or Mathematica for finding the value of r that gives the best solution
• r=0.19903 gives an algorithm using O*(5.283n) time– Analysis of number of m.i.s. of certain sizes,
etc.
• Other technique (2005): O*(2.4423n)
Exponential time algorithms16
Coloring with dynamic programming
• Lawler, 1976: using DP for solving graph coloring.
(G) = minS is m.i.s. in G 1+X(G[V-S])• Tabulate chromatic number of G[W] over
all subsets W– In increasing size– Using formula above– 2n * 1.4423n = 2.8868n
Exponential time algorithms17
Coloring
• Lawler 1976: 2.4423n (improved analysis)
• Eppstein, 2003: 2.4151n
• Byskov, 2004: 2.4023n – All using O*(2n) memory– Improvements on DP method
• Björklund, Husfeld, 2005: 2.3236n
• 2006 ….
Exponential time algorithms18
Inclusion-exclusion
• Björklund and Husfeld, 2006, and independently Koivisto, 2006
• O*(2n) time algorithm for coloring
• Expression: number of ways to cover all vertices with k independent sets
Exponential time algorithms19
First formula
• Let ck(G) be the number of ways we can cover all vertices in G with k independent sets, where the stable sets may be overlapping, or even the same– Sequences (V1,…,Vk) with the union of the Vi’s
= V, and each Vi independent
• Lemma: G is k-colorable, if and only if ck(G) > 0
Exponential time algorithms20
Counting independent sets
• Let s(X) be the number of independent sets that do not intersect X, i.e., the number of independent sets in G(V-X).
• We can compute all values s(X) in O*(2n) time. – s(X) = s(X {v}) + s(X {v} N(v)) for v X
• Count IS’s with v and IS’s without v
– Now use DP and store all values
– Polynomial space slower algorithm also possible, by computing s(X) each time again
Exponential time algorithms21
Expressing ck in s
• s(X)k counts the number of ways to pick k independent sets from V-X
• If a pick covers all vertices, it is counted in s()• If a pick does not cover all vertices, suppose it covers all
vertices in V-Y, then it is counted in all X that are a subset in Y– With a +1 if X is even, and a -1 if X is odd– Y has equally many even as odd subsets: total contribution is 0
VX
kXk XsGc )()1()( ||
Exponential time algorithms22
Explanations
• Consider the number of k-tuples (W(1), … , W(k)) with each W(i) an independent set in G
• If we count all these k-tuples, we count all colourings, but also some wrong k-tuples: those which avoid some vertices
• So, subtract from this number all k-tuples of independent sets that avoid a vertex v, for all v
• However, we now subtract too many, as k-tuples that avoid two or more vertices are subtracted twice
• So, add for all pairs {v,w}, the number of k-tuples that avoid both v and w
• But, then what happens to k-tuples that avoid 3 vertices???• Continue, and note that the parity tells if we add or subtract…• This gives the formula of the previous slide
Exponential time algorithms23
The algorithm
• Tabulate all s(X)
• Compute values ck(G) with the formula
• Take the smallest k for which ck(G) > 0
Exponential time algorithms24
Maximum independent set
• Branch and reduce algorithm (folklore)
• Uses:– Branching rule– Reduction rules
Exponential time algorithms25
Two simple reduction rules
• Reduction rule 1: if v has degree 0, put v in the solution set and recurse on G-v
• Reduction rule 2: if v has degree 1, then put v in the solution set. Suppose v has neighbor w. Recurse on G – {v,w}.– If v has degree 1, then there is always a
maximum independent set containing v
Exponential time algorithms26
Idea for branching rule
• Consider some vertex v with neighbors w1, w2, … , wd. Suppose S is a maximum independent set. One of the following cases must hold:
1. v S. Then w1, w2, … , wd are not in S.
2. For some i, 1 id, v, w1, w2, … , wi-1
are not in S and wi S. Also, no neighbor of wi is in S.
Exponential time algorithms27
Branching rule
• Take a vertex v of minimum degree.• Suppose v has neighbors w1, w2, …, wd.• Set best to 1 + what we get when we recurse on G – {v, w1,
w2, …, wd}. (Here we put v in the solution set.)• For i = 1 to d do
– Recurse on G – {v, w1, w2, …, wi} – N(wi). Say, it gives a solution of value x. (N(wi) is set of neighbors of wi. Here we put wi in S.)
– Set best = max (best, x+1).• Return best
Using some bookkeeping gives the corresponding set S
Using some bookkeeping gives the corresponding set S
Exponential time algorithms28
Analysis
• Say T(n) is the number of leaves in the search tree when we have a graph with n vertices.
• If v has degree d, then we have T(n) (d+1) T(n-d-1).– Note that each wi has degree d as v had minimum
degree, so we always recurse on a graph with at least d-1 fewer vertices.
• d > 1 (because of reduction rules).• With induction: T(n) 3n/3.• Total time is O*(3n/3) = O*(1.4423n).
Exponential time algorithms29
Number of maximal independent sets
• Suppose M(n) is maximum number of m.i.s.’s in graph with n vertices
• Choose v of minimum degree d.
• If v has degree 0: number of m.i.s.’s is at most 1* M(n)
• If v has degree 1: number of m.i.s.’s is at most 2* M(n-2)
• If v has degree d>1: number of m.i.s’s is at most (d+1)* M(n – d – 1)
• M(n) 3n/3 with induction
Exponential time algorithms30
Some remarks
• Can be done without reduction step
• Bound on number of m.i.s.’s sharp: consider a collection of triangles
Exponential time algorithms31
A faster algorithm
• Reduction rule 3: if all vertices of G have degree at most two, solve problem directly. (Easy in O(n+m) time.)
• New branching rule:– Take vertex v of maximum degree– Take best of two recursive steps:
• v not in solution: recurse of G – {v}
• v in solution: recurse on G – {v} – N(v); add 1.
Exponential time algorithms32
Analysis
• Time on graph with n vertices T(n).• We have T(n) T(n – 1) + T(n – 4) +
O(n+m)– As v has degree at least 3, we loose in the
second case at least 4 vertices
• Induction: T(n) = O*(1.3803n)– Solve (with e.g., Maple or Mathematica)
• x4 = x3 + 1
Exponential time algorithms33
Maximum Independent SetFinal remarks
• More detailed analysis gives better bounds• Current best known: O(1.1844n) (Robson, 2001)
– Extensive, computer generated case analysis!
– Includes memorization (DP)
• 2005: Fomin, Grandoni, Kratsch: the measure and conquer technique for better analysis of branch and reduce algorithms – Much simpler and only slightly slower compared to
Robson
Exponential time algorithms34
Analysis
• More complicated measure (measure and conquer)
• Very simplified example here:• Suppose algorithm costs p(n)* aq * br time if
we have a graph with q vertices of degree three and r vertices with degree at least four
• Suppose a b• Case analysis: …
Exponential time algorithms35
If largest degree is five or more
• If we do not select, r decreases by 1
• If we select, r decreases by at least 6
• For T(q,r) p(n)* aq * br to hold as induction hypothesis, we want:– aq * br aq * br-1 + aq * br-6 or
– b6 b5 + 1
Exponential time algorithms36
If largest degree is four (1)
• If we do not select the vertex, we loose one vertex of degree four, and for each of its neighbors, we get a vertex that gets a degree one smaller– If the degree was 2, it has degree one, and is removed,
and then its neighbor gets a degree one smaller
– So, 4 vertices of degree 3 or 4 get a degree one smaller
– If all neighbors have degree 4, then we get r decreases by 5, and q increases by 3
– Other cases similar
Exponential time algorithms37
If largest degree is four (2)
• If we select the vertex, then it, and its neighbors will be removed. The possible case is that these form a clique, and no other vertices are affected.– Here you see a possibility to improve the algorithm…
– If all neighbors have degree four, then we get that we want that aq * br aq+3 * br-4 + aq * br-4 which gives b4 a3 +1
– Other cases …
Exponential time algorithms38
If largest degree is 3
• Exercise set 8, and similar to previous cases:
• We get in both cases that q drops by four, and r stays the same (namely, 0).
• As condition, we get that a4 2
Exponential time algorithms39
Analysis
• Now, solve the set of equations
• Example: with Excel
• We get here as optimum 1.285n
Exponential time algorithms40
Polynomial space algorithm for coloring
• Using the current best polynomial space algorithm to compute s(X) for each X, we can get an algorithm that uses O*(2.2461n) time and polynomial space for coloring
Exponential time algorithms41
Remarks on measure and conquer
• More complicated manner to count number of subproblems
• For each degree d, we have a cost cd , e.g., vertices with degree 2 cost 0.7 (c2 = 0.7), etc.
• To an instance, associate the sum over all vertices of their cost (depending on their degree)
• From the rules, get a recurrence on these values, and bound it
• To select the costs that give the best bounds, a complicated problem has to be solved (quadratic programming)
• 2006, van Rooij: design by measure and conquer
Exponential time algorithms42
Dominating set
• Now: exact algorithm for Dominating Set– Given: Graph G=(V,E)
– Question: find a minimum size set S V, such that for all v V: v S or v has a neighbor in S
• Translate to Set Cover:– Given: Collection Z of subsets of V
– Question: find a minimum subcollection such that each v V is in at least one set of the subcollection
• Note: Z is set of sets
Exponential time algorithms43
Reduction rules for set cover: 1
• If S R, S and R both in Z, then there is an optimal solution without S– Replace S by R and we cover V with equally
many sets
• Rule 1: If S R, S and R both in Z, remove S from Z
Exponential time algorithms44
Reduction rules for set cover: 2
• If, for v V, there is only one set in the collection Z that contains v, say S, then any set cover must include S
• Rule 2: If, for v V, there is only one set in Z that contains v, say S, then – Make a new instance, where we set all R Z to
R-S ={w R | w S}– Solve new instance recursively, and add 1
Exponential time algorithms45
All sets of size 2
• If all sets have size at most 2, then the problem can be solved in polynomial time– After rules 1 and 2 cannot be applied, all sets
have size 2– The problem can be solved by maximum
matching• A maximum matching gives the largest number of
sets that cover two vertices each• All vertices not in the matching cost 1 entire set
Exponential time algorithms46
Algorithm
• while rule 1 or rule 2 can be applied do– Apply rule 1 or 2
• if all sets have size at most 2– Solve problem exactly by matching
• else– Take a set S from collection of maximum size– Recurse: put S in the solution (remove S from Z, set
each R Z to R-S, add 1 to result)– Recurse: remove S from Z (S not in solution)– Output the best of these two
Exponential time algorithms47
Simple analysis (1)
• Take as size: |V| + |Z|• Size starts at 2n• Rules 1 and 2, and all sets of degree 2 cost
polynomial time• Branching: two subproblems with sizes
decreased by 1 and by at least 4• Let T(n) be the time when size is n• T(n) T(n – 1) + T(n – 4)
Exponential time algorithms48
Solving T(n) = T(n – 1) + T(n – 4)
• cn = cn-1 + cn-4
• Divide by cn-4
• c4 = c3 + 1 has a unique positive solution: namely c = 1.3082…
• Use e.g., Mathematica, Maple or Excel• So T(n) 1.3083n
• Algorithm for Dominating set uses at most O*(1.30832n) O*(1.7116n) time
Exponential time algorithms49
More complicated cost measure (1)
• Idea: smaller sets are nicer than larger sets– Closer to reduction rules
– Similarly: elements that appear in few sets are better than elements that appear in many sets
• Frequency of element v: number of sets R Z with v R
• Write ni = number of sets in Z with size i
• Write mi = number of elements with frequency i
Exponential time algorithms50
More complicated cost measure (2)
• For some values vi and wj, that will be chosen in a clever way later, we take as cost measure
• The values will be between 0 and 1, and increasing (larger is worse)
• Elements of frequency 1 and sets of size 1 are all removed by rules 1 and 2
• Set v1 = w1 = 0
22 j
jji
ii mvnw
Exponential time algorithms51
Analysis• If the largest set S has size k, then
– Branch 1: choosing S in the solution, reduces nk by 1and k vertices are removed
• We gain wk for removing S• We gain k times a value vj (different js depending on frequency of
vertices in S, but each j>1)• Each v in S with frequency j means that j sets will have frequency one
smaller
– Branch 2: not choosing S in the solution, reduces nk by 1, and k vertices have frequency 1 smaller
• We gain wk for removing S• If one of these vertices had frequency 2, then it gets now frequency 1
and will be removed
Exponential time algorithms52
Details, details
• Different cases for frequencies and sizes give a large collection of recurrences
• To simplify, all values for 6 and larger are set to 1 (appears not to contribute anyhow to better analysis)
• For a choice of vis and wjs we get a collection of recurrences
• A program in Mathematica or Maple (or for simpler instances Excell) can give the values where these recurrences give the largest value
Exponential time algorithms53
(Near) Optimal values
• w2 = 0.3774
• w3 = 0.7548
• w4 = 0.9095
• w5 = 0.9764
• wk = 1 if k > 5
• v2 = 0.3996
• v3 = 0.7677
• v4 = 0.9300
• v5 = 0.9856
• vk = 1 if k > 5
Exponential time algorithms54
And then
• From the recurrences, we see that a cost n instance solves at most 1.236n subproblems
• Initial cost is at most 2n (n elements, and n sets, each of cost at most 1)
• So, total time of this algorithm for Dominating Set is bounded by O*(1.2362n ) = O*(1.5263n)
Exponential time algorithms55
Improvement
• van Rooij, 2006• Design by measure and conquer method:
– Analyze which of the recurrences is sharp– Look at that case, and try to build a new rule
that reduces that case
• For DS: one addition rule gives O*(1.5134n) and, using DP and storing some subproblems O*(1.5086n)
Exponential time algorithms56
Held-Karp algorithm for TSP
• O*(2n) algorithm for TSP• Uses Dynamic programming• Take some starting vertex s• For set of vertices R (s R), vertex w R, let
– B(R,w) = minimum length of a path, that• Starts in s
• Visits all vertices in R (and no other vertices)
• Ends in w
Exponential time algorithms57
TSP: Recursive formulation
• B({s},s) = 0
• If |S| > 1, then– B(S,w) = minv S – {w}B(S-{w}, v}
• If we have all B(V,v) then we can solve TSP.
• Gives requested algorithm using DP-techniques.
Exponential time algorithms58
Exponential time algorithms59
Final remarks
• Techniques for designing exponential time algorithms
• Other techniques, e.g., local search
• Combination of techniques
• Several interesting open problems