1 NTUEE Algorithms Algorithms 演算法 Professor Chien-Mo James Li 李建模 Graduate Institute of Electronics Engineering National Taiwan University NP-Completeness and Approximation Algorithms 2 NTUEE Algorithms Outline • NP-completeness, CH34 ♦ polynomial time verification ♦ NP-completeness and reducibility ♦ NP-completeness proofs ♦ NP-complete problems • Approximation Algorithms, CH35
30
Embed
NP-Completeness and Approximation Algorithmsb97032/LN11_NPC.pdf · ♦ NP-complete problems • Approximation Algorithms, CH35 Algorithms NTUEE 22 How to Prove NP-Completeness? •
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1NTUEEAlgorithms
Algorithms演算法
Professor Chien-Mo James Li 李建模
Graduate Institute of Electronics EngineeringNational Taiwan University
NP-Completeness andApproximation Algorithms
2NTUEEAlgorithms
Outline• NP-completeness, CH34
♦ polynomial time verification
♦ NP-completeness and reducibility
♦ NP-completeness proofs
♦ NP-complete problems
• Approximation Algorithms, CH35
3NTUEEAlgorithms
Stephen Cook (1939~, USA)• Stephen Arthur Cook is a renowned American-
Canadian computer scientist and mathematician who has made major contributions to the fields of complexity theory and proof complexity.
• He is currently a University Professor at the University of Toronto, Department of Computer Science and Department of Mathematics.
• Cook formalized the notion of NP-completeness in his seminal 1971 paper "The Complexity of Theorem Proving Procedures", where he proved the existence of an NP-complete problem by showing that the satisfiability problem is NP-complete
• In 1982, Cook received the prestigious Turing award for his contributions to complexity theory.
4NTUEEAlgorithms
Decision Problems vs. Optimization Problems• Decision problems: find answers to yes/no questions
♦ MST: Given a graph G=(V, E) and a bound K, is there a spanning tree with a cost at most K?
♦ TSP: Given cities, distance between each pair of cities, and a bound B, is there a route that starts and ends at a given city, visits every city exactly once, and has total distance at most B ?
• Optimization problems: find a legal answer to minimize cost
♦ MST: Given a graph G=(V, E), find the cost of a minimum spanning tree of G
♦ TSP: Given a set of cities and that distance between each pair of cities, find the distance of a “minimum route” starts and ends at a given city and visits every city exactly once
• Could apply binary search on decision problems to obtain solutions to optimization problems
♦ Assume a cost C, decide if solution exist
♦ If no, increase cost; if yes, reduce cost
♦ Repeat until we get satisfactory answers
5NTUEEAlgorithms
Hamiltonian Cycles (HC)• Hamiltonian cycle: a simple cycle that contains every vertex in an
undirected graph
• Hamiltonian-cycle problem: given a graph G, does it have a Hamiltonian cycle?
♦ Decision problem
• Example: Fig 34.2 (a) yes (b) no
6NTUEEAlgorithms
Complexity Classes P, NP• Complexity class P: the set of problems that can be solved in
polynomial time in the size of input
♦ O(nk) for some constant k
∗ Polynomial time: O(n2), O(n3), O(1), O(n lg n)
− Problems in P are considered tractable
∗ Not polynomial time: O(2n), O(nn), O(n!)
− considered intractable
♦ Closed under addition, multiplication, composition, complement..
• Complexity class NP (Nondeterministic Polynomial): the set of problems that can be verified in polynomial time in the size of input
♦ we know P ⊆ NP, but P = NP ?
Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correct. If a solution exists, computer always guesses it.One way to imagine it: a parallel computer that can freely spawnan infinite number of processes.
7NTUEEAlgorithms
HC ∈ NP• Verification algorithm: a 2-argument algorithm A, where one
argument is an input x and the other is y (called a certificate). Averifies x if there exists y s.t. A answers “yes”
• Example: Is HC ∈ NP?
♦ Given a list of vertices (certificate)
♦ Check if the tour visits every vertex exactly once
♦ Check whether each edge along the cycle actually exists
♦ Check if the first vertex = last vertex
♦ All can be done in O(n) time, so HC ∈ NP
8NTUEEAlgorithms
The P = NP Question• P = NP? is one of biggest open questions in computer science
♦ Most theorists believe P ≠ NP
• Complexity class NP-complete (NPC):
♦ The set of problems that can be verified in polynomial time
∗ but not sure if can be solved in polynomial time
♦ NPC is at crux of P = NP question
♦ If any NPC problem can be solved in polynomial time,
∗ then all problems in NP can be solved in polynomial time
• a US$1 million reward to anyone who has a formal proof
P=NP=NPC P NPC
NP
not likely more likely
9NTUEEAlgorithms
Complexity Class co-NP• Is class NP closed under complement?
• Class co-NP problems = whose complement problems are in NP
• Example: HC-Complement
♦ Given a set of vertices and a graph, do these vertices NOT form a Hamiltonian cycle?
♦ HC-Complement ∈ NP, therefore, HC ∈ co-NP
• Fig 34.3 Four possibilities
not likely
10NTUEEAlgorithms
Outline• NP-completeness, CH34
♦ polynomial time verification
♦ NP-completeness and reducibility
♦ NP-completeness proofs
♦ NP-complete problems
• Approximation Algorithms, CH35
11NTUEEAlgorithms
Polynomial-time Reduction• Let L1 and L2 be two decision problems
♦ Suppose algorithm A2 can solve L2. Can we use A2 to solve L1?
• Polynomial-time reduction from L1 to L2: L1 ≤P L2
♦ Function f reduces input for L1 into an input for L2
∗ s.t. reduced input is a “yes” input for L2 iff the original input is a “yes” input for L1
∗ f is computable in polynomial time
♦ L1 is no more than a polynomial factor harder than L2
yesyes
no no
12NTUEEAlgorithms
Lemma 34.3• (Lemma 34.3) If L1 ≤P L2 then L2 ∈ P implies L1 ∈ P
• Fig 34.5
♦ F is a reduction algorithm that computes the reduction function ffrom L1 to L2 in polynomial time
♦ A2 is a polynomial time algorithm that decides L2
♦ A1 decides whether x∈L1 by using F to transform input x into f(x)
13NTUEEAlgorithms
FFT • it takes more time to compute L1 than L2
• why L1 ≤P L2 ? not L1 ≥P L2 ?
14NTUEEAlgorithms
Example: HC ≤P TSP (step 1)• Define a reduction function f for HC ≤P TSP
• Given an arbitrary HC instance G = (V, E) with n vertices
․ Create a set of n cities labeled with names in V
․ Assign distance between u and v
․ Set bound B = n
․ f can be computed in O(V2) time.
15NTUEEAlgorithms
HC ≤P TSP (Step 2)• G has an HC iff the reduced instance has a TSP with distance ≤ B
♦ If x is a solution to HC , then f(x) must be a solution to TSP
∗ Suppose the HC is x = <v1, v2, …, vn, v1>. Then, x is also a tour in the transformed TSP instance
∗ The distance of the tour h is n = B since there are nconsecutive edges in E, and so has distance 1 in f(x)
∗ Thus, f(x) is a solution to TSP
− f(x) has a TSP tour with distance ≤ B
16NTUEEAlgorithms
HC ≤P TSP (step 2, cont’d)• G has an HC iff the reduced instance has a TSP with distance ≤ B
♦ If f(x) is a solution to TSP, then x must be a solution to HC
∗ Suppose there is a TSP tour with distance ≤ n = B
− Let it be <v1, v2, …, vn, v1>
∗ Since distance of the tour ≤ n and there are n edges in the TSP tour, the tour contains only edges in E
∗ Thus, <v1, v2, …, vn, v1> is a Hamiltonian cycle
17NTUEEAlgorithms
NP-Completeness• A decision problem L is NPC if both conditions are true
1. L ∈ NP
2. L' ≤ P L for every L' ∈ NP
• (Theorem 34.4) Suppose L ∈ NPC
♦ If L ∈ P, then there exists a polynomial-time algorithm for every L' ∈ NP (i.e., P = NP)
♦ If L ∉ P, then there exists no polynomial-time algorithm for any L' ∈ NPC (i.e., P ≠ NP)
• NPC is at crux of deciding whether P = NP
NP-completeproblems
intractabletractable??
A problem is called NP-hard if only condition 2 is true
18NTUEEAlgorithms
Combinational Circuit• Inverter ¬x x’
• AND x∧y xy
• OR x∨y x+y
• XOR x⊕y x’y+xy’
• XNOR x ≡y x↔y xy+x’y’
♦ if and only if
• implication x→y x’+xy
x
19NTUEEAlgorithms
Circuit-SAT• Circuit-Satisfiability Problem (Circuit-SAT):
♦ Given a combinational circuit C of AND, OR, and NOT gates
♦ Is there an assignment of Boolean values to the inputs that makes output of C = 1?
• A circuit is satisfiable if there exists a set of Boolean input values that makes output of the circuit =1
♦ Circuit (a) is satisfiable since x1, x2, x3= 1, 1, 0 makes output =1
♦ how about (b)?
=1?
20NTUEEAlgorithms
Circuit-SAT is NPC• To prove Circuit-SAT is NPC, we need two conditions
1. L ∈ NP
2. L' ≤ P L for every L' ∈ NP
• Condition 1: (Lemma 34.5) Circuit-SAT belongs to NP
♦ given inputs, takes only polynomial time to verify if output is 1
• Condition 2: (Lemma 34.6) Circuit-SAT is NP-hard
♦ Informal proof see textbook
• Based on two lemmas, Circuit Satisfiability is NP-complete
♦ our first NPC problem
♦ will prove other NPC problems based on this one
21NTUEEAlgorithms
Outline• NP-completeness, CH34
♦ polynomial time verification
♦ NP-completeness and reducibility
♦ NP-completeness proofs
♦ NP-complete problems
• Approximation Algorithms, CH35
22NTUEEAlgorithms
How to Prove NP-Completeness?• Five steps for proving that L is NP-complete:
1. Prove L ∈ NP
2. Select a known NP-complete problem L'
3. Construct a reduction function f that maps every instance of L' to an instance of L
4. Prove that x ∈ L' iff f(x) ∈ L for all input x
5. Prove that f is done in polynomial time
• So far, thousands of problems has been proven NPC
♦ if ρ(n) = constant, we call it ρ(n)-approximation algorithm
• Relative error bound, ε(n) ≤ ρ(n)-1
♦ if ε(n) = constant, we call it (1+ ε) approximation algorithm
)(,max*
*n
C
C
C
C ρ≤⎟⎟⎠
⎞⎜⎜⎝
⎛
)(*
|*|n
C
CC ε≤−
37NTUEEAlgorithms
APPROX-VERTEX-COVER(G)1 C = ∅2 E’= G.E4 while E’ ≠ ∅5 let (u,v) be an arbitrary edge of E’6 C = C∪{u,v}7 remove from E’ every edge incident on either u or v8 return C
Approx. Alg. for Vertex Cover• Complexity O(V+E)
• Example: Fig 35.1
♦ pick edge (b,c)
♦ pick edge (e,f)
♦ pick edge (d,g)
♦ Size = 6
• optimal solution = {b,d,e}
♦ Size = 3
38NTUEEAlgorithms
Ratio Bound = 2• (Theorem 35.1) APPROX-VERTEX-COVER is 2-approximation
algorithm
♦ Proof:
∗ Let A = set of edges picked ; C = set of vertices picked
− |C| = 2|A|
∗ no edges in A share an endpoint, so no two edges in A are covered by the same vertex from C*
− |A|≤|C*|
∗ so |C|≤2|C*|
39NTUEEAlgorithms
Outline• NP-completeness, CH34
• Approximation Algorithms, CH35
♦ Approximate Vertex-cover 35.1
♦ Approximate TSP 35.2
♦ Approximate Set-Covering 35.3
40NTUEEAlgorithms
Traveling Salesman Problem• TSP: Given an undirected graph G=(V,E) that has non-negative
integer cost c(u,v) associated with each edge, find a Hamiltonian cycle with minimum cost
• Let A = set of edges chosen
• triangle inequality
∑∈
=Avu
vucAc),(
),()(
),(),(),( wvcvucwuc +≤
41NTUEEAlgorithms
Approx-TSP –Tour with Triangle Inequality
• root= vertex a
• MST
• preorder walk
♦ a b c h d e f g
• approx. solution
♦ cost = 19
• optimal solution
♦ cost = 14.7
G.V
H*H
T W
42NTUEEAlgorithms
Ratio Bound = 2• (Theorem 35.2) APPROX-TSP is a polynomial time 2-approximation
algorithm for TSP with triangle inequality
♦ Proof:
∗ Let T = MST
∗ Let H* be the optimal tour.
∗ H* is formed by some tree plus an edge
− c(T)≤ c(H*)
∗ Let W = a full walk along T
− e.g. a, b, c, b, h, b, a, d, e, f, e, g, e, d, a
∗ since W traverse every edge of T twice
− c(W) ≤ 2 c(T)
∗ H = removed from W all but first visit to each vertex
− e.g. a, b, c, h, d, e, f, g
∗ triangle inequality: removing vertex does not increase cost
− c(H) ≤ c(W)
∗ so, c(H) ≤ 2 c(H*)
W
H
43NTUEEAlgorithms
General TSP without Triangle Inequality (* not in exam)
• (Theorem 35.3) If P≠NP, there is no polynomial time ρ-approximation algorithm for the general TSP problem (ρ≥1)
♦ Proof: by contradiction
∗ Suppose there is a polynomial time ρ-approximation for TSP
∗ Suppose we want to find an HC in graph G =(V,E)
∗ Let G’(V,E’) be a complete graph, where E’ contains all edges
∗ apply the ρ-approximation algorithm for TSP, we are guaranteed to find the HC in polynomial time
− because the other edges are too costly to choose
∗ then we get a polynomial time HC algorithm!
− this is wrong since we already proved HC is NPC
otherwise
),( if
1
1 ),(
Evu
Vvuc
∈
⎩⎨⎧
+=
ρ
V VV 1)-V(1)V( ρρρ >+=++
44NTUEEAlgorithms
Outline• NP-completeness, CH34
• Approximation Algorithms, CH35
♦ Approximate Vertex-cover 35.1
♦ Approximate TSP 35.2
♦ Approximate Set-Covering 35.3
45NTUEEAlgorithms
Set-Covering Problem• Given a finite set X, and a family F of subsets of X
• subset S∈ F covers its elements
• Set covering problem: find a minimum number of subset C⊆ F that cover all X
• Optimization problem
♦ minimum size of C
• Decision problem
♦ exist a covering C ≤ size k?
♦ exercise: prove set covering is NPC (Hint: use vertex covering)
• Applications of Set-Covering
♦ find minimum of people (C) that perform all tasks (X)
♦ minimum 2-level combinational circuit design
UFS
S X∈
=
UCS
S X∈
=
46NTUEEAlgorithms
Greedy-Set-Cover• Each iteration chooses the subset that covers most elements
♦ running time O(|X||F| min(|X|,|F|))
• example: |X|=12, |F|=6
♦ greedy solution
∗ {S1, S4, S5, S6} size =4
♦ optimal solution
∗ C={S3, S4, S5} size=3
• (Corollary 35.5) GREEDY-SET-COVER is a polynomial time (ln|X|+1)-approximation algorithm
47NTUEEAlgorithms
SOP Minimization * not in exam
• Two-level AND-OR Combinational Circuit
• Sum of product (SOP)
♦ f = a’c’d + bc’d + bcd’ + acd’
• Disjunctive normal form (DNF)
♦ f=(¬a∧¬c ∧d)∨(b∧¬c ∧d)∨(b∧c ∧ ¬ d)∨(a∧c ∧¬d)
• Very important EDA problem:
♦ Given f, find minimize number of gates
♦ if gates are the same, minimize number of inputs
48NTUEEAlgorithms
Quine-McCluskey Algorithm• Invented in 1956, one of the earliest EDA algorithms
• Two-step process
♦ 1. Generate all Prime Implicants (PI)
∗ PI = product term that cannot be combined with other terms
♦ 2. Select minimum number of PI
∗ set-covering
49NTUEEAlgorithms
• f(abcd)=Σm(4,5,6,8,9,10,13)+d(0,7,15)
♦ at least ONE of the following terms must be covered