CSCI 3130: Formal languages and automata theory

CSCI 3130: Formal languages and automata theory

Andrej Bogdanovhttp://www.cse.cuhk.edu.hk/~andrejb/

csc3130

The Chinese University of Hong Kong

NP and NP-completeness

Fall 2011

Some more problems

1 2

3 4

Graph G

A clique is a subset of vertices that are all interconnected

{1, 4}, {2, 3, 4}, {1} are cliques

An independent set is a subset of vertices so that no pair is connected{1, 2}, {1, 3}, {4} are independent setsthere is no independent set of size 3

A vertex cover is a set of vertices that touches (covers) all edges{2, 4}, {3, 4}, {1, 2, 3} are vertex covers

How to solve themCLIQUE = { 〈 G, k 〉 : G is a graph with a clique of k vertices}

M: On input 〈 G, k 〉 :For all subsets S of vertices of size k: If for every pair u, v in S (u, v) is an edge in G, acceptOtherwise, reject.

1 2

3 4

input: 〈 G, 3 〉 subsets: {123} {124} {134} {234}

all edges in? NO NO NO YES

Running time analysisCLIQUE = { 〈 G, k 〉 : G is a graph with a clique of k vertices}

M: On input 〈 G, k 〉 :For all subsets S of vertices of size k: If for every pair u, v in S (u, v) is an edge in G, acceptOtherwise, reject.

( )nk subsets

≈ n2 pairs

≈ n2( )nk

≈ 2n when k = n/2

Status of these problems

CLIQUE = { 〈 G, k 〉 : G is a graph with a clique of k vertices}

IS = { 〈 G, k 〉 : G is a graph with an independent set of k vertices}

VC = { 〈 G, k 〉 : G is a graph with a vertex cover of k vertices}

running timeof best-known algorithm

problem CLIQUE

≈ 2n

IS

≈ 2n

VC

≈ 2n

What do these problems have in common?

Checking solutions efficiently• We don’t know how to solve them efficiently

• But if someone told us the solution, we would be able to verify it very quickly

1

2

3

4

56

78

9

10

11

12

1314

15

Is (G, 5) in CLIQUE?1,5,9,12,14

Example:

The class NP• A verifier for L is a TM V such that

– s is a potential solution for x– We say V runs in polynomial time if on every input

x, it runs in time polynomial in |x| (for every s)

NP is the class of all languages that have polynomial-time verifiers

x ∈ L V accepts 〈 x, s 〉 for some s

Example

CLIQUE is in NP:On input 〈 G, k 〉 and a set of vertices C, V :=If C has size k and all edges between vertices ofC are present in G, accept, otherwise reject.

running time ≈ O(k2)✔

P versus NP

because the verifier can ignore the solution

Conceptually, finding solutions can only be harder than checking them

P (efficient)

decidable

L01PATH

CLIQUE

SAT ISNP (efficiently checkable)

VC

P is contained in NP

Millenium prize problems• Recall how in 1900, Hilbert gave 23 problems

that guided mathematics in the 20th century

• In 2000, the Clay Mathematical Institute gave 7 problems for the 21st century

1 P versus NP2 The Hodge conjecture3 The Poincaré conjecture4 The Riemann hypothesis5 Yang–Mills existence and mass gap6 Navier–Stokes existence and smoothness7 The Birch and Swinnerton-Dyer conjecture

$1,000,000Hilbert’s 8th problem

Perelman 2006

computer science

P versus NP• The answer to the question

is not known. But one reason it is believed to be negative is because, intuitively, searching is harder than verifying

• For example, solving homework problems (searching for solutions) is harder than grading (verifying the solution is correct)

Is P equal to NP?

$1,000,000

Searching versus verifyingMathematician: Given a mathematical claim, come up with a proof for it.

Scientist: Given a collection of data on some phenomena, find a theory explaining it.

Engineer: Given a set of constraints (on cost, physical laws, etc.) come up with a design (of an engine, bridge, etc.) which meets them.

Detective: Given the crime scene, find “who’s done it”.

P and NP

languages that can be decided on a TMwith polynomial running time

P =

(problems that admit efficient algorithms)

NP = languages whose solutions can be verifiedon a TM with polynomial running time

(solutions can be checked efficiently)

decidable

NPP

We believe that NP is bigger than P,but we are not 100% sure

Evidence that NP is bigger than P

• These (and many others) are in NP– Their solutions, once found, are easy to verify

• But no efficient solutions are known for any of them– The fastest known programs take time ≈2n

CLIQUE = { 〈 G, k 〉 : G is a graph with a clique of k vertices}

IS = { 〈 G, k 〉 : G is a graph with an independent set of k vertices}

VC = { 〈 G, k 〉 : G is a graph with a vertex cover of k vertices}

Equivalence of certain NP languages• We strongly suspect that problems like

CLIQUE, SAT, etc. require time ≈2n to solve

• We do not know how to prove this, but what we can prove is that

If any one of them can be solved efficiently, then all of them can be solved efficiently

Equivalence of some NP languages• All these problems are as hard as one another

• Moreover, they are at the “frontier” of NP– They are at least as hard as any problem in NP

clique

independent set

vertex-cover

NP

P

Polynomial-time reductions• What do we mean when we say, for

example,

• We mean that

• Or, we can convert any polynomial-time TM for IS into one for CLIQUE

“ IS is at least as hard as CLIQUE”

If CLIQUE has no polynomial-time TM, thenneither does IS

Polynomial-time reductions

• Theorem

If IS has a polynomial-time TM, so does CLIQUE

CLIQUE = { 〈 G, k 〉 : G is a graph with a clique of k vertices}

IS = { 〈 G, k 〉 : G is a graph with an independent set of k vertices}

{1, 4}, {2, 3, 4}, {1} are independent sets{1, 2}, {1, 3}, {4} are cliques

1 2

3 4

Polynomial-time reductions

• Proof: Suppose IS has an poly-time TM A

• We want to use it to solve CLIQUE

If IS has a polynomial-time TM, so does CLIQUE

〈 G, k 〉

reject if not

accept if G has clique of size kA

for IS

〈 G’, k’ 〉

reject if not

accept if G’ has IS of size k

Reducing CLIQUE to IS

• We look for a polynomial-time TM R that turns the question:

into: RG, k G’, k’“ Does G have a clique of size k?”

“ Does G’ have an IS of size k’?”

1 2

3 4

G

cliques of size k ISs of size k’

1 2

3 4

G’flip all edges

k’ = k

Reducing CLIQUE to IS

RG, k G’, k’On input 〈 G, k 〉 Construct G’ by flipping all edges of G Set k’ = k Output 〈 G’, k’ 〉

cliques in G independent sets in G’

If G’ has an IS of size k, then G has a clique of size k

If G’ does not have an IS of size k, then G has no clique of size k ✓

✓

Reduction recap• We showed that

by converting an imaginary TM for IS into one for CLIQUE

• To do this, we came up with a reduction that transforms instances of CLIQUE into ones of IS

If IS has a polynomial-time TM, so does CLIQUE

Polynomial-time reductions• Language L polynomial-time reduces to L’ if

there exists a polynomial-time TM R that takes an instance x of L into instance y of L’ s.t.

x ∈ L if and only if y ∈ L’

L L’(CLIQUE) (IS)

x y= 〈 G, k 〉 = 〈 G’, k’ 〉

x ∈ L y ∈ L’(G has clique of size k) (G’ has IS of size k’)

R

The meaning of reductions• Saying L reduces to L’ means L is no harder than L’

– In other words, if we can solve L’, then we can also solve L

• Therefore

If L reduces to L’ and L’ ∈ P, then L ∈ P

x y

x ∈ L y ∈ L’

R poly-time TM for L’

acc

rej

TM accepts

The direction of reductions• The direction of the reduction is very

important– Saying “A is easier than B” and “B is easier than

A” mean different things

• However, it is possible that L reduces to L’ and L’ reduces to L– This means that L and L’ are as hard as one

another– For example, IS and CLIQUE reduce to one

another

Boolean formula satisfiability• A boolean formula is an expression made up of

variables, ands, ors, and negations, like

• The formula is satisfiable if one can assign values to the variables so the expression evaluates to true

(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)

x1 = F x2 = F x3 = T x4 = TAbove formula is satisfiable because this assignment makes it true:

3SAT

SAT = { 〈〉 : is a satisfiable Boolean formula}3SAT = { 〈〉 : is a satisfiable Boolean formula in conjunctive normal form with 3 literals per clause}

CNF: AND of ORs of literals

literal: xi or xi

(conjunctive normal form)

3CNF: CNF with 3 literals per clause

(x1∨x2∨x2 ) ∧ (x2∨x3∨x4)clauseliterals

(repetitions are allowed)

3SAT and NP

(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)=

Verifying a solution:FFTT

substitutex1 = F x2 = F x3 = T x4 = T

evaluate formula(F ∨T ) ∧ (F ∨ T ∨ F) ∧ (T)=

can be done in linear time

Finding a solution:Try all possible assignments

FFFFFFFTFFTFFFTT

FTFFFTFTFTTFFTTT

TFFFTFFTTFTFTFTT

TTFFTTFTTTTFTTTT

For n variables, there are 2n possible assignmentsTakes exponential time

The Cook-Levin Theorem

• So every problem in NP is no harder than SAT

• But SAT itself is in NP, so SAT must be the “hardest problem” in NP:

Every L ∈ NP reduces to SAT

If SAT ∈ P, then P = NP

SAT = {: is a satisfiable Boolean formula}(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)E.g.

P

SATNP

NP-completeness• A language C is NP-complete if:

• Cook-Levin Theorem:

1. C is in NP, and2. For every L in NP, L reduces to C.

P

CNP

SAT is NP-complete

NP-complete

Our picture of NP

SAT

NPCLIQUEIS

P

0n1nPATH

context-free

A BA reduces to B

More NP-complete problems

P

3SAT

NP

CLIQUEIS

0n1nPATH

NP-complete A BA reduces to B

In practice, most of the NP-problems areeither in P (easy) or NP-complete (probably hard)

Interpretation of Cook-Levin Theorem• Optimistic view:

• Pessimistic view:

If we manage to solve SAT, then we can also solve CLIQUE and many other things

Since we do not believe P = NP, it is unlikely thatwe will ever have a fast algorithm for SAT

The ubiquity of NP-complete problems• We saw a few examples of NP-complete

problems, but there are many more

• A surprising fact of life is that most CS problems are either in P or NP-complete

• A 1979 book by Garey and Johnsonlists 100+ NP-complete problems