NP-completeness Class of hard problems. Jaruloj ChongstitvatanaNP-complete Problems2 Outline Introduction Problems and Languages Turing machines and.

NP-completeness

Class of hard problems

Jaruloj Chongstitvatana NP-complete Problems 2

Outline Introduction Problems and Languages

Turing machines and algorithms P and NP

Satisfiability problem Reduction Other NP-complete problems


Problems A problem can be defined as a mapping from

the instance to the answer. Examples:

Shortest path problem{<graph G, node u, node v>} {a shortest path P}

0/1 Knapsack problem{<set S of objects, capacity C>} {the maximum value}

Closest-pair-of-points problem{<set P of points>} {<point q, point r>, which is the closest pair of points in P}.

Satisfiable problem{<boolean formula>} {T, F}


Abstract problemsA set of instances

A set of solutions


Decision Problems A decision problem is a problem whose

answer is either “yes” or “no”.

Example: Satisfiable problem{<boolean formula>} {T, F}

Any problem can be described in term of a decision problem.


Reformulate Problems as Decision Problems Shortest path problem

{<graph G, node u, node v, int k>} {T, F}

The answer is T iff there is no path between u and v, which is shorter than k.

0/1 Knapsack problem {<set S of objects, capacity C, value V>}

{T, F} The answer is T iff there is no selection of

objects fitted in the sack which yields higher value than V.


Encoding of Decision Problems An encoding of a problem is a mapping from the set

of instances of the problem to a set of strings.

Example: shortest path The encoding of <graph G, node u, node v, path p> in

the Shortest path problem is a string which is used to describe G, u, v, and p.

G can be described by a string of its adjacency matrix.

u and v can be described by two strings, indicating the node labels.

p can be described by a sequence of node in the path.


Abstract/ Concrete problems

Mapping(encoding)

Mapping(encoding)

A set of instances

A set of solutions

Abstract problem Concrete

problem


Encoding of Problems Is X a prime ?

f:{X |X>0 and is a binary number}{Y,N}f(x) = Y if X is primef(x) = N otherwise

Is a 3-CNF expression X satisfiable?f:{X | X is a 3-CNF expression} {Y, N}

f(x) = Y if X is satisfiablef(x) = N otherwise


Why encoding is important? Encoding determines the instance size. Example: an integer n

n encoded as one number => instance size = 1

n encoded as binary number => instance size = log2 n

n encoded as unary number => instance size = n

Problems and Languages


Languages and Decision Problems Let be an alphabet.

A language over is a set of strings created from symbols in .

B={0,1}* is the set of all binary strings.

{ is a palindrome} is a language over B.

A yes-instance (or no-instance) is an instance whose answer is “yes” (or “no”).

An instance must be either a yes- or a no-instance.


Languages and Decision Problems A decision problem can be encoded in a

language {| is an encoding of a yes- instance}. The shortest path problem can be encoded as

{e(G, u, v, p>)| p is the shortest path between u and v in the graph G}.

Turing Machines


Turing machines (TM) A Turing machine is a machine which takes

a string on its input tape, and decides whether to accept or to reject the string.

A Turing machine contains A finite-state control with

A start state A halt state Transition function: state x symbol state x symbol x

{R,L}

A tape with a Read/Write head which can move left and right.


Deterministic/ Nondeterministic Turing machines

Difference A transition is deterministic or nondeterministic.

What is nondeterminism? Given a state of a machine/program, there is

more than one possible next step. Examples

Deterministic: functions. Nondeterministic: relations.


Example of a DTM

1/,R

q2

h

q1/1

,L@

/,R

/,L

s

p1

p4 p2

p3

/@

,R0

/,R

0/0

,R1

/1,R

/

,L0/,L

0/0,

L

1/1,

L/,R


How a DTM works

1/,R

q2

h

q1

/1,L

@/,R

/,Ls

p1

p4 p2

p3

/@,R

0/,R

0/0,R1/1,R

/,L0/,L

0/0,L1/1,L

/,R0 1 0 0 0 0 0 @ 1

On the input 0001000,the TM halts.


How a DTM works

1/,R

q2

h

q1

/1,L

@/,R

/,Ls

p1

p4 p2

p3

/@,R

0/,R

0/0,R1/1,R

/,L0/,L

0/0,L1/1,L

/,R0 0 0 0 0 0 0 @

On the input 0000000, the TM hangs.


Example of NTM Let L={ww| w{0,1}*}

s

p u

q0 t0

r0

h

/@

,R 0/0,L1/1,L

@/,R

0/,L

1/,L

0/0,L

1/1,L

/,L

0/@,R

/,R

0/0,R1/1,R

/,R

0/,L

1/,L

q1 t1

r1

@/,R

0/0,L1/1,L/,L 1/@

,R

0/0,R1/1,R

/,R

@/,Lv0/0,R1/1,R

/@,L


Accept/Decide Let T be a Turing machine. For a string over ,

T accepts iff T halts on with output “1”. T rejects iff T halts on with output “0”. T hangs on iff T dose not halts on .

For a language L over , T accepts L iff, for any string in L, T accepts . T decides iff

for any string in L, T accepts , and for any string inL, T rejects .


Time complexity A language L is accepted/decided in O(f(n))

by a Turing machine T if, for any length-n string in L, is accepted/decided in k·f(n) by a Turing machine T, for some constant k.

A language L is accepted/decided in polynomial time by a Turing machine T if, L is accepted/decided in O(nk) by a Turing machine T, for some constant k.


Church-Turing Thesis If there is a Turing machine deciding a

problem in O(f), then there is an algorithm solving in O(f).

Turing machine algorithm

Complexity classes


Class of P If is a problem in P, given any instance I of

, there is an algorithm which solves I in O(nk), or there is a Turing machine which accepts e(I) in

O(nk) steps,

where n is the size of e(I) and k is a constant.

What does this mean? If is in P, is easy because it can be solved in

polynomial time.


Class of NP If is a problem in P, given any instance I

and any certificate C of , there is an algorithm which verifies, in O(nk), that

C is the answer of I, for any C and I. there is a Turing machine which verifies, in O(nk),

that C is the answer of I, for any C and I. there is a nondeterministic Turing machine which

accepts e(I), in O(nk), for any I.

where n is the size of e(I) and k is a constant.

What does this mean? If is in NP, is not necessarily easy because it

can only be verified in polynomial time.


Easy problems / Hard problems Can be solved in

polynomial time Prove by showing the

algorithm.

Cannot be solved in polynomial time Prove by showing that

there is no polynomial-time algorithm.

Can verify the answer in polynomial time.

Can usually be solved by an exponential-time algorithm. Brute-force algorithm: try

every possible answers.


Class of co-NP L is in co-NP ifL is in NP.

NP co-NPP

Reduction

Is X more difficult than Y?


Reduction Let L1 and L2 be languages over 1 and 2,

respectively. L1 is (polynomial-time) reducible to

L2, denoted by L1L2, if there is a TM M computing a

function f: 1*2

* such that wL1 f(w)L2 in

polynomial time.

Let P1 and P2 be problems. P1 is (polynomial-time)

reducible to P2, denoted by P1P2, if there is an

algorithm computing a function f: 1*2

* such that

w is a yes-instance of P1 f(w) is a yes-instance of

P2 in polynomial time.


Meaning of ReductionP1 is reducible to P2

if DTM computing, in polynomial time, a function f: 1

*2* such that w is a yes-instance of P1 f(w) is

a yes-instance of P2. If you can map yes-instances of problem A to yes-

instances of problem B, then we can solve A in polynomial time if we can solve B

in polynomial time. it doesn’t mean we can solve B in polynomial time

if we can solve A in polynomial time.

Example of Reductuion


Satisfiability problem (SAT) Given a Boolean

expression , is satisfiable?

(A~BC)(~A~C)B~C is not satisfiable.

A B C A~BC ~A~C

0 0 0 0 1 0

0 0 1 1 1 0

0 1 0 0 1 0

0 1 1 1 1 0

1 0 0 1 1 0

1 0 1 1 0 0

1 1 0 0 1 0

1 1 1 1 0 0


3CNF-SAT Problem Given an expression in 3CNF, is

satisfiable? Given an expression in 3CNF, is there a

truth assignment of propositions in which makes true?

Example: =(PQR)(PQR)(PSR)(PSR) is satisfiable when

P=t, Q=f, R=f, S=t or P=f, Q=f, R=f, S=t


Clique or Complete Subgraph A graph G=(V,E) has a clique if there is W V such

that for all nodes a and b in W, there is an edge between a and b.

A

BC

D

E F


Clique Problem Given a graphG=(V,E) and an integer k, is there a

clique of size k in G? Example: Is there a clique of size 3, 4, or 5, in the

graph below?

A

BC

D

E F


3CNF-SAT CliqueProof:To prove that 3CNF-SAT Clique,

Find a function f such that a 3CNF expression is satisfiable

a graph G has a clique of size k, where <G,k>=f() and

there is a TM computing f in polynomial time.

f can be defined as follows:Let = (111213)(212223)…(n1n2n3)

f() = <G,n>, where G=(V, E), V={ij| 0<i<n+1, 0<j<4}, and E={(ij, km)| i≠k and ij ≠~km}


Building a graph from a 3CNF exp.

(PQR)(PSR)(PSR)

P R~Q

P

S

~R

~P

~R

~S


3CNF-SAT Clique (cont’d) Show is satisfiableG has clique of size n Let G has a clique of size n. From the definition of G, there is no edge

between vertices in the same disjunctive clause, and representing a literal and its negation.

A link between two nodes means that the two literals can be true at the same time, and they are in different disjunctive clauses.


Satisfiability and Clique

(PQR)(PSR)(PSR)

P R~Q

P

S

~R

~P

~R

~S



(PQR)(PQR)(PQR)(PQR)

P RQ

P

Q

~R

~P

R

Q

~P ~RQ



(PQR)(PQR)(PQR)(PQR)

P RQ

P

Q

~R

~P

R

Q

~P ~RQ


3CNF-SAT Clique (cont’d) There is a TM computing f in polynomial time.

Properties of Reductions


Reflexivity of reductionTheorem:

Let L be a language over . L L.

Proof:

Let L be a language over .

Let f be an identity function from **.

Then, there is a TM computing f.

Because f is an identity function, wL

f(w)=wL.

By the definition, L L.


Property of reductionTheorem:

Let L1 and L2 be languages over .

If L1L2, thenL1L2.

Proof:

Let L1 and L2 be languages over .

Because L1L2, there are a function f such that wL1

f(w)L2, and a TM T computing f in polynomial

time.

wL1 f(w)L2.

By the definition,L1L2.


Transitivity of reductionTheorem: Let L1, L2 and L3 be languages over .

If L1 L2 and L2 L3, then L1 L3.

Proof: Let L1, L2 and L3 be languages over .

There is a function f such that wL1 f(w)L2, and a TM T1

computing f in polynomial time because L1 L2.

There is a function g such that wL2 g(w)L3, and a TM

T2 computing g in polynomial time because L2 L3.

wL1 f(w)L2 g(f(w))L3, and T1T2 computes g(f(w))

in polynomial time .

By the definition, L1 L3.


Using reduction to prove P/NPTheorem: If L2 is in P/NP, and L1L2, then L1 is also P/NP.

Proof:Let L1 and L2 be languages over , L1L2, and L2 be in P/NP.

Because L2 is in P/NP, there is a DTM/NTM T2 accepting L2 in polynomial time.

Because L1L2, there is a DTM T1 computing a function f such that wL1 f(w)L2 in polynomial time.

Construct a DTM/NTM T=T1T2. We show that T accepts L1 in polynomial time. If wL1, T1 in T computes f(w)L2 and T2 in T accepts f(w), in

polynomial time. If wL1, T1 in T computes f(w) L2 and T2 in T does not accept

f(w) in polynomial time.

Thus, L1 is also in P/NP.


Using reduction to prove co-NP.Theorem:

If L2 is in co-NP, and L1L2, then L1 is also in co-NP.

Proof:Let L1 and L2 be languages over , L1L2, and L2 be in co-NP.

Because L2 is in co-NP,L2 is in NP.

Because L1L2,L1L2. Then,L1 is in NP.

Thus, L1 is co-NP.

NP-completeness


NP-completeness A language (or problem) L is NP-hard if, for

any language L' in NP, L' ≤ L.

A language (or problem) L is NP-complete if it is in NP and is NP-hard.


Why interested in NP-complete problem Implication on the problem “is P=NP ?” If an NP-complete problem is in P, then

P=NP. Why?

If L is NP-complete, any language in NP is reducible to L.

Since L is in P, any language in NP is reducible to a language in P.

Then, any language in NP is in P.


Lemma

If L is a language such that L’ ≤ L for some L’ ∈ NPC, then L - is NP hard.

Moreover, if L ∈ NP, then L ∈ NPC. Proof

Since L’ - is NP complete, for all L’’ ∈ NP, we h ave L’’ ≤ L’ .

Because L’ ≤ L , L’’ ≤ L by transitivity.Then, L - is NP hard.

If L ∈ NP, we also have L ∈ NPC.

Ground Reduction

SAT is NP-complete


SAT is NP-complete To prove that SAT is NP-complete:

Prove SAT is in NP Prove any language in NP is reducible to SAT.

SAT is in NP: Informal proof Given a certificate which is a truth assignment of

propositions which makes the expression true, there is an algorithm that can verify the answer in polynomial time.


SAT is NP-complete (cont’d)Prove any language in NP is reducible to SAT.

Proof:

Let L be any language in NP.

Then, there is an NTM T which accepts L in polynomial time p(n), where n is the input size.

To show that L ≤ SAT, we find a function f such that x L f(x) SAT.

That is, to find a function f such that x L f(x) is satisfiable.


SAT is NP-complete (cont’d)Proof (cont’d):

That is, to find a function f such that T accepts x f (x) is satisfiable.

Let f (x)=g1(x)g2(x)g3(x)g4(x)g5(x)g6(x)g7(x).

f (x) describes the sequence of configuration changes which leads to the halt state of T.

f(x) contains propositions Qi,j, Hi,k, and Si,k,j .

Qi,j: T is in state qj after i moves.

Hi,k: T’s tape head is on square k after i moves.

Si,k,j: Symbol j is on square k after i moves.


SAT is NP-complete (cont’d)Proof (cont’d):g1(x): the initial configuration of T at the beginning.

g2(x): accepting configuration of T after N steps.

g3(x): possible transitions of T.

g4(x): hanging configuration.

g5(x): describe that T is in exactly one state at a time.

g6(x): describe that each square of the tape contains exactly one symbol.

g7(x): describe that change can only made on the tape square on which the tape head is located.


SAT is NP-complete (cont’d)Proof (cont’d):g1(x): describe the initial configuration of T at the

beginning.

n N

Q0,0 H0,0 S0,0,0 S0,k, ik S0,k, 0

k=1

k=n+1

∆ 0 1 1 0 1 ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆ ∆

q0



g2(x): describe accepting configuration of T after N steps.

QN,a

ha



g3(x): describe possible transitions of T.

For each transitionδ(qj, l)= (qt1, z1, D1), δ(qj, l)= (qt2, z2, D2)

(Qi,j Hi,k Si,k,l)((Qi+1,t1 Hi+1,d1(k) Si+1,k,z1)

V (Qi+1,t2 Hi+1,d2(k) Si+1,k,z2))



g4(x): describe hanging configuration.

For each transition δ(qj, l)= ø

(Qi,j Hi,k Si,k,l) (Qi+1,j Hi+1,k Si+1,k,l)


SAT is NP-complete (cont’d)Proof (cont’d):g5(x): T is in exactly one state at a time.

For each time i and states qj0, qj1, qj2, …, qj0 ≠ qj1

≠ qj2 ≠ … :

(Qi,j0 v Qi,j1 v Qi,j2 v …) At least one state

(~Qi,j0v~Qi,j1)(~Qi,j1v~Qi,j2)(~Qi,j0v~Qi,j2)… Never two state at the same time



g6(x): state that each square of the tape contains exactly one symbol.

For each time i, square k, and symbols 0, 1,

2, …, 0 ≠ 1 ≠ 2 ≠ … :

(Si,k,0 v Si,k,1 v Si,k,2 v …) At least one symbol in a cell

(~Si,k,0v~Si,k,1) (~Si,k,1v~Si,k,2) (~Si,k,0v~Si,k,2)

… Never two symbols in a cell at the same time


SAT is NP-complete (cont’d)Proof (cont’d):g7(x): state that change can only made on the

tape square on which the tape head is located.

For each i, k, and l :(~Hi,k Si,k,l) (Si+1,k,l)

Hi,k v ~Si,k,l v Si+1,k,l


SAT is NP-complete (cont’d)Proof (cont’d):From the definition of f, f(x) is true iff T

accepts x.f(x) can be computed in polynomial time.

Thus, SAT is NP-complete.


3CNF-SAT is NP-completeFirst, show 3CNF-SAT is in NP.

Let be a 3CNF expression, andtruth assignments C be a certificate for .

There is an algorithm to verify that is true under the assignments C, by substituting the value of each literal and evaluating the expression.

This algorithm takes polynomial time of the length of .


3CNF-SAT is NP-completeSecond, show SAT ≤ 3CNF-SAT.Let be a Boolean expression.

We will construct a 3CNF expression f() such that f() is satisfiable iff is satisfiable.

To create f():(A) Create an abstract syntax tree of so that internal

nodes are logical operators , v, ~, , or , and leaf nodes are literals (x or ~x).

Then, create , from the abstract syntax tree which is satisfiable iff is satisfiable.


3CNF-SAT is NP-complete = ((p q) (((~s v r) (q ~r)) v ~s)

= y1 (y1 (y2 y3)) (y2 (p q )) (y3 (y4 v ~s)) (y4 (y5 y6)) (y5 (~s v r )) (y6 (q ~r))

is satisfiable iff is satisfiable.

:y1

:y2

v:y5

v:y3

:y4

:y6

p q

q ~r~s r

~s


3CNF-SAT is NP-complete(B) Convert into CNF.

P Q is transformed to (P Q) (Q P).

P Q is transformed to ~P v Q.

~(P v Q) is transformed to ~P ~Q.

~(P Q) is transformed to ~P v ~Q.

= (y1 (y2 y3)) =(y1 (y2 y3)) ((y2 y3) y1)

=(~y1 v (y2 y3)) (~y2 v ~y3 v y1)

= (~y1 v y2) (~y1 v y3) (y1 v ~y2 v ~y3)

= (~y1 v y2 v y3) (~y1 v y2 v ~y3) (~y1 v y2 v y3)

(~y1 v ~y2 v y3) (y1 v ~y2 v ~y3)


3CNF-SAT is NP-completef() = , and f() is satisfiable iff is satisfiable.

Next, we need to show that f is computable in polynomial time.In step (A):

the abstract syntax tree can be created in polynomial time of the length of ,the expression can be created also in polynomial time of the number of node in the tree and the length of , which is also a polynomial time of the length of .

In step (B): can be created, using Boolean logic identities, in polynomial time of the length of , which is also a polynomial time of the length of .

That is, SAT ≤ 3CNF-SAT.


CLIQUE is NP-completeProof:

3CNF-SAT is NP-complete.

3CNF-SAT ≤ CLIQUE.

CLIQUE is NP-complete.

NP-completeness Class of hard problems. Jaruloj ChongstitvatanaNP-complete Problems2 Outline Introduction Problems and Languages Turing machines and.

Documents

satisfiable problem

points problem

knapsack problem

instance size

log2 n n

integer n n

nnpcomplete problemsproblems

n otherwiseis