CMPUT 671 Hard Problems

02/01/11 CMPUT 671 Lecture 1 1

CMPUT 671Hard Problems

Winter 2002

Joseph Culberson Home Page

02/01/11 CMPUT 671 Lecture 1 2

What are you doing?

02/01/11 CMPUT 671 Lecture 1 3

What am I doing?

What are you doing?

02/01/11 CMPUT 671 Lecture 1 4

What is Hard?• Problems and Complexity

– P, NP, PSPACE; between and beyond

– Worst case, Randomized, Average case

What am I doing?

What are you doing?

02/01/11 CMPUT 671 Lecture 1 5

What is Hard?• Problems and Complexity

– P, NP, PSPACE; between and beyond

– Worst case, Randomized, Average case

• Instances and Algorithms– Some instances may be easy for some algorithms

– Hard instances for classes of algorithms

What am I doing?

What are you doing?

02/01/11 CMPUT 671 Lecture 1 6

What is Hard?• Problems and Complexity

– P, NP, PSPACE; between and beyond– Worst case, Randomized, Average case

• Instances and Algorithms– Some instances may be easy for some algorithms– Hard instances for classes of algorithms

• Ensembles and Phase Transitions- Most instances may be easy- Hard instances may be “concentrated”- Hard random instances for classes of algorithms- Generation

What am I doing?

What are you doing?

02/01/11 CMPUT 671 Lecture 1 7

Background Asymptotic notation and algorithm analysis Problems, Complexity Classes, Reductions

and Completeness (P to NPc) Backtrack Search, Stochastic Search Graphs and some graph theory Probability (as background to random

graphs and structures)

02/01/11 CMPUT 671 Lecture 1 8

Problems A Problem is a set of instances and queries.

An instance is a structure (versus language)

Queries are universal e.g. Graph k-coloring

Instance: A graph G and integer k Query: Is G k-colorable?

e.g. MST union TSP Instance: A weighted graph G, integers B1,B2

Query: Does G have a spanning tree of weight B1 or a tour of weight B2? (Set one to 0)

02/01/11 CMPUT 671 Lecture 1 9

Measuring Resources Time

Instance I n = Size(I); Formally, the number of bits in an

encoding (logarithmic model) but mostly we will use the uniform model, e.g. the number of vertices and edges of a graph.

T(I) = number of steps in computation to answer the query. Formally, on a model of computation, typically a RAM.

02/01/11 CMPUT 671 Lecture 1 10

Measuring Resources Time (continued)

Worst Case T(n) = max{ T(I) : size(I) = n }

Average Case A(n) = sum T(I)* Prob(I) : size(I) = n Note: distribution dependent! Typically all instances of size n equally likely, but this

may not be representative of the “real world”

02/01/11 CMPUT 671 Lecture 1 11

Measuring Resources SPACE

Similar, but measures number of memory cells required.

Formally, each cell must be of fixed size, e.g. number of bits fixed.

For certain purposes, e.g. O(log n) memory requirements, we exclude size of input memory. (Likely not relevant to this course)

02/01/11 CMPUT 671 Lecture 1 12

Asymptotic Notation

€

f ,g : N + → R*

O( f )= {g |∃n0,c > 0,∀n > n0,g(n) < cf (n)}

Ω( f )= {g |∃n0,c > 0,∀n > n0,g(n) > cf (n)}

Θ( f ) = O( f ) ∩ Ω( f )

ω( f ) = {g | limn →∞

f (n) /g(n) = 0}

o( f ) = {g | limn →∞

g(n) / f (n) = 0}

02/01/11 CMPUT 671 Lecture 1 13

Types of Problems Optimization

Given G find the minimum number of colors required to color it.

Functional Given G find an optimal coloring Given G and k find a k-coloring if one exists

Enumeration and Generation (listing) Decision (Yes/No)

Given G and k is there a k-coloring of G?

02/01/11 CMPUT 671 Lecture 1 14

The Class P P is the set of decision problems for

which there exists a polynomial time algorithm. i.e. O(nk) for some fixed k. n is the size of the input. See Garey and Johnson, Brassard and

Bratley, Sipser, Cormen et al.

02/01/11 CMPUT 671 Lecture 1 15

The Class NP NP is the set of decision problems for which

there exists a non-deterministic algorithm that solves the yes instances of the problem in polynomial time. (G&J, B&B) The algorithm non-deterministically

writes out a solution (certificate) and in time polynomial in the input size verifies that the answer is yes.

E.g. Coloring: Certificate= c:V -> {1..k}; verify for each edge e = (u,v), c(u) not = c(v)

02/01/11 CMPUT 671 Lecture 1 16

Class CO-NP The set of problems whose NO instances are

solved by a poly-time non-deterministic algorithm; the complement of NP wrt Yes/No

In Language Terms Note that P is in the intersection of NP and

CO-NP … Why?

€

CO − NP = {Σ* − L : L ∈ NP}

02/01/11 CMPUT 671 Lecture 1 17

We no of know certificate we can use to verify that G is not k-colorable in poly-time.

The Questions: P ?= NP ?= CO-NP ? The first asks if there is a polynomial time

deterministic algorithm to solve all NP problems The second asks is there a polynomial time non-

deterministic algorithm to solve all of CO-NP

02/01/11 CMPUT 671 Lecture 1 18

Reductions

Polynomial time many one reduction (transformation) from problem P to problem Q f:P->Q Maps instances x in Y(P) iff f(x) in Y(Q) f is computable by a polynomial time

deterministic algorithm.

02/01/11 CMPUT 671 Lecture 1 19

Reductions

Example: Graph 3-coloring to 3-SAT

€

f :[G(V , E),3] → S(U,C)

∀v ∈V →{xv1, xv2, xv3}⊂U

< xv1∨xv 2 ∨xv3 >∈ C

⎧ ⎨ ⎩

⎫ ⎬ ⎭

∀e = {u,v}∈ E →

{< xv1∨xu1 >,< xv 2 ∨xu2 >,< xv3 ∨xu3 >}⊂C

Can also add 2-clauses restricting each vertex to at most one color

02/01/11 CMPUT 671 Lecture 1 20

Class NPc A subclass of NP, Q in NPc iff for every

problem P in NP there is a polynomial time many reduction from P to Q.

Cook’s theorem showed that SAT is in NPc. Create a set of clauses which model the

possible execution of the NDTM that solves the problem.

02/01/11 CMPUT 671 Lecture 1 21

Showing a Problem is in NPc

To show a problem Q is NP-complete Show Q is in NP, that is, specify a certificate that

can be verified in polynomial time Choose a problem P in NPc and provide a

polynomial time reduction from P to Q.

The second alone shows it is NP-hard

02/01/11 CMPUT 671 Lecture 1 22

• Only applies to worst case