Complexity Theory CSE 331 Section 2 James Daly. Reminders Project 4 is out Due Friday Dynamic programming project Homework 6 is out Due next week (on.

Complexity Theory

CSE 331Section 2James Daly

Reminders

• Project 4 is out• Due Friday• Dynamic programming project

• Homework 6 is out• Due next week (on the last day of class)• Last homework• Covers greedy algorithms and dynamic

programming

Review: Sorting

• We discussed several sorting algorithms in class• Insertion Sort• Selection Sort• Quick Sort• Merge Sort• Heap Sort

• Each has various tradeoffs• Some are more efficient then others

Algorithm Theory

• An algorithm is a solution to a problem• Some problems have no efficient

algorithms to solve them• Some problems have no solution at all• The complexity of a problem is the

complexity of the best algorithm for solving the problem• Finding the max of an unsorted array: O(n)• Comparison-based sorting: O(n log n)

Finite Automata (FA)

• Very simple machine• Fixed number of states

• One starting state• Some “accepting” states

• Reads a string one symbol at a time• Moves to a new state based on the symbol• Returns whether it ends in an accepting

state

Example

SA

F11

1

00

0

0 1 0 0

Accept!

Example

SA

F11

1

00

0

0 1 1

Reject!

Nondeterministic Finite Automata (NFA)

• Special type of FA• States can have multiple transitions with

the same symbol• Takes all of them!

• Is in a superposition of states

• Accepts if any of them accept

Example

a b

*

Accepts any string ending in ab

Turing Machine

• An idealized computer capable of running any program

• Only slightly more powerful than an actual computer (they have infinite memory)

• Also has deterministic and non-deterministic versions

Class P

• An algorithm runs in polynomial time if its running time is O(nk) for some constant k• Polynomial time is better than exponential

time: O(2n)

• P is the class of problems solvable in polynomial time• We consider these to have “efficient” solutions

Class NP

• Class of problems solvable in polynomial time on a non-deterministic machine

• Can be verified but not necessarily solved in polynomial time on a normal (deterministic) machine

• Does not stand for “non-polynomial”

Decision Problems

• Yes / No (Boolean) problem• Can turn an optimization problem into a

decision problem by adding a target number

• Example:• Shortest path problem: Optimization problem• Decision version: Is there a path of size k or

less?

NP-Complete (NPC)

• Class of decision problems• Properties

• Must be in class NP (verifiable in polynomial time)

• Must be in class NP-Hard (at least as difficult as anything in NP)

NP

P

NP-Hard

NPC

P ≠ NP

NP-Hard

P = NP = NPC

P = NP

Completeness Theory

• If any NPC problem can be solved in polynomial time, then all NPC problems have polynomial time solutions (P = NP)• Whether this is the case or not is unknown

• One of the most important open problems• Millennium Prize: $1 million if you can answer it

• Most believe that P ≠ NP

Satisfiability (SAT)

• Let x1, x2, …, xn be Boolean variables

• Given a Boolean expression in conjunctive normal form (CNF), does there exist a variable assignment that makes it true?

• First known NP-complete problem

Maximum Clique

• A clique is a sub-graph of G where each vertex shares an edge with every other vertex

• Maximum clique problem: find the clique containing the largest number of vertices in G

• Decision problem: does a clique of at least k vertices exist within G?

Maximum Clique

Knapsack problem

• You can carry up to k kg in your backpack• There are a variety of items, each with

some value and weight• Maximize the value of the items you put in

your bag

15 kg capacity

$412 kg

$22 kg

$21 kg

$11 kg

$104 kg

Value: $15, Weight: 7 kg

Travelling Salesman Problem

• Give a list of cities and roads between them (a graph), find the shortest tour that visits each city

A B

C D

20

34

12

4230 35

Proving Hardness

• It is often helpful to prove a problem is NP-complete

• This is normally done by converting between problems• If you can solve the new problem, you can

solve the original

Proving Hardness

• If you can convert from an NP-complete problem to your problem, it must be NP-hard.

• If you can also convert back, then it is NP-complete

Maximum Independent Set (MIS)

• In a graph, an independent set is a set of vertices where no edge connects two vertices in the set.

• The MIS is the largest such set in the graph

Conversion

• Finding an independent set on G is the same as finding a clique on the complement of G (reversing whether edges exist)

• Since max clique is NP-complete, so is MIS

Max Clique MIS