Prof. Erik Demaine - Massachusetts Institute of Technology · – Does the Rubik’s Cube always have a solution in 20 moves? – Given a sequence of cards, is there a Crazy Eights

6.006IntroductiontoAlgorithms

Lecture25:ComplexityProf.ErikDemaine

Today• Reductionsbetweenproblems• Decisionvs.optimizationproblems• Complexityclasses

–P,NP,co‐NP,PSPACE,EXPTIME,…• NP‐completeness

Reductions

http://cdn.zedomax.com/blog/wp‐content/uploads/2010/02/reuse_reduce_recycle.jpg

design by Gary Andersonhttp://en.wikipedia.org/wiki/File:Recycle001.svg

balsamicreductionhttp://www.allthingsolive.ca/2011/01/pecorino‐di‐fossa‐cheese‐pears‐and‐cinnamon‐pear‐balsamic/

HowtoDesignanEfficient Algorithm?

1. Definecomputational problem2. Abstract irrelevant detail3. Reducetoaproblemyoulearnhere

(or6.046oralgorithmicliterature)4. Elsedesignusing“algorithmictoolbox”5. Analyzealgorithm’sscalability6. Implement & evaluate performance7. Repeat(optimize,generalize)

Reductions• Insteadofsolvingaproblemfromscratch,convertyourproblemintoaproblemyoualreadyknowhowtosolve

• Examples:– Min‐productpath shortestpath(takelogs)– Longestpath shortestpath(negateweights)– Minmultiple‐of‐5path shortestpath( )– Unweighted weightedshortestpath(weight1)– 2Dpathplanning shortestpath(visibilitygraph)

Polynomial‐TimeReductions

• Considertwoproblems &• Polynomial‐timereduction :

– Solutionto usingsolutionto– Polynomial‐timealgorithmfor ,withfreecallstosubroutinetosolve

• Write :“ isnoharderthan ”(uptopolynomialoverhead)

One‐CallReductions• Commonpolynomial‐timereduction :

1. Giveninputtoproblem2. Polynomial‐timepreprocessing3. Onecalltosolveproblem4. Polynomial‐timepostprocessing

• Moreinterestingthan“reduceanyproblemtobasicoperationsinmodelofcomputation”– Example: sortingreducestoelementcomparisons

DecisionProblems• Decisionproblem=anyproblemwhoseanswerisonebit:“yes”or“no”

• Examples:– Dotheselinesegmentshaveanintersection?– DoesthisSuperMarioBros.levelhaveasolution?– Doesthe Rubik’sCubealwayshaveasolutionin20moves?

– Givenasequenceofcards,isthereaCrazyEightssubsequencetrickofatleast17cards?

– Doesgivenweightedgraphhaveanegativecycle?

Hamlet(1948)

Optimization Decision• Anyoptimizationproblemcanbeconvertedintoadecisionproblem

• Addinput :boundonoptimalsolution– Maximizationproblem isalowerbound– Minimizationproblem isanupperbound

• Examples:– Givensequenceofcards&number ,isthereaCrazyEightstrickof cards?

– Givenaweightedgraph,vertices & ,number ,istherean pathofweight ?

WhyDecisionProblems?• Metaclaim: Everycomputationalproblemhasadecisionversionofroughlyequalcomputationaldifficulty

• Maximization/minimizationproblems:binarysearchonbound tofindoptimal– Logarithmicoverhead

• Example: Iskey inthisbinarysearchtree?• Example: Givenunsorted and & ,does sorttorank inasortedarray?

KarpReductions• Fordecisionproblems and• Simplest(andmostcommon)typeofpolynomial‐time reduction:1. Giveninputtoproblem2. Polynomial‐timepreprocessing3. Onecalltosolveproblem4. Returnthesameanswer (nopostprocessing)

RichardKarp

ComplexityClasses

TimeClasses• P =classofalldecisionproblemssolvablebypolynomial‐timealgorithms–

• EXPTIME =classofalldecisionproblemssolvablebyexponential‐timealgorithms–

• Ingeneral,if ,then

(TimeHierarchyTheorem)

SpaceClasses• PSPACE =classofalldecisionproblemssolvableusingpolynomialstoragespace

• Example: IsthereasolutiontoagivenRubik’sCubeusing moves?

• EXPSPACE =classofalldecisionproblemssolvableusingexponentialstoragespace

• SpaceHierarchyTheorem:if,then

NP• NP =classofalldecisionproblemssolvablebya“lucky”polynomial‐timealgorithm– In time,canguess betweentwochoices– Attheend,report“yes”or“no”– Ifanywaytosay“yes”,actuallyreturn“yes”(alwaysmakerightchoice)

• Example: IsthereasolutiontoagivenRubik’sCubeusing moves?

– Guessfirstmove,secondmove,…, th move– Return“yes”ifsolved

NPRemix• Equivalently,NP =classofalldecisionproblemsverifiable inpolynomialtime– Every“yes”inputhasapolynomial‐lengthcertificate (mightbeveryhardtofind)

– Giveninputandcertificate,canconfirmthatansweris“yes”inpolynomialtime

• Example: IsthereasolutiontoagivenRubik’sCubeusing moves?

– Certificate=sequenceof movestosolution

NPProblems• Givenagraph,doesithavevertexcoverofsize ?

• Givenagraph,isthereasimplepathoflength ?

• Givenagraph,canitbecoloredwith3colors?

• GivenaBooleancircuit,arethereinputsthatmaketheoutputtrue? OR

ANDOR

NP‐Completeness• NP‐complete =NPproblemthatisatleastashardasallproblemsinNP– Formally: and forall– HardestproblemsinNP,allessentiallyequivalent

• Ifthereareanyproblemsin ,thenNP‐completeproblemsareamongthem

• IfanyNP‐completeproblemhasapolynomial‐timealgorithm,then

StephenCook

ComplexityClasses

PowerofReduction• If isNP‐complete, ,and ,then isNP‐complete

• Proof:– forall–– forall

ProvingNP‐Completeness• StartwithanyknownNP‐completeproblem– Givenagraph,doesithavevertexcoverofsize ?

– Givenagraph,isthereasimplepathoflength ?

– Givenagraph,canitbecoloredwith3colors?

– GivenaBooleancircuit,arethereinputsthatmaketheoutputtrue?

• Prove and

IndependentSet• Independentset =subsetofverticesinducingnovertices

• Problem: Givenagraph andinteger ,isthereanindependentsetofsize ?

VertexCoverIndependentSet

• hasavertexcoverofsizeifandonlyif

hasanindependentsetofsize• So reduces• VCisNP‐complete ISis NP‐complete

VertexCover IndependentSet

DominatingSet• Dominatingset =subsetofverticessuchthateveryothervertexisadjacenttosomeoneinthesubset

• Problem: Givenagraph andinteger ,isthereadominatingsetofsize ?

VertexCoverDominatingSet

• hasavertexcoverofsizeifandonlyif

hasadominatingsetofsize• So reduces• VCisNP‐complete DSis NP‐complete

VertexCover DominatingSet

Phutball[Conway]

“Matein1”(canIwininonemove?)NP‐complete[Demaine,Demaine,Eppstein 2000]

a b c

a b c

a

a

b

b

c

c

(a b c) (a b c)

Bad&GoodNews• ManyproblemsareNP‐complete• Canoftenfindapproximatesolutionsinpolynomialtime– Within factorofoptimal(e.g.,VertexCover)

– Within0.0001%ofoptimal(e.g.,VertexCoverinplanargraphs)