Games and Complexity, Guangzhou 2009. Peter van Emde Boas KNOW YOUR NUMBERS ! The Impact of Complexity a short intro in Complexity theory Peter van Emde Boas ILLC-FNWI-UvA Guangzhou 2009
Dec 26, 2015
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
KNOW YOUR NUMBERS !The Impact of Complexity
a short intro in Complexity theory
Peter van Emde Boas
ILLC-FNWI-UvA
Guangzhou 2009
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Moore’s Law in action
© Scientifi American 274, 199601, p45
Computation speeddoubles every 1.5 years
Corresponding rate ofincrease ofMemory capacity
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
1982 1985
4 MHZ, 64K, 1.2M (FD) ? MHZ, 256K, 10/20 M (HD)
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
1986 1988
8 MHZ, 512K, 20/30M 16 MHZ, 1 M, 20/80M
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
1990 1992
12/33 MHZ, 1/4 M, 40/150M 25/40 MHZ, 4/32 M, 125/210M
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
1994 1996
25/60 MHZ, 4/8 M, 212/527M 166 MHZ, 8/32 M, 1/4 G
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
1998 2000
533/733 MHZ, 64/1024 M, 9/10 G166/300 MHZ, 32/64 M, 2.1/4 G
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
2003 2006
2/3 GHZ, 512 M, 40/80 G 1.8/2.6 GHZ, 512/1024 M, 80/320 G
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
2008
? GHZ, 2 G, ? G
2009
2.5 GHZ, 4 G, 250 G
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Is it really so nice ?
sum := 0for i = 1 to N do
sum := sum + a[i] ;print ( sum )
Back in 1977 this would work in less than a second for N = 10 000Can we deal with N = 400 000 000 today within a second ?
sum := 0for i = 1 to N do
j:= index[i] ; sum := sum + a[j] ;print ( sum )
And what about this one ?
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Information must be moved!
Source: Ph.d. Thesis Stefan ManegoldUnderstanding, Modeling, and Improving
Main-Memory Database PerformanceUvA Dec 16, 2002
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Information must be moved!
Source: Ph.d. Thesis Peter A. BonczMonet; A next-Generation
DBMS Kernel for Query-IntensiveApplications
UvA Mei 31, 2002
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Know your numbers !# Instructions Time
1 000 0.00001 sec1 000 000 0.01 sec Invisible10 000 000 0.1 sec Perceptive100 000 000 1 sec1 000 000 000 10 sec10 000 000 000 100 sec Coffee100 000 000 000 16.7 min Chat1 000 000 000 000 2 hr 47 min WHAARGH!10 000 000 000 000 27 hr the Limit!!100 000 000 000 000 11.25 days Hopeless1 000 000 000 000 000 4 months10 000 000 000 000 000 3.33 years Unrealistic100 000 000 000 000 000 33 years1 000 000 000 000 000 000 3.3 century
Processing times on a 100 MIPS system
1 day = 86400 sec 1 000 000 sec ≈ 11 days1 year ≈ 31 536 000 sec 1 000 000 000 sec ≈ 33 years
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Notation for asymptotic order of growth
f(x) = O( g(x) ) means f(x) / g(x) is bounded for x -->
f(x) = o( g(x) ) means f(x) / g(x) --> 0 for x -->
f(x) = ( g(x) ) means f(x) / g(x) > C > 0 for x -->
f(x) = ( g(x) ) means f(x) / g(x) is > C > 0 and bounded for x -->
f(x) = ( g(x) ) means f(x) / g(x) is unbounded for x -->
All functions here are non-negativeSimilar for integer argument n -->
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
WHAT IS COMPLEXITY ?
Relation between size of the input instance and the time required for solving this instance.
Traversing a list Linear O(n)Sorting O(n.logn)Multiplication Quadratic O(n2)Matrix multiplication Cubic O(n3)Truth table checking Exponential O(2n)Naive TSP O(n!)
What can be done in principle can become undoable in practice
Order of growth is more relevant than the implicit constants
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
The Order of growth
10 20 50 100 200 1000
n .00001 .00002 .00005 .0001 .0002 .001
n.logn .00003 .00008 .00027 .0006 .0016 .01
n2 .0001 .0004 .0025 .01 .04 1
n3 .001 .008 .125 1 8 16 min
2n .001 1 31 y ?? !!?*
Time on a 1 MIPS system in seconds (unless stated otherwise)
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Impact of a 1000 times faster machine
n 1000 times as large
n.logn 500 - 1000 times as large
n2 31 times as large
n3 10 times as large
2n add 10 to input size
That’s why Edmonds stated his thesis:Effective ==> Polynomial Time Complexity
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
A Fallacy ??
If computation speed doubles every 2 yearswe can perform (2N - 1) M operations in 2Nyears, where M is the number of operationsperformed in the first 2 years.
So why worry about exponential timealgorithms ??
What is wrong with this argument ?
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
The Fundamental Complexity Classes
A program Mj belongs to the Complexity Class of programs F(t)iff its run-time TMj (x) < t( |x| ) for all sufficiently large inputs x(here |x| denotes the length of x, and t is a function from N to N)
A function (set) belongs to the Complexity Class C(t) iffit is computed (recognized) by some program in the class F(t)
This explains classes like F( 60.n ) or C(n.log(n))
Similarly one can define Space bounded Complexity ClassesH(t) and S(t) .
NF(t) , NC(t) , NH(t) , and NS(t) also allow Nondeterministic Programs
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
DetailsThe fundamental distinction between a run-time TMj (x) and just an arbitrary computable function f(x) is that for a run-time the predicate:
TMj (x) < k is decidable,
whereas for general computable f(x) thisis not the case (simple reduction from the Halting Problem).(Second Blum Axiom in Abstract Complexity Theory)
Another complication results from the fact that runtimes are boundedin terms of the input length rather than the input itself:TMj (x) < t( |x| ) , rather than TMj (x) < t( x ) There are many inputs of the same length; the complexity isdetermined by the worst case input. If one input leads to divergencethe no finite bound applies.
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
The Fundamental Complexity ClassesLOGSPACE = S(k.log(n)) for k N
NLOG = NS(k.log(n)) for k N
P = C( nk ) for k N
NP = NC( nk ) for k N
PSPACE = S ( nk ) for k N
NPSPACE = NS( nk ) for k N
E = C( 2kn ) for k N
NE = NC( 2kn ) for k N
EXP = C( 2^ nk ) for k Netc....
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
The Fundamental Complexity Classes
The following Inclusions follow from these definitions:
LOGSPACE NLOG P NP PSPACE NPSPACE EXP
It is known that PSPACE = NPSPACE (Savitch Theorem)For all remaining inclusions it is unknown whether the inclusionis strict or not; E.G., the Notorious P = NP problem(still good for US $ 1 000 000 if you solve this question)
Some of the inclusions are known to be strict since LOGSPACE PSPACE and P E EXPAll inclusions are conjectured to be strict.
Problem: To what extent are these classes Machine Model Dependent ?
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
The Invariance Thesis
Reasonable Sequential MachinesSimulate Each Other with
Polynomial overhead in Time andConstant Factor Overhead in Space
M’ simulates M : …. ; no definition given
TM’( x ) ≤ k.TM( x )k
SM’( x ) ≤ k.SM( x )
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
ConsequenceThe classes in the fundamental ComplexityHierarchy:
LOGSPACE NLOGSPACE P NP PSPACE EXPTIME NEXPTIME
are machine independent.(Classes like LIN and NLIN are not)LIN = C( k.n ) for k N
NLIN = NC( k.n ) for k N
Games and Complexity, Guangzhou 2009. Peter van Emde Boas
Parallel Computation Thesis
// PTIME = // NPTIME = PSPACE
The symbol // refers to machines involvingParallellismTrue for Computational Models which combineExponential Growth potential withUniform Behavior.
The Second Machine Class