Theory of Computer Algorithms (4005-800-01): Introduction Prof. Richard Zanibbi
Theory of Computer Algorithms (4005-800-01):
IntroductionProf. Richard Zanibbi
Course Administrative Information
Course Web Page
http://www.cs.rit.edu/~rlaz/algorithms20082
Course Syllabus and Tentative Schedule
Available from course web pages; handout
Resources
Additional textbooks/references/URLs listed on course web pages; some books to be placed on 2hr reserve in Watson library 2
Why and How do We Study Algorithms?
How we study algorithms: Space and Time
The Time-Space Tradeoff
Generally speaking, the more space used to store information, the less time needed to compute desired information, and vice versa.
• Simple example: table lookup for exponents, vs. function computing exponents iteratively
• Depending on the problem, at times we also want to consider other resources (e.g. power on mobile devices)
4
Emphasis for Algorithm Analysis:Worst-Case Time RequirementsWhy emphasize time?
In general, unless the space requirements are truly excessive, we want the fastest algorithm
• Analyses related to time often generalize quite directly to space and other resource types
• Note: while fast algorithms are useful, speed is not the only criteria for selecting an algorithm
Worst-case? Why so pessimistic?
Worst-case analyses are useful in practice and provide guarantees 5
A Benchmark:Brute-Force Search
Operation
Brute-force search enumerates the set of possible solutions, and checks each one
• e.g. to sort a list of numbers, generate all permutations until a sorted list is found
• Note: while inelegant and inefficient, brute-force search is correct, making it a useful for understanding and comparison
Our Goal
Preserve correctness, while finding faster solutions; apply knowledge to produce more informed algorithms 6
!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./011%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"
!"#$%&'(#$)*+,-.&"/%$)0($12-3,-/)0425
2 3456'7$8%9(8"4#(:9($6(:;<"'9$=;<(!"#$#%&$&'(0
2 >"'?6'%=;@"(6?$";(<'=A9($8"(47;"(&"$A"";(A8=$(79(?"=97&4"(=;<(A8=$(79(7%#6997&4"0
2 3456'7$8%7@(%=$8"%=$7@9(#'6B7<"9(=($#)*+#*,?6'($=4C7;5(=&6:$(#'65'=%(&"8=B76'0
2 >"'?6'%=;@"(79($8"("+--,)"( 6?(@6%#:$7;50
2 D8"(4"996;9(6?(#'65'=%(#"'?6'%=;@"(5";"'=47E"($6(6$8"'(@6%#:$7;5('"96:'@"90(
2 !#""<(79(?:;F
Measuring Algorithm Performance
T(n)
The time, in “primitive” computations needed by an algorithm for input size n (normally worst-case analysis)
• Primitive computations might be: assembly instructions, a line of C/Java, a basic operation in pseudo code (e.g. assignment, addition)
• Goal: machine-independent performance measure
8
!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./0+,1%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"
!"#$%&'(&)#)*+%,%
-'.%/01)%,2&123245567
8 A1"7(9 %4:;%2%($;%"(<=(45><';$?%(<@(4@6(;@#2$(<=(3;A"("0
34,.)5,01)%,2&13<%"$;%"37
8 A1"7(9 ":#"B$"C($;%"(<=(45><';$?%(<D"'(455(;@#2$3(<=(3;A"("0
8 E""C(4332%#$;<@(<=(3$4$;3$;B45(C;3$';&2$;<@(<=(;@#2$30
6,%/01)%,2&1&<>237
8 F?"4$(G;$?(4(35<G(45><';$?%($?4$(G<'H3(=43$(<@(0%/> ;@#2$0
Efficient AlgorithmsPolynomial Time Algorithm
If there exist constants c and d (c, d > 0) such that running time is bounded by cnd for all input sizes (n)
Efficient Algorithm (Page 33 K&T)
An algorithm is efficient if it has a polynomial running time
• This measure (normally) reflects efficiency of algorithms and tractability of problems in practice
• Problems with polynomial solutions usually require low order polynomials (e.g. n, n2, n3) 10
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
Example: The Problem of Stable Matching
(Chapter 1.1, K&T)
Problem Definition
Task
Design a procedure to match individuals in two sets that is self-enforcing (i.e. stable)
Example
Matching job applicants to employers (e.g. for co-op positions)
13
Problem FormulationIssues in the General Case
Asymmetric matching: companies need multiple employees, applicants need one job
Sizes: we may have a different sized sets
Simplification
Sets to be matched are of the same size, and each individual is in exactly one match (pair)
All individuals have a preference list ranking matches with individuals of the other set
14
Matches
Perfect Match
All individuals in both sets (e.g. men and women) are paired with a unique partner
Stable Match (Our Goal)
A matching that is perfect and stable, where no two individuals mutually prefer to leave their matching in order to join together
15
Copyright © 2005 Pearson Addison-Wesley. All rights reserved. 1-2
A Solution:Gale-Shapley Algorithm
17Copyright © 2005 Pearson Addison-Wesley. All rights reserved. 1-4
Copyright © 2005 Pearson Addison-Wesley. All rights reserved. 1-3
Properties of theG-S Algorithm
Upper Bound on Worst-Case Run-time (Performance)
(1.3) G-S Terminates after at most n2 iterations of the while loop
Match Properties (Correctness)
(1.1) Each woman remains engaged from their first proposal, and the sequence of partners to which she becomes engaged gets better and better (according to her preference list)
(1.4) If m is free at some point, then there is a woman to whom he has not proposed
(1.5) The set S returned at termination is a perfect matching
(1.6) The set S returned at termination is a stable matching19
Additional Properties
(1.7) Every execution of the G-S algorithm yields the same matching S*, where S* = {(m, best(m)) : m ε M)}.
(1.8) In stable matching S*, each woman is paired with her worst valid partner
valid partner: partner in a stable matching
best valid partner: highest ranked partner in a stable matching
20
Overview: Five Representative Problems
(Chapter 1.2 K&T)
I. Interval Scheduling Problem
Task
Assign a resource (e.g. lecture hall) to requests with known time intervals, maximizing the number of satisfied requests that do not conflict
Solved By
Greedy algorithm: sort requests, then single pass produces solution
22
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
2. Weighted Interval Scheduling Problem
Modification
Requests have an associated weight vi > 0. New goal is to maximize weight of satisfied requests that do not conflict
Special Case
If for all i vi = 1, instance of regular Interval Scheduling problem
Solved By (not greedy alg!)
Dynamic Programming: build optimal value over all possible solutions using an efficient table-based strategy 23
3. Bipartite Matching Problem
Task
Find a the largest set of edges producing disjoint pairs of nodes in a bipartite graph
• e.g. each x paired with a unique y
Solution (not greedy or dynamic!)
Augmentation: Inductively construct larger and larger matchings with backtracking (used in network flow problems)
24
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
4. Independent Set Problem
Task
Identify max no. of nodes not joined by an edge in a graph
• Edges represent ‘conflicts’
• Interval scheduling, bipartite matching instances of I.Set problem
Solution (general case)
No efficient algorithm believed to exist (can use brute force). Problem is NP-complete 25
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
5. Competitive Facility Location
Task
Two companies take turns selecting locations (nodes) forming an independent set, trying to maximize the value of selected nodes. Is there a strategy for player 2 guaranteeing a node set with at least value B?
26
Copyright © 2005 Pearson Addison-Wesley. All rights reserved.
(Comp. Facil. Location, Cont’d)
Solution
No short ‘proof’ for a solution; requires detailed case analysis (i.e. game traces); problem is PSPACE-complete (believed harder than NP-complete problems)
Many game playing and planning problems belong to PSPACE
27
Asymptotic Order of Growth(Section 2.2, K&T)
Asymptotic ComplexityDefinition
Characterizes worst-case running time of an algorithm in the limit, i.e. as input size goes to infinity
• Rate of growth defined as proportional to some function f(n) (i.e. within a constant multiple of f(n))
• f(n) normally simple (e.g. n2), not a detailed characterization such as: 1.62n2 + 3.5n + 8
• Consider upper (big ‘O’), lower (Ω), and tight (Θ) bounds
(One) Purpose
Identify sets of algorithms with similar behavior 29
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/*
!"#$%&'&()*+'&,&('+
=012$3$421(56##"'(&261789:
;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/
;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/0
!"#$%&'&()*+'&,&('+
=123$4$532(67##"'(&37289:;
<"(='5$"(>68:(>(=6'68:: 5?($@"'"("A59$(B329$42$9(? C(,+(8, C(, 97B@($@4$(,(! >68:(! ?'68: ?3'(4DD(8 " 8,/
<"(='5$"(>68:(>(=6'68:: 5?($@"'"("A59$(B329$42$9(? C(,+(8, C(, 97B@($@4$(,(! >68:(! ?'68: ?3'(4DD(8 " 8,/
-.!/01-2 *8* >(=680: 6? >()+(8, >(*:
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/-
!"#$%&'&()*+'&,&('+
=012$3$421(56##"'(&261789:
;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/
;"(<'4$"(>589(=(=5'5899 4>($?"'"("@48$(A218$31$8(? B(,+(8, B(, 86A?($?3$(,(! >589(! ?'589 >2'(3CC(8 " 8,/
-.!/01-2 *8* =(=58D9 5? =()+(8, =(*9
>@8?)&"8;A*8")*B7:@5;
>@88$A*C"85/D7$E5F@7:&)$
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/0
!"#$%"&'('#')($)&$*+()#,#')(
=1'1822 3(4(>182(5 $6"'"("789$(:;<9$=<$9(? >(,+(8, >(, 9?:6($6=$(,(! >182(! ?'182@;'(=AA(8 " 8, B
=1'1822 3(4(>182(5 $6"'"("789$(:;<9$=<$9(? >(,+(8, >(, 9?:6($6=$(,(! >182(! ?'182@;'(=AA(8 " 8, B
-./012-3 *8* # =18C2
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/),
!"#$%&'()'*+*(*+%,
!"#$%#&'"#( 0(1"$(23(4(56'%784('"#'"1"3$1(43(43639%671(573:$263(23($;"(1"$/
=<8=(>(8? @(><8*=(
%"431(
=<8=(>(8? @((<8=
56'(16%"((<8=(! ><8*= /
-./!01-2
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/))
!"#$%&'()'*+*(*+%,
!"#$%#&'"#( 0(1"$(23(4(56'%784('"#'"1"3$1(43(43639%671(573:$263(23($;"(1"$/
8* <(==8>(?(==8*>
%"431
56'(439(>=8>(! ==8>@
8* <(>=8>(?((=8>
56'(16%"((=8>(! ==8*> /
-./!01-2
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)0
!!"#$%$&#"'()#*+,'-#."/01
=123$4$532(56(42(>##5%/?">89 23$4$532/((7$(%48"6(23(6"26"($3(649(@:8; 56(4$(<"46$(=:8*;/
!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F
!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)0
!!"#$%$&#"'()#*+,'-#."/01
=123$4$532(56(42(>##5%/?">89 23$4$532/((7$(%48"6(23(6"26"($3(649(@:8; 56(4$(<"46$(=:8*;/
!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F
!"':8;; =(>(@:8;(? $@"'"("A56$(B326$42$6(A C(,+(8, C(, 6DB@($@4$(,(# A':8;(# @:8;E3'(4<<(8 $ 8, F
23456728 :A =()+(8, =()G;;:<H88 !%
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)-
!!"#$%$&#"'($&)*$'+#,"-./
!"'0811(2(30'0811((# $0'0811!"'0811(2(30'0811((# $0'0811
f(n) = Θ(g(n)) means that both f(n) = O(g(n)) and f(n) = Ω(g(n))
!"#$"%&"'()*+(*,,- !"#$%&'()*+ ,--./0*1%&2*34*3567&85*789*!(7%:5;*14*<5&;5%;"8 .*/)0
!!"#$%$&#"'($&)*$'+#,"-./
!"'1822(3(41'1822((# $1'1822!"'1822(3(41'1822((# $1'1822
21* **
*
)888 !%&01234506
f(n) = Θ(g(n)) means that both f(n) = O(g(n)) and f(n) = Ω(g(n))
!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./0++1%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"
Θ!"#$%$&#"
1 2'3#(43563'7"'($"'%89(:;<3'"(4"=7:<;(>3<8$=<$80
1 ?@=%#4"A(B"B(C(D,"5 E -" C(F,GF(H(ΘI"BJ
!"#$%ΘI-I"JJ(H(K(A*I"J A $L"'"("@:8$(#38:$:M"(>3<8$=<$8((/*((+*(=<7(
", 8N>L($L=$(,(≤ (/(-I"J(≤ A*I"J(≤ (+(-I"JO3'(=44(" ≥ ", P
&'()'**+)'(%
!"#$"%&"'()*(+,,- !"#$%&'(#)%"*#%*+,-%$)#./0 ./0+11%23$)-.#*4 566789*:$);*<=*<>/?)"> ?"&*1.?$,>0*:=*@>)0>$0%"
!"#$%&'&()*%+,-',$./)+
"
A2"3
",
4 5"(6789:;<=$(>?<8'"(@6A%#$8$>B@::A(6:8C"'(@:?8'>$7%6*(78C"D"'0
4 E"@:FC8':;(;"6>?<(6>$9@$>8<6(8G$"<(B@::(G8'(@(B@'"G9:(&@:@<B><?(8G("<?><""'><?(8&H"B$>D"60
4 I6A%#$8$>B(@<@:A6>6(>6(@(96"G9:($88:($8(7":#($8(6$'9B$9'"(89'($7><J><?0
57"<(" ?"$6(:@'?"("<89?7*(@(Θ2"+3 @:?8'>$7%(?,B?30 &"@$6(@(Θ2"13 @:?8'>$7%0