7/25/2019 01 Iterative methods
1/356
1
Iterative Algorithms
& Loop Invariants
Jeff Edmonds
York University COSC 3101LectureLecture11
Understanding Algorithms Abstractly
Home to School Problem
Steps in an Iterative Algorithm
Assertions
Insertion SortDesigning an Algorithm
Finding Errors
ypical Loop Invariants
!inary Search Li"e E#ampleshe Partition Problem
$reatest %ommon Divisor
!c"et '(ic") Sort *or Hmans
+adi#,%onting Sort
Lo-er !ond *or Sort
7/25/2019 01 Iterative methods
2/356
.
he $oal is to
U/DE+SA/D
Algorithms
Fndamentally to their core
7/25/2019 01 Iterative methods
3/356
0
+epresentation
Understand the relationship bet-een
di**erent representations o* the same
in*ormation or idea
1
2
3+dich ---2discretemath2com
7/25/2019 01 Iterative methods
4/356
3
Di**erent +epresentations
o* Algorithms4 %ode
4 +nning e#ample
4 DFA and ring 5achines
4 Higher level abstract vie-
7/25/2019 01 Iterative methods
5/356
6
%ode
+epresentation o* an Algorithm class InsertionSortAlgorithm e#tends SortAlgorithm 7
void sort'int a89) thro-s E#ception 7
*or 'int i : 1; i < a2length; i==) 7
int > : i;
int ! : a8i9;
-hile ''> ? @) && 'a8>19 ? !)) 7
a8>9 : a8>19;
>; B
a8>9 : !;
BB Pros and %onsC
7/25/2019 01 Iterative methods
6/356
%ode
+epresentation o* an Algorithm
4 +ns on compters
4 Precise and sccinct
4 Perception that being
able to code is the
only thing needed toget a >ob2 '*alse)
4 I am not a compter
4 I need a higher level o*
intition2
4 Prone to bgs
4 Langage dependent
Pros %ons
7/25/2019 01 Iterative methods
7/356
+nning E#ample
+epresentation o* an Algorithm
ry ot a problem or
soltion on small e#amples2
7/25/2019 01 Iterative methods
8/356
+nning E#ample
+epresentation o* an Algorithm
13
G.6
.
6.
G
0@
.0
01
13
G
7/25/2019 01 Iterative methods
9/356G
+nning E#ample
+epresentation o* an Algorithm
Pros and %onsC
7/25/2019 01 Iterative methods
10/3561@
+nning E#ample
+epresentation o* an Algorithm
4 %oncrete
4 Dynamic
4 isal
4 +elies on yo to *ind the
pattern2
4 Does not e#plain -hy it
-or"s2
4 Demonstrates *or onlyone o* many inpts2
Pros %ons
7/25/2019 01 Iterative methods
11/35611
DFA and ring 5achines
+epresentation o* an Algorithm
Pros and %onsC
7/25/2019 01 Iterative methods
12/3561.
DFA and ring 5achines
+epresentation o* an Algorithm
4 $ood *or theorems
abot algorithms
4 /ot good *or designing
algorithms
Pros %ons
7/25/2019 01 Iterative methods
13/35610
Higher Level Abstract ie-
+epresentation o* an Algorithm
i1i
i
i
@=
1
7/25/2019 01 Iterative methods
14/35613
Levels o* Abstraction
It is hard to thin" o* love in terms o* the *iring o*
nerons2
vs
So*t-are developers vie- sbsystems as entities -ithseparate personalities roles and interactions
not details o* code2
vs
7/25/2019 01 Iterative methods
15/35616
Higher Level Abstract ie-
+epresentation o* an Algorithm
4 Intitive *or hmans
4 Use*l *or
J thin"ing abot
J designing
J describing algorithms
4 ie- *rom -hich
correctness is sel*
evident2
4 5athematical mmbo
>mbo
4 oo abstract
4 Stdents resist it
Pros %ons
7/25/2019 01 Iterative methods
16/3561
Abstract a-ay the inessential *eatres o* a problem
=
ale Simplicityale Simplicity
$oal Understand and thin" abot comple#algorithm in simple -ays2
DonKt tne ot2 here are deep ideas -ithin
the simplicity2
7/25/2019 01 Iterative methods
17/3561
Levels o* Abstraction
4 $iven a concrete algorithm eg2 5ergeSortand a concrete inpt eg2
7/25/2019 01 Iterative methods
18/3561
Levels o* Abstraction
4 $iven a ne- comptational problem on e#amdesign a ne- concrete algorithm that solves it
*ollo-ing the M5eta AlgorithmN steps2
4 I* yo do not get all the details o* the ne-algorithm correct it -ont be a disaster2
4 I* yo do not get the steps o* the M5eta
Algorithm correct it -ill be a disaster24 here are only a *e- M5eta AlgorithmsN
Learn themO
7/25/2019 01 Iterative methods
19/3561G
Levels o* Abstraction
4 M5eta AlgorithmsNJIterative Algorithms
J+ecrsive Algorithms
J$reedy AlgorithmsJ+ecrsive !ac" rac"ing
JDynamic Programming2
J/P%ompleteness
7/25/2019 01 Iterative methods
20/356.@
Levels o* Abstraction
4 M5eta 5athNJE#istential and Universal (anti*iers
JAsymptotic $ro-th
JAdding 5ade Easy Appro#imationsJ+ecrrence +elations
7/25/2019 01 Iterative methods
21/356.1
ale %orrectness
4 DonKt tac" on a *ormal proo* o* correctness
a*ter coding to ma"e the pro*essor happy2
4 It need not be mathematical mmbo >mbo2
4 $oal o thin" abot algorithms in sch -aythat their correctness is transparent2
7/25/2019 01 Iterative methods
22/356..
E#plaining an Algorithm
4 $oal o provide a nmber o*
di**erent notations analogies and
abstractions -ithin -hich todevelop thin" abot and
describe algorithms2
7/25/2019 01 Iterative methods
23/356
.0
-o ey ypes o* Algorithms
4 Iterative Algorithms
4 +ecrsive Algorithms
i l i h
7/25/2019 01 Iterative methods
24/356
.3
4 A good -ay to strctremany programs
4 Store the "ey in*ormationyo crrently "no- in somedata strctre2
4 In the main loop ta"e a step*or-ard to-ards destination
4 by ma"ing a simple changeto this data2
Iterative Algorithmsloop 'ntil done)
ta"e step
end loop
7/25/2019 01 Iterative methods
25/356
.6
I li"e nderstanding thingssing a story2
I -ill no- tell one2
7/25/2019 01 Iterative methods
26/356
.
he $etting to School Problem
7/25/2019 01 Iterative methods
27/356
.
Problem Speci*ication4Pre condition location o* home and school
4Post condition traveled *rom home to school
7/25/2019 01 Iterative methods
28/356
.
Algorithm4Qhat is an AlgorithmC
7/25/2019 01 Iterative methods
29/356
.G
Algorithm4he algorithm de*ines the comptation rote2
7/25/2019 01 Iterative methods
30/356
0@
Algorithm4Is this reasonableC
7/25/2019 01 Iterative methods
31/356
01
%omple#ity4here are an in*inite nmber o* inpt instances
4Algorithm mst -or" *or each2
7/25/2019 01 Iterative methods
32/356
0.
%omple#ity4Di**iclt to predict -here comptation might be
in the middle o* the comptation2
7/25/2019 01 Iterative methods
33/356
00
4he crrent MStateN o* comptation is
determined by vales o* all variables2
Location o* %omptation
7/25/2019 01 Iterative methods
34/356
03
4Sppose the comptation ends p hereC
Location o* %omptation
7/25/2019 01 Iterative methods
35/356
06
DonKt Panic4Qhere ever the comptation might be
ta"e best step to-ards school2
$ l i i l
7/25/2019 01 Iterative methods
36/356
0
$eneral Principle4Do not -orry abot the entire comptation2
4a"e one step at a timeO
D *i i Al i h
7/25/2019 01 Iterative methods
37/356
0
De*ining Algorithm4Qherever the comptation might be
ta"e best step to-ards school2
"
7/25/2019 01 Iterative methods
38/356
0
a"e a step4Qhat is reRired o* this stepC
A 5 * P
7/25/2019 01 Iterative methods
39/356
0G
A 5easre o* Progress
G "m
to school
6 "m
to school
D *i i Al i h
7/25/2019 01 Iterative methods
40/356
3@
G "m 6 "m
De*ining Algorithm4Is this s**icient to de*ine a -or"ing algorithmC
Q "i Al i h
7/25/2019 01 Iterative methods
41/356
31
Qor"ing Algorithm4%omptation steadily approaches goal
G "m
to school
6 "m
to school
D *i i Al ith
7/25/2019 01 Iterative methods
42/356
3.
De*ining Algorithm4E#tra reRirements
2GGG "m
G "m
@ "mkm
G "m 6 "m
7/25/2019 01 Iterative methods
43/356
30
4Qherever the comptation might be
ta"e best step to-ards school2
De*ine a Step
7/25/2019 01 Iterative methods
44/356
33
4Some location too con*sing *or algorithm
4Algorithm too lay to de*ine step *or every
location2
%omptation Stc"
7/25/2019 01 Iterative methods
45/356
36
4Algorithm speci*ies *rom -hich locations
it "no-s ho- to step2
Sa*e Locations
i
7/25/2019 01 Iterative methods
46/356
3
4Mhe comptation is presently in a sa*e location2N
45aybe tre and maybe not2 I* not something has gone -rong2
Loop Invariant
D *i i Al ith
7/25/2019 01 Iterative methods
47/356
3
De*ining Algorithm4From every sa*e location
de*ine one step to-ards school2
" t
7/25/2019 01 Iterative methods
48/356
3
a"e a step4Qhat is reRired o* this stepC
5 i i L I i
7/25/2019 01 Iterative methods
49/356
3G
4I* the comptation is in a sa*e location
it does not step into an nsa*e one2
5aintain Loop Invariant
5 i i L I i
7/25/2019 01 Iterative methods
50/356
6@
4%an -e be assred that the comptation
-ill al-ays be in a sa*e locationC
4I* the comptation is in a sa*e locationit does not step into an nsa*e one2
5aintain Loop Invariant
5 i i L I i
7/25/2019 01 Iterative methods
51/356
61
4%an -e be assred that the comptation
-ill al-ays be in a sa*e locationC
4I* the comptation is in a sa*e locationit does not step into an nsa*e one2
5aintain Loop Invariant
/o2 Qhat i* it is not initially treC
E t bli hi L I i t
7/25/2019 01 Iterative methods
52/356
6.
From the Pre%onditions on the inpt instance
-e mst establish the loop invariant2
Establishing Loop Invariant
5 i t i L I i t
7/25/2019 01 Iterative methods
53/356
60
5aintain Loop Invariant
4%an -e be assred that the
comptation -ill al-ays bein a sa*e locationC
4!y -hat principleC
5 i t i L I i t
7/25/2019 01 Iterative methods
54/356
63
5aintain Loop Invariant4!y Indction the comptation -ill
al-ays be in a sa*e location2
+
S
i S i S i
i S i
' )
' ) ' )
' )
@
1
7/25/2019 01 Iterative methods
55/356
66
Ending he Algorithm
E#itDe*ine E#it %ondition
ermination@ "m E#it
Qith s**icient progress
the e#it condition -ill be met2
E#it
*rom these -e mst establish
the post conditions2
E#it
Qhen -e e#it -e "no-4e#it condition is tre4loop invariant is tre
LetKs +ecap
7/25/2019 01 Iterative methods
56/356
6
Let s +ecap
Designing an Algorithm
Is this s**icientC
7/25/2019 01 Iterative methods
57/356
6
Designing an AlgorithmDe*ine Problem De*ine Loop
Invariants
De*ine 5easre o*Progress
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km
G "m
to school
E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
Is this s**icientC
%onsider an Algorithm
7/25/2019 01 Iterative methods
58/356
6
%onsider an Algorithm
km E#it@ "m E#it
E#it
G "m 6 "m
E#it
Loop Invariant 5aintained
7/25/2019 01 Iterative methods
59/356
6G
Loop Invariant 5aintained
E#it
%omptation can al-ays proceed
7/25/2019 01 Iterative methods
60/356
@
%omptation can al-ays proceed
%omptation al-ays ma"es progress
7/25/2019 01 Iterative methods
61/356
1
%omptation al-ays ma"es progress
G "m 6 "m
G "m 6 "m
E#it
%omptation erminates
7/25/2019 01 Iterative methods
62/356
.
%omptation erminates
km
G"m 6 "m
@ "m
@ "m
@ "m E#it
E#it
%omptation erminates
7/25/2019 01 Iterative methods
63/356
0
%omptation erminates
E#it
E#it
%onsider an Algorithm
7/25/2019 01 Iterative methods
64/356
3
%onsider an Algorithm
his is s**icientO
km E#it@ "m E#it
E#it
G "m 6 "m
E#it
7/25/2019 01 Iterative methods
65/356
6
his completesTne Higher Level Abstract ie-
o* Iterative Algorithms
7/25/2019 01 Iterative methods
66/356
Assertions
Qe -ill no- redo
the previos ideasa little more *ormally
Paradigm Shi*t
7/25/2019 01 Iterative methods
67/356
Paradigm Shi*t
Is theblac"the *orm and the greenthe bac"grondC
Is the greenthe *orm and theblac"the bac"grondC
It is help to have di**erent -ays o* loo"ing at it2
A SeRence o* Actions
7/25/2019 01 Iterative methods
68/356
R
5a#' abc )
i*' b?m )
m : b
endi*
m : a
i*' c?m )
m : c
endi*
retrn'm)
I* yo are describing an alg to a
*riend is this -hat yo sayC
Do yo thin" abot an algorithm asa seRence o* actionsC
Do yo e#plain it by saying
MDo this2 Do that2 Do thisNC
Do yo get lost abot
-here the comptation is
and -here it is goingC
Qhat i* there are many pathsthrogh many i*s and loopsC
Ho- do yo "no- it -or"s
*or every pathon every inptC
A SeRence o* Actions
7/25/2019 01 Iterative methods
69/356
G
R
5a#' abc )
i*' b?m )
m : b
endi*
m : a
i*' c?m )
m : c
endi*
retrn'm)
At least tell me -hat the algorithm
is spposed to do2
4 PreconditionsAny assmptions that mst be treabot the inpt instance2
4 Postconditions
he statement o* -hat mst be tre-hen the algorithm,programretrns2
Mpre%ond Inpt has 0 nmbers2N
Mpost%ond
retrn ma# in 7abcBN
A SeRence o* Actions
7/25/2019 01 Iterative methods
70/356
@
R
5a#' abc )
i*' b?m )
m : b
endi*
m : a
i*' c?m )
m : c
endi*
retrn'm)
Ho- can yo possibly nderstand
this algorithm -ithot "no-ing
-hat is tre -hen the comptationis hereC
Mpre%ond Inpt has 0 nmbers2N
Mpost%ond
retrn ma# in 7abcBN
A SeRence o* Actions
7/25/2019 01 Iterative methods
71/356
1
R
5a#' abc )
i*' b?m )
m : b
endi*
m : a
i*' c?m )
m : c
endi*
retrn'm)
Ho- can yo possibly nderstand
this algorithm -ithot "no-ing
-hat is tre -hen the comptationis hereC
Tell me!
Massert m is ma# in 7abBN
Mpre%ond Inpt has 0 nmbers2N
Mpost%ond
retrn ma# in 7abcBN
7/25/2019 01 Iterative methods
72/356
.
ale %orrectness
4 DonKt tac" on a *ormal proo* o* correctness
a*ter coding to ma"e the pro*essor happy2
4 It need not be mathematical mmbo >mbo24 $oal o thin" abot algorithms in sch -ay
that their correctness is transparent2
A SeRence o* Actionss
7/25/2019 01 Iterative methods
73/356
0
5a#' abc )
Mpre%ond Inpt has 0 nmbers2N
i*' b?m )
m : b
endi*
Massert m is ma# in 7aBN
m : a
i*' c?m )
m : c
endi*
Massert m is ma# in 7abBN
Massert m is ma# in 7abcBN
retrn'm)
Mpost%ond
retrn ma# in 7abcBN
vs
A SeRence o* Assertions
It is help to havedi**erent -ays o*
loo"ing at it2
7/25/2019 01 Iterative methods
74/356
3
Prpose o* Assertions
Use*l *or
Jthin"ing abot algorithmsJdeveloping
Jdescribing
Jproving correctness
7/25/2019 01 Iterative methods
75/356
6
De*inition o* Assertions
An assertion is a statement
abot the crrent state o* the data
strctre that is either tre or *alse2
eg2 the amont in yor ban" accont
is not negative2
7/25/2019 01 Iterative methods
76/356
De*inition o* Assertions
It is made at some particlar point dringthe e#ection o* an algorithm2
It shold be tre independent o*the path *ollo-ed throgh the code
and independent o* the inpt2
I* it is *alse then something has gone-rong in the logic o* the algorithm2
7/25/2019 01 Iterative methods
77/356
De*inition o* Assertions
An assertion is not a tas" *or the
algorithm to per*orm2
It is only a comment that is added
*or the bene*it o* the reader2
7/25/2019 01 Iterative methods
78/356
Dont !e Frightened
An assertion need not consist o* *ormal
mathematical mmble >mble
Use an in*ormal description
and a pictre
5a#' A )Speci*ying a %omptational
7/25/2019 01 Iterative methods
79/356
G
5a#' A )
Mpre%ond
Inpt is array A81n9
o* n vales2N
i : 1
m : A819
loope#it -hen 'i:n)
i : i = 1
m : ma#'mA8i9)
endloopretrn'm)
Mpost%ond
m is ma# in A81n9N
4 PreconditionsAny assmptions that mst be
tre abot the inpt instance2
4Postconditionshe statement o* -hat mst
be tre -hen the
algorithm,program retrns2
Problem
7/25/2019 01 Iterative methods
80/356
@
De*inition o* %orrectness
7/25/2019 01 Iterative methods
81/356
1
7/25/2019 01 Iterative methods
82/356
.
7/25/2019 01 Iterative methods
83/356
0
7/25/2019 01 Iterative methods
84/356
3
7/25/2019 01 Iterative methods
85/356
6
7/25/2019 01 Iterative methods
86/356
' )
Mpre%ond
Inpt has 0 nmbers2N
m : aMassert m is ma# in 7aBN
i*' b?m)
m : b
endi*
Massert m is ma# in 7abBN
i*' c?m)
m : c
endi*
Massert m is ma# in 7abcBNretrn'm)
Mpost%ond
retrn ma# in 7abcBN
E#ample
7/25/2019 01 Iterative methods
87/356
E#ample
7/25/2019 01 Iterative methods
88/356
E#ample
7/25/2019 01 Iterative methods
89/356
G
Mpre%ond
Inpt has 0 nmbers2N
m : aMassert m is ma# in 7aBN
i*' b?m)
m : b
endi*
Massert m is ma# in 7abBN
i*' c?m)
m : c
endi*
Massert m is ma# in 7abcBNretrn'm)
Mpost%ond
retrn ma# in 7abcBN
E#ample
7/25/2019 01 Iterative methods
90/356
G@
Loop Invariants pre%ond Inpt is array A81n9
o* n vales2N
i : 1
m : A819
loop
e#it -hen 'i:n)
i : i = 1
m : ma#'mA8i9)endloop
retrn'm)
Mpost%ond
m is ma# in A 1n N
Any assmptions that
mst be tre abot the
state o* comptation
every time it is at the
top o* the loop2
Loop Invariants5a#' A )Mpre%ond
7/25/2019 01 Iterative methods
91/356
G1
Loop Invariants pre%ond Inpt is array A81n9
o* n vales2N
i : 1
m : A819
loop
Mloopinvariant m is ma# in A81i9N
e#it -hen 'i:n)
i : i = 1
m : ma#'mA8i9)endloop
retrn'm)
Mpost%ond
m is ma# in A 1n N
Any assmptions that
mst be tre abot the
state o* comptation
every time it is at the
top o* the loop2
Loop Invariants5a#' A )Mpre%ond
7/25/2019 01 Iterative methods
92/356
G.
4 /ot >st to please the pro*2
4 /ot >st to prove correctness24 Use them to thin" abot yor
algorithmO
4 !e*ore yo start coding2
4 Qhat do yo -ant to betre in the middle o* yorcomptationCJ Wo needto "no-2
J Let yor reader "no-2
Loop Invariants pre%ond Inpt is array A81n9
o* n vales2N
i : 1
m : A819
loop
Mloopinvariant m is ma# in A81i9N
e#it -hen 'i:n)
i : i = 1
m : ma#'mA8i9)endloop
retrn'm)
Mpost%ond
m is ma# in A 1n N
Iterative Algorithms
7/25/2019 01 Iterative methods
93/356
G0
7/25/2019 01 Iterative methods
94/356
G3
7/25/2019 01 Iterative methods
95/356
G6
g
Loop Invariantspre%ond
Inpt is array A81n9
o* n vales2N
i : 1
m : A819
loop
Mloopinvariant m is ma# in A81i9N
e#it -hen 'i:n)
i : i = 1
m : ma#'mA8i9)endloop
retrn'm)
Mpost%ond
m is ma# in A 1n N
Step @
7/25/2019 01 Iterative methods
96/356
G
Step 1
7/25/2019 01 Iterative methods
97/356
G
Last Step
7/25/2019 01 Iterative methods
98/356
G
Proves that IFthe program terminates then it -or"s
7/25/2019 01 Iterative methods
99/356
GG
g
Wo need to de*ine some
5easre o* progress
to prove that yor algorithmeventally terminates2
7/25/2019 01 Iterative methods
100/356
1@@
Simple E#ample
Insertion Sort Algorithm
+econsidering Simple E#amples
7/25/2019 01 Iterative methods
101/356
1@1
+econsidering Simple E#amples
A good martial arts stdent -ill attentively repeat eachA good martial arts stdent -ill attentively repeat each
*ndamental techniRe many times2*ndamental techniRe many times2
In contrast many college stdents tne ot -hen a conceptIn contrast many college stdents tne ot -hen a concept'e2g2 depth *irst search) is taght more than once2'e2g2 depth *irst search) is taght more than once2
he better stdents listen care*lly in order to re*ine andhe better stdents listen care*lly in order to re*ine and
develop their nderstanding2develop their nderstanding2
Repetition s !n Opportunity+dich ---2discretemath2com
%ode
7/25/2019 01 Iterative methods
102/356
1@.
%ode
+epresentation o* an Algorithm class InsertionSortAlgorithm e#tends SortAlgorithm 7 void sort'int a89) thro-s E#ception 7
*or 'int i : 1; i < a2length; i==) 7
int > : i;
int ! : a8i9;
-hile ''> ? @) && 'a8>19 ? !)) 7
a8>9 : a8>19;
>; B
a8>9 : !;
BB
7/25/2019 01 Iterative methods
103/356
1@0
Higher Level Abstract ie-
+epresentation o* an Algorithm
G "m
6 "m
Designing an Algorithm*i bl *i *i *
7/25/2019 01 Iterative methods
104/356
1@3
De*ine Problem De*ine LoopInvariants
De*ine 5easre o*Progress
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km
G "m
to school
E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
Problem Speci*ication
7/25/2019 01 Iterative methods
105/356
1@6
4Precondition he inpt is a list o* nvales
-ith the same vale possibly repeated2
4Post condition he otpt is a list consisting
o* the same nvales in nondecreasing order2
13
G.6.
6.
G
Location o* %omptation
7/25/2019 01 Iterative methods
106/356
1@
4MStateN o* comptation determined
by vales o* all variables2
Location o* %omptation
7/25/2019 01 Iterative methods
107/356
1@
13
G.6 .
6.
G
4MStateN o* comptation determined
by vales o* all variables2
13
7/25/2019 01 Iterative methods
108/356
1@
13
G.6 .
6.
G
.0
4Location too con*sing *or algorithm
4Algorithm too lay to de*ine step *orevery location2
13
7/25/2019 01 Iterative methods
109/356
1@G
4Algorithm speci*ies *rom -hich locations
it "no-s ho- to step2
13
G.6 .
6.
G
De*ine Loop Invariant
7/25/2019 01 Iterative methods
110/356
11@
13
G.6 .
6.
G
13
G.6 .
G
4Some sbset o* the elements are sorted
4he remaining elements are o** to the side2
Location /ot Determined
7/25/2019 01 Iterative methods
111/356
111
13
G.6 .
G
.0
G.6
01
Qhich sbset o* the elements are sorted is
not speci*ied2
7/25/2019 01 Iterative methods
112/356
11.
De*ining 5easre o* Progress
13
G
.6 .
G
elements
to school
7/25/2019 01 Iterative methods
113/356
110
De*ine Step
G.6 .
G
.0016.130@
7/25/2019 01 Iterative methods
114/356
113
De*ine Step
4 Select arbitrary element *rom side24 Insert it -here it belongs2
G.6 .
G
.0016.
13
G.6
G
130@
5a"ing progress -hile
E#it
7/25/2019 01 Iterative methods
115/356
116
5aintaining the loop invariant
G
.6 .
G
.0016.
13
G.6
G
0@
.0016..
130@
G "m 6 "m
6 elements
to school
elements
to school
Sorted sblist
!eginning &Ending
km E#it@ "m E#it
7/25/2019 01 Iterative methods
116/356
11
88 14
98256
2
52
79
3023
31
8814
98256
2
52
79
3023
31nelements
to school
14,23,25,30,31,52,62,79,88,
14,23,25,30,31,52,62,79,88,98
@ elements
to school
Ending
7/25/2019 01 Iterative methods
117/356
11
+nning imeInserting an element into a list o* sie i
ta"es 'i) time2
otal : 1=.=0==n
G.6 .
G
.0016.
13
G.6
G
0@
.0016..
130@
: C
7/25/2019 01 Iterative methods
118/356
Adding 5ade Easy
7/25/2019 01 Iterative methods
119/356
11G
1 & 2 & 3 & " " " & n'1 & n = S
n & n'1 & n'2 & " " " & 2 & 1 = S
1 2 " " " " " " " " n
= num#er of ye((o$ dots
n " " " " " " " 2 1
= num#er of $%ite dots
Adding 5ade Easy
7/25/2019 01 Iterative methods
120/356
1.@
1 & 2 & 3 & " " " & n'1 & n = S
n & n'1 & n'2 & " " " & 2 & 1 = S
= num#er of $%ite dots
= num#er of ye((o$ dots
n&1 n&1 n&1 n&1 n&1
n
n
n
n
n
n
= n)n&1* dots in t%e
+rid
2Sdots
S:n'n=1)
.
Adding 5ade Easy
7/25/2019 01 Iterative methods
121/356
1.1
Adding 5ade Easy
7/25/2019 01 Iterative methods
122/356
1..
+ i i
7/25/2019 01 Iterative methods
123/356
1.0
+nning ime
Inserting an element into a list o* sie ita"es 'i) time2
otal : 1=.=0==n
G.6 .
G
.0016.
13
G.6
G
0@
.0016..
130@
: 'n.)
Algorithm De*inition %ompletedDe*ine Problem De*ine Loop De*ine 5easre o*
7/25/2019 01 Iterative methods
124/356
1.3
De*ine Problem De*ine LoopInvariants
De*ine 5easre o*Progress
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km
G "m
to school
E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
Wet Another Abstract ie-
7/25/2019 01 Iterative methods
125/356
1.6
Wet Another Abstract ie-
4Wor >ob is only to
4rst -ho passes yo the baton
4$o arond once
4Pass on the baton
i1 i
ii
@=
1
A +elay +ace
Person @ %arries baton region to sa*e region
E t bli hi L I i t
7/25/2019 01 Iterative methods
126/356
1.
7/25/2019 01 Iterative methods
127/356
1.
i1 i
7/25/2019 01 Iterative methods
128/356
1.
=1
7/25/2019 01 Iterative methods
129/356
1.G
as a +elay +ace
13
G.6 .
6.
G
01
Establish Loop Invariant
7/25/2019 01 Iterative methods
130/356
10@
p
*rom preconditions
@
.0
13
G.6 .
6.
G
0@01
5 i t i i L I i t
7/25/2019 01 Iterative methods
131/356
101
5aintaining Loop Invariant
1
13
G.6 .
6.
G
01
5 i t i i L I i t
7/25/2019 01 Iterative methods
132/356
10.
5aintaining Loop Invariant
i1 i
13
G.6 .
6.
G
01
5 i t i i L I i t
7/25/2019 01 Iterative methods
133/356
100
5aintaining Loop Invariant
i
13
G.6 .
6.
G
01
5 i t i i L I i t
7/25/2019 01 Iterative methods
134/356
103
5aintaining Loop Invariant
i1 i
13
G.6 .
6.
G
01
5aintaining Loop In ariant
7/25/2019 01 Iterative methods
135/356
106
5aintaining Loop Invariant
13
G.6 .
6.
G
01
i
5aintaining Loop Invariant
7/25/2019 01 Iterative methods
136/356
10
5aintaining Loop Invariant
i1 i
13
G.
6.
G
.601
5aintaining Loop Invariant
7/25/2019 01 Iterative methods
137/356
10
5aintaining Loop Invariant
i
13
G.
6.
G
.601
5aintaining Loop Invariant
7/25/2019 01 Iterative methods
138/356
10
5aintaining Loop Invariant
i1 i
13
G.
G
.6016.
5aintaining Loop Invariant
7/25/2019 01 Iterative methods
139/356
10G
5aintaining Loop Invariant
%lean p loose ends
7/25/2019 01 Iterative methods
140/356
13@
%lean p loose ends
=1
T"
7/25/2019 01 Iterative methods
141/356
131
I "no- yo "ne- Insertion Sort
!t hope*lly yo are beginning to appreciate
Loop Invariants*or describing algorithms
E#plaining Insertion Sort
7/25/2019 01 Iterative methods
142/356
13.
Qe maintain a sbset o* elements sorted-ithin a list2 he remaining elements areo** to the side some-here2 Initiallythin" o* the *irst element in the array as
a sorted list o* length one2 Tne at a time-e ta"e one o* the elements that is o** tothe side and -e insert it into the sortedlist -here it belongs2 his gives a sorted
list that is one element longer than it -asbe*ore2 Qhen the last element has beeninserted the array is completely sorted2
E#plaining Selection Sort Qe maintain that the ksmallest o* the elements
7/25/2019 01 Iterative methods
143/356
130
are sorted in a list2 he larger elements are in a set
on the side2 Initially -ith k:@ all the elementsare in this set2 Progress is made by *inding thesmallest element in the remaining set o* largeelements and adding this selected element at theend o* the sorted list o* elements2 his increases kby one2 Stop -ith k:n2 At this point all theelements have been selected and the list is sorted2
13.0.60@01 ects
7/25/2019 01 Iterative methods
185/356
16
E#tra
p y >
Qe have read in the *irst iob>ects2
Qe -ill pretend that this pre*i#
is the entire inpt2Qe have a soltion *or this pre*i#
pls some additional in*ormation
Soltion
Loop Invariants andDeterministic Finite AtomatonL : 7 7@1B] ^ has length at most three and the
nmber o* 1s is odd B2
7/25/2019 01 Iterative methods
186/356
1
B
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
187/356
1
E#tra
p y >
Qe read in the i=1stob>ect2
Qe -ill pretend that this larger pre*i#
is the entire inpt2Qe e#tend the soltion -e have
to one *or this larger *or pre*i#2
' l dditi l i * ti )
Soltion
E#traSoltion
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
188/356
1
E#tra
p y >
Soltion
E#traSoltion
G "m 6 "m
E#it
ito i=1
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
189/356
1G
p y >
End the end -e have read in the entire inpt2
he LI gives s that -e have a soltion
*or this entire inpt2
Soltion
E#it
Insertion Sort
he inpt consistso* an array o* integers
7/25/2019 01 Iterative methods
190/356
1G@
p y g
Qe have read in the *irst iob>ects2
Qe -ill pretend that this pre*i#
is the entire inpt2Qe have a soltion *or this pre*i#
Soltion
.0016.
Insertion Sort
he inpt consistso* an array o* integers
7/25/2019 01 Iterative methods
191/356
1G1
.0016.
Qe read in the i=1stob>ect2
Qe -ill pretend that this larger pre*i#
is the entire inpt2Qe e#tend the soltion -e have
to one *or this larger *or pre*i#2
.0.6016.
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
192/356
1G.
A stdent midterm ans-er
Each iteration considers the ne#t inpt item2
Loop Invariants are pictres
o* crrent state2/ot actionsO/ot algorithmsO
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
193/356
1G0
A stdent midterm ans-er
Qe have considered the ithinpt element2
Qhat abot elements 81222i9C
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
194/356
1G3
A stdent midterm ans-er
Qe have considered inpt elements 8122i92
SoC
5ore o* the InptLoop Invariant
he inpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
195/356
1G6
Qe have read in the *irst iob>ects2
Qe have a soltion *or each2
Soltion
A stdent midterm ans-er
Each ob>ect does not need a separate soltion
he inpt as a -hole needs a soltion2
7/25/2019 01 Iterative methods
196/356
Longest !loc" o* Tnes
7/25/2019 01 Iterative methods
197/356
1G
Alg reads the digits one at a time and remembers
enogh abot -hat read so that it does not need
to reread anything2 'i2e2 a DFA)
@@1@1111@@11@@@11111@@@@11@@1
Longest !loc" o* Tnes
7/25/2019 01 Iterative methods
198/356
1G
@@1@1111@@11@@@11
Qhen it has read this mch
-hat does it rememberC
4Largest bloc" so *ar2
111
+ead the ne#t character &redetermine the largest bloc" so *ar2
Longest !loc" o* Tnes
7/25/2019 01 Iterative methods
199/356
1GG
@@1@1111@@11@@@11
Qhen it has read this mch
-hat does it rememberC
4Largest bloc" so *ar2
111
+ead the ne#t character &redetermine the largest bloc" so *ar
& crrent largest2
4Sie o* crrent bloc"2
ypical ypes o* Loop Invariants
7/25/2019 01 Iterative methods
200/356
.@@
4 5ore o* the inpt
4 5ore o* the otpt
4 /arro-ed the search space4 %ase Analysis
4 Qor" Done
5ore o* the TtptLoop Invariant
he otpt consistso* an array o* ob>ects
7/25/2019 01 Iterative methods
201/356
.@1
I have prodced the *irst iob>ects2
Prodce the i=1stotpt ob>ect2
E#itG "m 6 "m
E#it
ito i=1 Done -hen
otpt nob>ects2
Selection Sort
he otpt consists o* an array o* [email protected]
Inpt
7/25/2019 01 Iterative methods
202/356
.@.
he otpt consistso* an array o* [email protected]
I have prodced the *irst iob>ects2
Prodce the i=1stotpt ob>ect2
E#it
Done -hen
otpt nob>ects2
ypical ypes o* Loop Invariants
7/25/2019 01 Iterative methods
203/356
.@0
4 5ore o* the inpt
4 5ore o* the otpt
4 /arro-ed the search space4 %ase Analysis
4 Qor" Donehree
!inary Search li"e
e#amples
De*ine Problem !inary Search
7/25/2019 01 Iterative methods
204/356
.@3
4 Pre%onditions
Jey .6
JSorted List
4 Post%onditions
JFind "ey in list 'i* there)2
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
De*ine Loop Invariant
7/25/2019 01 Iterative methods
205/356
.@6
4 5aintain a sblist
4 Sch that
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
De*ine Loop Invariant
7/25/2019 01 Iterative methods
206/356
.@
4 5aintain a sblist2
4 I* the "ey is contained in the original list
then the "ey is contained in the sblist2
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
De*ine Step
7/25/2019 01 Iterative methods
207/356
.@
4 5a"e Progress
4 5aintain Loop Invariant
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
De*ine Step
7/25/2019 01 Iterative methods
208/356
.@
4 %t sblist in hal*2
4 Determine -hich hal* "ey -old be in2
4 eep that hal*2"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
De*ine Step
7/25/2019 01 Iterative methods
209/356
.@G
4 %t sblist in hal*2
4 Determine -hich hal* the "ey -old be in2
4 eep that hal*2"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
De*ine Step
7/25/2019 01 Iterative methods
210/356
.1@
4 It is *aster not to chec" i* the middle
element is the "ey2
4 Simply contine2
"ey 30
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
5a"e Progress G "m 6 "mE#it
7/25/2019 01 Iterative methods
211/356
.11
4 he sie o* the list becomes smaller2
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
G "m
6 "m
Initial %onditionskm
7/25/2019 01 Iterative methods
212/356
.1.
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
4 I* the "ey is contained inthe original list
then the "ey is containedin the sblist2
4 he sblist is theentire original list2
n "m
Ending AlgorithmE#it
"ey .6
7/25/2019 01 Iterative methods
213/356
.10
4 I* the "ey is contained in theoriginal list
then the "ey is contained in thesblist2
4 Sblist contains one element2
E#it
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
@ "m
4 I* the "ey iscontained in theoriginal list
then the "ey is atthis location2
"ey .6
I* "ey not in original list
7/25/2019 01 Iterative methods
214/356
.13
4 I* the "ey is contained inthe original list
then the "ey is contained
in the sblist2
4 Loop invariant treeven i* the "ey is notin the list2
4 I* the "ey iscontained in theoriginal list
then the "ey is atthis location2
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
"ey .3
4 %onclsion still solvesthe problem2
Simply chec" this onelocation *or the "ey2
+nning imehe sblist is o* sie n n,. n,3 n,1
7/25/2019 01 Iterative methods
215/356
.16
he sb list is o* sie n ,. ,3 ,1
Each step '1) time2
otal :'log n)
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
%ode
7/25/2019 01 Iterative methods
216/356
.1
Algorithm De*inition %ompletedDe*ine Problem De*ine Loop
Invariants
De*ine 5easre o*Progress
G "m
7/25/2019 01 Iterative methods
217/356
.1
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km
to school
E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
Stdy
7/25/2019 01 Iterative methods
218/356
.1
4 5any e#perienced programmers -ere as"ed
to code p binary search2
@X got it -rong
$ood thing is -as not *or a
nclear po-er plant2
5a"e Precise De*initions
7/25/2019 01 Iterative methods
219/356
.1G
4 5aintain a sblist -ith end points i&j2
"ey .6
i j
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
5a"e Precise De*initions
7/25/2019 01 Iterative methods
220/356
..@
4 5aintain a sblist -ith end points i&j2
"ey .6
i j
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
Does not matter -hich
bt yo need to be consistent2
5a"e Precise De*initions
7/25/2019 01 Iterative methods
221/356
..1
4 I* the sblist has even length
-hich element is ta"en to be midC
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
mid or mid C. .
i j i j+ + = =
5a"e Precise De*initions
7/25/2019 01 Iterative methods
222/356
...
4 I* the sblist has even length
-hich element is ta"en to be midC
"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
Shold not matter
%hoose right2mid
.
i j+ =
mid
Common ,u+s
7/25/2019 01 Iterative methods
223/356
..0
4 %t sblist in hal*2
4 Determine -hich hal* the "ey -old be in2
4 eep that hal*2"ey .6
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
Common ,u+s
7/25/2019 01 Iterative methods
224/356
..3
4 I* the middle element is the "ey
it can be s"ipped over2
"ey 30
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
E#it
Common ,u+s4
7/25/2019 01 Iterative methods
225/356
..6
Fi# the bg2
"ey 30
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
Common ,u+s4
7/25/2019 01 Iterative methods
226/356
..
Second *i#by ma"ing the le*t hal* slightly bigger2
"ey 30
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
I* "ey \ mid
then "ey is in
le*t hal*2
I* "ey ? mid
then "ey is in
right hal*2
4/e- bgC
Common ,u+s4
G "m 6 "m
E#it
7/25/2019 01 Iterative methods
227/356
..
Second *i#by ma"ing the le*t hal* slightly bigger2
"ey 30
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
4/e- bgC
0 6 10 1 .1 .1 .6 0 30 3G 61 60 @ . 3 0 G1 G6
/o progress is made2
Qhy is !inary Search so Easy to $et QrongC
7/25/2019 01 Iterative methods
228/356
..
C+.
T i j+
T+?C
mid.
i j+ =
1
i* "ey L'mid).
4 Ho- many
possible
algorithmsC
4Ho- manycorrect
algorithmsC
4 Probability o*
sccess bygessingC
0i: mid
else
j: mid 1
T+
i: mid = 1
else
j: mid
5oral
7/25/2019 01 Iterative methods
229/356
..G
4 Use the loop invariant method to thin"
abot algorithms2
4 !e care*l -ith yor de*initions2
4 !e sre that the loop invariant is al-ays
maintained2
4!e sre progress is al-ays made2
7/25/2019 01 Iterative methods
230/356
.0@
Loop Invariants*or
Iterative Algorithms
A second
!inary Search li"e
e#ample
De*ine Problem !inary Search ree4
7/25/2019 01 Iterative methods
231/356
.01
0
.6
1
3 .1
01
. 06
61
3.
3@ 3G
0
66 1
JA binary search tree2
J
7/25/2019 01 Iterative methods
232/356
.0.
0
.6
1
3 .1
01
. 06
61
3.
3@ 3G
0
66 1
\ \
De*ine Loop Invariant4 5aintain a sbtree2
7/25/2019 01 Iterative methods
233/356
.00
4 I* the "ey is contained in the original treethen the "ey is contained in the sbtree2
"ey 1
0
.6
1
3 .1
01
. 06
61
3.
3@ 3G
0
66 1
De*ine Step4 %t sbtree in hal*2
4
7/25/2019 01 Iterative methods
234/356
.03
Determine -hich hal* the "ey -old be in24 eep that hal*2
"ey 1
0
.6
1
3 .1
01
. 06
61
3.
3@ 3G
0
66 1
I* "ey < root
then "ey is
in le*t hal*2
I* "ey ? root
then "ey is
in right hal*2
I* "ey : root
then "ey is
*ond
Algorithm De*inition %ompletedDe*ine Problem De*ine Loop
Invariants
De*ine 5easre o*Progress
G "m
7/25/2019 01 Iterative methods
235/356
.06
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km
to school
E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
7/25/2019 01 Iterative methods
236/356
.0
Loop Invariants*or
Iterative Algorithms
A third
!inary Search li"e
e#ample
%ard ric"
7/25/2019 01 Iterative methods
237/356
.0
4 A volnteer please2
Pic" a %ard
7/25/2019 01 Iterative methods
238/356
.0
Done
Loop Invariant
he selected card is one o*
these2
7/25/2019 01 Iterative methods
239/356
.0G
Qhich colmnC
7/25/2019 01 Iterative methods
240/356
.3@
le*t
7/25/2019 01 Iterative methods
241/356
Selected colmn is placed
in the middle2
7/25/2019 01 Iterative methods
242/356
.3.
I -ill rearrange the cards2
7/25/2019 01 Iterative methods
243/356
.30
+ela# Loop Invariant
I -ill remember the same
abot each colmn2
7/25/2019 01 Iterative methods
244/356
.33
Qhich colmnC
7/25/2019 01 Iterative methods
245/356
.36
right
Loop Invariant
he selected card is one o*
these2
7/25/2019 01 Iterative methods
246/356
.3
Selected colmn is placed
in the middle2
7/25/2019 01 Iterative methods
247/356
.3
I -ill rearrange the cards2
7/25/2019 01 Iterative methods
248/356
.3
Qhich colmnC
7/25/2019 01 Iterative methods
249/356
.3G
le*t
Loop Invariant
he selected card is one o*
these2
7/25/2019 01 Iterative methods
250/356
.6@
Selected colmn is placed
in the middle2
7/25/2019 01 Iterative methods
251/356
.61
7/25/2019 01 Iterative methods
252/356
he MPartitioningN Problem'sed in Ric"sort)
7/25/2019 01 Iterative methods
253/356
.60
13G.6.
6.
G
01
p:6.
13
.60@
.001
G
.G
\ 6. \
Inpt Ttpt
De*ine Loop Invariant
p 6.
7/25/2019 01 Iterative methods
254/356
.63
or
01 .0 .6 G 0@ 13 . G
p:6.
\ p p \
01 .0 1 .6 G 0@ . G
p:6.
\ p p \
De*ining 5easre o* Progress
p:6.
7/25/2019 01 Iterative methods
255/356
.66
3 elements
not
loo"ed at
01 .0 .6 G 0@ 13 . G
p:6.
De*ine Step
45 " P
7/25/2019 01 Iterative methods
256/356
.6
5a"e Progress4 5aintain Loop Invariant
01 .0 .6 G 0@ 13 . G
p:6.
\ p p \
De*ine Step
45 " P
7/25/2019 01 Iterative methods
257/356
.6
5a"e Progress4 5aintain Loop Invariant
01 .0 .6 G 0@ 13 . G
p:6.
\ p p \
01 .0 13 .6 G 0@ . G
p:6.
\ p p \
De*ine Step
01 .0 .6 G 0@ 13 . G
p:6.
For cases
01 .0 13 .6 G 0@ . G
p:6.
7/25/2019 01 Iterative methods
258/356
.6
01 .0 .6 G 0@ 13 . G
01 .0 13 .6 G 0@ . G
01 .0 13 .6 G 0@ . G
01 .0 13 .6 G 0@ . G
01 .0 .6 G 0@ 1 . G
p:6.
01 .0 .6 G 0@ 1 . G
01 .0 1 .6 G 0@ . G
p:6.
01 .0 .6 G 0@ 1 . G
13
.6 .
6.
0@01
!eginning &Ending
km E#it@ "m E#it
p:6.
7/25/2019 01 Iterative methods
259/356
.6G
G.6 .
G
01p:6.
.6 01 G . 13 0@ G .0
.0 0@ .6 01 13 . G G
p:6.
.6
0@01
13.0
.G
G
\ 6. \
n1 elements
not
loo"ed at
@ elements
not
loo"ed at
+nning imeEach iteration ta"es '1) time2
7/25/2019 01 Iterative methods
260/356
.@
otal : 'n)
01 .0 .6 G 0@ 13 . G
p:6.
\ p p \
01 .0 13 .6 G 0@ . G
p:6.
\ p p \
Algorithm De*inition %ompletedDe*ine Problem De*ine Loop
Invariants
De*ine 5easre o*Progress
G "m
7/25/2019 01 Iterative methods
261/356
.1
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km
to school
E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
p:6. [email protected]
7/25/2019 01 Iterative methods
262/356
..
racing an
E#ample
7/25/2019 01 Iterative methods
263/356
$%D'ab)
Inpt
7/25/2019 01 Iterative methods
264/356
.3
: 3Ttpt $%D'ab)
5aintain vales
7/25/2019 01 Iterative methods
265/356
.6
:
7/25/2019 01 Iterative methods
266/356
.
7/25/2019 01 Iterative methods
267/356
.
Speci*ies ho- the rnning time depends on
the sie o* the inpt2
A *nction mapping
MsieN o* inpt
MtimeN 'n) e#ected 2
De*inition o* ime
7/25/2019 01 Iterative methods
268/356
.
4 _ o* seconds 'machine dependent)2
4 _ lines o* code e#ected2
4 _ o* times a speci*ic operation is per*ormed'e2g2 addition)2
QhichC
De*inition o* ime
7/25/2019 01 Iterative methods
269/356
.G
4 _ o* seconds 'machine dependent)2
4 _ lines o* code e#ected2
4 _ o* times a speci*ic operation is per*ormed'e2g2 addition)2
hese are all reasonable de*initions o* timebecase they are -ithin a constant *actor o*
each other2
Sie o* Inpt InstanceC
0G.@
7/25/2019 01 Iterative methods
270/356
.@
0G.@
Sie o* Inpt Instance
0G.@
6
1KK
7/25/2019 01 Iterative methods
271/356
.1
4 Sie o* paper4 _ o* bits
4 _ o* digits
4 ale
n : . in.
n : 1 bits
n : 6 digits
n : 0G.@
.KK
0G.@ 1
Qhich are reasonableC
Sie o* Inpt Instance
0G.@
6
1KK
7/25/2019 01 Iterative methods
272/356
..
4 Sie o* paper4 _ o* bits
4 _ o* digits
4 ale
n : . in.
n : 1 bits
n : 6 digits
n : 0G.@
4 Intitive
.KK
0G.@ 1
Sie o* Inpt Instance
0G.@
6
1KK
7/25/2019 01 Iterative methods
273/356
.0
4 Sie o* paper4 _ o* bits
4 _ o* digits
4 ale
n : . in.
n : 1 bits
n : 6 digits
n : 0G.@
4 Intitive4 Formal
.KK
0G.@ 1
Sie o* Inpt Instance
0G.@
6
1KK
7/25/2019 01 Iterative methods
274/356
.3
4 Sie o* paper4 _ o* bits
4 _ o* digits
4 ale
n : . in.
n : 1 bits
n : 6 digits
n : 0G.@
4 Intitive4 Formal
4 +easonable
_ o* bits : 020. ] _ o* digits
.KK
0G.@ 1
Sie o* Inpt Instance
0G.@
6
1KK
7/25/2019 01 Iterative methods
275/356
.6
4 Sie o* paper4 _ o* bits
4 _ o* digits
4 ale
n : . in.
n : 1 bits
n : 6 digits
n : 0G.@
4 Intitive4 Formal
4 +easonable
4 Unreasonable
_ o* bits : log.'ale)
ale : ._ o* bits
.KK
0G.@ 1
$%D'ab)
Inpt
7/25/2019 01 Iterative methods
276/356
.
7/25/2019 01 Iterative methods
277/356
.
7/25/2019 01 Iterative methods
278/356
.
7/25/2019 01 Iterative methods
279/356
.G
:
7/25/2019 01 Iterative methods
280/356
.@
!c"et '(ic") Sort *or Hmans
Inpt A pile o* things to sort2
'E# E#ams)
Ttpt Sort them
7/25/2019 01 Iterative methods
281/356
.1
Ttpt Sort them+eRirements
4Easy to e#ecte by hmans4Tnly can have piles on des"4%an move one e#am 'or pile)
at a time24Fast J T'nlog'n)) time2
Li"ely the only algorithm
in this corse -hich yo -ill
e#ecte by hand yorsel*2
!c"et '(ic") Sort *or Hmans
8A`9Inpt
7/25/2019 01 Iterative methods
282/356
..
Denotes an nsorted pile o* e#ams
-ith last names starting in the range 8A`9
!c"et '(ic") Sort *or Hmans
8A`9
7/25/2019 01 Iterative methods
283/356
.0
Psychology stdy
Hmans can only thin" abot6 things at once2
Is the *irst letter
o* the nameon the *irst e#am
8AE9
8F98LT98P98U`9
in
!c"et '(ic") Sort *or Hmans
8A`9
7/25/2019 01 Iterative methods
284/356
.3
8AE9 8F9 8LT9 8P9 8U`9
!c"et Sort4Pt e#ams one at a time
in the correct bc"et2
!c"et '(ic") Sort *or Hmans
8AE98F9
8LT98P 9
89
t d
7/25/2019 01 Iterative methods
285/356
.6
8 98P98U`9
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
sorted
8AE98F9
8LT98P 9
89
t d
!c"et '(ic") Sort *or Hmans
7/25/2019 01 Iterative methods
286/356
.
8 98P98U`9
sorted
8A9 8!9 8%9 8D9 8E9
!c"et Sort4Pt e#ams one at a time
in the correct bc"et2
89
sorted
8A98!98%98D9
!c"et '(ic") Sort *or Hmans
7/25/2019 01 Iterative methods
287/356
.
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
sorted
8F98LT9
8P98U`9
8 98E9
89
sorted
8A98!98%98D9
!c"et '(ic") Sort *or Hmans
7/25/2019 01 Iterative methods
288/356
.
sorted8F98LT9
8P98U`9
8 98E9
8AAAE98AFA98ALAT98APA98AUA`9
89
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8AAAE9
8AUA`9
L
7/25/2019 01 Iterative methods
289/356
.G
sorted8F9
8U`9
8E9L
L
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
89
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8AAAE9
8AUA`9
L
7/25/2019 01 Iterative methods
290/356
.G@
sorted8F9
8U`9
8E9L
L
8AAAE9
89
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8AFA9
8AUA`9
L
7/25/2019 01 Iterative methods
291/356
.G1
sorted8F9
8U`9
8E9L
L
8AAAE9
sorted
Qhen s**iciently small
sort by hand
8AAAE9
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8AFA9
8AUA`9
L
7/25/2019 01 Iterative methods
292/356
.G.
sorted8F9
8U`9
8E9L
L
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
8AAAE9
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8AFA9
8AUA`9
L
7/25/2019 01 Iterative methods
293/356
.G0
sorted8F9
8U`9
8E9L
L
8AFA9
sorted
8AAA9
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8ALAT9
8AUA`9
L
7/25/2019 01 Iterative methods
294/356
.G3
sorted8F9
8U`9
8E9L
L
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
8AAA9
sorted8!9
!c"et '(ic") Sort *or Hmans
L
8ALAT9
8AUA`9
L
7/25/2019 01 Iterative methods
295/356
.G6
sorted8F9
8U`9
8E9L
L
8ALAT98APA9
8AUA`9sorted
sortedsorted
8F9
8!9
8E9
!c"et '(ic") Sort *or Hmans
L
L8AAA`9
sorted
7/25/2019 01 Iterative methods
296/356
.G
8U`9
Lsorted
8F9
8!9
8E9
!c"et '(ic") Sort *or Hmans
L
L
8A9
sorted
7/25/2019 01 Iterative methods
297/356
.G
8U`9
L
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
sorted
8A9
8F9
8!9
8E9
!c"et '(ic") Sort *or Hmans
L
L
sorted
7/25/2019 01 Iterative methods
298/356
.G
8U`9
L
8!A!E98!F!98!L!T9 8!P!98!U!`9
sorted
8A9
8%9
!c"et '(ic") Sort *or Hmans
L
8!A!E9
8!U!`9
L
sorted
7/25/2019 01 Iterative methods
299/356
.GG
8F9
8U`9
8E9L
L
A stac" o* piles4each pile nsorted4all in one ile be*ore all in ne#t
A sorted pile
4be*ore all the rest
'pside do-n)
sorted
8A`9
!c"et '(ic") Sort *or Hmans
8`U``9
sorted
7/25/2019 01 Iterative methods
300/356
0@@
8`U``9
sorted
sorted
8A`9
!c"et '(ic") Sort *or Hmans
sorted
7/25/2019 01 Iterative methods
301/356
0@1
E#it
sorted
Inpt A o* stac" o*Npnch cards2 Each card contains ddigits2
Each digit bet-een @&'k1)
+adi#Sort 0331.6
000
103..3
1.6..3
..6
0.6
000
7/25/2019 01 Iterative methods
302/356
0@.
Ttpt Sort the cards2 ..3
003
130
..6
0.6
.30
!c"et Sort 5achineSelect one digit
Separates cards into kpiles2
-rt selected digit2
000
103
003
033
130
.30
Stable sort I* t-o cards are the same *or that digit
their order does not change2
+adi#Sort
033
1.6000 Sort -rt -hich
1.6
103130 Sort -rt -hich
1.6
..3
..6
7/25/2019 01 Iterative methods
303/356
0@0
000
103
..3
003
130
..6
0.6.30
digit *irstC
he most
signi*icant2
130
..3
..6
.30
033
000
0030.6
digit SecondC
he ne#t most
signi*icant2
..6
0.6
103
000
003
130
.30033
All meaning in *irst sort lost2
+adi#Sort
033
1.6000 Sort -rt -hich Sort -rt -hich
000
130.30
..3
1.6..6
7/25/2019 01 Iterative methods
304/356
0@3
000
103
..3
003
130
..6
0.6.30
digit *irstC digit SecondC
he least
signi*icant2
.30
033
103
..3
003
1.6
..60.6
he ne#t least
signi*icant2
..6
0.6
000
103
003
130
.30033
+adi#Sort
033
1.6000 Sort -rt -hich Sort -rt -hich
000
130.30
. .3
1 .6. .6
7/25/2019 01 Iterative methods
305/356
0@6
000
103
..3
003
130
..6
0.6.30
digit *irstC digit SecondC
he least
signi*icant2
.30
033
103
..3
003
1.6
..60.6
he ne#t least
signi*icant2
. .6
0 .6
0 00
1 03
0 03
1 30
. 300 33
Is sorted -rt *irst idigits2
. .3
1 .6. .6 Is sorted -rt
1 .6
1 031 30 Is sorted -rt
+adi#Sort
7/25/2019 01 Iterative methods
306/356
0@
Sort -rt i=1stdigit2
. .6
0 .6
0 00
1 03
0 03
1 30
. 300 33
Is sorted -rt
*irst idigits2
1 30
. .3
. .6
. 30
0 .6
0 000 03
0 33
Is sorted -rt
*irst i=1 digits2
i=1
hese are in the
correct order
becase sorted
-rt high order digit2
. .3
1 .6. .6 Is sorted -rt
1 .6
1 031 30 Is sorted -rt
+adi#Sort
7/25/2019 01 Iterative methods
307/356
0@
Sort -rt i=1stdigit2
. .6
0 .6
0 00
1 03
0 03
1 30
. 300 33
Is sorted -rt
*irst idigits2
1 30
. .3
. .6
. 30
0 .6
0 000 03
0 33i=1
hese are in the
correct order
becase -as sorted &
stable sort le*t sorted2
Is sorted -rt
*irst i=1 digits2
%ontingSortInpt
Ttpt @ @ @ @ @ 1 1 1 1 1 1 1 1 1 . . 0 00
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
7/25/2019 01 Iterative methods
308/356
0@
InptNrecords each labeled -ith a digit
bet-een @&'k1)2
Ttpt Stable sort the nmbers2
Algorithm %ont to determine -here records go2
$o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ @ @ @ @ 1 1 1 1 1 1 1 1 1 . . . 00
Inde# [email protected]@ 1. 10 13 16 1 1 1
7/25/2019 01 Iterative methods
309/356
0@G
Stable sort I* t-o records are the same *or that digit
their order does not change2here*ore the 3threcord in inpt -ith digit 1 mst be
the 3th record in otpt -ith digit 12
It belongs in otpt inde# becase records go be*ore itie 6 records -ith a smaller digit &
0 records -ith the same digit
%ont
hese
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
7/25/2019 01 Iterative methods
310/356
01@
ale v_ o* records -ith digit v 0.1@ .0G6
Nrecords kdi**erent vales2 ime to contC'/)
'/ ")
Qe have conted _ o* each vale
in the *irst ivales2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
7/25/2019 01 Iterative methods
311/356
011
ale v_ o* records -ith digit v
_ o* records -ith digit < v
1136@
Nrecords kdi**erent vales2 ime to contC 'k.)
oo mch
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
7/25/2019 01 Iterative methods
312/356
01.
ale v_ o* records -ith digit v
_ o* records -ith digit < v
1136@
%ompted
Nrecords kdi**erent vales2 ime to contC 'k)
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ @ @ @ @ 1 1 1 1 1 1 1 1 1 . . . 00
7/25/2019 01 Iterative methods
313/356
010
ale v_ o* records -ith digit < v 0.1@ 1136@Location o* *irst record
-ith digit v2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
1@ C
7/25/2019 01 Iterative methods
314/356
013
ale v 0.1@1136@Location o* *irst record
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
1
7/25/2019 01 Iterative methods
315/356
016
ale v 0.1@1136@Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 1
7/25/2019 01 Iterative methods
316/356
01
ale v 0.1@113@Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ @ 1
7/25/2019 01 Iterative methods
317/356
01
ale v0.1@1131Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@
7/25/2019 01 Iterative methods
318/356
01
ale v0.1@ 113.Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 0
7/25/2019 01 Iterative methods
319/356
01G
ale v0.1@ 113.Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 1 0
7/25/2019 01 Iterative methods
320/356
0.@
ale v0.1@ 113.Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 1 01
7/25/2019 01 Iterative methods
321/356
0.1
ale v0.1@ 113.Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 001 1
7/25/2019 01 Iterative methods
322/356
0..
ale v0.1@ 113G.Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 1 01 1 0
7/25/2019 01 Iterative methods
323/356
0.0
ale v0.1@ 1G13G.Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ @ 1 1 1 0 0
7/25/2019 01 Iterative methods
324/356
0.3
ale v0.1@ [email protected] o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ .1 1 1 0 0@
7/25/2019 01 Iterative methods
325/356
0.6
ale v 0.1@ 1G131@0Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 11 1 1 0 0@ .
7/25/2019 01 Iterative methods
326/356
0.
ale v 0.1@ 1G161@0Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 11 1 1 0 0@ .
7/25/2019 01 Iterative methods
327/356
0.
ale v 0.1@ 1G161@0Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
%ontingSortInpt
Ttpt
Inde# [email protected]@ 1. 10 13 16 1 1 1
1 @ @ 1 0 1 1 0 1 @ . 1 @ 1 1 . . 1 @
@ 11@ 11 1 1 0 0@ .@ @ 1 1 1 . .
7/25/2019 01 Iterative methods
328/356
0.
ale v 0.1@ 1G1136Location o* ne#trecord
-ith digit v2
Algorithm $o throgh the records in order ptting them -here they go2
'/)ime : '%onting Tps)
'log / = log ") 'bit ops)'/=")otal :
+adi#,%onting SortInpt Nnmbers eachLbits long2
Ttpt Sort the nmbers2
1111@111@1@@1@1@@@1@1
N 1@1@@1@1@1@@@1@11@111
7/25/2019 01 Iterative methods
329/356
0.G
L
N 1@1@@1@1@1@@@1@11@11111@1@@@11@1@@1111@1@1
111 1@1 11@ 1@@ 1@1 @@@ 1@11@1 @@1 @1@ 1@@ @1@ 11@ 111
11@ 1@@ @11 @1@ @11 11@ 1@1
d:L , lo k
N
: log k 'k-ill be set toN)
+adi#,%onting SortInpt Nnmbers eachLbits long2
Each card contains ddigits2
Each digit bet-een @&'k1)Ttpt Sort the nmbers2
7/25/2019 01 Iterative methods
330/356
00@
111 1@1 11@ 1@@ 1@1 @@@ 1@11@1 @@1 @1@ 1@@ @1@ 11@ 11111@ 1@@ @11 @1@ @11 11@ 1@1
d:L , log k
N
: log k
digit bet-een @&'k1)
sing %onting Sort2
p
Use +adi# Sort Sort -rt each digit
+adi#,%onting Sort
7/25/2019 01 Iterative methods
331/356
001
+adi#,%onting Sort
ime :: '_ o* digits) 'ime o* %onting Sort)
'ime o* +adi# Sort)
7/25/2019 01 Iterative methods
332/356
00.
' g ) ' g )
: L,log k 'N=k) 'logN= log k) bit ops
Set kto minimie time2
Qants kbig2 +eally -ants ksmall bt does not
care as long as kN2
Set k:N
+adi#,%onting Sort
ime :: '_ o* digits) 'ime o* %onting Sort)
'ime o* +adi# Sort)
7/25/2019 01 Iterative methods
333/356
000
' g ) ' g )
: L,log k 'N=k) 'logN= log k) bit ops
: L,logN N logN bit ops
: 'LN) bit ops
Sie : Sie o* Nnmbers eachLbits long2
: 'LN) : n
: 'n) bit ops
MLinearN ime,ut sortin+ s%ou(d t/ke )N(o+ N* time...
+adi#,%onting Sortime : 'LN) bit ops
Sie : Sie o* Nnmbers eachLbits long2: 'LN)
'NlogN) bit ops
7/25/2019 01 Iterative methods
334/356
003
5erge or (ic" Sortime : 'NlogN) comparisons
Sie : Sie o* Nnmbers each arbitrarily long2: 'NlogN) bits2
L:? logN i* yo -antNdistinct nmbers2
: 'n)
5erge (ic" and Heap Sort can sort/nmbers
sing T'/ log /)comparisons bet-een the vales2
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
335/356
006
heorem /o algorithm can sort *aster2
A I 8 A'I) =P'I) or ime'AI) lo-er'^I^)9
I have an algorithm Athat Iclaim -or"s and is *ast2
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
336/356
00
I -in i* Aon inpt Igives4the -rong otpt or
4rns slo-2
claim -or"s and is *ast2
Th yeah I have an inpt I
*or -hich it does not 2
4 Sorting
J Inpt
J Ttpt
630.1
11100.103.0ales
Inde#es
3.013303.0.110110ales
Inde#es
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
337/356
00
4 Lo-er !ond
J Inpt An Algorithm *or Sorting A
J Ttpt An instance I
on -hich alg Aeither
4 a"es too mch time
4 Tr give the -rong ans-er
630.1
11100.103.0ales
Inde#es
3.013
303.0.110011ales
Inde#es
Algorithm De*inition %ompletedDe*ine Problem De*ine LoopInvariants
De*ine 5easre o*Progress
D *i St D *i E it % diti 5 i t i L I
G "m
to school
7/25/2019 01 Iterative methods
338/356
00
De*ine Step De*ine E#it %ondition 5aintain Loop Inv
5a"e Progress Initial %onditions Ending
km E#it
E#it
G "m 6 "m
E#it
E#it
@ "m E#it
I give yo algorithm A
I claim it sorts2
I mst otpt an instance
630.1
11100.103.0alesInde#es
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
339/356
00G
on -hich alg Aeither4a"es too mch time
4Tr give the -rongans-er
It might as -ell be a
permtation o* 122/
'inde#es not sho-n)01.63
I give yo algorithm A
I claim it sorts2A Lo-er !ond on Sorting
/eed to "no-
-hat the algorithm does
be*ore -e can "no-
-hat inpt to give it
7/25/2019 01 Iterative methods
340/356
03@
-hat inpt to give it2
/eed to give the
algorithm an inpt
be*ore -e can "no-
-hat the algorithm does inpt2
!rea" this cycleone iteration at a time2
A Lo-er !ond on Sorting
+estrict the search space
7/25/2019 01 Iterative methods
341/356
031
1 . 0 3 6
I maintain a set o* instances
'permtations o* 122n)
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
342/356
03.
1 . 0 3 6
0 . 6 1 3
3 6 . 1 0
6 3 . 0 1
Th dearO
he *irst t time steps o* my
algorithm Aare the same given
any o* these instancesOOO
Initially I consider all /O
permtations o* 122/
A Lo-er !ond on Sorting
1 . 0 3 6
7/25/2019 01 Iterative methods
343/356
030
Th dearO
he *irst @ time steps o* my
algorithm Aare the same given
any o* these instancesOOO
1 . 0 3 6
0 . 6 1 3
3 6 . 1 0
6 3 . 0 1
Initially I consider all /O
permtations o* 122/
A Lo-er !ond on Sorting
1 . 0 3 6
7/25/2019 01 Iterative methods
344/356
033
he measre o* progressis the nmber o* instances2
Initially there are /O o* them2
G "m km
1 . 0 3 6
0 . 6 1 3
3 6 . 1 0
6 3 . 0 1
1 . 0 3 6
5yt=1sttime step
a< a.
6th bit o* a = a.
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
345/356
036
1 . 0 3 6
0 . 6 1 3
3 6 . 1 0
6 3 . 0 1
!ecase the *irst t time steps
are the same -hat the alg
"no-s is the same2
Hence its t=1ststep is the same2
6 bit o* a= a.Any yes,no
Restion
1 . 0 3 6
I partition my instances based
on -hat they do on this t=1st
step25yt=1sttime step
a< a.
6th bit o* a = a.
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
346/356
03
1 . 0 3 6
0 . 6 1 3
3 6 . 1 0
6 3 . 0 1
!ecase the *irst t time steps
are the same -hat the alg
"no-s is the same2
Hence its t=1ststep is the same2
6 bit o* a= a.Any yes,no
Restion
1 . 0 3 6
I "eep the larger hal*2
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
347/356
03
1 . 0 3 6
0 . 6 1 3
3 6 . 1 0
6 3 . 0 1
1 . 0 3 6
I "eep the larger hal*2
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
348/356
03
1 . 0 3 6
0 . 6 1 3
Th dearO
he *irst t=1 time steps o* myalgorithm Aare the same given
any o* these instancesOOO
1 . 0 3 6
I "eep the larger hal*2
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
349/356
03G
0 6
0 . 6 1 3
G "m
Initially I have nO Permtations2
A*ter t time steps I have /O,.t
1 . 0 3 6
I "eep the larger hal*2
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
350/356
06@
0 . 6 1 3
Qe e#it -hen there are t-o
instances2E#it
Initially I have nO Permtations2
A*ter t time steps I have /O,.t
: log'/O)
A Lo-er !ond on Sorting
N/ *actors each at least N/
7/25/2019 01 Iterative methods
351/356
061
Qe e#it -hen there are t-o
instances2E#it : log'/O)
/O : 1 . 0 N/2 N
/ *actors each at most /2
/2*actors each at least /
22
! //
: '/ log'/))2
I give yo algorithm A
I claim it sorts2A Lo-er !ond on Sorting
E#it
I mst otpt an instance
on -hich alg Aeither4a"es too mch time
630.1
11100.103.0ales
Inde#es
7/25/2019 01 Iterative methods
352/356
06.
a"es too mch time
4Trgive the -rongans-er
A Lo-er !ond on SortingE#it
I mst otpt an instance
on -hich alg Aeither4a"es too mch time
630.1
11100.103.0ales
Inde#es
TopsO
7/25/2019 01 Iterative methods
353/356
060
%ase 1 he algorithm does notstop at time on these t-o
instances2
E#it : log'/O) : '/ log'/))2
a"es too mch time
4Tr give the -rongans-er
TopsO
I mst give
the -rong
ans-er on oneo* theseO
I mst otpt an instance
on -hich alg Aeither4a"es too mch time
630.1
11100.103.0ales
Inde#es
A Lo-er !ond on SortingE#it
7/25/2019 01 Iterative methods
354/356
063
a"es too mch time
4Tr give the -rongans-er
%ase . he algorithm stops at time and gives an ans-er2
Qe e#it -hen there are t-o
instancesE#itand these need di**erent ans-ers2
he *irst time steps o* alg Aare
the same on these t-o instances
and hence the same ans-er is
Theorem: For every sorting algorithmA
on the orst "ase in#$t instan"eI%Nlog&N'"om#arisons %or other bit o#erations'
d b d N b
A Lo-er !ond on Sorting
7/25/2019 01 Iterative methods
355/356
066
need to be exe"$ted to sortNobje"ts(
A I A%I' =)%I' or Time%AI' N log&N
)roo*: )rover+Adversary ,ame
End Iterative Algorithms
7/25/2019 01 Iterative methods
356/356
End Iterative Algorithms
5ath +evie-
http://01.5-short_math.ppt/http://01.5-short_math.ppt/