Algorithmic Analysis Measuring “Work” OperationCounter Class.

Algorithmic Analysis Algorithmic Analysis

Measuring “Work”Measuring “Work”

OperationCounter ClassOperationCounter Class

Algorithms and Running TimeAlgorithms and Running Time

Some programs are fastSome programs are fast Other programs are slowOther programs are slow Even if they’re doing the “same thing”Even if they’re doing the “same thing” Even if they’re on the same computerEven if they’re on the same computer Even if they’re written in the same languageEven if they’re written in the same language It’s a question of how much work they’re It’s a question of how much work they’re

doingdoing

Print the Numbers From 1 to NPrint the Numbers From 1 to N

Stupid methodStupid method

1.1. Print the number 1Print the number 1

2.2. RepeatRepeata.a. Count how many numbers we already printedCount how many numbers we already printedb.b. If it’s N, stop (* we’re done *)If it’s N, stop (* we’re done *)c.c. Add 1 to the number from step a.Add 1 to the number from step a.d.d. Find the end of the listFind the end of the liste.e. Print the number you got in step c.Print the number you got in step c.

How Much Work to Print to 3?How Much Work to Print to 3?

1.1. Print number 1Print number 1

2.2. Count number 1Count number 1

3.3. Test if 1 = 3Test if 1 = 3

4.4. Add 1 to 1 Add 1 to 1 2 2

5.5. Skip over number 1Skip over number 1




9.9. Test if 2 = 3Test if 2 = 3

10.10. Add 1 to 2 Add 1 to 2 3 3







17.17. Test if 3 = 3Test if 3 = 3


Better methodBetter method

1.1. Set k to 1Set k to 1

2.2. While k While k N Na.a. Print kPrint kb.b. Add 1 to kAdd 1 to k

How Much Work to Print to 3?How Much Work to Print to 3?

1.1. Set k to 1 (k = 1)Set k to 1 (k = 1)

2.2. Test k Test k 3 3

3.3. Print kPrint k

4.4. Add 1 to k (k = 2)Add 1 to k (k = 2)


6.6. Print kPrint k

7.7. Add 1 to k (k = 3)Add 1 to k (k = 3)


9.9. Print kPrint k

10.10. Add 1 to k (k = 4)Add 1 to k (k = 4)


6 less steps than 6 less steps than StupidStupid

Faster and SlowerFaster and Slower

Second way clearly betterSecond way clearly better• Six less steps for N = 3Six less steps for N = 3

For N = 1000For N = 1000• Better algBetter algthmthm takes 999,997 less steps takes 999,997 less steps• If If BetterBetter took 1 second to print to 1000… took 1 second to print to 1000…• ……StupidStupid would take over 5 minutes would take over 5 minutes

How do I know?How do I know?

Three Ways to Find FasterThree Ways to Find Faster

Stupid wayStupid way• Write out all the steps required & count themWrite out all the steps required & count them

Better wayBetter way• Write a program to calculate & count stepsWrite a program to calculate & count steps

Best way?Best way?

Counting in the ProgramCounting in the Program

Create a variable to count the operationsCreate a variable to count the operations• have method return that value at the endhave method return that value at the end

Make a loop to call the method with Make a loop to call the method with different numbers to count todifferent numbers to count to• count to 0, count to 1, count to 2, …count to 0, count to 1, count to 2, …• record the amount of workrecord the amount of work• look at the numberslook at the numbers• figure out the patternfigure out the pattern


Stupid methodStupid method

1.1. Print the number 1Print the number 1

2.2. RepeatRepeata.a. Count how many … already printedCount how many … already printedb.b. If it’s N, stop (* we’re done *)If it’s N, stop (* we’re done *)c.c. Add 1 to the number from step a.Add 1 to the number from step a.d.d. Find the end of the listFind the end of the liste.e. Print the number you got in step c.Print the number you got in step c.

1 step

k steps

1 step

1 step

1 step

k steps

(assume printed k numbers so far)

fillStupidly with opCountfillStupidly with opCountpublic static public static intint fillStupidly(int n, int v[]) { fillStupidly(int n, int v[]) { int opCount = 0;int opCount = 0; int a;int a; int c = 1;int c = 1; ++opCount;++opCount; v[0] = 1;v[0] = 1; //step 1 (asgn)//step 1 (asgn) while (true) {while (true) { for (a = 0; for (a = 0; v[a] != 0v[a] != 0; ++a) {; ++a) { ++opCount;++opCount; // step 2a (comp)// step 2a (comp) }} ++opCount;++opCount; if (if (v[a]==nv[a]==n) ) // step 2b (comp)// step 2b (comp) return opCount;return opCount; ++opCount;++opCount; c = a + 1c = a + 1;; // step 2c (asgn)// step 2c (asgn) for (a = 0; for (a = 0; v[a] != 0v[a] != 0; ++a) {; ++a) { ++opCount;++opCount; // step 2d (comp)// step 2d (comp) }} ++opCount;++opCount; v[a] = cv[a] = c;; // step 2e (asgn)// step 2e (asgn)} }} }


Better methodBetter method



1 step

1 step

1 step

1 step

fillIntelligently with opCountfillIntelligently with opCount

private static private static intint fillIntelligently(int n, int[] v) { fillIntelligently(int n, int[] v) { int opCount = 0;int opCount = 0; ++opCount++opCount;; int k = 1int k = 1;; // step 1 (asgn)// step 1 (asgn) while (true) {while (true) { ++opCount++opCount;; if (if (k > nk > n)) // step 2 (comp)// step 2 (comp)

return opCount;return opCount; ++opCount++opCount;; v[k-1] = kv[k-1] = k;; // step 2a // step 2a

(asgn)(asgn) ++opCount++opCount;; ++k++k;; // step 2b// step 2b }}}}

Head-to-HeadHead-to-Head

Number of Steps taken (program count):Number of Steps taken (program count):NN StupidStupid BetterBetter11 33 5522 99 8833 1717 111144 2727 141455 3939 1717

1010 129129 3232100100 10,29910,299 302302

The bigger N gets, the worse Stupid looks

Growth PatternGrowth Pattern

Number of Steps taken (program count):Number of Steps taken (program count):NN StupidStupid (change)(change)

BetterBetter(change)(change)11 33 5522 99 +6+6 88 +3+333 1717 +8+8 1111 +3+344 2727 +10+10 1414

+3+355 3939 +12+12 1717

+3+31010 129129 3232

100100 10,29910,299 302302Stupid grows faster and fasterBetter grows at a steady rate

Three Ways to Find FasterThree Ways to Find Faster

Stupid wayStupid way• Write out all the steps required & count themWrite out all the steps required & count them

Better wayBetter way• Write a program to calculate & count stepsWrite a program to calculate & count steps

Best wayBest way• Figure out how to do algorithmic analysisFigure out how to do algorithmic analysis• Apply it to this problemApply it to this problem

Algorithmic AnalysisAlgorithmic Analysis

Figuring out how much work it will doFiguring out how much work it will do• how many steps it will take to finishhow many steps it will take to finish

In terms of how “big” the problem isIn terms of how “big” the problem is• size of problem is called “N”size of problem is called “N”• in our example, N is the number to print toin our example, N is the number to print to

At its best – a “closed-form” solutionAt its best – a “closed-form” solution• exact number of steps it’ll takeexact number of steps it’ll take

In any case, an “order of magnitude”In any case, an “order of magnitude”

The Better Printing AlgorithmThe Better Printing Algorithm



One step to set k One step to set k For each number to For each number to

print, 3 stepsprint, 3 steps• Test kTest k• Print kPrint k• Add 1 to kAdd 1 to k

Also need to test Also need to test N+1N+1

Number of steps:1 + 3N + 1

= 3N + 2

Reality CheckReality Check

1.1. Set k to 1Set k to 12.2. Test k Test k N N3.3. Print kPrint k4.4. Add 1 to kAdd 1 to k5.5. Test k Test k N N6.6. Print kPrint k7.7. Add 1 to kAdd 1 to k8.8. Test k Test k N N9.9. Print kPrint k10.10. Add 1 to kAdd 1 to k11.11. Test k Test k N N

For N = 1For N = 1• Stops at step 5Stops at step 5• 5 = 3(1) + 25 = 3(1) + 2

For N = 2For N = 2• Stops at step 8Stops at step 8• 8 = 3(2) + 28 = 3(2) + 2

For N = 3For N = 3• Stops at step 11Stops at step 11• 11 = 3(3) + 2 11 = 3(3) + 2

f(N) = 3N + 2 is goodf(N) = 3N + 2 is good

N = 1

N = 2

N = 3

The Stupid Printing AlgorithmThe Stupid Printing Algorithm



3.3. Test if 1 = NTest if 1 = N

4.4. Add 1 to 1 Add 1 to 1 2 2






Stops at “Test if” stepStops at “Test if” step Steps just to print k?Steps just to print k?

• Add 1 to k–1 Add 1 to k–1 k k• Skip over k–1 numbersSkip over k–1 numbers• Print kPrint k• Count k numbersCount k numbers• Test kTest k

1 + (k–1) + 1 + k + 11 + (k–1) + 1 + k + 1 = 2k + 2= 2k + 2

k = 2

Spot CheckSpot Check

9.9. Add 1 to 2 Add 1 to 2 3 3








Works for k = 3Works for k = 3• 8 = 2(3) + 28 = 2(3) + 2

Off-by-one for k = 1Off-by-one for k = 1• Only takes 3 stepsOnly takes 3 steps• Special caseSpecial case

Check for k = 7Check for k = 7• ……

ExerciseExercise

Stupid Stupid takes 3 steps to print the 1, 6 steps to takes 3 steps to print the 1, 6 steps to print the 2, 8 steps to print the 3, …print the 2, 8 steps to print the 3, …• how many steps to print from 1 to 3?how many steps to print from 1 to 3?• how many steps to print the 4?how many steps to print the 4?• how many steps to print 1 to 4?how many steps to print 1 to 4?• how many steps to print the 5?how many steps to print the 5?• how many steps to print 1 to 5?how many steps to print 1 to 5?

Work in the Stupid AlgWork in the Stupid Algthmthm

For printing 1 to NFor printing 1 to N Let W be the # of steps to print 1 to NLet W be the # of steps to print 1 to N

• W = 3 + 6 + 8 + 10 + 12 + … + (2N + 2)W = 3 + 6 + 8 + 10 + 12 + … + (2N + 2) Do we know what this sum is?Do we know what this sum is? What sum do we know that it’s What sum do we know that it’s likelike??

• how can we use how can we use thatthat sum? sum?

Sum From 1 to NSum From 1 to N

Sum from 1 to N is a common seriesSum from 1 to N is a common series• S = 1 + 2 + 3 + 4 + 5 + … + NS = 1 + 2 + 3 + 4 + 5 + … + N

• S = S = i=1i=1NN i i

• S = N (N + 1) / 2S = N (N + 1) / 2 Also variationsAlso variations

• S = 1 + 2 + 3 + … + (N+1) + (N+2) = ?S = 1 + 2 + 3 + … + (N+1) + (N+2) = ?• S = 2 + 4 + 6 + 8 + 10 + … + 2N = ?S = 2 + 4 + 6 + 8 + 10 + … + 2N = ?• S = 3 + 4 + 5 + 6 + 7 + … + N = ?S = 3 + 4 + 5 + 6 + 7 + … + N = ?

Work in the Stupid AlgWork in the Stupid Algthmthm

WW == 3 + 6 + 8 + 10 + 12 + … + (2N + 2)3 + 6 + 8 + 10 + 12 + … + (2N + 2)

W + 1W + 1 == 4 + 6 + 8 + 10 + 12 + … + (2N + 2)4 + 6 + 8 + 10 + 12 + … + (2N + 2)

(W+1)/2(W+1)/2 == 2 + 3 + 4 + 5 + 6 + … + (N + 1)2 + 3 + 4 + 5 + 6 + … + (N + 1)

1 + (W+1)/21 + (W+1)/2 == 1 + 2 + 3 + 4 + … + (N + 1)1 + 2 + 3 + 4 + … + (N + 1)

== (N+1)(N+2)/2(N+1)(N+2)/2

== (N(N22 + 3N + 2)/2 + 3N + 2)/2

2 + (W+1)2 + (W+1) == NN22 + 3N + 2 + 3N + 2

WW == NN22 + 3N – 1 + 3N – 1

Reality CheckReality Check

For N = 1For N = 1• NN22 + 3N – 1 = (1) + 3N – 1 = (1)22 + 3(1) – 1 = 3 + 3(1) – 1 = 3 correctcorrect



Prediction for N=4: 27 stepsPrediction for N=4: 27 steps check itcheck it

Theory and PracticeTheory and Practice

Number of steps taken (program count):Number of steps taken (program count):NN StupidStupid BetterBetter55 3939 1717

1010 129129 3232100100 10,29910,299 302302

Number of steps taken (alg. anal.):Number of steps taken (alg. anal.):NN NN22 + 3N – 1 + 3N – 1 3N + 23N + 2

Comparing the AlgorithmsComparing the Algorithms

Work for Work for BetterBetter = 3N + 2 = 3N + 2 Work for Work for StupidStupid = N = N22 + 3N – 1 + 3N – 1 For N = 1000For N = 1000

• BetterBetter: 3(1000) + 2 = 3,002 steps: 3(1000) + 2 = 3,002 steps• StupidStupid: (1000): (1000)22 + 3(1000) – 1 = 1,002,999 steps + 3(1000) – 1 = 1,002,999 steps

For N = 1, For N = 1, StupidStupid actually takes less steps… actually takes less steps… For N > 1, For N > 1, BetterBetter takes less steps takes less steps

ExerciseExercise

Calculate the formula for the amount of Calculate the formula for the amount of work in the following algorithm:work in the following algorithm:

Sum numbers from 1 to NSum numbers from 1 to N1.1. set i to 1set i to 12.2. set sum to 0set sum to 03.3. while i while i N N

a.a. add i to sumadd i to sumb.b. add 1 to iadd 1 to i

But, But, But….But, But, But….

The above was rather informalThe above was rather informal Lots of valid complaintsLots of valid complaints

• You ignored variables in You ignored variables in StupidStupid, not in , not in BetterBetter• Some steps will take longer than othersSome steps will take longer than others• And just how did you decide what the “steps” And just how did you decide what the “steps”

were, anyway?were, anyway? Mostly, we ignore these problemsMostly, we ignore these problems

• It has been shown that things work out, anywayIt has been shown that things work out, anyway

DesiderataDesiderata

We want to get an idea of how fast the We want to get an idea of how fast the algorithmalgorithm is is• Want to ignore different speed machinesWant to ignore different speed machines• Want to ignore differences in machine languageWant to ignore differences in machine language• Want to ignore compiler issuesWant to ignore compiler issues• Want to ignore O/S issuesWant to ignore O/S issues

Running time on an “ideal” computerRunning time on an “ideal” computer

FudgesFudges

Can’t deal with fine timing issuesCan’t deal with fine timing issues• Multiplication takes longer than additionMultiplication takes longer than addition• Pretend they take the same timePretend they take the same time

Can’t deal with memory limitsCan’t deal with memory limits• Assume we have infinite memoryAssume we have infinite memory• No paging in/outNo paging in/out

Note: those issues Note: those issues may needmay need to be to be considered sometimesconsidered sometimes

Calculating WorkCalculating Work

Want a function that tells us how many Want a function that tells us how many “steps” an algorithm takes for a problem of “steps” an algorithm takes for a problem of a given sizea given size

Any Any simple simple instruction takes one “step”instruction takes one “step”• complex instructions must be counted as complex instructions must be counted as

multiple steps (e.g. counting numbers printed)multiple steps (e.g. counting numbers printed) Bigger problems naturally take longerBigger problems naturally take longer

Time and SpaceTime and Space

Above is Above is time time complexitycomplexity• How long it takes to do somethingHow long it takes to do something

Also interested in Also interested in space space complexitycomplexity• How much memory is requiredHow much memory is required• Also expressed in terms of problem sizeAlso expressed in terms of problem size

More interested in time than space, tho’More interested in time than space, tho’

Orders of MagnitudeOrders of Magnitude

Work for Work for BetterBetter = 3N + 2 = 3N + 2 Work for Work for StupidStupid = N = N22 + 3N – 1 + 3N – 1 For N = 1000For N = 1000

• BetterBetter: 3(1000) + 2 = 3,002 steps: 3(1000) + 2 = 3,002 steps• StupidStupid: (1000): (1000)22 + 3(1000) – 1 = 1,002,999 steps + 3(1000) – 1 = 1,002,999 steps

WorkWorkBetterBetter(1000) (1000) 3,000 = 3N 3,000 = 3N

WorkWorkStupidStupid(1000) (1000) 1,000,000 = N 1,000,000 = N22

Lower Order TermsLower Order Terms

The extra two steps for The extra two steps for BetterBetter don’t matter don’t matter much when N is “big”much when N is “big”• the 3N term the 3N term dominatesdominates the formula the formula

The extra 3N – 1 steps for The extra 3N – 1 steps for StupidStupid don’t don’t matter much, eithermatter much, either• the Nthe N22 term term dominates dominates its formulaits formula

We’re mostly interested in the We’re mostly interested in the dominantdominant term of the formulaterm of the formula

Graph of 3N Graph of 3N vsvs. N. N22

0

500

1000

1500

2000

2500

10 20 30 40 50

3NN^2

Leading ConstantsLeading Constants

Leading constants (3N Leading constants (3N vsvs. N) are more . N) are more important, but still secondaryimportant, but still secondary

If you If you doubledouble the size of the problem… the size of the problem… ……N and 3N both double the workN and 3N both double the work

• while Nwhile N22 takes four times as much work takes four times as much work ““How fast does it grow?”How fast does it grow?”

Graph of N Graph of N vsvs. 3N . 3N vsvs. N. N22

0

500

1000

1500

2000

2500

10 20 30 40 50

N3NN^2

Big Oh NotationBig Oh Notation

We often just state the “order of magnitude” of We often just state the “order of magnitude” of an algorithman algorithm• the dominant term of the formula…the dominant term of the formula…• ……without any constants addedwithout any constants added

Written with capital OWritten with capital O• N + 75 = O(N)N + 75 = O(N) order Norder N• 3N + 2 = O(N)3N + 2 = O(N) order Norder N• NN22 + 3N – 1 = O(N + 3N – 1 = O(N22)) order N squaredorder N squared

ExercisesExercises

What are the orders of magnitude of the What are the orders of magnitude of the following formulas?following formulas?• 21N + 54.621N + 54.6• 121N121N22 + 5N + 2000 + 5N + 2000• 33N + 2233N + 2222

• 2700N + 3N2700N + 3N22 + 54 + 54• 12N12N3/23/2 + N + N33 + 9 + 9• 53N + 2 log N + 553N + 2 log N + 5

Standard Orders of MagnitudeStandard Orders of Magnitude

O(1)O(1) constantconstant time time O(log N)O(log N) logarithmic timelogarithmic time O(N)O(N) linearlinear time time O(N log N)O(N log N) order N log Norder N log N O(NO(N22)) quadraticquadratic time time O(NO(Nkk)) polynomial timepolynomial time O(2O(2NN)) exponential timeexponential time

Comparison of Orders Comparison of Orders

11 log Nlog N NN NN22 22NN

11 00 11 11 22

11 11 22 44 44

11 22 44 1616 1616

11 33 88 6464 256256

11 44 1616 256256 65,53665,536

11 55 3232 10241024 4,294,967,2964,294,967,296

11 66 6464 40964096 2*10 2*101919

Next TimeNext Time

StacksStacks Algorithmic Analysis of StacksAlgorithmic Analysis of Stacks More on Algorithmic AnalysisMore on Algorithmic Analysis

Extra: the OperationCounter classExtra: the OperationCounter class


Class for keeping track of operationsClass for keeping track of operations• assignmentsassignments• comparisonscomparisons• exchanges (swaps)exchanges (swaps)• otherother

Client responsible for saying what to countClient responsible for saying what to count• those operations involving elements from the those operations involving elements from the

list being searched or sorted, for examplelist being searched or sorted, for example

Where is It?Where is It?

Need to download Young.jarNeed to download Young.jar• get it from sample code for D06get it from sample code for D06

Need to add JAR file to your projectNeed to add JAR file to your project• linux/mac, copy to program’s folder, then use:linux/mac, copy to program’s folder, then use:javac javac -cp Young.jar -cp Young.jar MyProg.javaMyProg.javajava java –cp Young.jar:. –cp Young.jar:. MyProgMyProg

Need to import it into your program!Need to import it into your program!import young.csci2341.OperationCounter;import young.csci2341.OperationCounter;


Methods to add to operation countsMethods to add to operation counts• counter.incrementAssignments()counter.incrementAssignments()• counter.incrementComparisons()counter.incrementComparisons()• counter.incrementExchanges()counter.incrementExchanges()• counter.incrementOther()counter.incrementOther()

Optional argument: how much to incrementOptional argument: how much to increment• defaults to 1defaults to 1

counter.reset() – puts them all back to 0counter.reset() – puts them all back to 0


Methods to get countsMethods to get counts• counter.numberOfAssignments()counter.numberOfAssignments()• counter.numberOfComparisons()counter.numberOfComparisons()• counter.numberOfExchanges()counter.numberOfExchanges()• counter.numberOfOther() counter.numberOfOther() // note: no s// note: no s

Methods for name of other operations:Methods for name of other operations:• counter.nameOfOther()counter.nameOfOther()• counter.setNameOfOther( string )counter.setNameOfOther( string )


Methods to display the counts on screen:Methods to display the counts on screen:• counter.display(true)counter.display(true)Summary of Number of Operations PerformedSummary of Number of Operations PerformedAssignments = Assignments = ##Comparisons = Comparisons = ##Exchanges = Exchanges = ##NameOfOtherNameOfOther = = ##

• counter.display(false)counter.display(false)» as above, but ops with zero counts not printedas above, but ops with zero counts not printed

Displayed on screen; no way to redirectDisplayed on screen; no way to redirect

Using an OperationCounterUsing an OperationCounter

Can use it as a local variableCan use it as a local variablepublic static int doThis() {public static int doThis() { OperationCounter counter = new OperationCounter();OperationCounter counter = new OperationCounter(); … … counter.incrementAssignments();counter.incrementAssignments(); … … return counter.numberOfAssignments();return counter.numberOfAssignments();}}

Getting All the ComparisonsGetting All the Comparisons

Counting beforeCounting beforecounter.incrementComparisons();counter.incrementComparisons();if (v[i] == item)if (v[i] == item)return i;return i;

Counting afterCounting afterif (v[i] == item) {if (v[i] == item) {counter.incrementComparisons();counter.incrementComparisons();return i;return i;

}}counter.incrementComparisons();counter.incrementComparisons();

Loop ComparisonsLoop Comparisons

Counting beforeCounting beforecounter.incrementComparisons();counter.incrementComparisons();while (item != sentinel)while (item != sentinel){{……counter.incrementComparisons();counter.incrementComparisons();

}}

Why two calls?Why two calls?

Loop ComparisonsLoop Comparisons

Counting afterCounting afterwhile (item != sentinel)while (item != sentinel){{counter.incrementComparisons();counter.incrementComparisons();……

}}counter.incrementComparisons();counter.incrementComparisons();

Why two calls?Why two calls?

On Counting OperationsOn Counting Operations

Only need one operation counterOnly need one operation counter• only run one algorithm at a time, so re-use itonly run one algorithm at a time, so re-use it

Needed in many different functionsNeeded in many different functions• if you’re using functions properly!if you’re using functions properly!

““Incidental” to the functionIncidental” to the function• not part of what it’s designed to donot part of what it’s designed to do

Good sign that static variable may be usedGood sign that static variable may be used

Using an OperationCounterUsing an OperationCounter

Or can use it as a class final variableOr can use it as a class final variable• we only actually need one of themwe only actually need one of themprivate static finalprivate static final

OperationCounter oc = new OperationCounter();OperationCounter oc = new OperationCounter();……// in main or some other method// in main or some other method oc.reset();oc.reset(); doSomething();doSomething(); // uses oc without creating it// uses oc without creating it oc.display();oc.display();……

QuestionsQuestions

Algorithmic Analysis Measuring “Work” OperationCounter Class.

Documents