Asymptotes and Algorithms What You’ve Forgotten Since University

Asymptotes and Algorithms What You’ve Forgotten Since UniversityBy Gary ShortDeveloper EvangelistDeveloper Express

DPR302

Agenda

IntroductionWhat is an Algorithm?Why Optimization is ImportantOptimisation is Child’s PlayWhy do we Use Mathematics?One Tactic for Optimizing an AlgorithmBut .Net Saves me from all that, Right?Questions.

Introduction

Gary ShortDeveloper Evangelist DevExpressMicrosoft MVP in C#[email protected]@garyshort

What is an Algorithm?

… a finite list of well defined steps to complete a task or calculation.

There are simple ones…

There are complex ones…

Prove no three positive integers a, b, and c can satisfy the equation an + bn = cn for any integer

value of n greater than two

Why is optimization important?

http://www.flickr.com/photos/mushjazzlexima-leakpaalex/2237327967/

Optimization is Child’s Play

Take This Algorithm…

Stop current taskWash handsSit at table

Foreach item of food in MainCourseConsume food

Wait for dessertConsume dessertWait for meal completionForeach dish in dishes

Clear dish from tableForeach dish in dishes

Wash dishReturn to previous task

Optimized by my 4 Year Old Daughter to…

Pause current gameSet velocity = MAX_INTMove to tableTakeSliceBread(2)Foreach item of food in MainCourse

Place item on BreadSlice(1)Place BreadSlice(2) on topLeave tableResume current game

And it’s not a skill we lose in adulthood

Morning Algorithm

RiseShowerBoil kettleMake teaToast breadSpread butter on toastConsume breakfastBrush teethCommute to work

Optimized for Smallville…

RiseRush downstairsPut kettle on to boilPut toast in toasterRush back upstairsShowerRush downstairsMake tea and spread toastWatch SmallvilleConsume tea and toast

The Only Thing Hard About This is…

The language we chose to use to describe it

𝐴=𝜋 𝑟2

(𝑥+𝑎 )𝑛=∑𝑘=0

𝑛

(𝑛𝑘)𝑥𝑘𝑎𝑛−𝑘

𝑓 (𝑥 )=𝑎0+∑𝑛=1

∞

(𝑎𝑛cos𝑛𝜋 𝑥𝐿 +𝑏𝑛 sin

𝑛𝜋 𝑥𝐿 )

𝑎 2+𝑏 2=𝑐 2𝑥=−𝑏±√𝑏2−4𝑎𝑐

2𝑎

𝑒𝑥=1+ 𝑥1 !

+𝑥22 !

+𝑥33 !

+…,−∞<𝑥<∞

𝐴=𝜋 𝑟2

𝐴=𝜋 𝑟2

𝑎 2+𝑏 2=𝑐 2

𝑎 2+𝑏 2=𝑐 2

𝑎2+𝑏2=𝑐

2

𝐴=𝜋 𝑟2

(𝑥+𝑎)𝑛 =∑

𝑘=0

𝑛 (𝑛𝑘)𝑥𝑘 𝑎𝑛

−𝑘

So Why do we use Math?

http://www.flickr.com/photos/truelifeimages/2102930162/

http://www.flickr.com/photos/29019866@N00/4783131736/

Our Model Has…

Single processorRAMUnderstands arithmetical instructions

AddSubtractPopPush

All instructions executed sequentiallyAll instructions take constant time to complete.

demo

Running Time is Then…

Sum running times for each statementProduct of the cost and the timeSo…T(n) = c1n + c2(n-1) + c3(n-1) + c4(sum of tj for 1 <= j <= n) + c5(sum of tj -1 for 1 <= j <= n) + c6(sum of tj -1 for 1 <= j <= n) + c7(n-1).

Best Case Running time

When the collection is already sortedT(n) = c1n + c2(n-1) + c3(n-1) + c4(n-1) + c7(n-1-)T(n) = (c1+c2+c3+c4+c7)n – (c2+c3+c4+c7)Which is a function in the form an + bThus T(n) is a linear function of n.

Let’s Quickly Prove That…

Worst Case Running Time

Reverse sorted orderT(n) = c1n + c2(n-1) + c3(n-1) + c4(n(n+1)/2 -1) + c5(n(n-1)/2) + c6(n(n-1)/2) + c7(n-1)T(n) = (c4/2 + c5/2 + c6/2)n^2 + (c1 + c2 + c3 + c4/2 – c5/2 – c6/2 + c7)n – (c2 + c3 + c4 + c7)Which is a function in the form an^2 + bn + cThus T(n) is a quadratic function of n.

Again Let’s Quickly Prove That

So Now We Can Prove Speed

an + bIs faster thanan^2 + bn + c

Can we Make The Notation More Readable?

Asymptotic Notation

Big ‘O’ NotationFor the functions

an + ban^2 + bn + c

If n is sufficiently largeThen the largest term of n dominates the function

SoT(n) = an + b = O(n) T(n) = an^2 + bn + c + O(n^2).

So Now we Can Say…

Insertion Sort isBest Case

O(n)Worst Case

O(n^2)

But we only care about the worst case

Let’s Optimize Our Search Algorithm

Let’s do What we Did When we Were Kids

Divide the problem up into smaller parts and conquer each of those smaller parts before combining the solutions.

Demo

Okay, so How Fast is That Algorithm?

Merge Sort in 3 Easy Parts

DivideComputes the middle of the arrayConstant time

D(n) = O(1)

ConquerRecursively solve two n/2 sized sub problems

Each contribute 2T(n/2)

CombineCombine step is linear

C(n) = O(n)

So worst case2T(n/2) + O(n)

Use The Master Method to Solve This Recursion

What is The Master Method?

‘Recipe’ for solving recurrences in the formT(n) = aT(n/b) + f(n)Where

The constants a >= 1 and b > 1And f(n) is asymptotically positive

Our function 2T(n/2) + O(n) is in this form

How Does it Work?

Recall it works with function of the formaT(n/b) + f(n)

We compare f(n) with n log baThe larger of the two determines the solutionIf n log ba is larger then

T(n) = O(nlogba)If f(n) is larger then

T(n) = O(f(n))If they are equal then we multiply by log factor

T(n) = O(f(n) log n)

So in Our Example

We’re in case threeWorst case Merge Sort is

O(n log n)Which is much better than Insertion Sort

O(n^2)

But .net Handles all this for me

List<T> - Add

List<T> - Growing

List<T> - EnsureCapacity

List<T> - Capacity

List<T> - AddRange

List<T> - AddRange

List<T> - InsertRange

So What are the Performance Implications?

Data Provider

Tester

Results

10 100 1000 10000 100000 10000000

5000

10000

15000

20000

25000

30000

Performance: Add Versus AddRange

AddAddRange

Number of Elements Added

Num

ber o

f Tic

ks

What We’ve Learned

Performance is importantOptimization comes naturally to usDivide and conquerRunning time is the sum of product of cost and execution Maths syntax makes this all seem scaryAsymptotic notation simplifies languageThis stuff matters, even in the age of .net.

Questions?

[email protected]@garyshort

DPR Track Resources

http://www.microsoft.com/visualstudio http://www.microsoft.com/visualstudio/en-us/lightswitch http://www.microsoft.com/expression/http://blogs.msdn.com/b/somasegar/http://blogs.msdn.com/b/bharry/http://www.microsoft.com/sqlserver/en/us/default.aspxhttp://www.facebook.com/visualstudio

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

http://northamerica.msteched.com

Connect. Share. Discuss.

Complete an evaluation on CommNet and enter to win!

Scan the Tag to evaluate this session now on myTech•Ed Mobile

Click icon to add picture