Top Banner
Introduction to Computer Science CSCI 109 Andrew Goodney Fall 2017 China – Tianhe-2 Readings St. Amant, Ch. 4, Ch. 8 Lecture 5: Data Structures & Algorithms 9/25, 2017 “An algorithm (pronounced AL-go-rith- um) is a procedure or formula for solving a problem. The word derives from the name of the mathematician, Mohammed ibn-Musa al-Khwarizmi, who was part of the royal court in Baghdad and who lived from about 780 to 850.”
53

Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Mar 24, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Introduction to Computer ScienceCSCI109

AndrewGoodneyFall2017

China– Tianhe-2

ReadingsSt.Amant,Ch.4,Ch.8

Lecture5:DataStructures&Algorithms9/25,2017

“An algorithm (pronounced AL-go-rith-um) is a procedure or formula forsolving a problem. The word derivesfrom the name of the mathematician,Mohammed ibn-Musa al-Khwarizmi,who was part of the royal court inBaghdad and who lived from about780 to 850.”

Page 2: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Reminders

u Quiz2today(coverslecturematerialfrom1/30and2/6)u Nolecturenextweek(Feb20)duetoPresidents’day

u Quiz3onFeb27(coverslecturematerialfromtoday(2/13))u HW2dueon2/27

u Midtermon3/20

1

Page 3: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Where are we?

2

Date Topic Assigned Due Quizzes/Midterm/Final

21-Aug IntroductionWhatiscomputing,howdidcomputerscometobe?

28-Aug ComputerarchitectureHowisamoderncomputerbuilt?Basicarchitectureandassembly

HW1

4-Sep Laborday

11-Sep DatastructuresWhyorganizedata?Basicstructuresfororganizingdata

HW1

12-Sep

18-Sep DatastructuresTrees,GraphsandTraversals

HW2Quiz1onmaterialtaughtinclass8/21-8/28

25-SepMoreAlgorithms/DataStructures Recursionandrun-time

2-OctComplexityandcombinatorics

How"long"doesittaketorunanalgorithm.

HW2Quiz2onmaterialtaughtinclass9/11-9/25

6-Oct

9-Oct Algorithmsandprogramming(Somewhat)Morecomplicatedalgorithmsandsimpleprogrammingconstructs

Quiz3onmaterialtaughtinclass10/2

16-Oct Operatingsystems WhatisanOS?Whydoyouneedone? HW3Quiz4onmaterialtaughtinclass10/9

23-Oct Midterm MidtermMidtermonallmaterialtaughtsofar.

30-Oct ComputernetworksHowarenetworksorganized?HowistheInternetorganized?

HW3

6-Nov ArtificialintelligenceWhatisAI?Search,plannningandaquickintroductiontomachinelearning

Quiz5onmaterialtaughtinclass10/30

10-Nov

13-Nov Thelimitsofcomputation Whatcan(andcan't)becomputed? HW4Quiz6onmaterialtaughtinclass11/6

20-Nov RoboticsRobotics:backgroundandmodernsystems(e.g.,self-drivingcars)

Quiz7onmaterialtaughtinclass11/13

27-Nov Summary,recap,review Summary,recap,reviewforfinal HW4Quiz8onmaterialtaughtinclass11/20

8-DecFinalonallmaterialcoveredinthesemester

Finalexam11am-1pminSAL101

LastdaytodropaMonday-onlyclasswithoutamarkof“W”andreceivearefundorchangetoPass/NoPassorAuditforSession001

Lastdaytodropacoursewithoutamarkof“W”onthetranscript

Lastdaytodropaclasswithamarkof“W”forSession001

Page 4: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Data Structures and Algorithms

uAproblem-solvingviewofcomputersandcomputing

uOrganizinginformation:sequencesandtreesuOrganizinginformation:graphsuAbstractdatatypes:recursion

3

Reading:St.Amant Ch.4Ch.8(partial)

Page 5: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Overview

4

CPU,Memory,Disk,I/OProgram

Problems

Solution:Algorithms +DataStructures

Pseudocode

Low-levelinstructions

Executionsmanagedby

Compileto

Low-levelinstructionsLow-levelinstructions

ProgramProgramProgram

OperatingSystem

Page 6: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Sequences, Trees and Graphs

5

u Sequence:alistv Itemsarecalledelementsv Itemnumberiscalledtheindex

uGraph

u TreeEric

Emily Jane

Terry Bob

Jim

Mike Chris

Bob

Page 7: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Recursion: abstract data types

uDefiningabstractdatatypesintermsofthemselves(e.g.,treescontaintrees)

u SoatreeisEitherasinglevertex,ora vertexthatistheparent

ofoneormoretrees

6

Eric

Emily Jane

Terry Bob

Drew Pam

Kim

Page 8: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Recursion: algorithms

u Definingalgorithmsintermsofthemselves(e.g.,quicksort)Checkwhetherthesequencehasjustoneelement.Ifitdoes,stopCheckwhetherthesequencehastwoelements.Ifitdoes,andtheyareintherightorder,stop.Iftheyareinthewrongorder,swapthem,stop.Chooseapivotelementandrearrangethesequencetoputlower-valuedelementsononesideofthepivot,higher-valuedelementsontheothersideQuicksortthelowerelementsQuicksortthehigherelements

7

Page 9: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Recursion: algorithms

uHowdoyouwriteaselectionsortrecursively?uHowdoyouwriteabreadth-firstsearchofatreerecursively?Whataboutadepth-firstsearch?

8

Page 10: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Analysis of algorithms

uHowlongdoesanalgorithmtaketorun?timecomplexity

uHowmuchmemorydoesitneed?spacecomplexity

9

Page 11: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Estimating running time

uHowtoestimatealgorithmrunningtime?vWriteaprogramthatimplementsthealgorithm,runit,andmeasurethetimeittakes

vAnalyzethealgorithm(independentofprogramminglanguageandtypeofcomputer)andcalculateinageneralwayhowmuchworkitdoestosolveaproblemofagivensize

uWhichisbetter?10

Page 12: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Analysis of binary search

u n=8,thealgorithmtakes3stepsu n=32,thealgorithmtakes5stepsu Forageneraln,thealgorithmtakeslog2nsteps 11

Page 13: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Growth rates of functions

uLinearuQuadraticuExponential

12

Page 14: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Big O notation

u Characterizefunctionsaccordingtohowfasttheygrowu Thegrowthrateofafunctioniscalledtheorderofthefunction.(hencetheO)

u BigOnotationusuallyonlyprovidesanupperbound onthegrowthrateofthefunction

u Asymptoticgrowth

f(x)=O(g(x))asx->∞ifandonlyifthereexistsapositivenumberM suchthatf(x)≤M*g(x)forallx>x0

13

Page 15: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Examples

u f(n)=3n2 +70v Wecanwritef(n)=O(n2)v WhatisavalueforM?

u f(n)=100n2 +70v Wecanwritef(n)=O(n2)v Why?

u f(n)=5n+3n5

u Wecanwritef(n)=O(n5)

u Why?

14

u f(n)=nlognv Wecanwritef(n)=O(nlogn)v Why?

u f(n)=πnnv Wecanwritef(n)=O(nn)v Why?

u f(n)=(logn)5 +n5

u Wecanwritef(n)=O(n5)

u Why?

Page 16: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Examples

u f(n)=logan andg(n)=logbn arebothasymptoticallyO(logn)v Thebasedoesn’tmatterbecauselogan =logbn/logba

u f(n)=logan andg(n)=loga(nc)arebothasymptoticallyO(logn)v Why?

u f(n)=logan andg(n)=logb(nc)arebothasymptoticallyO(logn)v Why?

u Whataboutf(n)=2n and g(n)=3n ?v Aretheybothofthesameorder?

15

Page 17: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Conventions

uO(1)denotesafunctionthatisaconstantv f(n)=3,g(n)=100000,h(n)=4.7areallsaidtobeO(1)

u Forafunctionf(n)=n2 itwouldbeperfectlycorrecttocallitO(n2)or O(n3)(orforthatmatterO(n100))

uHoweverbyconventionwecallitbythesmallestordernamely O(n2)

16

Page 18: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Complexity

u (Binary)searchofasortedlist:O(log2n)u Selectionsort

u Quicksortu Breadthfirsttraversalofatree

u Depthfirsttraversalofatree

u Prim’salgorithmtofindtheMSTofagraphu Kruskal’s algorithmtofindtheMSTofagraph

u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes

17

Page 19: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Selection sort

u Puttingthesmallestelementinplacerequiresscanningallnelementsinthelist(andn-1comparisons)

u Puttingthesecondsmallestelementinplacerequiresscanningn-1elementsinthelist(andn-2comparisons)

u …u Totalnumberofcomparisonsis

v (n-1)+(n-2)+(n-3)+…+1v n(n-1)/2v O(n2)

u Thereisnodifferencebetweenthebestcase,worstcaseandaveragecase

18

Page 20: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Quicksort

u Bestcase:v Assumeanidealpivotv TheaveragedepthisO(logn)v Eachlevelofprocessesatmostn elementsv Thetotalamountofworkdoneonaverageistheproduct,O(n logn)

u Worstcase:v Eachtimethepivotsplitsthelistintooneelementandtherestv So,(n-1)+(n-2)+(n-3)+…(1)v O(n2)

u Averagecase:v O(n logn)[butprovingitisabitbeyondCS109]

19

Page 21: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

BF and DF traversals of a tree

u Abreadthfirsttraversalvisitstheverticesofatreelevelbylevel

u Adepthfirsttraversalvisittheverticesofatreebygoingdeepdownonebranchandexhaustingitbeforepoppinguptovisitanotherbranch

uWhatdotheyhaveincommon?

20

Page 22: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

BF and DF traversals of a tree

u Abreadthfirsttraversalvisitstheverticesofatreelevelbylevel

u Adepthfirsttraversalvisittheverticesofatreebygoingdeepdownonebranchandexhaustingitbeforepoppinguptovisitanotherbranch

uWhatdotheyhaveincommon?u Bothvisitall theverticesofatreeu IfatreehasV vertices,thenbothBFandDFareO(V)

21

Page 23: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Prim’s algorithm

u Initializeatreewithasinglevertex,chosenarbitrarilyfromthegraph

u Growthetreebyaddingonevertex.Dothisbyaddingtheminimum-weightedgechosenfromtheedgesthatconnectthetreetoverticesnotyetinthetree

u Repeatuntilallverticesareinthetree

u Howfastitgoesdependsonhowyoustoretheverticesofthegraph

u Ifyoudon’tkeeptheverticesofthegraphinsomereadilysortedorderthenthecomplexityisO(V2)wherethegraphhasVvertices

22

Page 24: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Kruskal’s algorithm

u Initializeatreewithasingleedgeoflowestweightu Addedgesinincreasingorderofweight

u Ifanedgecausesacycle,skipitandmoveontothenexthighestweightedge

u Repeatuntilalledgeshavebeenconsideredu Evenwithoutmuchthoughtonhowtheedgesarestored(aslongaswesortthemonceinthebeginning),thecomplexityisO(ElogE)wherethegraphhasEedges

23

Page 25: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Dijkstra’s algorithm

u Ateachiterationwerefinethedistanceestimatethroughanewvertexwe’recurrentlyconsidering

u InagraphwithV vertices,alooseboundisO(V2)

24

Page 26: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Reminders

u Homework#2duetodayu Quiz#2laterinlecture

25

Page 27: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Where are we?

26

Date Topic Assigned Due Quizzes/Midterm/Final

21-Aug IntroductionWhatiscomputing,howdidcomputerscometobe?

28-Aug ComputerarchitectureHowisamoderncomputerbuilt?Basicarchitectureandassembly

HW1

4-Sep Laborday

11-Sep DatastructuresWhyorganizedata?Basicstructuresfororganizingdata

HW1

12-Sep

18-Sep DatastructuresTrees,GraphsandTraversals

HW2Quiz1onmaterialtaughtinclass8/21-8/28

25-SepMoreAlgorithms/DataStructures Recursionandrun-time

2-OctComplexityandcombinatorics

How"long"doesittaketorunanalgorithm.

HW2Quiz2onmaterialtaughtinclass9/11-9/25

6-Oct

9-Oct Algorithmsandprogramming(Somewhat)Morecomplicatedalgorithmsandsimpleprogrammingconstructs

Quiz3onmaterialtaughtinclass10/2

16-Oct Operatingsystems WhatisanOS?Whydoyouneedone? HW3Quiz4onmaterialtaughtinclass10/9

23-Oct Midterm MidtermMidtermonallmaterialtaughtsofar.

30-Oct ComputernetworksHowarenetworksorganized?HowistheInternetorganized?

HW3

6-Nov ArtificialintelligenceWhatisAI?Search,plannningandaquickintroductiontomachinelearning

Quiz5onmaterialtaughtinclass10/30

10-Nov

13-Nov Thelimitsofcomputation Whatcan(andcan't)becomputed? HW4Quiz6onmaterialtaughtinclass11/6

20-Nov RoboticsRobotics:backgroundandmodernsystems(e.g.,self-drivingcars)

Quiz7onmaterialtaughtinclass11/13

27-Nov Summary,recap,review Summary,recap,reviewforfinal HW4Quiz8onmaterialtaughtinclass11/20

8-DecFinalonallmaterialcoveredinthesemester

Finalexam11am-1pminSAL101

LastdaytodropaMonday-onlyclasswithoutamarkof“W”andreceivearefundorchangetoPass/NoPassorAuditforSession001

Lastdaytodropacoursewithoutamarkof“W”onthetranscript

Lastdaytodropaclasswithamarkof“W”forSession001

Page 28: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Recap

u (Binary)searchofasortedlist:O(log2n)u Selectionsort:O(n2)

u Quicksort:O(nlogn)u Breadthfirsttraversalofatree:O(V)

u Depthfirsttraversalofatree:O(V)

u Prim’salgorithmtofindtheMSTofagraph:O(V2)u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)

u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)

27

Page 29: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

What do they have in common?

u (Binary)searchofasortedlist:O(log2n)u Selectionsort:O(n2)

u Quicksort:O(nlogn)u Breadthfirsttraversalofatree:O(V)

u Depthfirsttraversalofatree:O(V)

u Prim’salgorithmtofindtheMSTofagraph:O(V2)u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)

u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)

28

Page 30: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

A knapsack problem

u Youhaveaknapsackthatcancarry20lbs

u Youhavebooksofvariousweights

u Isthereacollectionofbookswhoseweightaddsuptoexactly20lbs?

u Canyouenumerateallcollectionsofbooksthatare20lbs

29

Book Weight

Book 1 2

Book 2 3

Book 3 13

Book 4 7

Book 5 10

Book 6 6

Page 31: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

A knapsack problem

u Youhaveaknapsackthatcancarry20lbs

u Youhavebooksofvariousweights

u Isthereacollectionofbookswhoseweightaddsuptoexactly20lbs?

u Canyouenumerateallcollectionsofbooksthatare20lbs

30

Book Weight

Book 1 2

Book 2 3

Book 3 13

Book 4 7

Book 5 10

Book 6 6

Page 32: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

A knapsack problem

u Youhaveaknapsackthatcancarry20lbs

u Youhavebooksofvariousweights

u Isthereacollectionofbookswhoseweightaddsuptoexactly20lbs?

u Canyouenumerateallcollectionsofbooksthatare20lbs

31

Book Weight

Book 1 2

Book 2 3

Book 3 13

Book 4 7

Book 5 10

Book 6 6

Page 33: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

How many combinations are there?

32

#ofbooks

Combinations Combinations

0 {} 1

1 {2}{3}{13}{7}{10}{6} 6

2 {2,3} {2,13}{2,7}{2,10}{2,6}{3,13}{3,7}{3,10}{3,6}{13,7}{13,10}{13,6}{7,10}{7,6}{10,6}

15

3 {2,3,13} {2,13,7}{2,7,10}{2,10,6}{2,3,7}{2,3,10}{2,3,6}{2,13,10}{2,13,6}{2,7,6}{3,13,7}{3,13,10}{3,13,6}{3,7,10}{3,7,6}{3,10,6}{13,7,10}{13,10,6}{13,7,6}{7,10,6}

20

4 {2,3,13,7} {2,3,13,10}{2,3,13,6}{2,3,7,10}{2,3,7,6}{2,3,10,6}{2,13,7,10}{2,13,10,6}{2,13,7,6}{2,7,10,6}{3,13,7,10}{3,13,10,6}{3,13,7,6}{3,7,10,6}{13,7,10,6}

15

5 {2,3,13,7,10} {3,13,7,10,6}{13,7,10,6,2}{7,10,6,2,3}{10,6,2,3,13}{6,2,3,13,7} 6

6 {2,3,13,7,10,6} 1

TOTAL 64

Page 34: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

How many combinations are there?

33

#ofbooks

Combinations Combinations

0 {} 1

1 {2}{3}{13}{7}{10}{6} 6

2 {2,3} {2,13}{2,7}{2,10}{2,6}{3,13}{3,7}{3,10}{3,6}{13,7}{13,10}{13,6}{7,10}{7,6}{10,6}

15

3 {2,3,13} {2,13,7}{2,7,10}{2,10,6}{2,3,7}{2,3,10}{2,3,6}{2,13,10}{2,13,6}{2,7,6}{3,13,7}{3,13,10}{3,13,6}{3,7,10}{3,7,6}{3,10,6}{13,7,10}{13,10,6}{13,7,6}{7,10,6}

20

4 {2,3,13,7} {2,3,13,10}{2,3,13,6}{2,3,7,10}{2,3,7,6}{2,3,10,6}{2,13,7,10}{2,13,10,6}{2,13,7,6}{2,7,10,6}{3,13,7,10}{3,13,10,6}{3,13,7,6}{3,7,10,6}{13,7,10,6}

15

5 {2,3,13,7,10} {3,13,7,10,6}{13,7,10,6,2}{7,10,6,2,3}{10,6,2,3,13}{6,2,3,13,7} 6

6 {2,3,13,7,10,6} 1

TOTAL 64

Page 35: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Subset sum problem

u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?

u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}

u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations

u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential– i.e.O(2NN)

34

Page 36: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Big O notation

u Characterizefunctionsaccordingtohowfasttheygrowu Thegrowthrateofafunctioniscalledtheorderofthefunction.(hencetheO)

u BigOnotationusuallyonlyprovidesanupperbound onthegrowthrateofthefunction

u Asymptoticgrowth

f(x)=O(g(x))asx->∞ifandonlyifthereexistsapositivenumberM suchthatf(x)≤M*g(x)forallx>x0

35

Page 37: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

What do they have in common?

u (Binary)searchofasortedlist:O(log2n)u Selectionsort:O(n2)

u Quicksort:O(nlogn)u Breadthfirsttraversalofatree:O(V)

u Depthfirsttraversalofatree:O(V)

u Prim’salgorithmtofindtheMSTofagraph:O(V2)u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)

u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)

36

Page 38: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Subset sum problem

u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?

u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}

u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations

u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential– i.e.O(2NN)

37

Page 39: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Travelling salesperson problem

u Givenalistofcitiesandthedistancesbetweeneachpairofcities,whatistheshortestpossibleroutethatvisitseachcityexactlyonceandreturnstotheorigincity?

u Givenagraphwhereedgesarelabeledwithdistancesbetweenvertices.Startataspecifiedvertex,visitallotherverticesexactlyonceandreturntothestartvertexinsuchawaythatsumoftheedgeweightsisminimized

u Therearen! routes(anumberontheorderofnn - muchbiggerthan2n)

u O(n!)

38

Page 40: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Enumerating permutations

u Listallpermutations(i.e.allpossibleorderings)ofnnumbers

uWhatistheorderofanalgorithmthatcandothis?

39

Page 41: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Enumerating permutations

u Listallpermutations(i.e.allpossibleorderings)ofnnumbers

uWhatistheorderofanalgorithmthatcandothis?uO(n!)

40

Page 42: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Analysis of problems

u Studyofalgorithmsilluminatesthestudyofclasses ofproblems

u Ifapolynomialtimealgorithmexiststosolveaproblemthentheproblemiscalledtractable

u Ifaproblemcannotbesolvedbyapolynomialtimealgorithmthenitiscalledintractable

u Thisdividesproblemsinto#?ree groups:known polynomialtimealgorithmbutnotyetproventobeintractable

41

Page 43: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Analysis of problems

u Studyofalgorithmsilluminatesthestudyofclasses ofproblems

u Ifapolynomialtimealgorithmexiststosolveaproblemthentheproblemiscalledtractable

u Ifaproblemcannotbesolvedbyapolynomialtimealgorithmthenitiscalledintractable

u Thisdividesproblemsintothreegroups:v Problemswithknownpolynomialtimealgorithmsv Problemsthatareproventohavenopolynomial-timealgorithmv Problemswithnoknownpolynomialtimealgorithmbutnotyetproventobeintractable

42

Page 44: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Tractable and Intractable

u Tractableproblems(P)v Sortingalistv Searchinganunorderedlistv Findingaminimumspanningtree

inagraph

43

u Intractablev Listingallpermutations(all

possibleorderings)ofnnumbers

u Mightbe(in)tractablev Subsetsum:givenasetof

numbers,isthereasubsetthataddsuptoagivennumber?

v Travellingsalesperson:ncities,n!routes,findtheshortestroute

Theseproblemshavenoknownpolynomialtimesolution

Howevernoonehasbeenabletoprovethatsuchasolutiondoesnotexist

Page 45: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Tractability and Intractability

u ‘Propertiesofproblems’(NOT ‘propertiesofalgorithms’)u Tractable:problemcanbesolvedbyapolynomialtimealgorithm(orsomethingmoreefficient)

u Intractable:problemcannotbesolvedbyapolynomialtimealgorithm(allsolutionsareproventobemoreinefficientthanpolynomialtime)

u Unknown:notknowniftheproblemistractableorintractable(noknownpolynomialtimesolution,noproofthatapolynomialtimesolutiondoesnotexist)

44

Page 46: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Tractability and Intractability

u ‘Propertiesofproblems’(NOT ‘propertiesofalgorithms’)u Tractable:problemcanbesolvedbyapolynomialtimealgorithm(orsomethingmoreefficient)

u Intractable:problemcannotbesolvedbyapolynomialtimealgorithm(allsolutionsareproventobemoreinefficientthanpolynomialtime)

u Unknown:notknowniftheproblemistractableorintractable(noknownpolynomialtimesolution,noproofthatapolynomialtimesolutiondoesnotexist)

45

Page 47: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Subset sum problem

u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?

u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}

u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations

u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential)

46

Page 48: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Take away

u Somesimpleproblemsseemtobeveryhardtosolvebecauseofexponentialorfactorialrun-time

u Notsosimpleinpractice:

47

Problem NaïveSolution(s) Best?

Knapsack 2N 2N,pseudopolynomial

Subset-sum N2N 2N/2,pseudopolynomial

TravellingSalesman N! N22N

Page 49: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

P and NP

u P:setofproblemsthatcanbesolvedinpolynomialtime

u Considersubsetsumv Noknownpolynomialtimealgorithmv However,ifyougivemeasolutiontotheproblem,itiseasyformetocheckifthesolutioniscorrect– i.e.Icanwriteapolynomialtimealgorithmtocheckifagivensolutioniscorrect

uNP:setofproblemsforwhichasolutioncanbecheckedinpolynomialtime

48

Easytosolve

Easytocheck

Page 50: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

P=NP?

49

uAllproblemsinP arealsoinNPuArethereanyproblemsinNP thatarenotalsoinP?

uInotherwords,is

P =NP ?uCentralopenquestioninComputerScience

Page 51: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

P=NP?

u Whydowecare?u “Asidefrombeinganimportantproblemincomputationaltheory,aproofeitherwaywouldhaveprofoundimplicationsformathematics,cryptography,algorithmresearch, artificialintelligence, gametheory,multimediaprocessing, philosophy, economics andmanyotherfields.”

50

Page 52: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Data Structures and Algorithms

uAproblem-solvingviewofcomputersandcomputing

uOrganizinginformation:sequencesandtreesuOrganizinginformation:graphsuAbstractdatatypes:recursion

51

Reading:St.Amant Ch.4Ch.8(partially)

Page 53: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called

Overview

52

CPU,Memory,Disk,I/OProgram

Problems

Solution:Algorithms +DataStructures

Pseudocode

Low-levelinstructions

Executionsmanagedby

Compileto

Low-levelinstructionsLow-levelinstructions

ProgramProgramProgram

OperatingSystem