Algorithm analysis basics

Algorithm Analysisg y

Muhammad Shoaib Farooq

AlgorithmAlgorithm

• An Algorithm is an ordered sequence ofAn Algorithm is an ordered sequence of precise steps for solving a particular problemproblem.

• Well defined computational procedure that takes some value as input and producetakes some value as input and produce same value as output

A R i A bli T• e.g. A Recipe, Assembling a Toy

M.Shoaib Farooq

AlgorithmAlgorithm

• In mathematics and computing anIn mathematics and computing, an algorithm is a procedure (a finite set of well-defined instructions) for accomplishing somedefined instructions) for accomplishing some task which, given an initial state, will terminate in a defined end-stateterminate in a defined end state

From Wikipediao ped a

M.Shoaib Farooq

Algorithm [webster.com]Algorithm [webster.com]• Main Entry: al·go·rithmy g

Pronunciation: 'al-g&-"ri-[th]&mFunction: nounEtymology: alteration of Middle English algorisme, from y gy g gOld French & Medieval Latin; Old French, from Medieval Latin algorismus, from Arabic al-khuwArizmi, from al-KhwArizmI fl A.D. 825 Arabian mathematician: a procedure for solving a mathematical problem (as of finding the greatest common divisor) in a finite number of steps that frequently involves repetition of an operation;p q y p p ;broadly : a step-by-step procedure for solving a problem or accomplishing some end especially by a computer

M.Shoaib Farooq

HistoryHistory• The word algorithm comes from the name of the

9th century Persian Muslim mathematician Abu9th century Persian Muslim mathematician Abu Abdullah Muhammad ibn Musa al-Khwarizmi.

• The word algorism originally referred only to the rules of performing arithmetic using Hindu-rules of performing arithmetic using HinduArabic numerals but evolved via European Latin translation of al-Khwarizmi's name into algorithmgby the 18th century.

M.Shoaib Farooq

Algorithms and ProgramsAlgorithm: a method or a process followed toAlgorithm: a method or a process followed to

solve a computational problem.

An algorithm takes the input to a problem (function) and transforms it to the output(function) and transforms it to the output.– A mapping of input to output.

A problem can have many algorithms.A computer program is an instance orA computer program is an instance, or

concrete representation, for an algorithm in some programming language

M.Shoaib Farooq

some programming language

Instance• An input sequence is called an instance of

algorithmalgorithm.e.g. A sorting AlgorithmInput : A sequence of n numbers

(a1,a2…an)Output: A permutation (reordering)

(a1’,a2’..an’) of the input sequence such that a1’<=a2’<=……<=an’

M.Shoaib Farooq

Algorithm PropertiesAlgorithm PropertiesAn algorithm possesses the following g p g

properties:– It must be correct.– It must be composed of a series of

concrete steps.Th b bi it t hi h– There can be no ambiguity as to which step will be performed next.It must be composed of a finite number of– It must be composed of a finite number of steps.

– It must terminateM.Shoaib Farooq

It must terminate..

Analyzing Algorithmy g g• Analysis means predicting the resources

that the algorithm requiresthat the algorithm requires.• Before analysis we must know the

implementation technology/ Model ofimplementation technology/ Model of computation.

- RAM (Random Access Machine)Concurrent Operations (Uni Processor)Concurrent Operations (Uni-Processor)

Standard generic single-processor machine.

- Parallel Machines (PRAM)

M.Shoaib Farooq

( )

Model Of ComputationRAM- A RAM is an idealized machine with an infinity y

large random-access memory.- Instruction are executed one by one (there isInstruction are executed one by one (there is

no parallelism).Abstract machine which has an unlimited- Abstract machine which has an unlimited number of registers of unlimited size which can be accessed randomlycan be accessed randomly

M.Shoaib Farooq

RAM (cont..)• The RAM model contains instructions

commonly found in real computers:• Arithmetic (Add, subtract, multiply, divide,

remainder floor, ceiling), , g),• Data movement (load, store, copy)• Control (Conditional and unconditional• Control (Conditional and unconditional

branch)

* Each instruction takes a constant amount of time

M.Shoaib Farooq

Distributed Systems• A collection of certain no of autonomous• A collection of certain no of autonomous,

dispersed, but interconnected computers viacommunication path that appear to the usercommunication path that appear to the userof the system as a single computer.

• Autonomous Controlling itselfAutonomous Controlling itself• Dispersed geographically scattered in

a room, building, Campusa room, building, Campus city, Continent or World-Over

• Communication Path Copper-wire, Optic-Fiber, pp , p ,Laser, Microwaves or Communication Satellite

• Single Computer virtual UniProcessor

Lost Update ProblemTransaction T:Bank$Withdraw(A,4);Bank$Deposit(B, 4)

Transaction U:Bank$Withdraw(C, 3);Bank$Deposit(B, 3) $ p ( , ) $ p ( , )

Banlance := A.Read() $100a a ce ead() $ 00A.Write (balance-4) $96

Balance := C.Read() $300C Write(balance 3) $297

Balance := B.Read() $200C.Write(balance-3) $297

Balance := B.Read() $200

B.Write ( balance +4 ) $204B.Write( balance + 3) $203

M.Shoaib Farooq

PRAM• PRAM stands for Parallel Random Access

Machine, which is an abstract machine for designing the algorithms applicable to parallel computers.

• It eliminates the focus on miscellaneous issuesIt eliminates the focus on miscellaneous issues such as synchronization and communication.

• In terms of Flynn‘s, PRAMs are multiple i t ti lti l d t t (MIMD )

M.Shoaib Farooq

instruction multiple data computers (MIMDs).

Algorithm Selection Criteria1- CorrectnessCompute valid input for a finite amount of time and

produce a right output2. Amount of Work Done

How much data is processed in a unit of time3. Amount of space usedp

How much memory is used4 Simplicity4. Simplicity5. Optimality

M.Shoaib FarooqOptimal Algorithm can save both time and space

Complexity of Algorithm• Complexity is a function T(n) which

measures the time and/or space used bymeasures the time and/or space used by an algorithm in term of the input size n.

• The running Time of an algorithm on a particular input is the number of primitive operations or “steps” executed.

M.Shoaib Farooq

Space-Time Trade OffSpace Time Trade Off

• Refers to choice between algorithm thatRefers to choice between algorithm that allows one to decrease the running time of an algorithm solution by increasing thean algorithm solution by increasing the space to store and vice-versa

M.Shoaib Farooq

Algorithmic Languageg g g1- Indentation indicates block structure. e.g

body of loopbody of loop2- Looping Constructs while, for and the

conditional if then elseconditional if-then-else3- The symbol ► indicates that the reminder

of the line is a comment.4- Arithmetic expressions may use usual

arithmetic op (+, -,*,/, ). Relational expression may use relational operators ( =, ,, and )

M.Shoaib Farooq

logical ops and, or and not.

5- Assignment statements are a ba ba b c ( Multiple Assignment)

6 S i6- Swappinga b

7- GOTOGoTo LableGoTo Lable

loop : if c the do sgoto loop

M.Shoaib Farooq

end

8- An exit statement may be used to terminate a while or for loopp

9- Return is used to indicate termination of an algorithmalgorithm.

10- Algorithm lang mostly relies on end of lines rather than semicolons Several shortlines rather than semicolons. Several short stmts(eg, assignment stmt) may be written on one line separated by semicolonsone line separated by semicolons.

11- Local variables are used in a procedure. W h ll t l b l i bl ith tWe shall not use global variables without explicit indication.

M.Shoaib Farooq

12- If A is a structure the |A| size of structure

If A is an Array the n=legth[A] upper bound of arrayof array.

13 All A l t d b13- All Array elements are accessed by name followed by index in square brackets.

A[i]14- Parameters are passed to a procedure by p p y

values.

M.Shoaib Farooq

Mathematical Notations & Functions• Floor Function

X called the floor of x, denotes the greatest integer that does not exceed x.

• Ceiling FunctionCeiling FunctionX called the ceiling of x, denotes the least integer that is not less than xinteger that is not less than x

• Integer and Absolute Function

M.Shoaib Farooq

Mathematical Notations & FunctionsPermutationPermutationSet of n elements is an arrangement of the

elements in given orderelements in given ordere.g. Permutation for elements are a, b, cabc, acb, bac, bca, cab, cba

- n! permutation exist for a set of elementspe utat o e st o a set o e e e ts5! = 120 permutation for 5 elements

M.Shoaib Farooq

Mathematics Review+= baba xxx

bl Logarithms

x

−= bab

a

xxx

blogblog

yby logx

ca

xb

=

==

2)(+

=nnn

abba xxlogloglog

alogg

ca

+= baab

122*2222

+==+

=+nnnn

nnn xxxlogloglog −= ba

ba

101024lg,12lg,01lglog)log(

==== abab

M.Shoaib Farooq

g,g,g

Mathematics Review

+n nn )1(Arithmetic Series

∑ +==++++

nnin2

)1(...321=i 1 2

Q d ti i

∑ ++==++++

n nnnin 22 )12)(1(...941Quadratic series

∑=i 1 6

M.Shoaib Farooq

Mathematics Review

Geometric Series

Harmonic series

M.Shoaib Farooq

Mathematics ReviewGeometric Series ProofGeometric Series Proof

1132 −+

∑ an n

:Proof1

1...10

32

−==+++++

=∑ a

aaaaaai

in

......1S

132

32

+++++=+++++=

+aaaaaaSaaaa

nn

n

11

1

1

−=

−=−+

+

aS

aSaSn

n

1−aS

M.Shoaib Farooq