Alg1

Universidad de los AndesUniversidad de los Andes

nalysis ofnalysis of

lgorithmslgorithms

OUTLINEOUTLINE

IntroductionWhat is an algorithm?

Semantic analysis

Correctness

Analysis of complexity

What is an algorithm?What is an algorithm?

• Well-defined computational procedure that takes Well-defined computational procedure that takes some value, or a set of values, as some value, or a set of values, as inputinput and produces and produces some value, or a set of values, as some value, or a set of values, as outputoutput. An . An algorithm is thus a algorithm is thus a sequence of computational of sequence of computational of stepssteps that transforms the input into the output. that transforms the input into the output. We can also view an algorithms as a tool for We can also view an algorithms as a tool for solving solving a well-specified computational problema well-specified computational problem[Cormen, 91].[Cormen, 91].

•A A set of instructionsset of instructions, clearly specified, that are , clearly specified, that are performed in order to performed in order to solve a problemsolve a problem [Baase, 91]. [Baase, 91].

•Unambiguous specification of a Unambiguous specification of a sequence of stepssequence of steps performed to performed to solve a problemsolve a problem [Aho, 95]. [Aho, 95].

AlgorithmAlgorithm

A sequence of computational of stepssequence of computational of steps for solving a well-specified computational problemwell-specified computational problem

The statement of the problem specifies in general terms the desired input/output relationship

The algorithm describes a specific computational procedure for achieving that input/ouput

relationship

AlgorithmAlgorithm

I: input / O:output

I Ostepssteps

INPUT: A sequence of numbers

<a1,a2,a3,...,an>

OUTPUT: A permutation

<a’1,a’2,a’3,...a’n> such that

a’1 ai2 ai3 ... ain

Sorting problemSorting problem

A possible input sequence is a instance of the sorting problem

<31,41,59,26,41,58>

A sorting algorithms must return: <26,31,41,41,58,59>

Instance of the sorting problemInstance of the sorting problem

Analysis of algorithmsAnalysis of algorithms

ComplexityComplexity

Time complexityTime complexity

Space complexitySpace complexityAnalysisAnalysis

Semantic Semantic AnalysisAnalysis Correctness Correctness

Semantic AnalysisSemantic Analysis What does an algorithm do? What does an algorithm do? (Discovering its function?)(Discovering its function?)

Is the study of the function associated to an algorithm:

• I I input space : input space : the space of values of the input variables defined in the algorithm.

• O O output space : output space : the space of the values of the output variables defined in the algorithm.

• X X state space : state space : the space of the values of the variables defined in the algorithm.

f :f : II O O { {}} function associated to the algorithmfunction associated to the algorithm

function mistery( n )function mistery( n ) i i 1 1 j j 0 0 for k for k 1 to n do1 to n do

j j i + j i + j i i j - i j - iend-for end-for **return jreturn j

Value in the variables at the point ** (n) (n) XX0 0 XX1 1 XX2 2 XX3 3 XX4 4 XX5 5 XX6 6 XX77 .… .… XXn n (j)(j) k k 0, 1, 2, 3, 4, 5, 6, 7,.…, 0, 1, 2, 3, 4, 5, 6, 7,.…, j j 0, 1, 1, 2, 3, 5, 8, 13,.…, 0, 1, 1, 2, 3, 5, 8, 13,.…, i i 1, 0, 1, 1, 2, 3, 5, 8,.…, 1, 0, 1, 1, 2, 3, 5, 8,.…,

ExamplExamplee

mistery( 0) = mistery( 0) = 00

mistery( 1) = mistery( 1) = 11

mistery( n ) =mistery( n ) = mistery( n - 1 ) + mistery( n - 1 ) + mistery( n - 2), for n > 2 .mistery( n - 2), for n > 2 .

““mistery( n ) computess the n-th Fibonacci’s mistery( n ) computess the n-th Fibonacci’s

number”number”

mistery( n ) =(mistery( n ) =(n - - n)) //5 with5 with

=(1 + =(1 + 5)/2 5)/2 [Golden ratio]

=(1 - =(1 - 5)/25)/2

following an algorithm.following an algorithm.

( ( I ) ) X X00 (O (O00) ) X X1 1 (O(O11) ) ...... X Xm m (O(Omm) ) ......

• If it stops in m steps the output is {OIf it stops in m steps the output is {O0,0, O O11 ... ... OOmm } }

• If it does not stop {OIf it does not stop {O0,0, O O11 ... ... OOmm … … }}

f :f : II O O { {}} with with I II , , O O OO , , XXii XX

I Of(X)

CorrectnessCorrectnessIs an algorithm correct for a problem?Is an algorithm correct for a problem?(Does it do a given function?)(Does it do a given function?)

Given a computational problem

f :f : II O O { {}}

and a algorithm

A .

The algorithm A is correct for the problem ff if the function that A calculates is ff.

Problem: Finding the minumun Problem: Finding the minumun element in an array of n realselement in an array of n reals

Input : T[1..n] : array of realInput : T[1..n] : array of real

Output : x Output : x min{T[i] | 1 min{T[i] | 1 i i n n}}

min : N x Rmin : N x RN R R

(n,T[1..n] T[1..n] ) ) min{T[i] | 1 min{T[i] | 1 i i n} n}

“In order to prove that a program or algorithm is correct, we need to show

that the program gives the correct answer for every possible input.”

function min( n , T[1..n] ) : function min( n , T[1..n] ) : realreal

x x T[1] T[1] for i for i 2 to n do2 to n do

if T[i] < x thenif T[i] < x then x x T[i] T[i]

end_forend_forreturn xreturn x

endend

function min( n , T[1..n] ) : realfunction min( n , T[1..n] ) : real if n =1 then if n =1 then return T[1]return T[1] else else return MIN(min(n-1, T[1..n-1] ),T[n])return MIN(min(n-1, T[1..n-1] ),T[n]) endend

function MIN(a,b ) : realfunction MIN(a,b ) : real if a < b then if a < b then return areturn a else else return breturn bendend

function min( n , T[1..n] ) : realfunction min( n , T[1..n] ) : real if n =1 then if n =1 then return T[1]return T[1] else if n=2 then else if n=2 then if T[1] < T[2] then if T[1] < T[2] then return T[1]return T[1] else else return T[2]return T[2] else else return min(2,[min(n-1, T[1..n-return min(2,[min(n-1, T[1..n-

1] ),T[n]])1] ),T[n]]) endend

Analysis of complexityAnalysis of complexity

How much resources are needed to run an algorithm as a function of the size of the problem it solves?

Is the study of the amount of resources:

• timetime, [ time complexity] t(n) t(n) ,

• memormemory, [ space complexity] s(n) s(n) ,

• …… [ other resources],

that an algorithm will require to process an input of size nn.

ComplexityComplexity

•Time complexity Time complexity t(n)t(n)

computational stepscomputational steps

•Space complexity Space complexity s(n)s(n)

storage - memorystorage - memory

AnalysisAnalysis

Size of a problem n Size of a problem n

A measure of the amount of data that an A measure of the amount of data that an

algorithm takes as input algorithm takes as input

Examples:

•Sorting

nn = number of elements in the input sequence

•Inverting a square matrix A

nn = number of elements in the matrix

•Finding the root of a function f(x)

nn = number of precise digits in the root

•Multiplying two integers

nn = number of bits needed to represent the input

•Finding the shortest path in graph, in this case the size of the input is described by two numbers rather than one.

nn = number of vertex

mm = number of edges

Best case Best case ttbb(n) (n) Minimum execution time for any input of size n

Worst case Worst case ttww(n) (n) Maximum execution time over all the inputs of size n

Average case Average case ttaa(n) (n) Average execution time over all the inputs of size n

Time Complexity Time Complexity t(n) t(n) Number of computational stepsNumber of computational steps

SortingSorting average time taken on the n! different average time taken on the n! different

ways of ways of initially ordering n elements (assuming initially ordering n elements (assuming

they arethey are all different)all different)

• Worst-case analysis is appropriate for an algorithm whose response time is critical

•In a situation where an algorithm is to be used many times on many different instances, it may be more important to know the average execution time on instances of time n

An operation whose execution time can be bounded above by a constant depending only on the particular implementation used (machine, programming language etc). In general we use RAM elementary instructions as the model.

Elementary OperationElementary Operation

Data types Data types Integer Integer

Up to values n uses c log n bits.

Floating point Floating point Bounded rage of values.

Elementary (primitive) instructions Elementary (primitive) instructions ArithmeticArithmetic

Add +, substrac -, multiply *, divide /, remainder %, floor , ceiling .

Data movement Data movement load, store, copy.

RAM ModelRAM ModelRandom Access ModelRandom Access Model

ControlControlConditional and unconditional branch, subroutine call, return.

Special exponentiation Special exponentiation 2k for k small k-shift left.

Each instructions takes a Each instructions takes a constant amount of timeconstant amount of time

Memory-hierarchy effects Memory-hierarchy effects main – cache – virtual main – cache – virtual

[demand paging] [demand paging] are considered only in special are considered only in special casescases

ExamplExamplee

function mistery( n function mistery( n )) i i 1 1 j j 0 0 for k for k 1 to n do1 to n do

j j i + j i + j i i j - i j - iend-for end-for return jreturn j

If the operations are elementary the algorithm takes a time equal to a+bn, for appropriate constants a and b, i.e.,

time in O(n).

We must attribute to the additions a cost proportional to the length of the operands

• For n=47 the addition “j i + j” causes arithmetic overflow on a 32-bit machine

• 45.496 bits are needed to hold the result corresponding to n=65.536

time in O(n2)

!!!But the operations are not elementary !!!

““Two different implementations of Two different implementations of the same algorithm will not differ in the same algorithm will not differ in

efficiency by more than some efficiency by more than some multiplicative constant”multiplicative constant”

Principle of InvariancePrinciple of Invariance

El orden de complejidad de un algoritmo

Es una expresion matematica que indica como los recursos

necesitados para correr un algoritmo se incrementan como

se incrementa el tamaño del problema (que el resuelve).

Orden de complejidad de un algoritmoOrden de complejidad de un algoritmo

Complejidad LinealComplejidad Lineal

La complejidad de un algoritmo se dice que es lineal si:

t(n)=a * n + bt(n)=a * n + b

Con a y b son constantes

function min( T[1..n] ) : realfunction min( T[1..n] ) : real x x T[1] T[1] /* b *//* b */

for i for i 2 to n do 2 to n do /* n times *//* n times */

if T[i] < x then if T[i] < x then /* a *//* a */ x x T[i] T[i]

end_forend_for

return xreturn x

ExamplExamplee

By the invariance principle we if we have two different implementations

(computer + language + compiler +code optimizations)

with worst case time complexity tw1(n) and tw2(n) then

tw1(n) = c tw2(n)

Worst case time complexity

tw(n) = a*n + b(n) = a*n + b