Advanced Algorithms Analysis and Design - CS702 Power Point Slides Lecture 45

Advanced Algorithms Analysis and Design

By

Dr. Nazir Ahmad Zafar

Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Lecture No. 45

Review Lecture


• Analysis independent of the variations in machine, operating system, language, compiler, etc.

• Our model was an abstraction of a standard generic single-processor machine, called a random access machine RAM– infinitely large random-access memory, – instructions execute sequentially

• Every instruction, a basic operation taking unit time.• We identified some weaknesses in our model of

computation but finally we proved that with all these weaknesses, our model is not so bad because it fulfils our needs in design and analysis of algorithms


Lecture No 1: Model of Computation

A Sequence of Mathematical Tools • Sets, Sequences, Cross Product, Relation,

Functions, Operators over above structuresLogic and Proving Techniques• Propositional Logic, Predicate Logic• Proofs Logical Equivalences, Contradiction, Rule of

InferenceMathematical Induction• Simple Induction• Strong Induction


Lecture No 2, 3, 4 & 5: Mathematical Tools

• Fibonacci Sequences• Recursion? Recursive Mathematical Models• First, second, higher order Linear Homogenous

Recurrences with Constant Coefficients• General Homogenous Recurrence when

– Roots distinct, repeated, multiplicity of root is k– many roots with different multiplicities

• Non-homogenous Recurrence, Characteristics and solution

• Recursive Tree methods• Substitution Method• Proof of Master Theorem


Lecture 6, 7, 8, & 9: Recursion

• Major Factors in Algorithms Design• Complexity Analysis• Growth of Functions• Asymptotic Notations• Usefulness of Notations• Reflexivity, Symmetry, Transitivity Relations over

, , O, and o• Relation between , and O• Various Examples Explaining each concept


Lecture 10 & 11: Asymptotic Notations

Brute Force Approach, • Checking primality• Sorting sequence of numbers • Knapsack problem• Closest pair in 2-D, 3-D and n-D• Finding maximal points in n-D

Divide and Conquer?• Merge Sort algorithm• Finding Maxima in 1-D, and 2-D• Finding Closest Pair in 2-D

Lecture 12, 13 & 14


Optimizations Problems and Dynamic Programming 1. Chain-Matrix Multiplication2. Assembly Line Scheduling Problem3. Generalization to n-Line Assembly Problem4. 0-1 Knapsack Problem5. Optimal Weight Triangulation6. Longest Common sub-sequence problem7. Optimal Binary Search Trees


Lecture 15 - 24: Dynamic Programming

Statement: The chain-matrix multiplication problem can be stated as below:

• Given a chain of [A1, A2, . . . , An] of n matrices for i = 1, 2, . . . , n, matrix Ai has dimension pi-1 x pi, find the order of multiplication which minimizes the number of scalar multiplications.

Objective Function• Let m[i, j] = minimum number of multiplications

needed to compute Ai..j, for 1 ≤ i ≤ j ≤ n• Objective function = finding minimum number of

multiplications needed to compute A1..n i.e. to compute m[1, n]


1. Chain Matrix Multiplication

• There are two assembly lines each with n stations• The jth station on line i is denoted by Si, j

• The assembly time at that station is ai,j.

• An auto enters factory, goes into line i taking time ei • After going through the jth station on a line i, the auto

goes on to the (j+1)st station on either line• There is no transfer cost if it stays on the same line• It takes time ti,j to transfer to other line after station Si,j

• After exiting the nth station on a line, it takes time xi for the completed auto to exit the factory.

• Problem is to determine which stations to choose from lines 1 and 2 to minimize total time through the factory.


2. Assembly-Line Scheduling

• Let fi[j] = fastest time from starting point station Si, j

• Objective function = f* = min(f1[n] + x1, f2[n] + x2)• l* = line no. whose nth station is used in fastest way.

Dr. Nazir A. Zafar Advanced Algorithms Analysis and Design

2. Assembly-Line Scheduling Problem

S1 j-1

8 5 6 5 7

In Out

2

4

3

2Si,j

ai,jt1,j-1

• There are n assembly lines each with m stations• The jth station on line i is denoted by Si, j

• The assembly time at that station is ai,j.

• An auto enters factory, goes into line i taking time ei • After going through the jth station on a line i, the auto

goes on to the (j+1)st station on either line• It takes time ti,j to transfer from line i, station j to line i’

and station j+1 • After exiting the nth station on a line i, it takes time xi for

the completed auto to exit the factory. • Problem is to determine which stations to choose from

lines 1 to n to minimize total time through the factory.Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

3. n Line Assembly Scheduling Problem

• Let fi[j] = fastest time from starting point to station Si, j • li[j] = Line no. 1 to n, for station j-1 used in fastest way• ti[j-1] = transfer time from station Si, j-1 to station Si,, j • a[i, j] = time of assembling at station Si, j • f* = is minimum time through any way• l* = line no. whose mth station is used in a fastest way


3. n-Line Assembly Problem

s11 s12 s13 s1m

s21 s22 s23 s2m

In

Outei

x1

xi

sn1 sn2 sn3 snm

enxn

sijsi 1 si 2 si 3 sim

t(1, j-1)

e1

Assumption• Each item must be put entirely in the knapsack or

not included at all that is why the problem is called 0-1 knapsack problem

Remarks• Because an item cannot be broken up arbitrarily, so

it is its 0-1 property that makes the knapsack problem hard.

• If an item can be broken and allowed to take part of it then algorithm can be solved using greedy approach optimally

4. 0-1 Knapsack Problem


• A triangulation of a convex polygon is a maximal set T of pair-wise non-crossing chords.

• It is easy to see that such a set subdivides interior of polygon into a collection of triangles, pair-wise disjoint

Problem Statement

• Given a convex polygon, determine a triangulation that minimizes sum of the perimeters of its triangles


5. Optimal Weight Triangulation Problem

Statement:• In the longest-common-subsequence (LCS) problem,

we are given two sequences X = <x1, x2, . . . , xm> and

Y = <y1, y2, . . . , yn> • And our objective is to find a maximum-length

common subsequence of X and Y.

Note:• This LCS problem can be solved using brute force

approach as well but using dynamic programming it will be solved more efficiently.

6. Longest Common Subsequence Problem


• If X = (x1, x2,. . ., xm), and Y = (y1, y2, . . ., yn) be sequences and let us suppose that Z = (z1, z2, . . ., zk) be a longest common sub-sequence of X and Y

1. if xm = yn, then zk = xm and Zk – 1 is LCS of Xm – 1, Yn-1.

2. If xm yn, then zk xm implies that Z is LCS of Xm – 1 and Y

3. If xm yn then zk yn implies Z is LCS of X and Yn – 1


Optimal Substructure of an LCS

ji

ji

y xand 0ji, if ))1,(),,1(max(

y xand 0ji, if 1)1,1(0 j OR 0 i if 0

),(

jicjic

jicjic

A translator from English to, say, Urdu.• Use a binary search tree to store all the words in our

dictionary, together with their translations.• The word “the” is much more likely to be looked up

than the word “ring”• So we would like to make the search time for the

word “the” very short, possibly at the expense of increasing the search time for the word “ring.”

Problem Statement: • We are given a probability distribution that

determines, for every key in the tree, the likelihood that we search for this key. Objective is to minimize expected search time of tree.


7. Optimal Binary Search Trees

List of Greedy Algorithms discussed in this Course 1. Activity Selection Problem2. Fractional Knapsack Problem3. Coin Change Making Problem4. Huffman Problem5. Road Trip Problem


Lecture 25-26: Greedy Algorithms

We went through the following steps in the above problem:1. Determine the suboptimal structure of the problem.2. Develop a recursive solution.3. Prove that at any stage of the recursion, one of the

optimal choices is the greedy choice. Thus, it is always safe to make the greedy choice.

4. Show that all but one of the sub-problems induced by having made the greedy choice are empty.

5. Develop a recursive algorithm that implements the greedy strategy.

6. Convert this recursive algorithm to an iterative one.


Steps Designing Greedy Algorithms

• In Huffman coding, variable length code is used• Data considered to be a sequence of characters.• Huffman codes are a widely used and very effective

technique for compressing data– Savings of 20% to 90% are typical, depending on the

characteristics of the data being compressed. • Huffman’s algorithm uses table of frequencies of

occurrence of characters to build up an optimal way of representing each character as a binary string.

• Objective in Huffman coding is to develop a code that represents given text as compactly as possible

Huffman Codes


Graph Theoretic Algorithms • Graph Concepts and types of graphs• Representation of graphs• Searching Algorithms• Backtracking, Branch and Bound Algorithms• Applications of Searching Algorithm• Minimal Spanning Tree Algorithms

– Kruskal’s Algorithm– Prim’s Algorithm

• Shortest Path Algorithms


Lecture 27-37: Graph Theoretic Algorithms

• Given a graph G = (V, E) such that– G is connected and undirected– w(u, v) weight of edge (u, v)

• T is a Minimum Spanning Tree (MST) of G if – T is acyclic subset of E (T E)– It connects all the vertices of G and

– Total weight, w(T) = is minimized.Tvu

vuw),(

),(

Minimum Spanning Tree



Shortest Path Problems

• Single-source shortest path– The Bellman-Ford Algorithm– Shortest Path in directed acyclic graphs– Dijkstra’s Algorithm

• Single-destination shortest path• Single-pair shortest path• All-pairs shortest-paths

– Matrix Multiplication– The Floyd-Warshall Algorithm– Johnson’s Algorithm


The Bellman-Ford Algorithm

BELLMAN-FORD (G, w, s)1 INITIALIZE-SINGLE-SOURCE (G, s)2 for i ← 1 to |V [G]| - 13 do for each edge (u, v) E[G]4 do RELAX (u, v, w)5 for each edge (u, v) E[G]6 do if d[v] > d[u] + w(u, v)7 then return FALSE8 return TRUE

(V)

} (V.E)

} O(E)

Total Running Time = O(V.E)Contd..


Algorithm : Shortest Path (dag)

DAG-SHORTEST-PATHS (G, w, s)1 topologically sort the vertices of G2 INITIALIZE-SINGLE-SOURCE (G, s)

3 for each vertex u, taken in topologically sorted order

4 do for each vertex v Adj[u]5 do RELAX (u, v, w)

(V+E)

Each iteration of for loop takes (1)

Total Running Time = (V+E)

(V)

(V) (E)

Input Given graph G(V, E) with source s, weights wAssumption• Edges non-negative, w(u, v) ≥ 0, (u, v) E• Directed, if (u, v) E then (v, u) may or may not EObjective: Find shortest paths from s to every u VApproach• Maintain a set S of vertices whose final shortest-path

weights from s have been determined• Repeatedly select, u V – S with minimum shortest

path estimate, add u to S, relax all edges leaving u.• Greedy, always choose light vertex in V-S , add to S


Dijkstra’s Algorithm


Lecture 38-40: Number Theoretic Algorithms

• Applications of Number Theory• Some Important Concepts and Fats useful in

number theoretic Algorithms• Modular Arithmetic• Finding GCD• Euclid’s Algorithm• Extended Euclid’s Algorithm• Residues and Reduced set of Residues• Chinese Remainder Theorem• RSA Cryptosystem


1. Choose two distinct large random prime numbers p and q such that p q

2. Compute n by n = pq, n is used as modulus3. Compute the totient function (n)4. Choose an integer e such that 1 < e < (n) and e

and (n) share no factors other than 15. Compute d to satisfy the congruence relation;

de ≡ 1 mod (n) i.e. de = 1 + k(n) for some integer k

6. Publish the pair P =(e, n) as his RSA public Key7. Keep secret pair S =(d, n) as his RSA secret Key

The RSA Public Key Cryptosystem


Lecture 41-44: Further Topics

• String Matching Problem– Naïve approach– Rabin Karp algorithm– String Matching using Finite automata

• Polynomials and Fast Fourier Transform• NP Completeness