Lecture 01

Copyright Li Zimao @ 2007-2008-1 SCUEC

Design and Analysis of Algorithms

Li ZimaoCollege of Computer Science

South Central University for Nationalites

Email: [email protected] or [email protected] Address: 9- 420 Office Hours: By Appointment


Textbook Cover

What does the cover include? Puzzles, Notions, and Stories of Algorithms.


Text Book and Reference Books

Text Book: Anany Levitin, Introduction to the Design and Analysis of Algori

thms, Tsinghua University Press, 2003, 39RMB.

Reference Books: M.H. Alsuwaiyel, Algorithms Design Techniques and Analysis,

Publishing House of Electronics Industry, 2003, 40RMB. Thomas H. Cormen etc., Introduction to Algorithms (Second Edi

tion), Higher Education Press & The MIT Press, 68RMB. Others, search on the website with keyword “Design and Analysi

s of algorithm”, you will find more...


Grading Schemes

Total 100 Class participation and Assignments: 5% Programming Projects: 10% Midterm Test: 15% Final Exam: 70%

Bonus is possible for those positive and active students


Caution

Please finish your assignments and projects independently. If you really meet problems, you can seek help from me or other

students, but remember not copying, CHEATING!

Please sign attendance when needed, but remember not signing for others, CHEATING!Power off (or ring off) your mobile phone

Feedback (positive or negative) is encouraged and welcome, and IMPORTANT. (ask questions!)Thinking, before, in and after class is encouraged and IMPORTANT.


Why Study this Course?

Donald E. Knuth stated “Computer Science is the study of algorithms” Cornerstone of computer science. Programs will not exist

without algorithms.

Closely related to our lives

Help to guide how others analyze and solve problems

Help to develop the ability of analyzing and solving problems via computers

Very interesting if you can concentrate on this course


Course Prerequisite

Data Structure

Discrete Mathematics

C, Java or other programming languages

Advanced Mathematics


Algorithm: A brief History

Muhammad ibn Musa al-Khwarizmi, one

of the most influential mathematicians in 9th century in Baghdad, wrote a textbook in Arabic about adding, multiplying, dividing numbers, and extracting square roots and computing π. www.lib.virginia.edu/science/parshall/khwariz.html

Many centuries later, decimal system was adopted in Europe, and the procedures in Al Khwarizmi’s book were named after him as “Algorithms”.

(image of Al Khwarizmi from http://jeff560.tripod.com/)


Let’s start our lecture formally

PedagogyWhat I hear, I forget.What I see, I remember.What I do, I understand.

What does this mean?


Expected Outcomes

Student should be able to Define algorithm formally and informally. Explain the idea of different algorithms for computing

the greatest common devisor Describe the process of algorithm design and analysis Design recursive and non-recursive algorithms for co

mputing a Fibonacci number


Notion: Algorithm

You all have learned the course DATA STRUCTURE or DISCRETE MATHEMATICS, and have your own sense on algorithm. to your opinion, what is algorithm?


Notion: Algorithms

“computer”

problem

algorithm

input output

An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.


Notion: Algorithm More precisely, an algorithm is a method or process to solve a problem satisfying the following properties:

Finiteness– terminates after a finite number of steps

Definiteness – Each step must be rigorously and unambiguously specified. -e.g., ”stir until lumpy”

Input– Valid inputs must be clearly specified.

Output– can be proved to produce the correct output given a valid can be proved to

produce the correct output given a valid input. Effectiveness

– Steps must be sufficiently simple and basic.-e.g., check if 2 is the largest integer n for which

there is a solution to the equation xn + yn = zn in positive integers x, y, and z


Examples

Is the following a legitimate algorithm?

i 1

While (i <= 10) do

a i + 1

Print the value of a

End of loop

Stop


Notion: Algorithm*

The exact definition of algorithm is given by Alonzo Church and Alan Turing in 1936. -calculus system of Church Deterministic Turning machine model Equivalent definition

For Turning machine model, see Michael Sipser, Introduction to the Theory of Computation, China Mach

ine Press. John E. Hopcroft, Rajeev Motwani, Introduction to Automata Theory, L

anguages, and Computation (Second Edition), Tsinghua University Press.


Notion: Algorithm*

According to the definition of Algorithms, “almost all” problems are unsolvable. For example: Determine if a polynomial with n (n > 1) variables has

an integral solution. Determine if there is a ‘1111111’ pattern in ’s

decimal format. Halting Problem:

– Determine if a program halts on an input


Examples of Algorithms – Computing the Greatest Common Divisor of Two Integers

gcd(m, n): the largest integer that divides both m and n.

First try -- Euclid’s algorithm: gcd(m, n) = gcd(n, m mod n)

Step1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.

Step2: Divide m by n and assign the value of the remainder to r. Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.


Methods of Specifying an Algorithm

Natural language Ambiguous

“Mike ate the sandwich on a bed.”

Pseudocode A mixture of a natural language and programming language-like

structures Precise and succinct. Pseudocode in this course

– omits declarations of variables– use indentation to show the scope of such statements as for, if, and while.– use for assignment


Pseudocode of Euclid’s Algorithm

Algorithm Euclid(m, n)

//Computes gcd(m, n) by Euclid’s algorithm//Input: Two nonnegative, not-both-zero integers m and n //Output: Greatest common divisor of m and n while n ≠ 0 do

r m mod nm nn r

return mQuestions: Finiteness: how do we know that Euclid’s algorithm actually comes to a stop? Definiteness: nonambiguity Effectiveness: effectively computable.


Second Try for gcd(m, n)

Consecutive Integer Algorithm Step1: Assign the value of min{m, n} to t.

Step2: Divide m by t. If the remainder of this division is 0, go to Step3;otherwise, go to Step 4.

Step3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step4.

Step4: Decrease the value of t by 1. Go to Step2.Questions Finiteness Definiteness Effectiveness Which algorithm is faster, the Euclid’s or this one?


Third try for gcd(m, n)

Middle-school procedure Step1: Find the prime factors of m.

Step2: Find the prime factors of n.

Step3: Identify all the common factors in the two prime expansions found in Step1 and Step2. (If p is a common factor occurring Pm and Pn times in m and n, respectively, it should be repeated in min{Pm, Pn} times.)

Step4: Compute the product of all the common factors and return it as the gcd of the numbers given.

Question Is this a legitimate algorithm?


Sieve of EratosthenesInput: Integer n ≥ 2

Output: List of primes less than or equal to n

for p ← 2 to n do

A[p] ← p

for p ← 2 to sqrt(n) do

if A[p] 0 //p hasn’t been previously eliminated from the list j ← p* p

while j ≤ n do

A[j] ← 0 //mark element as eliminated

j ← j + p

Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


What can we learn from the previous 3 examples?

Each step of an algorithm must be unambiguous.

The same algorithm can be represented in several different ways. (different pseudocodes)

There might exists more than one algorithm for a certain problem.

Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds.


Fundamentals of Algorithmic Problem Solving

Understanding the problem Asking questions, do a few examples by hand, think about special

cases, etc.

Deciding on Exact vs. approximate problem solving Appropriate data structure

Design an algorithmProving correctnessAnalyzing an algorithm Time efficiency : how fast the algorithm runs Space efficiency: how much extra memory the algorithm needs.

Coding an algorithm


Algorithm Design and Analysis Process


Two main issues related to algorithms

How to design algorithms

How to analyze algorithm efficiency


Algorithm Design Techniques/Strategies

Brute force

Divide and conquer

Decrease and conquer

Transform and conquer

Space and time tradeoffs

Greedy approach

Dynamic programming

Backtracking

Branch and bound


Analysis of Algorithms

How good is the algorithm? time efficiency space efficiency

Does there exist a better algorithm? lower bounds optimality


Example: Fibonacci NumberFibonacci’s original question: Suppose that you are given a newly-born pair of rabbits, o

ne male, one female. Rabbits are able to mate at the age of one month so that at

the end of its second month a female can produce another pair of rabbits.

Suppose that our rabbits never die. Suppose that the female always produces one new pair (on

e male, one female) every month.

Question: How many pairs will there be in one year? 1. In the beginning: (1 pair) 2. End of month 1: (1 pair) Rabbits are ready to mate. 3. End of month 2: (2 pairs) A new pair of rabbits are born. 4. End of month 3: (3 pairs) A new pair and two old pairs. 5. End of month 4: (5 pairs) ... 6. End of month 5: (8 pairs) ... 7. after 12 months, there will be 233 rabits

(image of Leonardo Fibonacci from http://www.math.ethz.ch/fibonacci)


Recurrence Relation of Fibonacci Number fib(n):

{0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …}


Algorithm: Fibonacci

Problem: What is fib(200)? What about fib(n), where n is any positive integer?

Algorithm 1 fib(n)if n = 0 then

return (0)if n = 1then

return (1)return (fib(n − 1) + fib(n − 2))

Questions that we should ask ourselves.1. Is the algorithm correct?2. What is the running time of our algorithm?3. Can we do better?


Answer of Questions

Is the algorithm correct? Yes, we simply follow the definition of Fibonacci numbers

How fast is the algorithm? If we let the run time of fib(n) be T(n), then we can formulate

T(n) = T(n − 1) + T(n − 2) + 3 1.6n

T(200) 2139

The world fastest computer BlueGene/L, which can run 248 instructions per second, will take 291 seconds to compute. (291 seconds = 7.85 × 1019 years )

Can Moose’s law, which predicts that CPU get 1.6 times faster each year, solve our problem?

No, because the time needed to compute fib(n) also have the same “growth” rate

– if we can compute fib(100) in exactly a year,– then in the next year, we will still spend a year to compute fib(101)– if we want to compute fib(200) within a year, we need to wait for 100 years.


Improvement

Can we do better? Yes, because many computations in the previous algorithm arere

peated.

Algorithm 2: fib(n)

comment: Initially we create an array A[0: n]

A[0] ← 0,A[1] ← 1

for i = 2 to n do

A[i] = A[i − 1] + A[i − 2]

return (A[n])


Important problem types

sorting

searching

string processing

graph problems

combinatorial problems

geometric problems

numerical problems


Fundamental data structures

list

array

linked list

string

stack

queue

priority queue

graph

tree

set and dictionary

Lecture 01

Documents

greatest common

common factors

prime factors

data structure

euclids algorithm

largest integer

correct output

algorithm