1 Decrease-and-Conquer Approach Lecture 06 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing.

1

Decrease-and-Conquer Approach

Lecture 06 ITS033 – Programming & Algorithms

Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.

School of Information, Computer and Communication Technology (ICT)

Sirindhorn International Institute of Technology (SIIT)

Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit

[email protected] 5013505 X 2005

2

ITS033Topic 01Topic 01 -- Problems & Algorithmic Problem SolvingProblems & Algorithmic Problem SolvingTopic 02Topic 02 – Algorithm Representation & Efficiency Analysis – Algorithm Representation & Efficiency AnalysisTopic 03Topic 03 - State Space of a problem - State Space of a problemTopic 04Topic 04 - Brute Force Algorithm - Brute Force AlgorithmTopic 05 - Divide and ConquerTopic 05 - Divide and ConquerTopic 06Topic 06 -- Decrease and ConquerDecrease and ConquerTopic 07Topic 07 - Dynamics Programming - Dynamics ProgrammingTopic 08Topic 08 -- Transform and ConquerTransform and ConquerTopic 09Topic 09 - Graph Algorithms - Graph AlgorithmsTopic 10Topic 10 - Minimum Spanning Tree - Minimum Spanning TreeTopic 11Topic 11 - Shortest Path Problem - Shortest Path ProblemTopic 12Topic 12 - Coping with the Limitations of Algorithms Power - Coping with the Limitations of Algorithms Power

http://www.siit.tu.ac.th/bunyarit/its033.phphttp://www.siit.tu.ac.th/bunyarit/its033.phpand and http://www.vcharkarn.com/vlesson/showlesson.php?lessonid=7http://www.vcharkarn.com/vlesson/showlesson.php?lessonid=7

3

This Week Overview Problem size reductionProblem size reduction Insertion SortInsertion Sort Recursive programmingRecursive programming ExamplesExamples

FactorialFactorial Tower of HanoiTower of Hanoi

4

Decrease & Conquer: ConceptLecture 06.1

ITS033 – Programming & Algorithms






5

Introduction TThe he decrease-and-conquer decrease-and-conquer technique is based on technique is based on

exploiting the relationship between a solution to a given exploiting the relationship between a solution to a given instance of a problem and a solution to a smaller instance instance of a problem and a solution to a smaller instance of of the same problemthe same problem. .

Once such a relationship is established, it can be exploited Once such a relationship is established, it can be exploited either top down (recursively) or bottom up (without a either top down (recursively) or bottom up (without a recursion). recursion).

6

Introduction There are three major variations of decrease-and-conquer:There are three major variations of decrease-and-conquer:

1. Decrease by a constant1. Decrease by a constant

2. Decrease by a constant factor2. Decrease by a constant factor

3. Variable size decrease3. Variable size decrease

7

Decrease by a constant In the In the decrease-by-a-constant decrease-by-a-constant variation, the size of an variation, the size of an

instance is reduced by the same constant on each iteration instance is reduced by the same constant on each iteration of the algorithm.of the algorithm.

Typically, this constant is equal to Typically, this constant is equal to 11

8

Decrease by a Constant

9

Decrease by a constant

Consider, as an example, the exponentiation problem of Consider, as an example, the exponentiation problem of computing computing an an for positive integer exponents. The for positive integer exponents. The relationship between a solution to an instance of size relationship between a solution to an instance of size n n and and an instance of size an instance of size n n - 1 is obtained by the - 1 is obtained by the

obvious formula: obvious formula: aann = = aann-1-1 x x aa

So the function So the function f (n) f (n) = = an an can be computed either “top down” can be computed either “top down” by using its recursive definitionby using its recursive definition

or “bottom up” by multiplying or “bottom up” by multiplying a a by itself by itself n n - 1 times.- 1 times.

10

Decrease by a Constant Factor The The decrease-by-a-constant-factor decrease-by-a-constant-factor technique suggests technique suggests

reducing a problem’s instance by the same constant factor reducing a problem’s instance by the same constant factor on each iteration of the algorithm.on each iteration of the algorithm.

In most applications, this constant factor is equal to two. In most applications, this constant factor is equal to two.

11

Decrease by a Constant Factor

12

Decrease by a Constant Factor

If the instance of size If the instance of size n n is to compute is to compute aann, the instance of half , the instance of half its size will be to compute its size will be to compute aan/n/22, with the obvious relationship , with the obvious relationship between the two: between the two: aann = = (a(an/n/22))22..

But since we consider instances of the exponentiation But since we consider instances of the exponentiation problem with integer exponents only, the former works only problem with integer exponents only, the former works only for even for even nn. If . If n n is odd, we have to compute is odd, we have to compute aann-1-1 by using the by using the rule for even-valued exponents and then multiply the result rule for even-valued exponents and then multiply the result byby

13

Variable Size Decrease the the variable-size-decrease variable-size-decrease variety of decrease-and-conquer, a variety of decrease-and-conquer, a

size reduction pattern varies from one iteration of an algorithm to size reduction pattern varies from one iteration of an algorithm to another.another.

Euclid’s algorithm for computing the greatest common divisor Euclid’s algorithm for computing the greatest common divisor provides a good example of such a situation.provides a good example of such a situation.

14

Decrease & Conquer: Insertionsort

Lecture 06.2 ITS033 – Programming & Algorithms






15

Insertion Sort we consider an application of the decrease-by-one we consider an application of the decrease-by-one

technique to sorting an array technique to sorting an array AA[0..[0..n n - 1]. - 1].

Following the technique’s idea, we assume that the smaller Following the technique’s idea, we assume that the smaller problem of sorting the array problem of sorting the array AA[0..[0..n n - 2] has already been - 2] has already been solved to give us a sorted array of size solved to give us a sorted array of size n n - 1: - 1: AA[0]= [0]= . . . . . . = = AA[[n n - 2]. - 2].

How can we take advantage of this solution to the smaller How can we take advantage of this solution to the smaller problem to get a solution to the original problem by taking problem to get a solution to the original problem by taking into account the element into account the element AA[[n n - 1]?- 1]?

16

Insertion Sort we can scan the sorted subarray from right to we can scan the sorted subarray from right to

left until the first element smaller than or equal left until the first element smaller than or equal to to AA[[n n - 1] is encountered and then insert - 1] is encountered and then insert AA[[n n - 1] - 1] right after that element. =>right after that element. =>straight insertion straight insertion sort sort or simply or simply insertion sortinsertion sort..

Or we can use binary search to find an Or we can use binary search to find an appropriate position for appropriate position for AA[[n n - 1] in the sorted - 1] in the sorted portion of the array. => portion of the array. => binary insertion sortbinary insertion sort..

17

Insertion Sort

18

Insertion Sort

19

Insertion Sort Demo

B R U T E F O R C E

unsorted activesorted

20

Insertion Sort Demo


B R U T E F O R C E

21

Insertion Sort Demo


B R U T E F O R C E

22

Insertion Sort Demo


B R U T E F O R C E

23

Insertion Sort Demo


B R UT E F O R C E

24

Insertion Sort Demo


B R UT E F O R C E

25

Insertion Sort Demo


B R UT E F O R C E

26

Insertion Sort Demo


B R UTE F O R C E

27

Insertion Sort Demo


B R UTE F O R C E

28

Insertion Sort Demo


B R UTE F O R C E

29

Insertion Sort Demo


B R UTE F O R C E

30

Insertion Sort Demo


B R UTE F O R C E

31

Insertion Sort Demo


B R UTE F O R C E

32

Insertion Sort Demo


B R UTE F O R C E

33

Insertion Sort Demo


B R UTE F O R C E

34

Insertion Sort

35

Analysis –Worst Case The basic operation of the algorithm is the key The basic operation of the algorithm is the key

comparison comparison AA[[j j ]]> v> v..

The number of key comparisons in this algorithm The number of key comparisons in this algorithm obviously depends on the nature of the input. obviously depends on the nature of the input.

In the worst case, In the worst case, AA[[j j ]]> v > v is executed the largest is executed the largest numbernumber

of times, i.e., for every of times, i.e., for every j j = = i i - 1, - 1, . . . . . . , 0. Since , 0. Since v v = = AA[[ii], it happens if and only if ], it happens if and only if AA[[j j ]]>A>A[[ii] for ] for j j = = i i - - 1, 1, . . . . . . , 0., 0.

36

Analysis –Worst Case In other words, the worst-case input is an array In other words, the worst-case input is an array

of strictly decreasing values. The number of key of strictly decreasing values. The number of key comparisons for such an input iscomparisons for such an input is

37

Analysis – Best Case In the best case, the comparison In the best case, the comparison AA[[j j ]]> v > v is is

executed only once on every iteration of the executed only once on every iteration of the outer loop. It happens if and only if outer loop. It happens if and only if AA[[i i - 1] = - 1] = AA[[ii] ] for every for every i i =1, =1, . . . . . . , , nn-1, i.e., if the input array is -1, i.e., if the input array is already sorted in ascending order. already sorted in ascending order.

Thus, for sorted arrays, the number of key Thus, for sorted arrays, the number of key comparisons iscomparisons is

38

Decrease & Conquer: Recursive Programming

Lecture 06.3 ITS033 – Programming & Algorithms






39

Concept of RecursionConcept of Recursion

A recursive definitionA recursive definition is one which uses the word is one which uses the word or concept being defined in the definition itselfor concept being defined in the definition itself

40

Factorials How is this recursive?How is this recursive?

nn! = ! = nn × ( × (nn-1) × (-1) × (nn-2) × … × 3 × 2 × 1-2) × … × 3 × 2 × 1

= (= (nn-1)!-1)!

So: So: nn! = ! = nn × ( × (nn-1) !-1) ! The factorial function is defined in terms of The factorial function is defined in terms of

itself (i.e. itself (i.e. recursivelyrecursively))

41

Recursive Calculation of Factorials

In order for this to work, we need a In order for this to work, we need a stop casestop case (the simplest (the simplest case)case)

Here: 0! = 1Here: 0! = 1

nn! = ! = nn × ( × (nn-1)!-1)!

42

This is Iterative Problem SolvingThis is Iterative Problem Solving

43

but, this is Recursionbut, this is Recursion

44

long Factorial(int n) { long fact = 1;

for (int i=2; i <= n; i++) fact = fact * i; return fact;}

n! = n (n-1) (n-2) … 1, if n > 0

Iterative SolutionIterative Solution

45

Recursive definition - definition in which something isdefined in terms of a smaller version of itself, e.g.

Programming with RecursionProgramming with Recursion

n! =n (n-1)!, if n > 0

1, if n = 0

46

Stopping Condition in a recursive definition is the case for which the solution can be stated nonrecursively

General (recursive) case is the case for which the solution is expressed in terms of a smaller version of itself.

Programming with RecursionProgramming with Recursion

n! =n (n-1)!, if n > 0

1, if n = 0

47

long MyFact (int n) { if (n == 0) return 1;

return (n * MyFact (n – 1));

}

Recursive call - a call made to the function from within the function itself;

Stopping cond.

recursive case

Recursion SolutionRecursion Solution

48

How does this work?

int int MyFactMyFact (int n) (int n) { if (n == 0) // The stop case{ if (n == 0) // The stop case return 1;return 1; elseelse return n * return n * MyFactMyFact (n-1);(n-1); } // factorial} // factorial

int x = MyFact(3);

MyFact(3)

3*MyFact(2)

2*MyFact(1)

1*MyFact(0)1

1

2

6

49

Example #1

#include <vcl.h>#include <vcl.h>#include <stdio.h>#include <stdio.h>#include <conio.h>#include <conio.h>

void iforgot_A(int n)void iforgot_A(int n){{ for (int i=1; i<=n; i++)for (int i=1; i<=n; i++) {{ printf("%d, I will remember to do my homework.\n",i);printf("%d, I will remember to do my homework.\n",i); }}

printf("Maybe NOT!");printf("Maybe NOT!");}}

void main()void main(){{ iforgot_A(5);iforgot_A(5); getch();getch();}}

Iterative programmingIterative programming

>> iforgot_A(5)1, I will remember to do my homework.2, I will remember to do my homework.3, I will remember to do my homework.4, I will remember to do my homework.5, I will remember to do my homework.Maybe NOT!

50

Example #2

#include <vcl.h>#include <vcl.h>#include <stdio.h>#include <stdio.h>#include <conio.h>#include <conio.h>

void iforgot_B(int n)void iforgot_B(int n){{ if (n>0)if (n>0) { { printf("%d, I will remember to do my homework.\n",n);printf("%d, I will remember to do my homework.\n",n); iforgot_B(n-1);iforgot_B(n-1); }} elseelse printf("Maybe NOT!");printf("Maybe NOT!");}}

void main()void main(){{ iforgot_B(5);iforgot_B(5); getch();getch();}}

Recursive programmingRecursive programming

>> iforgot_B(5)5, I will remember to do my homework.4, I will remember to do my homework.3, I will remember to do my homework.2, I will remember to do my homework.1, I will remember to do my homework.Maybe NOT!

51

Writing Recursive Functions

Get an exact definition of the problem to be solved.Get an exact definition of the problem to be solved.

Determine the size of the input of the problem.Determine the size of the input of the problem.

Identify and solve the Identify and solve the stopping condition(s)stopping condition(s) in which in which the problem can be expressed non-recursively.the problem can be expressed non-recursively.

Identify and solve the general case(s) correctly in Identify and solve the general case(s) correctly in terms of a smaller case of the same problem.terms of a smaller case of the same problem.

52

Divide or decrease problemDivide or decrease problem One “step” makes the problem smaller (but of the same type)One “step” makes the problem smaller (but of the same type)

Stopping case (solution is trivial)Stopping case (solution is trivial)

Concept 2 - Concept 2 - Recursive ThinkingRecursive Thinking

53

Recursion as problem solving technique Recursive methods defined in terms of themselvesRecursive methods defined in terms of themselves

In code - will see a call to the method itselfIn code - will see a call to the method itself Can have more than one “activation” of a method going at the Can have more than one “activation” of a method going at the

same timesame time Each activationEach activation

has own values of parametershas own values of parameters Returns to where it was called fromReturns to where it was called from

System keeps track of thisSystem keeps track of this

54

Stopping the recursion

The recursion must always STOP The recursion must always STOP Stopping condition is importantStopping condition is important

Recursive solutions to problemsRecursive solutions to problems

55

Stopping the recursion

General pattern is General pattern is test for stopping conditiontest for stopping condition

if not at stopping condition:if not at stopping condition: eithereither

do one step towards solutiondo one step towards solution call the method again to solve the restcall the method again to solve the rest

or or call the method again to solve most of the problemcall the method again to solve most of the problem do the final stepdo the final step

56

Implementation of Hanoi

See the implementation of Tower of Hanoi in the lecture

57

Advantages of Recursion

AdvantagesAdvantages Some problems have complicated iterative solutions, Some problems have complicated iterative solutions,

conceptually simple recursive onesconceptually simple recursive ones Good for dealing with dynamic data structures (size Good for dealing with dynamic data structures (size

determined at run time)determined at run time)

..

58

Disadvantages of Recursion

DisadvantagesDisadvantages Extra method calls use memory space & other Extra method calls use memory space & other

resourcesresources Thinking up recursive solution is hard at firstThinking up recursive solution is hard at first Believing that a recursive solution will workBelieving that a recursive solution will work

59

Why Program Recursively?

Recursive CodeRecursive Code Non-Recursive

Code typically has fewer Code typically has fewer lines.lines.

Code can be conceptually Code can be conceptually simpler, depending on your simpler, depending on your perspectiveperspective

Often easier to maintain!Often easier to maintain!

Code is longer.Code is longer. Code executes faster, Code executes faster,

depending on hardware depending on hardware and programming and programming language.language.

60

This Week’s Practice Write a recursive function to calculate Fibonacci numbersWrite a recursive function to calculate Fibonacci numbers

1 2

1 if 1 or 2

if 2nn n

n nf

f f n

What is the result of f(6) ?What is the result of f(6) ?

61

Your recursive functionYour recursive functionYour recursive functionYour recursive function

62

Fibonacci Recursive Tree

fibo(6)

fibo(5) + fibo(4)

fibo(4) + fibo(3) fibo(3) + fibo(2)

fibo(3) + fibo(2)

fibo(2) + fibo(1)

fibo(2) + fibo(1)fibo(2) + fibo(1)

1 1

1 1 1 1 1

1

63

Decrease & Conquer: Homework

ITS033 – Programming & Algorithms






64

Homework: Fake-Coin ProblemDesign an algorithm using Decrease and Conquer

approach to solve Fake Coin Problem

Among n identically looking coins, one is fake (lighter than genuine).

Using balance scale to find that fake coin.

How many time do you use the balance to find a fake coin from n coins ?

Is it optimum ?

65

End of Chapter 6

Thank you!

1 Decrease-and-Conquer Approach Lecture 06 ITS033 – Programming & Algorithms Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing.

Documents

constant factorthe decrease

introductionthe decrease

instance of size n

major variations of

constant factorif

constant variation

constant factor3

constantfactor technique