CSCE 222 Discrete Structures for Computing Solving Recurrences Dr. Hyunyoung Lee Based on slides by Andreas Klappenecker 1
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CSCE 222 Discrete Structures for Computing
!
Solving Recurrences
Dr. Hyunyoung Lee !!!!!!
Based on slides by Andreas Klappenecker
�1
Motivation
We frequently have to solve recurrence relations in computer science.
For example, an interesting example of a heap data structure is a Fibonacci heap. This type of heap is organized with some trees. It’s main feature are some lazy operations for maintaining the heap property. Analyzing the amortized cost for Fibonacci heaps involves solving the Fibonacci recurrence. We will outline a general approach to solve such recurrences.
The running time of divide-and-conquer algorithms requires solving some recurrence relations as well. We will review the most common method to estimate such running times.
�2
Generating Functions
Given a sequence (a0, a1, a2, a3,...) of real numbers, one can form its generating function, an infinite series given by
!
!
The generating functions is a formal power series, meaning that we treat it as an algebraic object, and we are not concerned with convergence questions of the power series.
1X
k=0
akxk
�3
Example 1
Suppose that the sequence is given by
(k+1)k>=0
Then its generating function is given by
! 1X
k=0
(k + 1)xk = 1 + 2x+ 3x2 + · · ·
�4
Example 2
The generating function of the sequence (1, 1, 1, 1, ...)
is given by
1 + x+ x
2+ x
3+ · · · = 1
1� x
.
�5
Example 3
The generating function of the sequence (1, a, a
2, a
3, . . .)
is given by
1 + ax+ a
2x
2+ a
3x
3+ · · · = 1
1� ax
.
�6
Example 4
The generating function of (1, 1, 1, 1, 1, 1, 0, 0, 0, . . .)
is given by
1 + x+ x
2+ x
3+ x
4+ x
5.
x
6 � 1
x� 1= 1 + x+ x
2 + x
3 + x
4 + x
5.
�7
Sum of Sequences
Let (ak)k�0 and (bk)k�0 be sequences with generat-
ing functions a(x) and b(x), respectively.
Then (ak + bk)k�0 has the generating function
a(x) + b(x).
�8
Products
Let a(x) =1X
k=0
a
kx
k and b(x) =1X
k=0
b
kx
k.
Then
a(x)b(x) =1X
k=0
0
@kX
j=0
ajbk�j
1
Ax
k
�9
Example
Recall that the constant sequence (1, 1, 1, 1, . . .) has
the generating function 1/(1� x).
What is the sequence corresponding to the gen-
erating function
1
(1� x)
2=
1
1� x
1
1� x
?
1
1� x
1
1� x
=1X
k=0
0
@kX
j=0
1
1
Ax
k =1X
k=0
(k + 1)xk
�10
Table of Generating Functions
Study the table on page 542!
�11
Solving Recurrences
�12
Shifting Sequences
Let G(x) be the generating function of the sequence
(ak)k. Then
xG(x) =
1X
k=0
akxk+1
=
1X
k=1
ak�1xk.
�13
Solving a RecurrenceSuppose we have the recurrence system withinitial condition a0 = 2 andrecurrence ak = 3ak�1 for k � 1.
G(x)� 3xG(x) =1X
k=0
akxk � 3
1X
k=1
ak�1xk
= a0 +
1X
k=1
(ak � 3ak�1)xk
= 2.
Thus, G(x)� 3xG(x) = (1� 3x)G(x) = 2. Hence,
G(x) =2
1� 3x.
�14
Solving a Recurrence
Since we know that 1/(1-ax)=1+ax+a2x2+...,
we have
G(x) = 2(1+3x+32x2+...).
Therefore, a sequence solving the recurrence is given by
(2,2x3,2x32,...)=(2x3k)k>=0
�15
Fibonacci Numbers (1)
The Fibonacci numbers satisfy the recurrence:
f0 = 0
f1 = 1
fn = fn�1 + fn�2 for n � 2
�16
Fibonacci Numbers (2)
The Fibonacci numbers satisfy the recurrence:
f0 = 0
f1 = 1
f2 = f1 + f0f3 = f2 + f1f4 = f3 + f2
.
.
.
�17
Fibonacci Numbers (3)
Let F (x) =
P1k=0 fkx
kbe the generating function of
the Fibonacci sequence. The recurrence
fn = fn�1 + fn�2
or
fn � fn�1 � fn�2 = 0
suggests that we should consider
F (x)� xF (x)� x
2F (x)
If it were not for the initial conditions, the latter sum
would be identically 0.
�18
Fibonacci Numbers (4)
The value of F (x)� xF (x)� x
2F (x) is
f0 + f1x+ f2x2 + f3x
3 + · · ·�(f0x+ f1x
2 + f2x3 + · · · )
�(f0x2 + f1x
3 + · · · )
f0 + (f1 � f0)x
Since f0 + (f1 � f0)x = x, we get
F (x)� xF (x)� x
2F (x) = F (x)(1� x� x
2) = x.
Hence,
F (x) =x
1� x� x
2
�19
Fibonacci Numbers (5)
We found the generating function
F (x) =
x
1� x� x
2
This function is of a simple form.
If we can find the coe�cient of x
nin the power
series of F (x), we have found a closed form solution
to the Fibonacci numbers.
�20
Fibonacci Numbers (6)
Since x/(1�x�x
2) is a rational function, we can
use the method of partial fractions, which you might
know from calculus.
Factor the denominator
1� x� x
2= (1� ↵1x)(1� ↵2x),
where
↵1 =
1 +
p5
2
, ↵2 =
1�p5
2
.
Then
x
1� x� x
2=
A1
1� ↵1x+
A2
1� ↵2x.
�21
Fibonacci Numbers (7)
We can plug in various values for x to obtain
linear equations that the real numbers A1 and A2
must satisfy. Solving these equations, we get
x
1� x� x
2=
A1
1� ↵1x+
A2
1� ↵2x
with
A1 =
1p5
, A2 = � 1p5
.
so
F (x) =
x
1� x� x
2=
1p5
✓1
1� ↵1x� 1
1� ↵2x
◆
�22
Fibonacci Numbers (8)
We can get the closed form by observing that
F (x) =
1p5
⇣1
1�↵1x� 1
1�↵2x
⌘
=
1p5
�(1 + ↵1x+ ↵
21x
2+ · · · )� (1 + ↵2x+ ↵
22x
2+ · · · )
�
Therefore,
f
n
=
1p5
(↵
n
1�↵
n
2 ) =1p5
1 +
p5
2
!n
� 1�
p5
2
!n
!
�23
Recurrence Relations and Characteristic Polynomials
�24
Linear Homogeneous Recurrence Relations
A linear homogeneous recurrence relation of de-
gree k with constant coe�cients is a recurrence rela-
tion of the form
an = c1an�1 + c2an�2 + · · ·+ ckan�k,
where c1, . . . , ck are real numbers, and ck 6= 0.
linear: an is a linear combination of ak’shomogeneous: no terms occur that aren’t multiples of ak’sdegree k: depends on previous k coefficients.
�25
Example
fn = fn-1+fn-2 is a linear homogeneous recurrence relation of degree 2.
gn = 5 gn-5 is a linear homogeneous recurrence relation of degree 5.
gn = 5 gn-5 + 2 is a linear inhomogeneous recurrence relation.
gn = 5 (gn-5)2 is a nonlinear recurrence relation.
�26
Remark
Solving linear homogeneous recurrence relations can be done by generating functions, as we have seen in the example of Fibonacci numbers.
Now we will distill the essence of this method, and summarize the approach using a few theorems.
�27
Fibonacci Numbers
Let F(x) be the generating function of the Fibonacci numbers.
Expressing the recurrence fn = fn-1+fn-2 in terms of F(x) yields
F(x) = xF(x)+x2F(x) + corrections for initial conditions
(the correction term for initial conditions is given by x).
We obtained: F(x)(1-x-x2) = x or F(x) = x/(1-x-x2)
We factored 1-x-x2 in the form (1-α1x)(1-α2x) and expressed the generating function F(x) as a linear combination of
1/(1-α1x) and 1/(1-α2x)
�28
A Point of Confusion
Perhaps you might have been puzzled by the factorization
p(x) = 1-x-x2 = (1-α1x)(1-α2x)
Writing the polynomial p(x) backwards (i.e. reciprocal polynomial),
c(x) = x2p(1/x) = x2-x-1 = (x-α1)(x-α2)
yields a more familiar form. We call c(x) the characteristic polynomial of the recurrence fn = fn-1+fn-2
�29
Characteristic Polynomial
Let
an = c1an�1 + c2an�2 + · · ·+ ckan�k
be a linear homogeneous recurrence relation. The
polynomial
x
k � c1xk�1 � · · ·� ck�1x� ck
is called the characteristic polynomial of the recur-
rence relation.
Remark: Note the signs!
�30
Theorem
Let c1, c2 be real numbers. Suppose that
r2 � c1r � c2 = 0
has two distinct roots r1 and r2. Then a sequence
(an) is a solution of the recurrence relation
an = c1an�1 + c2an�2
if and only if
an = ↵1rn1 + ↵2r
n2
for n � 0 for some constants ↵1,↵2.
�31
Example
Solve the recurrence system
an = an�1 + 2an�2
with initial conditions a0 = 2 and a1 = 7.
�32
The characteristic equation of the recurrence is
r2 � r � 2 = 0.
The roots of this equation are r1 = 2 and r2 = �1.Hence, (an) is a solution of the recurrence i↵
an = �12n + �2(�1)n
for some constants �1 and �2. From the initial con-ditions, we get
a0 = 2 = �1 + �2
a1 = 7 = �12 + �2(�1)
Solving these equations yields �1 = 3 and �2 = �1.Hence,
an = 3 · 2n � (�1)n.
�33
Further Reading
Our textbook discusses some more variations of the same idea. For example:
- How to solve recurrences which have characteristic equations with repeated roots
- How to solve recurrence of degree > 2
- How to solve recurrences of degree > 2 with repeated roots.
- How to solve certain inhomogeneous recurrences.
�34
Divide-and-Conquer Algorithms and Recurrence Relations
�35
Divide-and-Conquer
Suppose that you wrote a recursive algorithm that divides a problem of size n into
- a subproblems,
- each subproblem is of size n/b.
Additionally, a total of g(n) operations are required to combine the solutions.
How fast is your algorithm?
�36
Divide-and-Conquer Recurrence
Let f(n) denote the number of operations required to solve a problem of size n. Then
f(n) = a f(n/b) + g(n)
This is the divide-and-conquer recurrence relation.
�37
Example: Binary Search
Suppose that you have a sorted array with n elements. You want to search for an element within the array. How many comparisons are needed?
Compare with median to find out whether you should search the left n/2 or the right n/2 elements of the array. Another comparison is needed to find out whether terms of the list remain.
Thus, if f(n) is the number of comparisons, then
f(n) = f(n/2) + 2
�38
Example: Mergesort
• DIVIDE the input sequence in half
• RECURSIVELY sort the two halves
• basis of the recursion is sequence with 1 key
• COMBINE the two sorted subsequences by merging them
�39
Mergesort Example
1 32 42 5 662 64 5 1 2 63
5 2 64 3 1 625 2 64
2 5 643 1
1 362
62
5 62 45 62 34 1 62
3 1 62
�40
Recurrence Relation for Mergesort
• Let T(n) be worst case time on a sequence of n keys
• If n = 1, then T(n) = Θ(1) (constant)
• If n > 1, then T(n) = 2 T(n/2) + Θ(n)
• two subproblems of size n/2 each that are solved recursively
• Θ(n) time to do the merge
�41
Theorem
Let f(n) be an increasing function satisfying the re-
currence
f(n) = af(n/b) + c
whenever n is divisible by b, a � 1, b > 1 an integer,
and c a positive real number. Then
f(x) =
(O(nlogb a
) if a > 1
O(log n) if a = 1
�42
Proof
Suppose that n = bk for some positive integer k.
f(n) = af(n/b) + g(n)= a2f(n/b2) + ag(n/b) + g(n)= a3f(n/b3) + a2g(n/b2) + ag(n/b) + g(n).
.
.
= akf(n/bk) +Pk�1
j=0 ajg(n/bj)
�43
Suppose that n = bk. For g(n) = c, we get
f(n) = akf(1) +
k�1X
j=0
ajc.
For a = 1, this yields
f(n) = f(1) + ck = f(1) + clogb n = O(log n).
For a > 1, this yields
f(n) = akf(1) + cak � 1
a� 1
= O(nlogb a).
If n is not a power of b, then estimate with the next
power of b to get the claimed bounds.
�44
Related Documents
