242-535 ADA: 1. Induction 1
• Objectiveo to introduce mathematical induction through
examples
Algorithm Design and Analysis
(ADA)242-535, Semester 1 2014-2015
1. Mathematical
Induction
242-535 ADA: 1. Induction 2
1. Motivation2. Induction Defined3. Maths Notation Reminder4. Four Examples5. More General Induction Proofs6. A Fun Tiling Problem
Overview
242-535 ADA: 1. Induction 3
• Induction is used in mathematical proofs of many recursive algorithmso e.g. quicksort, binary search
• Induction is used to mathematically define recursive data structureso e.g. lists, trees, graphs
1. Motivation
continued
242-535 ADA: 1. Induction 4
• Induction is often used to derive mathematical estimates of program running timeo timings based on the size of input data
• e.g. time increases linearly with the number of data items processed
o timings based on the number of times a loop executes
242-535 ADA: 1. Induction 5
• Induction is used to solve problems such as:o is S(n) correct/true for all n values?
• usually for all n >= 0 or all n >=1
• Example:
o let S(n) be "n2 + 1 > 0"o is S(n) true for all n >= 1?
2. Induction Defined
continued
S(n) can be muchmore complicated,such as a programthat reads in an nvalue.
242-535 ADA: 1. Induction 6
• How do we prove (disprove) S(n)?
• One approach is to try every value of n:o is S(1) true?o is S(2) true?o ...o is S(10,000) true?o ... forever!!!
Not very practical
242-535 ADA: 1. Induction 7
• Induction is a technique for quickly proving S(n) true or false for all no we only have to do two things
• First show that S(1) is trueo do that by calculation as before
Induction to the Rescue
continued
242-535 ADA: 1. Induction 8
• We prove that S(n) S(n+1) is true• Read as "if S(n) is true then S(n+1) is true"
• When S(1) is true and "S(n) S(n+1)" is true, then S(n) is true for all n>=1.
• Why?
continued
" stands for "implies"
242-535 ADA: 1. Induction 9
• With S(1) and S(n) S(n+1)then S(2) is true o S(1) S(2) when n == 1
• With S(2) and S(n) S(n+1)then S(3) is true o S(2) S(3) when n == 2
• With S(3) and S(n) S(n+1)then S(4) is true o S(3) S(4) when n == 3
• and so on, for all n
242-535 ADA: 1. Induction 10
• We prove the implication by:
• 1. Assume that S(n) is true; write it down
• 2. Write down some extra maths, which depends on the problemo e.g. (n+1)! = n! * (n+1)
• 3. Write down S(n+1)
• 4. Combine S(n) and the maths to obtain S(n+1)
Proving S(n) S(n+1)
242-535 ADA: 1. Induction 11
• Prove S(n): "n2 + 1 > 0" for all n >= 1.
• First task: show S(1) is true by calculationo S(1) == 12 + 1 == 2, which is > 0o so S(1) is true
• Second task: prove S(n) S(n+1) is true
Let’s do it
continued
242-535 ADA: 1. Induction 12
• Assume S(n) is trueo S(n): n2 + 1 > 0
• Extra maths: n > 0
• Write down S(n+1)o S(n+1) == (n+1)2 + 1
== n2 + 2n + 1 +1
== (n2 + 1) + 2n + 1
continued
242-535 ADA: 1. Induction 13
• Use S(n) and the extra maths to show that S(n+1) is trueo S(n) means that n2 + 1 > 0, and the extra maths is n
> 0,
so (n2 + 1) + 2n + 1 > 0o this means that S(n+1) is trueo so S(n) S(n+1)
continued
242-535 ADA: 1. Induction 14
• We have used induction to show two things:o S(1) is trueo S(n) S(n+1) is true
• From these it follows that S(n) is true for
all n >= 1
242-535 ADA: 1. Induction 15
• Three pieces:o 1. A statement S(n) to be proved
• the statement must be about an integer n
o 2. A basis for the proof. This is the statement S(b) for some integer.
Often b = 0 or b = 1.
Induction More Formally
continued
242-535 ADA: 1. Induction 16
• 3. An inductive step for the proof. We prove the statement “S(n) S(n+1)”
o The statement S(n), used in this proof, is called the inductive hypothesis
o We conclude that S(n) is true for all n >= b• S(n) might not be true for some n < b
242-535 ADA: 1. Induction 17
• Summation: means 1+2+3+4+…+n
o e.g. means 4+9+16+…+m2
• Product: means 1*2*3*…*n
3. Maths Notation Reminder
n
i
i1
m
j
j2
2
n
i
i1
242-535 ADA: 1. Induction 18
• Prove the statement S(n):for all n >= 1o e.g. 1+2+3+4 = (4*5)/2 = 10
• Basis. S(1), n = 1so 1 = (1*2)/2
4. Example 1
2
)1(
1
nni
n
i
1
1
1i
i
continued
242-535 ADA: 1. Induction 19
• Inductive Step. Prove S(n) S(n+1)
• 1. Assume S(n) is true:
• 2. The extra maths involve summations:
• 3. We want to calculate S(n+1):
2
)1(
1
nni
n
i(1)
1
1 1
)1(n
i
n
i
nii (2)
2
)2)(1(
2
)11(11
1
nnnni
n
i
(3)
continued
242-535 ADA: 1. Induction 20
• To prove S(n) S(n+1), combine S(n) (equation 1) and the extra maths (equation 2) to calculate S(n+1) (equation 3).
continued
242-535 ADA: 1. Induction 21
• Substitute the right hand side (rhs) of (1) for the first operand of (2), to give:
= (n2 + n + 2n + 2) /2
= (n2+3n+2)/2
)1(2
)1(1
1
nnn
in
i
which is (3)2
)2)(1(1
1
nni
n
i
continued
242-535 ADA: 1. Induction 22
• We have shown:o S(1) is trueo S(n) S(n+1) is true
• This means that S(n) is true for all n ≥ 1• Finished.
continued
242-535 ADA: 1. Induction 23
• Prove the statement S(n):for all n >= 0o e.g. 1+2+4+8 = 16-1
• Basis. S(0), n = 0
so 20 = 21 -1
Example 2
122 1
0
nn
i
i
0
0
022i
i
continued
242-535 ADA: 1. Induction 24
• Inductive Step. Prove S(n) S(n+1)
• 1. Assume S(n) is true:
• 2. The extra maths involve summations:
• 3. We want to calculate S(n+1):
(1)
(2)
(3)
continued
122 1
0
nn
i
i
122 21
0
nn
i
i
1
0 0
1222n
i
n
i
nii
242-535 ADA: 1. Induction 25
• To prove S(n) S(n+1), combine S(n) (equation 1) and the extra maths (equation 2) to calculate S(n+1) (equation 3).
continued
242-535 ADA: 1. Induction 26
• Substitute the right hand side (rhs) of (1) for the first operand of (2), to give:
= 2*(2n+1) - 1
which is (3)
111
0
2122
nnn
i
i
122 21
0
nn
i
i
242-535 ADA: 1. Induction 27
• We have shown:o S(0) is trueo S(n) S(n+1) is true
• This means that S(n) is true for all n ≥ 0• Finished.
continued
242-535 ADA: 1. Induction 28
• Prove the statement S(n): n! >= 2n-1
for all n >= 1
o e.g. 5! >= 24, which is 120 >= 16
• Basis. S(1), n = 1: 1! >= 20
so 1 >= 1
Example 3
continued
242-535 ADA: 1. Induction 29
• Inductive Step. Prove S(n) S(n+1)
• 1. Assume S(n) is true: n! >= 2n-1
• 2. The extra maths involve factorials: (n+1)! = n! * (n+1)
• 3. We want to calculate S(n+1): (n+1)! >= 2(n+1)-1
>= 2n
(1)
(2)
(3)
continued
242-535 ADA: 1. Induction 30
• To prove S(n) S(n+1), combine S(n) (equation 1) and the extra maths (equation 2) to calculate S(n+1) (equation 3).
continued
242-535 ADA: 1. Induction 31
• Substitute the right hand side (rhs) of (1) for the first operand of (2), to give:
(n+1)! >= 2n-1 * (n+1)
>= 2n-1 * 2 since (n+1) >= 2
(n+1)! >= 2n which is (3)
why?
242-535 ADA: 1. Induction 32
• We have shown:o S(1) is trueo S(n) S(n+1) is true
• This means that S(n) is true for all n ≥ 1• Finished.
continued
242-535 ADA: 1. Induction 33
• Prove the statement S(n):for all n >= 1o This proof can be used to
show that. the limit of the sum:is 1
• Basis. S(1), n = 1so 1/2 = 1/2
Example 4
1)1(
1
1
n
n
ii
n
i
1
1 11
1
)1(
1
i ii
continued
...4*3
1
3*2
1
2*1
1
242-535 ADA: 1. Induction 34
• Inductive Step. Prove S(n) S(n+1)
• 1. Assume S(n) is true:
• 2. The extra maths involve summations:
• 3. We want to calculate S(n+1):
(1)
(2)
(3)
continued
1)1(
1
1
n
n
ii
n
i
)2)(1(
1
)1(
1
)1(
11
1 1
nniiii
n
i
n
i
1)1(
)1(
)1(
11
1
n
n
ii
n
i
242-535 ADA: 1. Induction 35
• To prove S(n) S(n+1), combine S(n) (equation 1) and the extra maths (equation 2) to calculate S(n+1) (equation 3).
continued
242-535 ADA: 1. Induction 36
• Substitute the right hand side (rhs) of (1) for the first operand of (2), to give:
=1)1(
)1(
n
nwhich is (3)
)2)(1(
1
1)1(
11
1
nnn
n
ii
n
i
242-535 ADA: 1. Induction 37
• We have shown:o S(1) is trueo S(n) S(n+1) is true
• This means that S(n) is true for all n ≥ 1• Finished.
continued
242-535 ADA: 1. Induction 38
• There can be more than one basis case.
• We can use strong induction where the proof of S(n+1) may use any of S(b), S(b+1), …, S(n)o b is the lowest basis value
5. More General Inductive
Proofs
242-535 ADA: 1. Induction 39
• Show:o S(1) is trueo S(2) is trueo S(3) is true
o S(n-2) S(n+1) is true
• This means that S(n) is true for all n ≥ 1• Why?
Strong Induction Example
3 base cases
242-535 ADA: 1. Induction 40
• Every integer >= 24 can be written as 5a+7b for non-negative integers a and b.o note that some integers < 24 cannot be expressed this
way (e.g. 16, 23).
• Let S(n) be the statement (for n >= 24)“n = 5a + 7b, for a >= 0 and b >= 0”
Example
continued
242-535 ADA: 1. Induction 41
• We will show:o S(24) is trueo S(25) is trueo S(26) is trueo S(27) is trueo S(28) is true
o S(n-4) S(n+1) is true
• This means that S(n) is true for all n ≥ 24• Why?
Using Strong Induction
5 base cases
242-535 ADA: 1. Induction 42
• Basis. The 5 basis cases are 24 through 28.
o S(24) true because 24 = (5*2) + (7*2)o S(25) true because 25 = (5*5) + (7*0)o S(26) true because 26 = (5*1) + (7*3)o S(27) true because 27 = (5*4) + (7*1)o S(28) true because 28 = (5*0) + (7*4)
continued
242-535 ADA: 1. Induction 43
• Inductive Step. Prove S(n-4) S(n+1)
• 1. Assume S(n-4) is true: n - 4 = 5a + 7b
• 2. No extra maths
• 3. We want to calculate S(n+1):
n+1 = 5a' + 7b'
• To prove S(n-4) S(n+1), use S(n-4) (equation 1) to calculate S(n+1) (equation 2).
(1)
(2)
continued
242-535 ADA: 1. Induction 44
• Use S(n-4): n-4 = 5a + 7b
• Add 5 to both sides of the equation:o n-4 + 5 = 5a + 7b + 5o n + 1 = 5(a + 1) + 7bo n + 1 = 5a' + 7b (a' is some new variable)o which means that S(n+1) is true
242-535 ADA: 1. Induction 45
• We have shown:o S(24), S(25), S(26), S(27), S(28) are trueo S(n-4) S(n+1) is true
• This means that S(n) is true for all n ≥ 24• Finished.
continued
242-535 ADA: 1. Induction 46
• A right tromino is a “corner” shapemade of 3 squares:
• Use induction to prove that any number of right trominos can be used to tile (cover) any n*n size board of squareso but n must be a power of 2
• e.g. board sizes can be 2*2, 4*4, 8*8, 16*16, ...
• Show that a tiled board will always have 1 blank square.
6. A Tiling Problem
continued