CS121 Induction Alan J. Hu 1. Induction: The Big Picture We’ll develop your intuition about induction. We’ll see examples of weak, strong, and structural.

CS121Induction

Alan J. Hu

1

Induction: The Big Picture

• We’ll develop your intuition about induction.• We’ll see examples of weak, strong, and

structural induction.

Two different learning goals:1. Learning the CONCEPT of induction.

2. Learning the FORM of an induction proof.

• We’ll see why they are really all the same.

A Little Game

Are you all willing to stand up and move around the classroom?

(If not, I can still convey the ideas, but it won’t be as memorable.)

A Little Game

Everyone in this room fails CPSC 121, unless…

A Little Game

Everyone in this room fails CPSC 121, unless…

you make yourself “safe” by following these rules:

1. The podium at the front of class is safe.

2. Anyone who puts both hands on something or someone safe is also safe.

(3. There is no other way to be safe.)

How do you make everyone safe?

Podium

Do you have to fight for the podium?

Podium

You can be good Canadians…

Podium

You can be less structured…

Podium

How many people can you make safe?

Podium

Is this OK?

Podium

Or worse, this?

Podium

Who fails now?

Podium

Who fails here?

Podium

What’s the point?

• This is exactly how induction works in general!• Base Case: Some cases where you know the

property holds already.• Inductive Case: You must show that the property

holds for any given item, by showing how it is implied by the property on items closer to the base case.

• If you can show the inductive case for all elements in a set, the property holds for all of them – even an infinite number!

What’s the point?

• This is exactly how induction works in general!• If anyone breaks the chain, the proof fails.• If anyone misses the base case, the proof fails.• If there are any cycles (rather than getting closer

to base case), the proof fails.

On the other hand, if you get induction right, it’s a very easy and powerful way to write a proof!

Weak Induction

• This is what you are usually taught first.• Goal: Prove p(n) for all natural numbers n• Base Case: Prove p(0) (and maybe p(1), etc.)• Inductive Case: Assume p(k) for some arbitrary k

and prove that it implies p(k+1).

Weak Induction: What’s really happening?

• This is what you are usually taught first.• Goal: Prove p(n) for all natural numbers n• Base Case: Prove p(0) (and maybe p(1), etc.)• Inductive Case: Assume p(k) for some arbitrary k

and prove that it implies p(k+1).

This is just the polite Canadians version of the game!

p(x) just means x is safe. Base case is the podium.

Inductive case says that an arbitrary p(k) is true because he’s got both hands on p(k-1), who is closer to the base case.

Weak Induction: What your proof should look like

Problem: Prove that the solution to the n disk Towers of Hanoi takes 2^n-1 moves.

(Aside: Do you know this problem?)



Proof: The proof is by induction on n.

Base Case: n=0. In this case, 0 moves are required. 2^0 – 1 = 0, so the property holds on the base case.




Inductive Case: The inductive assumption is that moving a stack of n-1 disks takes 2^(n-1) – 1 moves. Moving a stack of n disks means moving n-1 disks to the intermediate stack, moving the nth disk from start to destination, and the moving the n-1 disks from the intermediate stack to the destination. (continued)




Inductive Case: (continued) Using the inductive assumption, the total number of moves is 2^(n-1) – 1 + 1 + 2^(n-1) – 1 which simplifies to 2^n – 1. QED




Inductive Case: (continued) Using the inductive assumption, the total number of moves is 2^(n-1) – 1 + 1 + 2^(n-1) – 1 which simplifies to 2^n – 1. QED

(Aside: This is usually taught as assuming n and proving n+1, which is OK, too.)

Strong Induction

• This is often taught as something special…• Goal: Prove p(n) for all natural numbers n• Base Case: Prove p(0) (and maybe p(1), etc.)• Inductive Case: Assume p(0) through p(k) for

some arbitrary k and prove that it implies p(k+1).

Strong Induction: What’s Really Happening

• This is often taught as something special…• Goal: Prove p(n) for all natural numbers n• Base Case: Prove p(0) (and maybe p(1), etc.)• Inductive Case: Assume p(0) through p(k) for

some arbitrary k and prove that it implies p(k+1).

This is just the unstructured version of the game!

As long as p(k+1) has as many hands as the rules require on p(0)…p(k) (who are closer to base case), the property holds.

Strong Induction: What your proof should look like

Problem: Prove that the nth Fibonacci number is less than 2^n.

(Aside: Do you know this problem?)



Proof: The proof is by (strong) induction on n.

Base Case: When n=1, the 1st Fibonacci number is 1, which is less than 2, which is 2^1. When n=2, the 2nd Fibonacci number is also 1, which is less than 4, which is 2^2. So the base cases hold.



Proof: The proof is by (strong) induction on n.

Inductive Case: For all n>2, the nth Fibonacci number is defined to be fib(n-1)+fib(n-2). By the inductive assumption, fib(n-1) < 2^(n-1) and fib(n-2) < 2^(n-2). Therefore, fib(n) = fib(n-1) + fib(n-2) < 2^(n-1) + 2^(n-2) < 2^(n-1) + 2^(n-1) = 2^n. QED

Fibonacci proof in pictures

fib(1) < 2^1

fib(3)<2^3

fib(4)<2^4)

fib(5)<2^5

fib(2) < 2^2

fib(6)<2^6)

fib(7)<2^7

Strong Induction: What went wrong?

Theorem: All horses are the same colour.Proof: By (strong) induction on n, the number of horses in

any set of horses.Base Case: n=1. 1 horse is the same colour as itself.Inductive Case: Assume all sets of horses of size from 1 to n-

1 are the same colour. Given any set of horses of size n, separate one from the herd. Call this horse A. By the inductive assumption, the remaining n-1 horses are the same colour. Now, put back the first horse and separate out a different horse, called B; similarly, the remaining n-1 horses are the same colour.

Strong Induction: What went wrong?

Theorem: All horses are the same colour.

Proof: By (strong) induction on n, the number of horses in any set of horses.

Base Case: n=1. 1 horse is the same colour as itself.

Inductive Case: (continued) But, those two sets of n-1 horses had n-2 horses in common, and horses don’t change colour! So A has the same colour as the n-2, who have the same colour as B. So, they all must be the same colour, by transitivity of equality. QED

Structural Induction

• This is extremely common in computer science!• Goal: Prove p(n) for all structures n.

(Aside: This only works when the set of all possible structures is defined recursively/inductively.)

• Base Case: Prove p for all the base case structures.• Inductive Case: For an arbitrary non-base-case

structure n, show that p(n) holding on the structure is implied by p holding on the sub-structures.

Structural Induction: What’s Really Happening

• This is extremely common in computer science!• Goal: Prove p(n) for all structures n.

(Aside: This only works when the set of all possible structures is defined recursively/inductively.)

• Base Case: Prove p for all the base case structures.• Inductive Case: For an arbitrary non-base-case

structure n, show that p(n) holding on the structure is implied by p holding on the sub-structures.

This is still the same game!

Structural Induction: What your proofs should look like

Problem: Prove that a perfect/complete n-ary tree of height h has (n^(h+1) – 1)/(n – 1) nodes.

Aside: Do you know what a tree is yet?

A tree is either a leaf (a node with no children), or a node whose children are trees.

In an n-ary tree, the each non-leaf node has n children.

In a perfect/complete tree, all leaves are at the same depth.



Aside: Do you know what a tree is yet?

A tree is either a leaf (a node with no children), or a node whose children are trees.

In an n-ary tree, the each non-leaf node has n children.

In a perfect/complete tree, all leaves are at the same depth.



Proof: The proof is a structural induction on trees.

Base Case: If the tree is a single leaf, then it has height 0 and 1 node. The formula evaluates as (n^(0+1) – 1)/(n – 1) = 1.




Inductive Case: If the tree is not a leaf, then it must have n children (because it is n-ary), each of which is a tree. Because it is perfect/complete, the height of each of these subtrees is h-1. (continued)




Inductive Case: (continued) By the inductive hypothesis, each of those trees, therefore, have (n^((h-1)+1) – 1)/(n – 1) nodes. Adding up all the nodes, we have n times the above, plus 1 for the root. Simplifying the math, we have…


1

1

1

1

1

11

11

1

1

1

11)1(

n

n

n

n

n

nn

n

nn

n

nn

h

h

hh

QED!

Induction Is Induction

• Weak, strong, and structural induction get taught as if they were separate concepts, but they are all the same. We will see:– Weak and strong induction are the same.

– Weak is a special case of structural induction.

Weak Induction = Strong Induction

• We are trying to prove some property P(n) for all natural numbers n.

• Base cases are the same: Prove P(0) or other small cases.

• In weak induction, we get to assume P(k-1) to prove P(k).

• In strong induction, we get to assume P(0) through P(k-1) to prove P(k).

• So, clearly, every weak induction proof is a strong induction proof. What about the other way around?

Weak Induction = Strong Induction

• We are trying to prove some property P(n) for all natural numbers n.

But we have no limitations on what P(n) is!• Given a strong induction proof for P(n), define Q(n)

to be the property:

Q(n) = “P(k) holds for all 0<=k<n”• The strong induction proof of P(n) becomes a weak

induction proof on Q(n).

Weak Induction Is Structural Induction

• Weak induction proves p(n) for all natural numbers.• Structural induction proves p(n) for all structures n,

which are defined inductively.

What are natural numbers, really?

Peano Postulates/Axioms

• By Giuseppe Peano, in 1889.• This is the formal way mathematicians define

natural numbers.


1. 0 is a natural number.

2. For every natural number n, S(n) is a natural number.

3. For every natural number n, S(n)!=0.

4. For all natural numbers m and n, if S(m)=S(n), then m=n.

5. If K is any set that obeys #1 and #2, then K contains all natural numbers.







Natural numbers are defined inductively!







Nothing else is a natural number. (So, if you do the structuralinduction, you’ve proven your result for all n.)

Weak Induction Is Structural Induction

Weak induction is just structural induction on the natural numbers (as they are formally defined).

Base Case: Prove the property for p(0).

Inductive Case: Any non-base-case number is of the form S(k) for some k. By the inductive hypothesis, p(k) holds. Use this fact to prove p(S(k)).

CS121 Induction Alan J. Hu 1. Induction: The Big Picture We’ll develop your intuition about induction. We’ll see examples of weak, strong, and structural.

Documents

induction proof

inductive case

weak induction

natural numbers n base

qed slide

strong induction

structured podium slide

stack of n disks