CPSC 221: Data Structures Lecture #23 Counting Steve Wolfman (stolen from Alan J. Hu who stole some from me which I stole from Kim Voll) 2011W2.

CPSC 221: Data StructuresLecture #23

Counting

Steve Wolfman(stolen from Alan J. Hu who stole some from me which I

stole from Kim Voll)2011W2

Learning GoalsAfter this unit, you should be able to:• Apply counting principles to determine the number of

arrangements or orderings of discrete objects, with or without repetition, and given various constraints.

• Use appropriate mathematical constructs to express a counting problem (e.g. counting passwords with various restrictions placed on the characters within).

• Identify problems that can be expressed and solved as a combination of smaller sub problems. When necessary, use decision trees to model more complex counting problems.

2

Counting: Motivation

• This is sometimes really useful, e.g.:– How many possible passwords, encryption keys, etc.– How many possible legal inputs, how many possible data

structure configurations, how many paths through a network, etc.

– Crucial step in computing probabilities.• Sometimes fun – like a puzzle or game.

Counting: Warning

• However, once you get beyond basic problems, it can be very subtle, and there are no simple tricks that always work.

Sometimes, you just have to think really, really hard and really really carefully!

• Epp textbook is pretty good. Must read on your own. (3rd ed 6.1-6.5, 4th ed. 9.1-9.6)

• Old slides follow textbook presentation closely.• I am going to try to de-mystify some of the formulas.

Outline

• Case-Splitting, Decision Trees• “Addition Rule” Deconstructed• “Multiplication Rule” Deconstructed• Alan’s “Division Rule”• Permutation and Combination Formulas• Dealing with Overlapping Cases: Inclusion-Exclusion

Counting

• The basic problem is to count how many items are in some set, which might be specified in a non-obvious way.

• E.g., how many different min-heaps are possible with the numbers 1,…,7?

• How many poker hands beat a pair of kings?• Etc.

Basic Technique: Case Splitting

• This is the most fundamental way to break a problem down into simpler parts.– The cases must be disjoint (no overlap).

(We’ll handle overlap later.)– Combined, the cases must cover all possibilities.

• Count each case separately, then add them all up.

• Whenever you are totally lost, always fall back to case-splitting.

Point of interest: disjoint but covering sets form a partition.

Case Splitting Example

• What’s the population of Canada?


• What’s the population of Canada?– What’s the population of BC?– What’s the population of AB?– What’s the population of SK?– What’s the population of MB?– etc.

• Add them all up!

Other case splits are possible

• What’s the population of Canada?– How many men in Canada?– How many women in Canada?


“Addition Rule” Demystified

• From Epp:“Suppose a finite set A equals the union of k distinct

mutually disjoint subsets A1, A2, …, Ak.Then N(A) = N(A1)+N(A2)+…+N(Ak).”

Addition Principle (Sum Rule)If X1, X2, …, Xt are pairwise disjoint sets (i.e., Xi ∩ Xj = ∅ ∀i ∀j s.t. i, j∈{1, 2, …, t }, i ≠ j), then the number of ways to select an element from any of X1 or

X2 or … or Xt is:| X1 | + | X2 | + … + | Xt |

Case Splitting Error: Missing Cases

• What’s the population of Canada?– Liberals– Conservatives– NDPs


• If we miss any cases (e.g., Greens, Rhinoceros Party, Marijuana Party, people without political parties, etc.), we’ll undercount.

Case Splitting Error: Overlap

• What’s the population of Canada?– Francophone Canadians– Anglophone Canadians


• Double counts anyone who speaks both.• (We’ll see how to deal with overlap later.)

Also misses anyone who speaks neither, of course!

Cases can be split into sub-cases

• What’s the population of Canada?– What’s the population of BC?

• What’s the population of Vancouver?• What’s the population of Burnaby?• What’s the population of Richmond?• etc. Add them all up.

– What’s the population of AB?– What’s the population of SK?– What’s the population of MB?– etc.


Decision Trees

• When you split cases into subcases, it’s often nice to draw this as a tree.

• These are informally called “decision trees”, although a proper decision tree has probabilities on branches. We’ll just ignore this. (Epp calls them “possibility trees” which is descriptive, but not standard.)


• How many different min-heaps are possible with the numbers 1,…,7?– How many have 1 at the root?– How many have 2 at the root?– How many have 3 at the root?– How many have 4 at the root?– How many have 5 at the root?– How many have 6 at the root?– How many have 7 at the root?



• How many different min-heaps are possible with the numbers 1,…,7?– How many have 1 at the root? ???– How many have 2 at the root? 0– How many have 3 at the root? 0– How many have 4 at the root? 0– How many have 5 at the root? 0– How many have 6 at the root? 0– How many have 7 at the root? 0


Case Splitting: Splitting into Sub-Cases

• How many different min-heaps are possible with the numbers 1,…,7?– How many have 1 at the root?

• And have 2 at left child?• And have 3 at left child?• And have 4 at left child?• And have 5 at left child?• And have 6 at left child?• And have 7 at left child?

• Add them all up!• People often draw these as trees.



• And have 2 at left child? ???• And have 3 at left child? ???• And have 4 at left child? ???• And have 5 at left child? ???• And have 6 at left child? 0• And have 7 at left child? 0



• And have 2 at left child?– And have 3 at right child?– And have 4 at right child?– And have 5 at right child?

• And have 3 at left child? ???• And have 4 at left child? ???• And have 5 at left child? ???



• And have 2 at left child?– And have 3 at right child?– And have 4 at right child?– And have 5 at right child?

» 6 and 7 must be leaves of right child» Therefore 3 and 4 are leaves of left child» Therefore 4 possibilities in this case.




• And have 2 at left child?– And have 3 at right child?– And have 4 at right child?

» Right subtree leaves are 5, 6:» Right subtree leaves are 5, 7:» Right subtree leaves are 6, 7:

– And have 5 at right child? 4• And have 3 at left child? ???• And have 4 at left child? ???• And have 5 at left child? ???



• And have 2 at left child?– And have 3 at right child?– And have 4 at right child?

» Right subtree leaves are 5, 6: 4» Right subtree leaves are 5, 7: 4» Right subtree leaves are 6, 7: 4

– And have 5 at right child? 4• And have 3 at left child? ???• And have 4 at left child? ???• And have 5 at left child? ???



• And have 2 at left child?– And have 3 at right child?– And have 4 at right child? 12– And have 5 at right child? 4




• And have 2 at left child?– And have 3 at right child?

» Leftmost leaf is 4:» Leftmost leaf is 5:» Leftmost leaf is 6:» Leftmost leaf is 7:

– And have 4 at right child? 12– And have 5 at right child? 4




• And have 2 at left child?– And have 3 at right child?

» Leftmost leaf is 4:» Leftmost leaf is 5:» Leftmost leaf is 6:» Leftmost leaf is 7:

– And have 4 at right child? 12– And have 5 at right child? 4



• How many have 1 at the root, 2 at left child, and 3 at right child:– Leftmost leaf is 4:– Leftmost leaf is 5:– Leftmost leaf is 6:– Leftmost leaf is 7:


• How many have 1 at the root, 2 at left child, and 3 at right child:– Leftmost leaf is 4:

• Next leaf is 5• Next leaf is 6• Next leaf is 7

– Leftmost leaf is 5:– Leftmost leaf is 6:– Leftmost leaf is 7:


• How many have 1 at the root, 2 at left child, and 3 at right child:– Leftmost leaf is 4:


– Leftmost leaf is 5:• Next leaf is 4• Next leaf is 6• Next leaf is 7

– Leftmost leaf is 6:– Leftmost leaf is 7:

Case Splitting: Splitting into Sub-Cases• How many have 1 at the root, 2 at left child, and 3

at right child:– Leftmost leaf is 4:



– Leftmost leaf is 6:• etc.


What do you call itwhen you add upthe same numbera bunch of times?

Multiplication!

E.g.: suppose there are 30 NHL teams, each with 23 players. How many players are there in total?

Multiplication!

If some case split results in a bunch of identical-sized sub-cases, the total is just the number of cases times the size of each sub-case.

Multiplication!

If some case split results in a bunch of identical-sized sub-cases, the total is just the number of cases times the size of each sub-case.

•Note that the cases don’t have to be identical, but just have the same number of things in them.

Multiplication Principle (Product Rule)If an operation consists of t steps and:Step 1 can be performed in n1 ways,Step 2 can be performed in n2 ways,…Step t can be performed in nt ways,then the entire operation can be performed in n1n2…nt different ways.

Example: How many postal codes begin with the letter V and end with the digit 4?__ __ __ __ __ __


at right child:– Leftmost leaf is 4:





4 cases with samenumber of elementsin each one.


at right child:– 4 times:

• The number of cases with leftmost leaf is 4:– Next leaf is 5– Next leaf is 6– Next leaf is 7




• The number of cases with leftmost leaf is 4:– Next leaf is 5– Next leaf is 6– Next leaf is 7




• 3 times:– The number of cases with next leaf 5




• 3 times:– 2 times

» 1

Permutations and Factorial

The number of ways to arrange n distinct items in a row is n!

– This comes straight from the product rule(which comes from case splitting)

These are called permutations of the n items.

Permutations P(n,r)

• What if we only want permutations of r elements out of the n objects?– E.g., If there are 20 people here, and I have 5 different

prizes to give away to people in this class, how many possible winning situations are there?

Permutations P(n,r)



– Product Rule: 20^5 if you can win repeatedly– Product Rule: 20*19*18*17*16 if you can win at most

one prize.

Permutations P(n,r)



– Product Rule: 20^5 if you can win repeatedly– Product Rule: 20*19*18*17*16 if you can win at most

one prize.

This is P(20,5), the number of permutations of 5 objectsout of 20.

Permutations P(n,r)

)!(!

)1()2)(1(),(

rnn

n

rnnnnrnPr

Note the underline below the r.This notation is due to Knuth and is probably the most standard notation for this, but nothing is really standard.

Permutations P(n,r)

)!(!

)1()2)(1(),(

rnn

n

rnnnnrnPr

You can understand this by canceling out terms. But there’s an intuitive way to understand this, too.

P(n,r)=n!/(n-r)! Intuition

• How many ways to line up 20 people?• If I only give prizes to first 5, the order of

remaining 15 doesn’t matter.• How many ways can I order 15 people?


• How many ways to line up 20 people?• If I only give prizes to first 5, the order of

remaining 15 doesn’t matter.• How many ways can I order 15 people?

• Therefore, the 20! is counting each winning possibility of the top 5 an extra 15! times.

• So, the answer is 20!/15!


• How many ways to line up n people?• If I only give prizes to first r, the order of

remaining (n-r) doesn’t matter.• How many ways can I order (n-r) people?

• Therefore, the n! is counting each winning possibility of the top r and extra (n-r)! times.

• So, the answer is n!/(n-r)!

Alan’s “Division Rule”

• If the things you are trying to count are really identical-sized groupings of things from a larger set, you can divide the size of the larger set by the size of the groupings.

• E.g., if I want to know how many cars drove over a sensor strip, I can divide the number of sensor inputs by 2. Why?

Steve also uses this but didn’t think to name it after himself. Drat!It’s often handy to think about this type of “overcounting”.

Division Rule and Combinations

• Suppose the 5 prizes are identical, so I don’t care to distinguish who gets which prize, just who the winners are.

• (This is usually phrased as choosing a committee of 5 out of the class of 20.)



• I already know how many ways to get 5 winners with distinct prizes: P(n,r)

• But if the prizes are the same, then any re-arranging of those 5 should only be counted once. So, I’m grouping together cases when the same 5 people get different prizes. How many cases are there?



• I already know how many ways to get 5 winners with distinct prizes: P(20,5)

• But if the prizes are the same, then any re-arranging of those 5 should only be counted once. So, I’m grouping together cases when the same 5 people get different prizes. How many cases are there? 5!

• Therefore, answer is P(20,5)/5!


• Suppose the r prizes are identical, so I don’t care to distinguish who gets which prize, just who the winners are.

• I already know how many ways to get r winners with distinct prizes: P(n,r)

• But if the prizes are the same, then any re-arranging of those r should only be counted once. So, I’m grouping together cases when the same r people get different prizes. How many cases are there? r!

• Therefore, answer is P(n,r)/r!

Combinations C(n,r)

• These are called “combinations” – the number of ways of choosing r items out of n distinct items, where the order of those r doesn’t matter.

• Pronounced “n choose r”.• AKA “binomial coefficients”

)!(!!),(rnr

nrn

rnC

Binomial Coefficients?

• If I compute (x+1)^n, what’s the coefficient of x^r• (x+1)(x+1)(x+1)…(x+1)

• It’s exactly C(n,r)

Dealing with Overlapping Cases

• Suppose we try to compute population of the Americas by adding up the population of “North America” (defined by NAFTA as US, Canada, and Mexico) plus “Latin America” defined as countries that are predominantly Spanish- or Portuguese-speaking.

• What’s wrong? How do we fix it?


• Suppose we try to compute population of the Americas by adding up the population of “North America” (defined by NAFTA as US, Canada, and Mexico) plus “Latin America” defined as countries that are predominantly Spanish- or Portuguese-speaking.

• What’s wrong? How do we fix it?• Subtract the population of Mexico!


• This goes by the fancy name “inclusion-exclusion principle”, but it’s just basic sets:

BABABA

Old Counting Slides

These are for practice and your reference.

(BTW, on my computer, many of the set symbols aren’t showing up properly. Most of them are the

set “is a member of” symbol.)

Today’s Outline

• Multiplication Principle• Addition Principle• Inclusion/Exclusion Principle• Combinations and Permutations

– r-permutations, no repetition– r-combinations, no repetition– r-permutation, with repetition– r-combinations, with repetition– variants

61

In CS, we often encounter situations where we want to count the number of possible outcomes of an event. For example, we may wish to determine: the number of possible paths to follow in a directed network (graph), the number of possible 5-8 character passwords, etc.

Suppose that the customers of a bank are asked to use 3-digit PINs to protect their accounts when using the bank’s ATM machines. If the other constraints are: no 2 digits can be the same, and the only allowable digits are 1, 2, 3, & 4, how many PINs are possible?(a) Describe the problem in mathematical notation.

(b) Model the problem using a tree diagram.

Multiplication Principle (Product Rule)If an operation consists of t steps and:Step 1 can be performed in n1 ways,Step 2 can be performed in n2 ways,…Step t can be performed in nt ways,then the entire operation can be performed in n1n2…nt different ways.

Example: How many postal codes begin with the letter V and end with the digit 4?__ __ __ __ __ __

Example:Use the multiplication principle to prove that the number of subsets of a set containing n elements is 2n.

Example: Suppose a programming language requires you to define variable names (identifiers) using exactly 3 different upper case characters. How many different identifiers contain either an A or a B, but not both?

Example: Let X be a set with n elements. How many ordered pairs (A, B) satisfy all of these constraints: A ⊆ X, B ⊆ X, and A ∩ B = ∅?

Today’s Outline



69

Addition Principle (Sum Rule)If X1, X2, …, Xt are pairwise disjoint sets (i.e., Xi ∩ Xj = ∅ ∀i ∀j s.t. i, j∈{1, 2, …, t }, i ≠ j), then the number of ways to select an element from any of X1 or

X2 or … or Xt is:| X1 | + | X2 | + … + | Xt |

Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters. How many different passwords are possible?

Back-of-the-envelope calculation. For the previous example, roughly how long would it take for a program to perform a brute-force attack to discover a user’s password, if we assume (guess?) that a million passwords can be tested each second?

Best case?Average case?Worst case?Side thought: Common passwords (when “128-bit” security is no better than 10-bit or 20-bit security).

Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters, but this time, at least 1 of the chars. has to be a number.How many possible combinations are there this time?

Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters, but this time, at least 1 of the chars. has to be a number.How many possible combinations are there if we know users are “minimally complex” in their password choices?

Example: Suppose a user is asked to create an alphanumeric password consisting of at least 6 characters, but no more than 8 characters, but this time, at least 1 of the chars. has to be a number.Now, will the passwords be more secure or less secure?

Today’s Outline



76

Inclusion-Exclusion PrincipleIf an object can be found in 2 or more sets at the same time, then we cannot use the addition principle. Why not?

If A and B are sets, then the total number of elements in either A or B is given by:

| A ∪ B | = | A | + | B | – | A ∩ B |Example using a Venn Diagram:

Example: How many 8-bit strings either start with “1” or end with “00”?

Today’s Outline



79

Permutations & CombinationsHow many different outcomes are there if you choose r balls from a barrel containing n different balls?

Example: r = 3Barrel = {A,B,C,D}

Order mattersABC ≠ CAB

Order doesn’tABC = CAB

Repetition not OKe.g., AAB is not counted

ABC, ABD, ACB, ACD, ADB, ADC, BAC, BAD, BCA, BCD, BDA, BDC, CAB, CAD, CBA, CBD, CDA, CDB, DAB, DAC, DBA, DBC, DCA, DCBr-permutations

ABC, ABD, ACD, BCD

r-combinations

Repetition OKe.g., AAB is counted

AAA, AAB, AAC, AAD, ABA, ABB, ABC, ABD, …

r-permutations w/ rep.

AAA, AAB, AAC, AAD, ABB, ABC, ABD, ACC, ACD, ADD, BBB, BBC, BBD, BCC, BCD, BDD, CCC, CCD, CDD, DDDr-combinations w/ rep.

(1) r-Permutations (Order matters, and repetition is not allowed.)

An r-permutation of n distinct elements x1, x2, …, xn is an ordering of an r-element subset of {x1, x2, …, xn}. The number of r-permutations is:

P(n, r) = n(r) =

Derivation:

Example 1: In how many ways can 5 electoral candidates finish in an election, assuming no ties?

Example 2: In how many ways can 7 girls and 3 boys line up, if the boys must stand next to each other?

Example 3: Suppose an operating system has a queue of 3 low priority and 5 high priority processes ready to run. In how many ways could these processes be ordered for execution if 2 low priority processes are not allowed to be executed back to back?

Today’s Outline



85

(2) r-Combinations (Order doesn’t matter, and repetition is not allowed.)An r-combination of n distinct elements x1, x2, …, xn is an r-element subset of {x1, x2, …, xn}. The number of r-combinations is:

C(n, r) = ( ) =

Example 4: A donut shop has 10 kinds of donuts. In how many ways can 6 distinct kinds of donuts be selected?

Example 5: Show how to derive a relationship between r-permutations and r-combinations.

Today’s Outline



87

(3) r-Permutations with Repetition (Generalized r-Permutations)Here, order matters, and repetition is allowed.

Suppose we have a set of n distinct elements x1, x2, …, xn and we select a sequence of r elements, allowing repeats. How many different sequences are possible?

Today’s Outline



89

(4) r-Combinations with Repetition (Generalized r-Combinations)Here, order doesn’t matter, and repetition is allowed.

Suppose we have a set of n distinct elements x1, x2, …, xn. The number of unordered r-element selections from this set, with repetition allowed, is:

Example 8: If a donut shop sells 3 kinds of donuts: plain, glazed, and jelly, then how many possible selections of 7 donuts are there?

Summary of Formulas

Order Matters & Repetition is not Allowed

Order does not Matter & Repetition is not Allowed

Order Matters &Repetition is Allowed

Order does not Matter & Repetition is Allowed

nr

n(r)

n-1

n( )r

r+n-1( )

Today’s Outline



93

Permutations of Indistinguishable ObjectsMultinomial Theorem: The number of different permutations of n objects where there are:n1 indistinguishable type 1 objectsn2 indistinguishable type 2 objects…nk indistinguishable type k objectsand n1 + n2 + … + nk = n is: Proof:

Example 9: How many strings can be made by reordering the letters PEPPER?

Theorem: The number of ways to place n distinguishable objects into r distinguishable boxes so that ni objects are placed into box i (where i = 1, 2, …, r and n1 + n2 + … + nr = n) is:

Example 10: How many ways are there to distribute hands of 5 cards to each of 4 players from a deck of 52 cards?

Learning GoalsAfter this unit, you should be able to:• Apply counting principles to determine the number of

arrangements or orderings of discrete objects, with or without repetition, and given various constraints.

• Use appropriate mathematical constructs to express a counting problem (e.g. counting passwords with various restrictions placed on the characters within).

• Identify problems that can be expressed and solved as a combination of smaller sub problems. When necessary, use decision trees to model more complex counting problems.

98

Coming Up

• Final Exam! Yay!

99

CPSC 221: Data Structures Lecture #23 Counting Steve Wolfman (stolen from Alan J. Hu who stole some from me which I stole from Kim Voll) 2011W2.

Documents