Stats: Counting Techniques
Fundamental Theorems
ArithmeticEvery integer greater than one is either prime or can
be expressed as an unique product of prime numbers
Algebra
Every polynomial function on one variable of degree n > 0 has
at least one real or complex zero.
Linear Programming
If there is a solution to a linear programming problem, then it
will occur at a corner point or on a boundary between two or more
corner points
Fundamental Counting Principle
In a sequence of events, the total possible number of ways all
events can performed is the product of the possible number of ways
each individual event can be performed.
The Bluman text calls this multiplication principle 2.
Factorials
If n is a positive integer, then
n! = n (n-1) (n-2) ... (3)(2)(1)
n! = n (n-1)!
A special case is 0!
0! = 1
Permutations
A permutation is an arrangement of objects without repetition
where order is important.
Permutations using all the objects
A permutation of n objects, arranged into one group of size n,
without repetition, and order being important is:
nPn = P(n,n) = n!Example: Find all permutations of the letters
"ABC"
ABC ACB BAC BCA CAB CBA
Permutations of some of the objects
A permutation of n objects, arranged in groups of size r,
without repetition, and order being important is:
nPr = P(n,r) = n! / (n-r)!Example: Find all two-letter
permutations of the letters "ABC"
AB AC BA BC CA CB
Shortcut formula for finding a permutation
Assuming that you start a n and count down to 1 in your
factorials ...
P(n,r) = first r factors of n factorial
Distinguishable Permutations
Sometimes letters are repeated and all of the permutations
aren't distinguishable from each other.
Example: Find all permutations of the letters "BOB"
To help you distinguish, I'll write the second "B" as "b"
BOb BbO OBb ObB bBO bOB
If you just write "B" as "B", however ...
BOB BBO OBB OBB BBO BBO
There are really only three distinguishable permutations
here.
BOB BBO OBB
If a word has N letters, k of which are unique, and you let n
(n1, n2, n3, ..., nk) be the frequency of each of the k letters,
then the total number of distinguishable permutations is given
by:
Consider the word "STATISTICS":
Here are the frequency of each letter: S=3, T=3, A=1, I=2, C=1,
there are 10 letters total
10! 10*9*8*7*6*5*4*3*2*1
Permutations = -------------- = -------------------- = 50400
3! 3! 1! 2! 1! 6 * 6 * 1 * 2 * 1
You can find distinguishable permutations using the TI-82.
Combinations
A combination is an arrangement of objects without repetition
where order is not important.
Note: The difference between a permutation and a combination is
not whether there is repetition or not -- there must not be
repetition with either, and if there is repetition, you can not use
the formulas for permutations or combinations. The only difference
in the definition of a permutation and a combination is whether
order is important.
A combination of n objects, arranged in groups of size r,
without repetition, and order being important is:
nCr = C(n,r) = n! / ( (n-r)! * r! )Another way to write a
combination of n things, r at a time is using the binomial
notation: Example: Find all two-letter combinations of the letters
"ABC"
AB = BA AC = CA BC = CB
There are only three two-letter combinations.
Shortcut formula for finding a combination
Assuming that you start a n and count down to 1 in your
factorials ...
C(n,r) = first r factors of n factorial divided by the last r
factors of n factorial
Pascal's Triangle
Combinations are used in the binomial expansion theorem from
algebra to give the coefficients of the expansion (a+b)^n. They
also form a pattern known as Pascal's Triangle.
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
1 7 21 35 35 21 7 1
Each element in the table is the sum of the two elements
directly above it. Each element is also a combination. The n value
is the number of the row (start counting at zero) and the r value
is the element in the row (start counting at zero). That would make
the 20 in the next to last row C(6,3) -- it's in the row #6 (7th
row) and position #3 (4th element).
Symmetry
Pascal's Triangle illustrates the symmetric nature of a
combination. C(n,r) = C(n,n-r)
Example: C(10,4) = C(10,6) or C(100,99) = C(100,1)
Shortcut formula for finding a combination
Since combinations are symmetric, if n-r is smaller than r, then
switch the combination to its alternative form and then use the
shortcut given above.
C(n,r) = first r factors of n factorial divided by the last r
factors of n factorial
TI-82
You can use the TI-82 graphing calculator to find factorials,
permutations, and combinations.
Tree Diagrams
Tree diagrams are a graphical way of listing all the possible
outcomes. The outcomes are listed in an orderly fashion, so listing
all of the possible outcomes is easier than just trying to make
sure that you have them all listed. It is called a tree diagram
because of the way it looks.
The first event appears on the left, and then each sequential
event is represented as branches off of the first event.
The tree diagram to the right would show the possible ways of
flipping two coins. The final outcomes are obtained by following
each branch to its conclusion: They are from top to bottom:
HH HT TH TT
Table of Contents3. Probability -- Counting Techniques Some
probability problems can be attacked by specifying a sample space
in which each simple event has probability (i.e. is "equally
likely"). Thus, if a compound event consists of simple events, then
. To use this approach we need to be able to count the number of
events in and in , and this can be tricky. We review here some
basic ways to count outcomes from "experiments". These approaches
should be familiar from high school mathematics.
General Counting Rules
There are two basic rules for counting which can deal with most
problems. We phrase the rules in terms of ``jobs" which are to be
done.
1. The Addition Rule:
Suppose we can do job 1 in ways and job 2 in ways. Then we can
do either job 1 or job 2, but not both, in ways.
For example, suppose a class has 30 men and 25 women. There are
ways the prof. can pick one student to answer a question.
1. The Multiplication Rule:
Suppose we can do job 1 in ways and an unrelated job 2 in ways.
Then we can do both job 1 and job 2 in ways.
For example, to ride a bike, you must have the chain on both a
front sprocket and a rear sprocket. For a 21 speed bike there are 3
ways to select the front sprocket and 7 ways to select the rear
sprocket.
This linkage of OR with addition and AND with multiplication
will occur throughout the course, so it is helpful to make this
association in your mind. The only problem with applying it is that
questions do not always have an AND or an OR in them. You often
have to play around with re-wording the question for yourself to
discover implied AND's or OR's.
Example: Suppose we pick 2 numbers at random from digits 1, 2,
3, 4, 5 with replacement. (Note: "with replacement" means that
after the first number is picked it is "replaced" in the set of
numbers, so it could be picked again as the second number.) Let us
find the probability that one number is even. This can be reworded
as: "The first number is even AND the second is odd, OR, the first
is odd AND the second is even." We can then use the addition and
multiplication rules to calculate that there are ways for this
event to occur. Since the first number can be chosen in 5 ways AND
the second in 5 ways, contains points. The phrase "at random" in
the first sentence means the numbers are equally likely to be
picked. When objects are selected and replaced after each draw, the
addition and multiplication rules are generally sufficient to find
probabilities. When objects are drawn without being replaced, some
special rules may simplify the solution.
Problems:
1. A course has 4 sections with no limit on how many can enrol
in each section. 3 students each randomly pick a section. Find the
probability:
1. they all end up in the same section
2. they all end up in different sections
3. nobody picks section 1.
2. Repeat (a) in the case when there are sections and students
.
2. Canadian postal codes consist of 3 letters alternated with 3
digits, starting with a letter (e.g. N2L 3G1). For a randomly
constructed postal code, what is the probability:
1. all 3 letters are the same?
2. the digits are all even or all odd? Treat 0 as being neither
even nor odd.
3. Suppose a password has to contain between six and eight
digits, with each digit either a letter or a number from 1 to 9.
There must be at least one number present.
1. What is the total number of possible passwords?
2. If you started to try passwords in random order, what is the
probability you would find the correct password for a given
situation within the first 1,000 passwords you tried?
Permutation Rules
Suppose that distinct objects are to be ``drawn" sequentially,
or ordered from left to right in a row.(Order matters; objects are
drawn without replacement)
4. The number of ways to arrange distinct objects in a row
is
Explanation: We can fill the first position in ways. Since this
object can't be used again, there are only ways to fill the second
position. So we keep having 1 fewer object available after each
position is filled.
Statistics is important, and many games are interesting largely
because of the extraordinary rate of growth of the function in For
example
012345678910
1126241207205040403203628803628800
which means that for many problems involving sampling from a
deck of cards or a reasonably large population, counting the number
of cases is virtually impossible. There is an approximation to
which is often used for large called Stirling's formula which says
that is asymptotic to Here, two sequences and are called
asymptotically equal if as (intuitively, the percentage error in
using Stirling's approximation goes to zero as For example the
error in Stirling's approximation is less than 1% if 5. The number
of ways to arrange objects selected from distinct objects is using
the same reasoning as in #1, and noting that for the selection,
objects have already been used. Hence there areways to make the
selection. We use the symbol to represent and describe this symbol
as " taken to terms". E.g. .
While only has a physical interpretation when and are positive
integers with , it still has a mathematical meaning when is not a
positive integer, as long as is a non-negative integer. For example
We will occasionally encounter such cases in this course but
generally and will be non-negative integers with . In this case, we
can re-write in terms of factorials.
Note that
The idea in using counting methods is to break the experiment
into pieces or ``jobs'' so that counting rules can be applied.
There is usually more than one way to do this.
Example: We form a 4 digit number by randomly selecting and
arranging 4 digits from 1, 2, 3,...7 without replacement. Find the
probability the number formed is (a) even (b) over 3000 (c) an even
number over 3000.
Solution: Let be the set of all possible 4 digit numbers using
digits 1, 2, ..., 7 without repetitions.
Then has points. (We could calculate this but it will be easier
to leave it in this form for now and do some cancelling later.)
6. For a number to be even, the last digit must be even. We can
fill this last position with a 2, 4, or 6; i.e. in 3 ways. The
first 3 positions can be filled by choosing and arranging 3 of the
6 digits not used in the final position. i.e. in ways. Then there
are ways to fill the final position AND the first 3 positions to
produce an even number.
Another way to do this problem is to note that the four digit
number is even if and only if (iff) the last digit is even. The
last digit is equally likely to be any one of the numbers 1, ..., 7
so 7. To get a number over 3000, we require the first digit to be
3, 4, 5, 6, or 7; i.e. it can be chosen in 5 ways. The remaining 3
positions can be filled in ways.
Another way to do this problem is to note that the four digit
number is over 3000 iff the first digit is one of 3, 4, 5, 6 or 7.
Since each of 1, ..., 7 is equally likely to be the first digit, we
get (number 3000) = .
Note that in both (a) and (b) we dealt with positions which had
restrictions first, before considering positions with no
restrictions. This is generally the best approach to follow in
applying counting techniques.
8. This part has restrictions on both the first and last
positions. To illustrate the complication this introduces, suppose
we decide to fill positions in the order 1 then 4 then the middle
two. We can fill position 1 in 5 ways. How many ways can we then
fill position 4? The answer is either 2 or 3 ways, depending on
whether the first position was filled with an even or odd digit.
Whenever we encounter a situation such as this, we have to break
the solution into separate cases. One case is where the first digit
is even. The positions can be filled in 2 ways for the first (i.e.
with a 4 or 6), 2 ways for the last, and then ways to arrange 2 of
the remaining 5 digits in the middle positions. This first case
then occurs in ways. The second case has an odd digit in position
one. There are 3 ways to fill position one (3, 5, or 7), 3 ways to
fill position four (2, 4, or 6), and ways to fill the remaining
positions. Case 2 then occurs in ways. We need case 1 OR case
2.
Another way to do this is to realize that we need only to
consider the first and last digit, and to find (first digit is 3
and last digit is even). There are different choices for (first
digit, last digit) and it is easy to see there are 13 choices for
which first digit , last digit is even ( minus the impossible
outcomes (4, 4) and (6, 6)). Thus the desired probability is .
Exercise: Try to solve part (c) by filling positions in the
order 4, 1, middle. You should get the same answer.
Exercise: Can you spot the flaw in the following? There are ways
to get an even number (part (a))There are ways to get a number 3000
(part (b)) By the multiplication rule there are ways to get a
number which is even and 3000. (Read the conditions in the
multiplication rule carefully, if you believe this solution.)
Here is another useful rule.
9. The number of distinct arrangements of objects when are alike
of one type, alike of a type, ..., alike of a type is For example:
We can arrange in ways. These are However, as soon as we remove the
subscripts on the , the second row is the same as the first row.
I.e., we have only 3 distinct arrangements since each arrangement
appears twice as the and are interchanged. In general, there would
be arrangements if all objects were distinct. However each
arrangement would appear times as the type was interchanged with
itself, times as the type was interchanged with itself, etc. Hence
only of the arrangements are distinct.
Example: 5 men and 3 women sit together in a row. Find the
probability that
10. the same gender is at each end
11. the women all sit together.
What are you assuming in your solution? Is it likely to be valid
in real life?
Solution: If we treat the people as being 8 objects -- 5 and 3,
our sample space will have points.
12. To get the same gender at each end we need either OR The
number of distinct arrangements with a man at each end is , since
we are arranging 's and 's in the middle 6 positions. The number
with a woman at each end is . Thus assuming each arrangement is
equally likely.
13. Treating as a single unit, we are arranging 6 objects -- 5's
and 1 . There are arrangements. Thus, Our solution is based on the
assumption that all points in are equally probable. This would mean
the people sit in a purely random order. In real life this isn't
likely, for example, since friends are more likely to sit
together.
Problems:
14. Digits 1, 2, 3, ..., 7 are arranged at random to form a 7
digit number. Find the probability that
1. the even digits occur together, in any order
2. the digits at the 2 ends are both even or both odd.
15. The letters of the word EXCELLENT are arranged in a random
order. Find the probability that
1. the same letter occurs at each end.
2. and occur together, in any order.
3. the letters occur in alphabetical order.
Combinations
This deals with cases where order does not matter; objects are
drawn without replacement.
The number of ways to choose objects from is denoted by (called
" choose "). For and both non-negative integers with , Proof: From
result 2 earlier, the number of ways to choose objects from and
arrange them from left to right is . Any choice of objects can be
arranged in ways, so we must have
(Number of way to choose objects from )
This gives as the number of ways to choose objects.
Note that loses its physical meaning when is not a non-negative
integer . However it is defined mathematically, provided is a
non-negative integer, by .
Example: In the Lotto 6/49 lottery, six numbers are drawn at
random, without replacement, from the numbers 1 to 49. Find the
probability that
16. the numbers drawn are 1, 2, 3, 4, 5, 6 (in some order)
17. no even number is drawn.
Solution:
18. Let the sample space consist of all combinations of 6
numbers from 1, ..., 49; there are of them. Since 1, 2, 3, 4, 5, 6
consist of one of these 6-tuples, , which equals about 1 in 13.9
million.
19. There are 25 odd and 24 even numbers, so there are choices
in which all the numbers are odd.
(no even number)=(all odd numbers)
=
20. which is approximately equal to 0.0127.
Example: Find the probability a bridge hand (13 cards picked at
random from a standard deck) has
21. 3 aces
22. at least 1 ace
23. 6 spades, 4 hearts, 2 diamonds, 1 club
24. a 6-4-2-1 split between the 4 suits
25. a 5-4-2-2 split.
Solution: Since order of selection does not matter, we take to
have points.
26. We can choose 3 aces in ways. We also have to choose 10
other cards from the 48 non-aces. This can be done in ways. Hence
27. Solution 1: At least 1 ace means 1 ace or 2 aces or 3 aces or 4
aces. Calculate each part as in (a) and use the addition rule to
get
28. Solution 2: If we subtract all cases with aces from the
points in we are left with all points having at least 1 ace. This
gives (The term can be omitted since , but was included here to
show that we were choosing of the 4 aces.)
29. Solution 3: This solution is incorrect, but illustrates a
common error. Choose 1 of the 4 aces then any 12 of the remaining
51 cards. This guarantees we have at least 1 ace, so The flaw in
this solution is that it counts some points more than once by
partially keeping track of order. For example, we could get the ace
of spades on the first choice and happen to get the ace of clubs in
the last 12 draws. We also could get the ace of clubs on the first
draw and then get the ace of spades in the last 12 draws. Though in
both cases we have the same outcome, they would be counted as 2
different outcomes.
(c)
30. Choose the 6 spades in ways and the hearts in ways and the
diamonds in ways and the clubs in ways.
31. The split in (c) is only 1 of several possible 6-4-2-1
splits. In fact, filling in the numbers 6, 4, 2 and 1 in the spaces
above each suit defines a 6-4-2-1 split. There are 4! ways to do
this, and then ways to pick the cards from these suits.
32. This is the same as (d) except the numbers 5-4-2-2 are not
all different. There are different arrangements of 5-4-2-2 in the
spaces .
INCLUDEPICTURE
"http://sas.uwaterloo.ca/%7Edlmcleis/s230/noteschap3.htm" \*
MERGEFORMATINET Problems:
33. A factory parking lot has 160 cars in it, of which 35 have
faulty emission controls. An air quality inspector does spot checks
on 8 cars on the lot.
1. Give an expression for the probability that at least 3 of
these 8 cars will have faulty emission controls.
2. What assumption does your answer to (a) require? How likely
is it that this assumption holds if the inspector hopes to catch as
many cars with faulty controls as possible?
34. In a race, the 15 runners are randomly assigned the numbers
. Find the probability that
1. 4 of the first 6 finishers have single digit numbers.
2. the fifth runner to finish is the 3rd finisher with a single
digit number.
3. number 13 is the highest number among the first 7
finishers.0.2in
Problems on Chapter 3
35. Six digits from 2, 3, 4, ..., 8 are chosen and arranged in a
row without replacement. Find the probability that
1. the number is divisible by 2
2. the digits 2 and 3 appear consecutively in the proper order
(i.e. 23)
3. digits 2 and 3 appear in the proper order but not
consecutively.
36. Suppose passengers get on an elevator at the basement floor.
There are floors above (numbered 1, 2, 3, ..., ) where passengers
may get off.
1. Find the probability
1. no passenger gets off at floor 1
2. passengers all get off at different floors .
2. What assumption(s) underlies your answer to (a)? Comment
briefly on how likely it is that the assumption(s) is valid.
37. There are 6 stops left on a subway line and 4 passengers on
a train. Assume they are each equally likely to get off at any
stop. What is the probability
1. they all get off at different stops?
2. 2 get off at one stop and 2 at another stop?
38. Give an expression for the probability a bridge hand of 13
cards contains 2 aces, 4 face cards (Jack, Queen or King) and 7
others. You might investigate the various permutations and
combinations relating to card hands using the Java applet at 39.
The letters of the word STATISTICS are arranged in a random order.
Find the probability
1. they spell statistics
2. the same letter occurs at each end.
40. Three digits are chosen in order from 0, 1, 2, ..., 9. Find
the probability the digits are drawn in increasing order; (i.e.,
the first the second the third) if
1. draws are made without replacement
2. draws are made with replacement.
41. The Birthday Problem. Note_1 Suppose there are persons in a
room. Ignoring February 29 and assuming that every person is
equally likely to have been born on any of the 365 other days in a
year, find the probability that no two persons in the room have the
same birthday. Find the numerical value of this probability for .
There is a graphic Java applet for illustrating the frequency of
common birthdays at
http://www-stat.stanford.edu/%7Esusan/surprise/Birthday.html
42. You have identical looking keys on a chain, and one opens
your office door. If you try the keys in random order then
what is the probability the th key opens the door?
what is the probability one of the first two keys opens the door
(assume )?
Determine numerical values for the answer in part (b) for the
cases .
From a set of consecutively numbered tickets, three are selected
at random without replacement. Find the probability that the
numbers of the tickets form an arithmetic progression. [The order
in which the tickets are selected does not matter.]
The 10,000 tickets for a lottery are numbered 0000 to 9999. A
four-digit winning number is drawn and a prize is paid on each
ticket whose four-digit number is any arrangement of the number
drawn. For instance, if winning number 0011 is drawn, prizes are
paid on tickets numbered 0011, 0101, 0110, 1001, 1010, and 1100. A
ticket costs $1 and each prize is $500.
What is the probability of winning a prize (i) with ticket
number 7337? (ii) with ticket number 7235? What advice would you
give to someone buying a ticket for this lottery?
Assuming that all tickets are sold, what is the probability that
the operator will lose money on the lottery?
There are 25 deer in a certain forested area, and 6 have been
caught temporarily and tagged. Some time later, 5 deer are caught.
Find the probability that 2 of them are tagged. (What assumption
did you make to do this?)
Suppose that the total number of deer in the area was unknown to
you. Describe how you could estimate the number of deer based on
the information that 6 deer were tagged earlier, and later when 5
deer are caught, 2 are found to be tagged. What estimate do you
get?
Lotto 6/49. In Lotto 6/49 you purchase a lottery ticket with 6
different numbers, selected from the set . In the draw, six
(different) numbers are randomly selected. Find the probability
that
Your ticket has the 6 numbers which are drawn. (This means you
win the main Jackpot.)
Your ticket matches exactly 5 of the 6 numbers drawn.
Your ticket matches exactly 4 of the 6 numbers drawn.
Your ticket matches exactly 3 of the 6 numbers drawn.
(Texas Hold-em) Texas Hold-em is a poker game in which players
are each dealt two cards face down (called your hole or pocket
cards), from a standard deck of 52 cards, followed by a round of
betting, and then five cards are dealt face up on the table with
various breaks to permit players to bet the farm. These are
communal cards that anyone can use in combination with their two
pocket cards to form a poker hand. Players can use any five of the
face-up cards and their two cards to form a five card poker hand.
Probability calculations for this game are not only required at the
end, but also at intermediate steps and are quite complicated so
that usually simulation is used to determine the odds that you will
win given your current information, so consider a simple example.
Suppose we were dealt 2 Jacks in the first round.
0. What is the probability that the next three cards (face up)
include at least one Jack?
1. Given that there was no Jack among these next three cards,
what is the probability that there is at least one among the last
two cards dealt face-up?
2. What is the probability that the 5 face-up cards show two
Jacks, given that I have two in my pocket cards?
MATHEMATICAL EXPECTATIONS
"Mathematical Expectation" is one of those few topics that is
rarely discussed in details in any curriculum, but is nevertheless
very important. This tutorial attempts to throw some light on this
topic by discussing few related mathematical and programming
problems.
Theory
Mathematical Expectation is an important concept in Probability
Theory. Mathematically, for a discrete variable X with probability
function P(X), the expected value E[X] is given by xiP(xi) the
summation runs over all the distinct values xi that the variable
can take. For example, for a dice-throw experiment, the set of
discrete outcomes is { 1,2,3,4,5,6} and each of this outcome has
the same probability 1/6. Hence, the expected value of this
experiment will be 1/6*(1+2+3+4+5+6) = 21/6 = 3.5. For a continuous
variable X with probability density function P(x) , the expected
value E[X] is given by xP(x)dx.
It is important to understand that "expected value" is not same
as "most probable value" - rather, it need not even be one of the
probable values. For example, in a dice-throw experiment, the
expected value, viz 3.5 is not one of the possible outcomes at
all.
The rule of "linearity of of the expectation" says that E[x1+x2]
= E[x1] + E[x2].
Problems
1. What is the expected number of coin flips for getting a
head?Ans: Let the expected number of coin flips be x. Then we can
write an equation for it -a. If the first flip is the head, then we
are done. The probability of this event is 1/2 and the number of
coin flips for this event is 1. b. If the first flip is the tails,
then we have wasted one flip. Since consecutive flips are
independent events, the solution in this case can be recursively
framed in terms of x - The probability of this event is 1/2 and the
expected number of coins flips now onwards is x. But we have
already wasted one flip, so the total number of flips is x+1.
The expected value x is the sum of the expected values of these
two cases. Using the rule of linerairty of the expectation and the
definition of Expected value, we get
x = (1/2)(1) + (1/2) (1+x)Solving, we get x = 2.
Thus the expected number of coin flips for getting a head is
2.
Q2. What is the expected number of coin flips for getting two
consecutive heads?
Let the expected number of coin flips be x. The case analysis
goes as follows:a. If the first flip is a tails, then we have
wasted one flip. The probability of this event is 1/2 and the total
number of flips required is x+1b. If the first flip is a heads and
second flip is a tails, then we have wasted two flips. The
probability of this event is 1/4 and the total number of flips
required is x+2c. If the first flip is a heads and second flip is
also heads, then we are done. The probability of this event is 1/4
and the total number of flips required is 2.
Adding, the equation that we get is -x = (1/2)(x+1) + (1/4)(x+2)
+ (1/4)2
Solving, we get x = 6.
Thus, the expected number of coin flips for getting two
consecutive heads is 6.
Q3. (Generalization) What is the expected number of coin flips
for getting N consecutive heads, given N?
Let the exepected number of coin flips be x. Based on previous
exercises, we can wind up the whole case analysis in two basic
parts
a) If we get 1st, 2nd, 3rd,...,n'th tail as the first tail in
the experiment, then we have to start all over again. b) Else we
are done.
For the 1st flip as tail, the part of the equation is (1/2)(x+1)
For the 2nd flip as tail, the part of the equation is
(1/4)(x+2)...For the k'th flip as tail, the part of the equation is
(1/(2k))(x+k)...For the N'th flip as tail, the part of the equation
is (1/(2N))(x+N)The part of equation corrresponding to case (b) is
(1/(2N))(N)
Adding,
x = (1/2)(x+1) + (1/4)(x+2) + ... + (1/(2^k))(x+k) + .. +
(1/(2^N))(x+N) + (1/(2^N))(N)
Solving this equation is left as an exercise to the reader. The
entire equation can be very easily reduced to the following
form:
x = 2N+1-2
Thus, the expected number of coin flips for getting N
consecutive heads is (2N+1 - 2).
Q4. Candidates are appearing for interview one after other.
Probability of each candidate getting selected is 0.16. What is the
expected number of candidates that you will need to interview to
make sure that you select somebody?
This is very similar to Q1, the only difference is that in this
case the coin is biased. (The probability of heads is 0.16 and we
are asked to find number of coin flips for getting a heads).Let x
be the expected number of candidates to be interviewed for a
selection. The probability of first candidate getting selected is
0.16 and the total number of interviews done in this case is 1. The
other case is that the first candidate gets rejected and we start
all over again. The probability for that is (1-0.16)*(x+1). The
equation thus becomes -
x = 0.16 + (1-0.16)*(x+1)
Solving, x = 1/0.16, i.e. x = 6.25
Q5. (Generalized version of Q4) - The queen of a honey bee nest
produces offsprings one-after-other till she produces a male
offspring. The probability of produing a male offspring is p. What
is the expected number of offsprings required to be produced to
produce a male offspring?
This is same as the previous question, except that the number
0.16 has been replaced by p. Observe that the equation now becomes
-
x = p + (1-p)*(x+1)Solving, x = 1/p
Thus, observe that in the problems where there are two events,
where one event is desirable and other is undesirable, and the
probability of desirable event is p, then the expected number of
trials done to get the desirable event is 1/p
Generalizing on the number of events - If there are K events,
where one event is desirable and all others are undesirable, and
the probability of desirable event is p, then also the expected
number of trials done to get the desirable event is 1/p
The next question uses this generalization -
Q6. what is the expected number of dice throws required to get a
"four"?
Let the expected number of throws be x. The desirable event
(getting 'four') has probability 1/6 (as each face is
equiprobable). There are 5 other undesirable events (K=5). Note
that the value of the final answer doesnot depend on K. The answer
is thus 1/(1/6) i.e. 6.
Q7.Candidates are appearing for interview one after other.
Probability of k-th candidate getting selected is 1/(k+1). What is
the expected number of candidates that you will need to interview
to make sure that you select somebody?
The result will be the sum of infinite number of cases -
case-1: First candidate gets selected. The probability of this
event is 1/2 and the number of interviews is 1. case-2. Second
candidate gets selected. The probability of this event is 1/6 (=
1/2 of first candidate not getting selected and 1/3 of second
candidate getting selected, multiplied together gives 1/6) and the
number of interviews is 2case-3. Third candidate gets selected. The
probability of this event is 1/2 * 2/3 * 1/4 = 1/12 (= first not
getting selected and second not getting selected and third getting
selected) and the number of interviews is 3....case-k. k'th
candidate gets selected. The probability of this event is 1/2 * 2/3
* 3/4 * ... * (k-1)/k * 1/(k+1). (The first k-1 candidates get
rejected and the k'th candidate is selected). This evaluates to
1/(k*(k+1)) and the number of interviews is k...
[ Note that similar to problem 4, here we can't just say - if
the first candidate is rejected, then we will start the whole
process again. This is not correct, because the probabilty of each
candidate depends on it's sequence number. Hence sub-experiments
are not same as the parent experiment. This means that all the
cases must be explicitly considered.]The resultant expression will
be
x = 1/(1*2) + 2/(2*3) + 3/(3*4) + 4/(4*5) + ... + k/(k*(k+1)) +
... = 1/2 + 1/3 + 1/4 + ...
This is a well-known divergent series, which means that sum
doesnot converge, and hence the expectation doesnot exist.
Q8: A random permutation P of [1...n] needs to be sorted in
ascending order. To do this, at every step you will randomly choose
a pair (i,j) where i < j but P[i] > P[j], and swap P[i] with
P[j]. What is the expected number of swaps needed to sort
permutation in ascending order. (Idea: Topcoder)This is a
programming question, and the idea is simple - since each swap has
same probability of getting selected, the total number of expected
swaps for a permutation P is
E[P] = ( 1/cnt ) * (E[Ps] + 1)
where cnt is the total number of swaps possible in permutation
P, and Ps is the permutation generated by doing swap 's'. Since all
swaps are equiprobable, we simply sum up the expected values of the
resultant permutations (of course add 1 to each to account for the
swap done already) and divide the result by the total number of
permutations. The base case will be for the array that has been
already sorted - and the expected number of permutations for a
sorted array is 0.
Coding this is left as a (trivial) exercise to the reader.
Q9. A fair coin flip experiment is carried out N times. What is
the expected number of heads?
Consider an experiment of flipping a fair coin N times and let
the outcomes be represented by the array Z = {a1, a2,... ,an} where
each ai is either 1 or 0 depending on whether the outcome was heads
or tails respectively. In other words, for each i we have -
ai = if the i'th experiment gave head then 1 else 0.
Hence we have: number of heads in z = a1+ a2 + ... + anHence
E[number of heads in z] = E[a1+ a2 + ... + an]= E[a1] + E[a2] + ...
+ E[an]
Since ai corresponds to a coin-toss experiment, the value of
E[ai] is 0.5 for each i. Adding this n times, the expected number
of heads in Z comes out to be n/2.
Q10: (Bernaulli Trials) n students are asked to choose a number
from 1 to 100 inclusive. What is the expected number of students
that would choose a single digit number? This question is based on
the concept of bernaulli trials.An experiment is called a bernaulli
trial if it has exactly two outcomes, one of which is desired. For
example - flipping a coin, selecting a number from 1 to 100 to get
a prime, rolling a dice to get 4 etc. The result of a bernaulli
trial can typically be represented as "yes/no" or
"success/failure". We have seen in Q5 above that if the probability
of success of a bernaulli trial is p then the expected number of
trials to get a success is 1/p. is
This question is based on yet another result related to
bernaulli trials - If the probability of a success in a bernaulli
trial is p then the expected number of successes in n trials is
n*p. The proof is simple -
The number of successes in n trials = (if 1st trial is success
then 1 else 0) + ... + (if nth trial is success then 1 else 0)The
expected value of each bracket is 1*p + 0*(1-p) = p. Thus the
expected number of successes in n trials is n*p.
In the current case, "success" is defined as the experiment that
chooses a single digit number. Since all choices are equiprobable,
the probability of success is 9/100. (There are 9 single digit
numbers in 1 to 100). Since there are n students, the expected
number of students that would contribute to success (i.e the
expected number of successes) is n*9/100
Q11. What is the expected number of coin flips to ensure that
there are atleast N heads?
The solution can easily be framed in a recursive manner -
N heads = if 1st flip is a head then N-1 more heads, else N more
heads.The probability of 1st head is 1/2. Thus
E[N] = (1/2)(E[N-1]+1)+ (1/2)(E[N] + 1)Note that each term has 1
added to it to account for the first flip.
The base case is when N = 1 : E[1] = 2 (As discussed in Q2)
Simplifying the recursive case,E[N] = (1/2)( E[N-1] +1 + E[N] +
1) = (1/2)( E[N-1] + E[N] + 2) => 2 * E[N] = ( E[N-1] + E[N] +
2) => E[N] = E[N-1] + 2
Since E[1] = 2, E[2] = 4, E[3] = 6,..., in general E[N] = 2N.
Thus, the expected number of coin flips to ensure that there are
atleast N heads in 2N.
The next problem discusses a generalization :
Q12. What is the expected number of bernaulli trials to ensure
that there are atleast N successes, if the probability of each
success is p?
The recursive equation in this case is -
E[N] = p(E[N-1]+1)+ (1- p)(E[N] + 1)
Solving, E[N]-E[N-1] = p. Writing a total of N-1 equations:
E[N]-E[N-1] = 1/pE[N-1]-E[N-2] = 1/pE[N-2]-E[N-3] =
1/p...E[2]-E[1] = 1/p
Adding them all, E[N] - E[1] = (n-1)/p. But E[1] is 1/p (lemma
-1). Hence E[N] = n/p.
Moral: If probability of success in a bernaulli trial is p, then
the expected number of trials to guaranttee N successes is N/p.
This completes the discussion on problems on Mathematical
Expectation.
Exercises:
Note: Some of these are non-trivial and require concepts not
discussed in this tutorial. If you are interested you could read
about probability distribution basics, more advanced topics such as
joint and bivariate distributions and transformations and a
tutorial on permutations and combinations1. A game involves you
choosing one number (between 1 to 6 inclusive) and then throwing
three fair dice simultaneously. If none of the dice shows up the
number that you have chosen, you lose $1. If exactly one, two or
three dice show up the number that you have chosen, you win $1, $3
or $5 respectively. What is your expected gain?
2. There are 10 flowers in a garden, exactly one of which is
poisonous. A dog starts eating all these flowers one by one at
random. whenever he eats the posionous flower he will die. What is
the expected number of flowers he will eat before he will die?
3. A bag contains 64 balls of eight different colours, with
eight of each colour. What is the expected number of balls you
would have to pick (without looking) to select three balls of the
same color?
4. In a game of fair dice throw, what is the expected number of
throws to make sure that all 6 outcomes appear atleast once?
5. What is the expected number of bernaulli trials for getting N
consecutive successes, given N, if the probability of each success
is p?
http://www.biostat.umn.edu/~sudiptob/pubh8429/ExpectationsBasic04.pdfhttp://www.math.wsu.edu/students/aredford/documents/Notes_Expectation.pdfhttp://www.mathsisfun.com/data/standard-normal-distribution.html