Math 141 - Lecture 4: Distributions Related To The Binomial

MultinomialGeometric

HypergeometricPoisson

Math 141Lecture 4: Distributions Related To The Binomial Distribution

Albyn Jones1

1Library [email protected]

www.people.reed.edu/∼jones/courses/141

Albyn Jones Math 141



Outline

ReviewThe Multinomial DistributionThe Geometric and Negative Binomial DistributionsThe Hypergeometric DistributionThe Poisson Distribution




Examples of Different Experiments

Binomial: Count the number of Heads in a fixed number oftosses.Geometric: Count the number of Tails before the first Head.Negative Binomial: Count the number of Tails beforebefore the k -th Head.Hypergeometric: Count the number of red cards dealt in apoker hand.Poisson A model for rare events.




Review: The Binomial

Dichotomous Trials: Each trial results in either a ‘Success’,S, or a ‘Failure’ F .Independence: Successive trials are independent; knowingwe got S on one trial does not help us predict the outcomeof any other trial.Constant probability: Each trial has the same probabilityP(S) = p, and P(F ) = 1− p.X counts the number of S’s.n the number of trials is fixed.




X ∼ Binomial(n,p) Probabilities

Let p = P(S) and q = 1− p = P(F ), then

P(X = k) =

(nk

)pk qn−k

And in R, the density function is given by

P(X = k) = dbinom(k ,n,p)




The Multinomial

Polychotomous Trials: Each trial results in one of a fixedset of possible outcomes E1,E2, . . .EN. Example: dierolls, Ω = 1,2,3,4,5,6.Independence: Successive trials are independent; knowingthe outcome of one trial does not help us predict theoutcome of any other trial.Constant probability: Each trial has the same probabilityfor each possibility.X1,X2, . . .XN count the number of events of each type.n the number of trials is fixed.




Multinomial Probabilities

Good news! The typical computation involves collapsingcategories to create a binomial.

Example: Roll a fair die 20 times. For each roll, there aresix possible outcomes, so we have a MultinomialDistribution.What is the probability of rolling three 6’s in 20 rolls?Let X be the number of 6’s in 20 rolls. What is thedistribution of X?

X ∼ Binomial(20,1/6)




But Since You Asked...

For n independent Multinomial(p1,p2, . . . ,pN ) trials, where theprobability of observing category i is pi . Let Xi be the count ofevents observed in category i , where

N∑i=1

Xi = n andN∑

i=1

pi = 1

P(X1 = k1, . . . ,XN = kN) =n!

k1!k2! . . . kN !pk1

1 pk22 . . . pkN

N

with R functions: rmultinom, dmultinom.




Example: Election Polls

Suppose we ask registered Republicans for their preference:Bachman, Gingrich, Perry, Romney, or ‘none of the above’ (RonPaul is invisible :-).

The Poll We ask 1000 randomly chosen Republicans fortheir preference, and let 9,290,108,395,198 be thecounts for those 5 options, in order.The Population Proportions Suppose that the actualprobabilities are (in order) .01, .3, .1, .4, .19.The Probability:dmultinom(c(9, 290, 108, 395 , 198),1000,

c(.01, .3, .1, .4, .19))[1] 2.572792e-06

That looks small, but even the most likely outcome hassmall probability (about 5× 10−6)!




The Geometric Distribution

Dichotomous Trials: Each trial results in either a ‘Success’,S, or a ‘Failure’ F .Independence: Successive trials are independent; knowingwe got S on one trial does not help us predict the outcomeof any other trial.Constant probability: Each trial has the same probabilityP(S) = p, and P(F ) = 1− p = q.X counts the number of F ’s before the first S.Question: What is the probability we see k failures beforethe first success?

Pr(X = k) = P(F1,F2,F3, . . . ,Fk ,S) = p · qk




Example

Roll a fair die repeatedly until getting the first 6.What are p and q here?What is the probability it comes on the 6th roll, that is wehave 5 non-sixes before the first 6?

P(X = 5) =

(56

)5 16≈ .067




The Negative Binomial Distribution

Like the Geometric: Dichotomous outcomes F ,S,independent trials, constant probability.X counts the number of F ’s before the r th S.Question: What is the probability we see k failures beforethe r th success?Hint The last trial must be an S, so we see k failures and(r − 1) successes (in any order), followed by a success.

Pr(X = k) =

(r − 1 + k

k

)·pr−1 ·qk ·p =

(r − 1 + k

k

)·qk ·pr




Example

Roll a fair die repeatedly until getting the third 6.What are p and q here?What is the probability it comes on the 10th roll, that is wehave 7 non-sixes before the third 6?

P(X = 7) =

(97

)(56

)7(16

)3

≈ .0465




Connections

The number of Failures observed before getting the r thSuccess is clearly the sum of

The number of Failures observed before the 1st SuccessThe number of Failures observed between the 1st and 2ndSuccessesThe number of Failures observed between the 2nd and 3rdSuccessesand so on.Theorem The sum of r independent Geometric(p) RV’s hasa NegativeBinomial(r ,p) distribution.




The Hypergeometric Distribution

Sampling from a finite population of two categories A and Bwithout replacement.

Non-Independence, Non-constant Probability: Successivetrials are dependent; every trial changes the sample spaceand probabilities for the subsequent trials.X counts the number of A’s.n the number of trials is fixed.




Hypergeometric Probabilities

Suppose we have a bag with A alabaster and B black marbles,well mixed. We extract n marbles. Let X be the number ofalabaster marbles drawn. For 0 ≤ k ≤ min(A,n), the probabilityof drawing k alabaster and n − k black marbles is given by

P(X = k) =

(Ak

)( Bn−k

)(A+Bn

)




Hypergeometric Example

Question: What is the probability that a 5 card poker handdealt from a well shuffled deck has 4 spades?There are A = 13 spades, B = 39 non-spades. Let X bethe number of spades dealt.

P(X = 4) =

(134

)(391

)(525

) ≈ 0.01




The Poisson DistributionA probability model for Rare Events

Origin: An analytical approximation for binomialprobabilities when n is large and p is small: let µ = np,then

P(X = k) ≈ µk

k !e−µ

Poisson Process A random process describing occuranceof events in time: let µ be the rate per unit time, then ifdisjoint time intervals are independent, and Xt counts thenumber of events occuring in an interval of length t ,

P(Xt = k) =(tµ)k

k !e−tµ




Poisson Example

Question: What is the probability that in a group of 40people, no two share a birthday? Or, the complement: atleast two share a birthday?Poisson Approximation: what is the relevant Poissondistribution? To use Poisson approximation to theBinomial, we need to know how many ‘trials’ there are, andthe probability of success on each trial.Trials: How many trials are there here?(

402

)=

40!

2! · 38!=

40 · 392

= 780

Probabilities: The probability two people share a birthdayis approximately

1365




Solution

The probability that no two of 40 people share a birthday isapproximately the probability that a Poisson RV X withparameter µ = 780/365 ≈ 2.137 is equal to 0.

P(X = 0) =µ0

0!e−µ = e−780/365 ≈ .12

Thus the probability that at least two people share a birthday isabout .88.




Sums of Poisson(µ) RV’s

Suppose that X and Y are independent Poisson(µ) RV’s. Whatis the distribution of X + Y ?

Hint: if X and Y are independent Binomial(n,p) RV’s, thenX + Y is Binomial(n + n,p).Suppose that n is large, and p small, and µ = np.Conclusion?A Binomial(2n,p) is close to a Poisson(2µ)In General: the sum of independent Poisson(µi ) RV’s isPoisson(µ), where µ =

∑µi .




Connecting the Poisson and Negative Binomial

Short version: The Negative Binomial is a good model for acollection of Poisson RV’s with variable rates µ.




R Functions

Distribution density CDF RNG

Binom(n,p) dbinom(k,n,p) pbinom(k,n,p) rbinom(N,n,p)

Geom(p) dgeom(k,p) pgeom(k,p) rgeom(N,p)

Hyper(A,B,n) dhyper(k,A,B,n) phyper(k,A,B,n) rhyper(N,A,B,n)

Poisson(µ) dpois(k,µ) ppois(k,µ) rpois(N,µ)




Summary

The Multinomial DistributionThe Geometric and Negative Binomial DistributionsThe Hypergeometric Distribution: sampling withoutreplacement from a finite population.The Poisson Distribution: rare events.


Math 141 - Lecture 4: Distributions Related To The Binomial

Documents