Rules of Probability. Recall: Axioms of Probability 1. P[E] ≥ 0. 2. P[S] = 1 3. Property 3 is called the additive rule for probability if E i ∩ E j =

Rules of Probability

Recall:

Axioms of Probability

1. P[E] ≥ 0.2. P[S] = 13. 1 2 3 1 2 3P E E E P E P E P E

Property 3 is called the additive rule for probability

if Ei ∩ Ej =

Note:for any event E

1 P S P E E P E P E

and S E E E E

hence

or 1P E P E

Note:

Hence 1 1 1 0P P S P S

S

Example –Probabilities for various events in Lotto 6/49

n Prize Prob0 winning 6,096,454 nil 0.43596497551 winning 5,775,588 nil 0.41301945052 winning 1,678,950 nil 0.12006379372 + bonus 172,200 5.00$ 0.01231423533 winning 246,820 10.00$ 0.01765040394 winning 13,545 80.00$ 0.00096861975 winning 252 2,500.00$ 0.00001802085 + bonus 6 100,000.00$ 0.00000042916 winning 1 4,000,000.00$ 0.0000000715Total 13,983,816

Let E = Event that no money is won

{0 winning #} {1 winning #} {2 winning #}

= 0.435965 + 0.413019 + 0.120064 = 0.969048

0 winning # 1 winning # 2 winning #P E P P P

money is won 1P E P P E = 1 - 0.969048 = 0.030952

Note: This also could have been calculated by adding up the probability of events where money is won.

1P E P E

If it is easier to compute the probability of the compliment of an event, than use the above equation to calculate the probability of the event E.

Example: The birthday problem

In a room of n unrelated individuals, what is the probability that at least 2 have the same birthday.

1P E P E

E

n(S) = total number of ways of assigning the n individuals birthdays = 365n

= the event no pair of individuals have the same birthday.

Let E = the event that at least 2 have the same birthday.

365!n E n

n

= total number of ways of assigning the n distinct birthdays

Hence

365!

1 1365n

nn

P E P E

365!

365n

nn E n

P En S

365!

!! 365 !

1365n

nn n

365!

1365 !365nn

365 364 363 365 11

365 365 365 365

n

Thus Two people have same birthday

in a room of people

Pn

365 364 363 365 11

365 365 365 365

n

n P [E ]

10 0.116920 0.411422 0.475723 0.507330 0.706340 0.8912

Table:

50%

Rule The additive rule

P[A B] = P[A] + P[B] – P[A B]

and

if P[A B] = P[A B] = P[A] + P[B]

Proof

and Ei Ej =

A B A B A B A B

1 2 3E E E

B

A

A B A BA B

Als0 1 2A A B A B E E

2 3B A B A B E E

Now P A B P A B P A B P A B

1 2 3P E P E P E

BA

A B A BA B

1 2P A P A B P A B P E P E 2 3P B P A B P A B P E P E 1 2 32P A P B P E P E P E

P A B P A B

Hence P A B P A P B P A B

Example:Saskatoon and Moncton are two of the cities competing for the World university games. (There are also many others). The organizers are narrowing the competition to the final 5 cities.There is a 20% chance that Saskatoon will be amongst the final 5. There is a 35% chance that Moncton will be amongst the final 5 and an 8% chance that both Saskatoon and Moncton will be amongst the final 5. What is the probability that Saskatoon or Moncton will be amongst the final 5.

Hence P A B P A P B P A B

Solution:Let A = the event that Saskatoon is amongst the final 5.Let B = the event that Moncton is amongst the final 5.Given P[A] = 0.20, P[B] = 0.35, and P[A B] = 0.08What is P[A B]?Note: “and” ≡ , “or” ≡ .

P A B P A P B P A B

0.20 0.35 0.08 0.47

Another Example – Bridge Hands

A Bridge hand is 13 cards chosen from a deck of 52 cards.

Total number of Bridge Hands

52 52 51 50 49 42 41 40

13 13 12 11 10 3 2 1N n S

635,013,559,600

1. What is the probability that the hand contains exactly 5 spades (8 non-spades)?

2. What is the probability that the hand contains exactly 5 hearts?

3. What is the probability that the hand contains exactly 5 spades and 5 hearts?

4. What is the probability that the hand contains exactly 5 spades or 5 hearts? (i. e. a five card major)

Some Questions

1. What is the probability that the hand contains exactly 5 spades (8 non-spades)?

Solutions

Let A = the event that the hand contains exactly 5 spades (8 non-spades)?

n(A) = the number of hands that contain exactly 5 spades (8 non-spades)?

13 391,287 61,523,748

5 8

79,181,063,676

the number of ways of choosing the 5 spades

the number of ways of choosing the 8 non-spades

79,181,063,676 = 0.124691926

635,013,559,600

n AP A

n S

2. What is the probability that the hand contains exactly 5 hearts?

Let B = the event that the hand contains exactly 5 hearts?

13 39

5 8 =

52

13

n BP B

n S

79,181,063,676= 0.124691926

635,013,559,600

3. What is the probability that the hand contains exactly 5 spades and 5 hearts?

A B= the event that the hand contains exactly 5 spades and 5 hearts?

13 13 26

5 5 3n A B

4,306,559,400

the number of ways of choosing the 5 spades

the number of ways of

choosing the 3 non-spades,hearts

1,287 1,287 61,523,748

the number of ways of choosing the 5 hearts

4,306,559,400and 0.006781838

635,013,559,600P A B

4. What is the probability that the hand contains exactly 5 spades or 5 hearts? (i. e. a five card major)

P A B P A P B P A B

0.124691926 0.124691926 0.006781838

0.2426

Thus there is a 24.26% chance that a bridge hand will contain at least one five card major.

Rule The additive for more than two events

P[A B C] = P[A] + P[B] + P[C] – P[A B]

then

if A B = A C = and B C =

P[A B C] = P[A] + P[B] + P[C]

For three events

– P[A C] – P[B C] + P[A B C]

A

B

C

E1

E3

E2

E6

E5

E4

E7

A B C = E1 E2 E3 E4 E5 E5 E7

A B = E1 E2

A C = E1 E4

B C = E1 E3

A = E1 E2 E4 E5

B = E1 E2 E3 E6

C = E1 E3 E4 E7

A B C = E1

When these three are added E2, E3 and E4 are counted twice and E1is counted three times

When these three are subtracted off the extra contributions of E2, E3 and E4 are removed and the contribution of E1is completely removed

To correct we now have to add back in the contribution of E1.

An Example

Three friends A, B and C live together in the same apartment.

For an upcoming “Stanley cup playoff game” there is

1. An 80% chance that A will watch.

2. A 60% chance that B will watch.

3. A 45% chance that C will watch.

4. A 50% chance that A and B will watch. (and possibly C)

5. A 30% chance that A and C will watch. (and possibly B)

6. A 25% chance that B and C will watch. (and possibly A)

7. A 15% chance that all three (A , B and C) will watch.

What is the probability that either A, B or C watch the upcoming “Stanley cup playoff game?”

Thus 1. P[A] = 0.80.

2. P[B] = 0.60.

3. P[C] = 0.45.

4. P[A B] = 0.50.

5. P[A C] = 0.30.

6. P[B C] = 0.25.

7. P[A B C] = 0.15.

P[A B C] = P[A] + P[B] + P[C] – P[A B]

– P[A C] – P[B C] + P[A B C]

= 0.80 + 0.60 + 0.45 – 0.50 – 0.30 – 0.25 + 0.15

= 0.95

Rule The additive rule for more than two events

then

Finally if Ai Aj = for all i ≠ j.

For n events

11

n n

i i i ji i ji

P A P A P A A

i j ki j k

P A A A

1

1 21n

nP A A A

11

n n

i iii

P A P A

Example: This is a Classic problem

Suppose that we have a family of n persons. • At Christmas they decide to put their names in a hat. • Each person will randomly pick a name. • This will be the only person for which they will buy

a present.

Questions1. What is the probability that at least one person picks

his (or her) own name? 2. How does this probability change as the size of the

family, n, increase to infinity?

Solution:

Let Ai = the event that person i picks his own name.

We want to calculate:

11

n n

i i i ji i ji

P A P A P A A

i j ki j k

P A A A

1

1 21n

nP A A A

1

n

ii

P A

Now because of the additive rule

11

n n

i i i ji i ji

P A P A P A A

i j ki j k

P A A A

1

1 21n

nP A A A

Note: N = n(S) = the total number of ways you can assign the names to the n people = n!

To calculate:

we need to calculate:

1 2, , , ,i i j i j k nP A P A A P A A A P A A A

Now

n(Ai) = the number of ways we can assign person i his own name (1) and arbitrarily assign names to the remaining n – 1 persons ((n – 1)!)

= 1 (n – 1)!

= (n – 1)! :

Thus:

1 ! 1

!i

i

n A nP A

n S n n

1 1

11

n n

ii i

P An

and

Now

n(Ai Aj) = the number of ways we can assign person i and person j their own name (1) and arbitrarily assign names to the remaining n – 2 persons ((n – 2)!)

= 1 (n – 2)!

= (n – 2)! :

Thus:

2 ! 1

! 1i j

i j

n A A nP A A

n S n n n

1 1

2 1 2!i ii j

nP A A

n n

and

Now

n(Ai Aj Ak) = the number of ways we can assign person i, person j and person k their own name (1) and arbitrarily assign names to the remaining n – 3 persons ((n – 3)!)

= 1 (n – 3)!

= (n – 3)! :Thus:

3 ! 1

! 1 2i j k

i j k

n A A A nP A A A

n S n n n n

1 1

3 1 2 3!i i ki j k

nP A A A

n n n

and

Continuing

n(A1 A2 A3 … An) = the number of ways we can assign all persons their own name

= 1

Thus:

1 2 3 1

!n

i j k

n A A A AP A A A

n S n

11

n n

i i i ji i ji

P A P A P A A

i j ki j k

P A A A

1

1 21n

nP A A A

Finally:

11 1 1 1 11 ( 1)

2! 3! 4! 5! !n

n

As the family size increases to infinity ( the number of terms become infinite)

1

1

1 1 1 1 11 1 1

2! 3! 4! 5!ii

P A ee

2 3 4 5

since 12! 3! 4! 5!

x x x x xe x

Summary of Rules to date

1. 1P E P E

2. P A B P A P B P A B

P A P B

Additive Rules

if A and B disjoint

11

3. n n

i i i ji i ji

P A P A P A A

i j ki j k

P A A A

1

1 21n

nP A A A

1

n

ii

P A

if Ai and Aj are all disjoint.

Conditional Probability

Conditional Probability

• Frequently before observing the outcome of a random experiment you are given information regarding the outcome

• How should this information be used in prediction of the outcome.

• Namely, how should probabilities be adjusted to take into account this information

• Usually the information is given in the following form: You are told that the outcome belongs to a given event. (i.e. you are told that a certain event has occurred)

Definition

Suppose that we are interested in computing the probability of event A and we have been told event B has occurred.

Then the conditional probability of A given B is defined to be:

P A BP A B

P B

if 0P B

Rationale:If we’re told that event B has occurred then the sample space is restricted to B.The probability within B has to be normalized, This is achieved by dividing by P[B]The event A can now only occur if the outcome is in of A ∩ B. Hence the new probability of A is:

P A BP A B

P B

BA

A ∩ B

An Example

The academy awards is soon to be shown.

For a specific married couple the probability that the husband watches the show is 80%, the probability that his wife watches the show is 65%, while the probability that they both watch the show is 60%.

If the husband is watching the show, what is the probability that his wife is also watching the show

Solution:

The academy awards is soon to be shown.

Let B = the event that the husband watches the show

P[B]= 0.80

Let A = the event that his wife watches the show

P[A]= 0.65 and P[A ∩ B]= 0.60

P A BP A B

P B

0.600.75

0.80

Another Example

Suppose a bridge hand (13 cards) is selected from a deck of 52 cards

Suppose that the hand contains 5 spades. What is the probability that it also contains 5 hearts.

Solution

N = n(S) = the total # of bridge hands =52

13

Let A = the event that the hand contains 5 hearts

Let B = the event that the hand contains 5 spades

13 39

13 39 5 8,

525 8

13

n B P B

13 13 26 52 13 26

5 5 3 13 5 3

13 39 52 39

5 8 13 8

P A BP A B

P B

13 13 26

13 13 26 5 5 3,

525 5 3

13

n A B P A B

Another Example

In the dice game – craps, if on the first roll you roll

i. a 7 or 11 you win

ii. a 2 or 12 you lose,

iii. If you roll any other number {3,4,5,6,8,9,10} that number is your point. You continue to roll until you roll your point (win) or a 7 (lose)

Suppose that your point is 5 what is the probability that you win.

Repeat the calculation for other values of your point.

The Sample Space S

N = n(S) = 36

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)

(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)

(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

n(B) = 4 + 6 = 10 10

36P B

Let B = the event {5} or {7}

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)

(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)

(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

n(A) = n(A ∩ B) = 4 4

36P A P A B

Let A = the event {5} = A ∩ B

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)

(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)

(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)

(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

4 36 40.4

10 36 10

P A BP A B

P B

Probability of winning Point = {3, 4, 5, 6, 8, 9, 10}

Point P [win]

3 0.250

4 0.3335 0.4006 0.4558 0.4559 0.40010 0.333

Independence

Definition

Two events A and B are called independent if

P A B P A P B

P A B P A P B

P A B P AP B P B

Note if 0 and 0 thenP B P A

and

P A B P A P BP B A P B

P A P A

Thus in the case of independence the conditional probability of

an event is not affected by the knowledge of the other event

Difference between independence and mutually exclusive

0 and 0. (also 0P A P B P A B

Two mutually exclusive events are independent only in the special case where

mutually exclusive

A B

Mutually exclusive events are highly dependent otherwise. A and B cannot occur simultaneously. If one event occurs the other event does not occur.

P A B P A P B

or

Independent events

A B

The ratio of the probability of the set A within B is the same as the ratio of the probability of the set A within the entire sample S.

P A B P A

P AP B P S

A B

S

P A B P A P B

To check if A and B are independent

compute , and P A P B P A B

and verify that

Example: A coin is tossed three times

S = sample space = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}

Each outcome is equally likely (prob = 1/8}Let A = the event that the first Head occurs on toss 2.

= {THH, THT}Let B = the event that the more Tails than Heads

= {HTT, THT, TTH, TTT}

Are A and B independent?

1 1 1since

8 4 2P A B P A P B

2 1 4 1 1, = = and

8 4 8 2 8P A P B P A B

Solution:

A and B are independent.

if 0

if 0

P A P B A P AP A B

P B P A B P B

The multiplicative rule of probability

and

P A B P A P B

if A and B are independent.

Example: Polya’s Urn model

An urn contains r red balls and b black balls.

A ball is selected at random, its colour is noted, it is replaced together with c balls of the same colour.

A second ball is selected.

What is the probability that you get first a black ball then a red ball?

What is the probability that a red ball is selected on the second draw?

Solution:

Let B1 = the event that the first ball is black.

Let R2 = the event that the second ball is red.

What is the probability that you get first a black ball then a red ball?

1

bP B

r b

2 1

rP R B

r b c

1 2 1 2 1

b rP B R P B P R B

r b r b c

What is the probability, that a red ball is selected on the second draw?

2P R

2 2 1 2 1: R R B R B Note

2 1 2 1 2P R P B R P B R 2 1R B

2R 1B

2 1R B

1 2 1 1 2 1P B P R B P B P R B

b r r r c

r b r b c r b r b c

r b r c r

r b c r b r b r b

Bayes Rule• Due to the reverend T.

Bayes• Picture found on website:

Portraits of Statisticians• http://www.york.ac.uk/

depts/maths/histstat/people/welcome.htm#h

P A P B AP A B

P A P B A P A P B A

Proof:

P A BP A B

P B

P A P B A

P A P B A P A P B A

P A B

P A B P A B

Example:

We have two urns. Urn 1 contains 14 red balls and 12 black balls. Urn 2 contains 6 red balls and 20 black balls.An Urn is selected at random and a ball is selected from that urn.

If the ball turns out to be red what is the probability that it came from the first urn?

Urn 1 Urn 2

Solution:

Note: the desired conditional probability is in the reverse direction of the given conditional probabilities. This is the case when Bayes rule should be used

14 6,

26 26P B A P B A

A

Let A = the event that we select urn 1

= the event that we select urn 2

1

2P A P A

Let B = the event that we select a red ball

We want .P A B

Bayes rule states

P A P B AP A B

P A P B A P A P B A

1 142 26

61 14 12 26 2 26

14 140.70

14 6 20

Example: Testing for a disease

Suppose that 0.1% of the population have a certain genetic disease.

A test is available the detect the disease.

If a person has the disease, the test concludes that he has the disease 96% of the time. It the person doesn’t have the disease the test states that he has the disease 2% of the time.

Two properties of a medical testSensitivity = P[ test is positive | disease] = 0.96

Specificity = P[ test is negative | no disease] = 1 – 0.02 = 0.98

A person takes the test and the test is positive, what is the probability that he (or she) has the disease?

Solution:

Note: Again the desired conditional probability is in the reverse direction of the given conditional probabilities.

0.96, 0.02P B A P B A

A

Let A = the event that the person has the disease

= the event that the person doesn’t have the disease

0.001, 1 0.001 0.999P A P A

Let B = the event that the test is positive.

We want .P A B

Bayes rule states

P A P B A

P A BP A P B A P A P B A

0.001 0.96

0.001 0.96 0.999 0.02

0.000960.0458

0.00096 .01998

Thus if the test turns out to be positive the chance of having the disease is still small (4.58%).Compare this to (.1%), the chance of having the disease without the positive test result.