-
Conditional Probability Total ProbabilityConditional
Probability, Total Probability Theorem and Bayes’ Rule
Berlin ChenBerlin ChenDepartment of Computer Science &
Information Engineering
National Taiwan Normal University
Reference:- D. P. Bertsekas, J. N. Tsitsiklis, Introduction to
Probability , Sections 1.3-1.4
-
Conditional Probability (1/2)
• Conditional probability provides us with a way to reason about
the outcome of an experiment based on partialabout the outcome of
an experiment, based on partial information– Suppose that the
outcome is within some given event , weBSuppose that the outcome is
within some given event , we
wish to quantify the likelihood that the outcome also belongs
some other given event
B
A
– Using a new probability law, we have the conditional
probability of given , denoted by , which is defined as:BA ( )BAP(
)
( ) ( )( )BBABA
PPP I=
A B
• If has zero probability, is undefined• We can think of as out
of the total probability of the
( )BP( )BP ( )BAP
( )BAP
Probability-Berlin Chen 2
We can think of as out of the total probability of the elements
of , the fraction that is assigned to possible outcomes that also
belong to
( )BAPB
A
-
Conditional Probability (2/2)
• When all outcomes of the experiment are equally likely, p q y
y,the conditional probability also can be defined as
( ) BAofelementsofnumber I( )BBABA
of elements ofnumber ofelementsofnumber I
=P
• Some examples having to do with conditional probability1. In
an experiment involving two successive rolls of a die, you are told
that
th f th t ll i 9 H lik l i it th t th fi t ll 6?the sum of the
two rolls is 9. How likely is it that the first roll was a 6?2. In
a word guessing game, the first letter of the word is a “t”. What
is the
likelihood that the second letter is an “h”?3. How likely is it
that a person has a disease given that a medical test was
negative?4. A spot shows up on a radar screen. How likely is it
that it corresponds to
Probability-Berlin Chen 3
yan aircraft?
-
Conditional Probabilities Satisfy the Three Axioms
• Nonnegative:g
• Normalization:
( ) 0≥BAP• Normalization:
( ) ( )( )( )( ) 1==
Ω=Ω
BB
BBB
PP
PPP I
• Additivity: If and are two disjoint events( ) ( )BB PP
1A 2A
( ) ( )( )BAABAA 21PP IUU( ) ( )( )( )( ) ( )( )BABA
BBAA
21
2121
PP
P
=
=
IUI
IUU
distributive
A ( )( ) ( )
( )BABA
B
21
PPP+
=
=
II disjoint sets1A 2A
Probability-Berlin Chen 4
( )( ) ( )BABA
B
21
PPP+=
B
-
Conditional Probabilities Satisfy General Probability
LawsProbability Laws
• Properties probability lawsp p y–
–
( ) ( ) ( )BABABAA 2121 PPP +≤U( ) ( ) ( ) ( )BAABABABAA 212121
IU PPPP −+=
– … ( ) ( ) ( ) ( )BAABABABAA 212121 IU PPPP +=
Conditional probabilities can also be viewed as a
probabilityConditional probabilities can also be viewed as a
probability law on a new universe , because all of the conditional
probability is concentrated on .
BB
Probability-Berlin Chen 5
-
Simple Examples using Conditional Probabilities (1/3)
Probability-Berlin Chen 6
-
Simple Examples using Conditional Probabilities (2/3)g ( )
Probability-Berlin Chen 7
-
Simple Examples using Conditional Probabilities (3/3)
SF FFF
SS FSN
S F
S
C
Probability-Berlin Chen 8
-
Using Conditional Probability for Modeling (1/2)
• It is often natural and convenient to first specify p
yconditional probabilities and then use them to determine
unconditional probabilities
• An alternative way to represent the definition of conditional
probability
( ) ( ) ( )BABBA PPP =I( ) ( ) ( )I
Probability-Berlin Chen 9
-
Using Conditional Probability for Modeling (2/2)
( )BAIP
( )( )cBAIP
( )BAc IP( )
( )cc BA IP
Probability-Berlin Chen 10
-
Multiplication (Chain) Rule
• Assuming that all of the conditioning events have g gpositive
probability, we have
( ) ( ) ( ) ( ) ( )III 1−nn AAAAAAAAA PPPPPTh b f l b ifi d b
iti
( ) ( ) ( ) ( ) ( )ILII 112131211 == = ni inni i AAAAAAAAA
PPPPP– The above formula can be verified by writing
( ) ( ) ( )( )( )( )
( )( )I
LIII
I 13212111==ni in
i iAAAAAAAA PPPPP( ) ( ) ( ) ( ) ( )III 1121111 −== = ni ii i
AAAAAA PPPPP
– For the case of just two events, the multiplication rule is
simply the definition of conditional probability
Probability-Berlin Chen 11
( ) ( ) ( )12121 AAAAA PPP =I
-
Multiplication (Chain) Rule: Examples (1/2)
• Example 1.10. Three cards are drawn from an ordinary 52-card
deck without replacement (drawn cards are not placed back in the
deck). We wish to find the probability that none of the three cards
is a “heart”.
( ) { } 1,2,3 ,heart anot is cardth the == iiAiP
( ) ( ) ( ) ( )213121321 = AAAAAAAAA III PPPP( ) ( ) ( ) ( )
5037
5138
5239 ⋅⋅=
?393C
Probability-Berlin Chen 12
505152 ?523C
-
Multiplication (Chain) Rule: Examples (2/2)
• Example 1.11. A class consisting of 4 graduate and 12
undergraduate students is randomly divided into 4 groups of 4
Whatundergraduate students is randomly divided into 4 groups of 4.
What is the probability that each group includes a graduate
student?
{ }diff tt2d1t d td tA { }{ }{ }diff4d321dd
groupsdifferent at are 3 and ,2, 1 students
graduategroupsdifferent at are2and1studentsgraduate
2
1==
AAA
{ }groupsdifferent at are4 and3,,2,1studentsgraduate3 =A( ) ( )
( ) ( ) ( )2131213213 == AAAAAAAAAA III PPPPP( )
( ) 81512
1 =
AA
A
P
P12
( )
( )134
14
213
12
=
=
AAA
AA
IP
P8
4
Probability-Berlin Chen 13
13
( )134
148
1512
3 ⋅⋅=∴ AP
4
-
Total Probability Theorem (1/2)
• Let be disjoint events that form a partition of the nAA ,,1 L
j psample space and assume that , for all . Then, for any event ,
we have
n1i
B( ) 0>iAP
( ) ( ) ( )( ) ( )
n BABAB PPP ++= ILI1( ) ( ) ( ) ( )nn ABAABA PPPP ++= L11
– Note that each possible outcome of the experiment (sample
space) is included in one and only one of the events nAA ,,1 L
Probability-Berlin Chen 14
-
Total Probability Theorem (2/2)
Figure 1.13:
Probability-Berlin Chen 15
-
Some Examples Using Total Probability Theorem (1/3)
Example 1.13.
Probability-Berlin Chen 16
-
Some Examples Using Total Probability Theorem (2/3)
Example 1.14.
(1,3),(1,4) (2,2),(2,3),(2,4) (4)
Probability-Berlin Chen 17
-
Some Examples Using Total Probability Theorem (3/3)
• Example 1.15. Alice is taking a probability class and at the
end of each week she can be either up-to-date or she may have
fallen p ybehind. If she is up-to-date in a given week, the
probability that she will be up-to-date (or behind) in the next
week is 0.8 (or 0.2, respectively) If she is behind in a given week
the probability thatrespectively). If she is behind in a given
week, the probability that she will be up-to-date (or behind) in
the next week is 0.4 (or 0.6, respectively). Alice is (by default)
up-to-date when she starts the class What is the probability that
she is up to date after three weeks?class. What is the probability
that she is up-to-date after three weeks?
( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) 4080
4080
111211212
222322323
BUBUPBUUPUU
.B.UBUPBUUPUU
⋅+⋅=+=
⋅+⋅=+=
PPPPP
PPPPPbehind:
date-to-up:
i
i
BU
( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( )( )0120806020
4080
011
111211212
111211212
UBU.B.UBBPBUBPBB
.B.UBUPBUUPUU
===
⋅+⋅=+=
⋅+⋅=+=
PPPPPPPP
PPPPP
know thatweAs Q
i
( ) ( ) ( )( )( )( ) 28060202080
72040208080012080
2
2
011
.....B.....U
.U.B,.U
=⋅+⋅==⋅+⋅==>
===
PP
PPP
know that we As Q
( ) ( ) ( )( ) ( ) ( ) 6020
4.08.0formuleaRecursion
1 ⋅+⋅=+BUBBUU iii
PPPPPP
Probability-Berlin Chen 18
( )( ) 688040280807203
2.....U =⋅+⋅=∴ P
( ) ( ) ( )( ) ( ) 2.0 ,8.0
6.02.0
11
1
==⋅+⋅=+
BUBUB iii
PPPPP
-
Bayes’ Rule
• Let be disjoint events that form a partition of nAAA ,,, 21 K
j pthe sample space, and assume that , for all . Then, for any
event such that we have
n21( ) 0≥iAP i
B ( ) 0>BP
( ) ( )( )i
i BBA
BAP
PP =
IMultiplication rule( )
( ) ( )( )
ii
BABA
PPP
= T t l b bilit th( )
( ) ( )( ) ( )
ii ABABPP
P
=
Total probability theorem
( ) ( )( ) ( )ii
nk kk
ABA
ABA
PP
PP∑ =1
Probability-Berlin Chen 19
( )( ) ( ) ( ) ( )nn ABAABA PPPP ++
=L11
-
Inference Using Bayes’ Rule (1/2)
惡性腫瘤
Figure 1.14:
良性腫瘤
Probability-Berlin Chen 20
-
Inference Using Bayes’ Rule (2/2)
• Example 1.18. The False-Positive Puzzle. – A test for a
certain disease is assumed to be correct 95% of the time: if
a person has the disease, the test with are positive with
probability 0.95 ( ), and if the person does not have the disease,
the test ( ) 95.0=ABP
⎞⎛results are negative with probability 0.95 ( ). A random
person drawn from a certain population has probability 0.001 ( ) of
having the disease. Given that the person just tested
( )
( ) 001.0=AP
95.0=⎟⎠⎞⎜
⎝⎛ cc ABP
( )positive, what is the probability of having the disease ( )
?
• : the event that the person has a diseaseh h h l i i
( )BAP
A• : the event that the test results are positiveB
( ) ( ) ( )( )= BABA
BAP
PPP
⎞⎛⎞⎛( )( ) ( )
( ) ( ) ( ) ( ) += cc ABAABAABA
B
PPPP
PPP
05.01 =⎟⎠⎞⎜
⎝⎛−=⎟
⎠⎞⎜
⎝⎛ ccc ABAB PP
Probability-Berlin Chen 21
0187.005.0999.095.0001.0
95.0001.0 =⋅+⋅
⋅=
-
Recitation
• SECTION 1.3 Conditional Probabilityy– Problems 11, 14, 15
• SECTION 1.4 Probability Theorem, Bayes’ Rule– Problems 17, 23,
24, 25
Probability-Berlin Chen 22
-
Homework -1
• Chapter 1 : Additional
Problemsp(http://www.athenasc.com/CH1-prob-supp.pdf)
– Problems 2, 8, 13, 18, 24
Probability-Berlin Chen 23