Conditional Probability Total ProbabilityConditional ...berlin.csie.ntnu.edu.tw/Courses/Probability/2009Lectures/PROB2009… · Conditional Probability (1/2) • Conditional probability
Post on 17-Mar-2021
18 Views
Preview:
Transcript
Conditional Probability Total ProbabilityConditional Probability, Total Probability Theorem and Bayes’ Rule
Berlin ChenBerlin ChenDepartment of Computer Science & Information Engineering
National Taiwan Normal University
Reference:- D. P. Bertsekas, J. N. Tsitsiklis, Introduction to Probability , Sections 1.3-1.4
Conditional Probability (1/2)
• Conditional probability provides us with a way to reason about the outcome of an experiment based on partialabout the outcome of an experiment, based on partial information– Suppose that the outcome is within some given event , weBSuppose that the outcome is within some given event , we
wish to quantify the likelihood that the outcome also belongs some other given event
B
A
– Using a new probability law, we have the conditional probability of given , denoted by , which is defined as:BA ( )BAP( )
( ) ( )( )BBABA
PPP I=
A B
• If has zero probability, is undefined• We can think of as out of the total probability of the
( )BP( )BP ( )BAP
( )BAP
Probability-Berlin Chen 2
We can think of as out of the total probability of the elements of , the fraction that is assigned to possible outcomes that also belong to
( )BAPB
A
Conditional Probability (2/2)
• When all outcomes of the experiment are equally likely, p q y y,the conditional probability also can be defined as
( ) BAofelementsofnumber I( )BBABA
of elements ofnumber ofelementsofnumber I
=P
• Some examples having to do with conditional probability1. In an experiment involving two successive rolls of a die, you are told that
th f th t ll i 9 H lik l i it th t th fi t ll 6?the sum of the two rolls is 9. How likely is it that the first roll was a 6?2. In a word guessing game, the first letter of the word is a “t”. What is the
likelihood that the second letter is an “h”?3. How likely is it that a person has a disease given that a medical test was
negative?4. A spot shows up on a radar screen. How likely is it that it corresponds to
Probability-Berlin Chen 3
yan aircraft?
Conditional Probabilities Satisfy the Three Axioms
• Nonnegative:g
• Normalization:
( ) 0≥BAP• Normalization:
( ) ( )( )( )( ) 1==
Ω=Ω
BB
BBB
PP
PPP I
• Additivity: If and are two disjoint events( ) ( )BB PP
1A 2A
( ) ( )( )BAABAA 21PP IUU( ) ( )( )( )( ) ( )( )BABA
BBAA
21
2121
PP
P
=
=
IUI
IUU
distributive
A ( )( ) ( )
( )BABA
B
21
PPP+
=
=
II disjoint sets1A 2A
Probability-Berlin Chen 4
( )( ) ( )BABA
B
21
PPP+=
B
Conditional Probabilities Satisfy General Probability LawsProbability Laws
• Properties probability lawsp p y–
–
( ) ( ) ( )BABABAA 2121 PPP +≤U( ) ( ) ( ) ( )BAABABABAA 212121 IU PPPP −+=
– … ( ) ( ) ( ) ( )BAABABABAA 212121 IU PPPP +=
Conditional probabilities can also be viewed as a probabilityConditional probabilities can also be viewed as a probability law on a new universe , because all of the conditional probability is concentrated on .
BB
Probability-Berlin Chen 5
Simple Examples using Conditional Probabilities (1/3)
Probability-Berlin Chen 6
Simple Examples using Conditional Probabilities (2/3)g ( )
Probability-Berlin Chen 7
Simple Examples using Conditional Probabilities (3/3)
SF FFF
SS FSN
S F
S
C
Probability-Berlin Chen 8
Using Conditional Probability for Modeling (1/2)
• It is often natural and convenient to first specify p yconditional probabilities and then use them to determine unconditional probabilities
• An alternative way to represent the definition of conditional probability
( ) ( ) ( )BABBA PPP =I( ) ( ) ( )I
Probability-Berlin Chen 9
Using Conditional Probability for Modeling (2/2)
( )BAIP
( )( )cBAIP
( )BAc IP( )
( )cc BA IP
Probability-Berlin Chen 10
Multiplication (Chain) Rule
• Assuming that all of the conditioning events have g gpositive probability, we have
( ) ( ) ( ) ( ) ( )III 1−nn AAAAAAAAA PPPPPTh b f l b ifi d b iti
( ) ( ) ( ) ( ) ( )ILII 112131211 == = ni inni i AAAAAAAAA PPPPP– The above formula can be verified by writing
( ) ( ) ( )( )( )( )
( )( )I
LIII
I 13212111==ni in
i iAAAAAAAA PPPPP( ) ( ) ( ) ( ) ( )III 1121111 −== = ni ii i AAAAAA PPPPP
– For the case of just two events, the multiplication rule is simply the definition of conditional probability
Probability-Berlin Chen 11
( ) ( ) ( )12121 AAAAA PPP =I
Multiplication (Chain) Rule: Examples (1/2)
• Example 1.10. Three cards are drawn from an ordinary 52-card deck without replacement (drawn cards are not placed back in the deck). We wish to find the probability that none of the three cards is a “heart”.
( ) { } 1,2,3 ,heart anot is cardth the == iiAiP
( ) ( ) ( ) ( )213121321 = AAAAAAAAA III PPPP( ) ( ) ( ) ( )
5037
5138
5239 ⋅⋅=
?393C
Probability-Berlin Chen 12
505152 ?523C
Multiplication (Chain) Rule: Examples (2/2)
• Example 1.11. A class consisting of 4 graduate and 12 undergraduate students is randomly divided into 4 groups of 4 Whatundergraduate students is randomly divided into 4 groups of 4. What is the probability that each group includes a graduate student?
{ }diff tt2d1t d td tA { }{ }{ }diff4d321dd
groupsdifferent at are 3 and ,2, 1 students graduategroupsdifferent at are2and1studentsgraduate
2
1==
AAA
{ }groupsdifferent at are4 and3,,2,1studentsgraduate3 =A( ) ( ) ( ) ( ) ( )2131213213 == AAAAAAAAAA III PPPPP( )
( ) 81512
1 =
AA
A
P
P12
( )
( )134
14
213
12
=
=
AAA
AA
IP
P8
4
Probability-Berlin Chen 13
13
( )134
148
1512
3 ⋅⋅=∴ AP
4
Total Probability Theorem (1/2)
• Let be disjoint events that form a partition of the nAA ,,1 L j psample space and assume that , for all . Then, for any event , we have
n1i
B( ) 0>iAP
( ) ( ) ( )( ) ( )
n BABAB PPP ++= ILI1( ) ( ) ( ) ( )nn ABAABA PPPP ++= L11
– Note that each possible outcome of the experiment (sample space) is included in one and only one of the events nAA ,,1 L
Probability-Berlin Chen 14
Total Probability Theorem (2/2)
Figure 1.13:
Probability-Berlin Chen 15
Some Examples Using Total Probability Theorem (1/3)
Example 1.13.
Probability-Berlin Chen 16
Some Examples Using Total Probability Theorem (2/3)
Example 1.14.
(1,3),(1,4) (2,2),(2,3),(2,4) (4)
Probability-Berlin Chen 17
Some Examples Using Total Probability Theorem (3/3)
• Example 1.15. Alice is taking a probability class and at the end of each week she can be either up-to-date or she may have fallen p ybehind. If she is up-to-date in a given week, the probability that she will be up-to-date (or behind) in the next week is 0.8 (or 0.2, respectively) If she is behind in a given week the probability thatrespectively). If she is behind in a given week, the probability that she will be up-to-date (or behind) in the next week is 0.4 (or 0.6, respectively). Alice is (by default) up-to-date when she starts the class What is the probability that she is up to date after three weeks?class. What is the probability that she is up-to-date after three weeks?
( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) 4080
4080
111211212
222322323
BUBUPBUUPUU
.B.UBUPBUUPUU
⋅+⋅=+=
⋅+⋅=+=
PPPPP
PPPPPbehind:
date-to-up:
i
i
BU
( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( )( )0120806020
4080
011
111211212
111211212
UBU.B.UBBPBUBPBB
.B.UBUPBUUPUU
===
⋅+⋅=+=
⋅+⋅=+=
PPPPPPPP
PPPPP
know thatweAs Q
i
( ) ( ) ( )( )( )( ) 28060202080
72040208080012080
2
2
011
.....B.....U
.U.B,.U
=⋅+⋅==⋅+⋅==>
===
PP
PPP
know that we As Q
( ) ( ) ( )( ) ( ) ( ) 6020
4.08.0formuleaRecursion
1 ⋅+⋅=+BUBBUU iii
PPPPPP
Probability-Berlin Chen 18
( )( ) 688040280807203
2.....U =⋅+⋅=∴ P
( ) ( ) ( )( ) ( ) 2.0 ,8.0
6.02.0
11
1
==⋅+⋅=+
BUBUB iii
PPPPP
Bayes’ Rule
• Let be disjoint events that form a partition of nAAA ,,, 21 K j pthe sample space, and assume that , for all . Then, for any event such that we have
n21( ) 0≥iAP i
B ( ) 0>BP
( ) ( )( )i
i BBA
BAP
PP =
IMultiplication rule( )
( ) ( )( )
ii
BABA
PPP
= T t l b bilit th( )
( ) ( )( ) ( )
ii ABABPP
P
=
Total probability theorem
( ) ( )( ) ( )ii
nk kk
ABA
ABA
PP
PP∑ =1
Probability-Berlin Chen 19
( )( ) ( ) ( ) ( )nn ABAABA PPPP ++
=L11
Inference Using Bayes’ Rule (1/2)
惡性腫瘤
Figure 1.14:
良性腫瘤
Probability-Berlin Chen 20
Inference Using Bayes’ Rule (2/2)
• Example 1.18. The False-Positive Puzzle. – A test for a certain disease is assumed to be correct 95% of the time: if
a person has the disease, the test with are positive with probability 0.95 ( ), and if the person does not have the disease, the test ( ) 95.0=ABP
⎞⎛results are negative with probability 0.95 ( ). A random person drawn from a certain population has probability 0.001 ( ) of having the disease. Given that the person just tested
( )
( ) 001.0=AP
95.0=⎟⎠⎞⎜
⎝⎛ cc ABP
( )positive, what is the probability of having the disease ( ) ?
• : the event that the person has a diseaseh h h l i i
( )BAP
A• : the event that the test results are positiveB
( ) ( ) ( )( )= BABA
BAP
PPP
⎞⎛⎞⎛( )( ) ( )
( ) ( ) ( ) ( ) += cc ABAABAABA
B
PPPP
PPP
05.01 =⎟⎠⎞⎜
⎝⎛−=⎟
⎠⎞⎜
⎝⎛ ccc ABAB PP
Probability-Berlin Chen 21
0187.005.0999.095.0001.0
95.0001.0 =⋅+⋅
⋅=
Recitation
• SECTION 1.3 Conditional Probabilityy– Problems 11, 14, 15
• SECTION 1.4 Probability Theorem, Bayes’ Rule– Problems 17, 23, 24, 25
Probability-Berlin Chen 22
Homework -1
• Chapter 1 : Additional Problemsp(http://www.athenasc.com/CH1-prob-supp.pdf)
– Problems 2, 8, 13, 18, 24
Probability-Berlin Chen 23
top related