Discrete Random Events Thurs. March 11, 1999 – Review with Examples – Discrete Outcome Sample Spaces – Influence Diagrams & Probability Trees – Moments (e.g., expectation, variance, kth central) – Probability Functions (Marginal, Joint, Conditional) Next week: Influence Diagrams (Tues.) and Dr. Anil Varma, Guest Speaker from GE’s AI/Expert Systems Group (Thurs.)
29
Embed
Discrete Random Events n Thurs. March 11, 1999 – Review with Examples – Discrete Outcome Sample Spaces – Influence Diagrams & Probability Trees – Moments.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Discrete Random Events
Thurs. March 11, 1999– Review with Examples– Discrete Outcome Sample Spaces– Influence Diagrams & Probability Trees– Moments (e.g., expectation, variance, kth central) – Probability Functions (Marginal, Joint, Conditional)
Next week: Influence Diagrams (Tues.) and Dr. Anil Varma, Guest Speaker from GE’s AI/Expert Systems Group (Thurs.)
Example: Venn Diagram and Independence
We know from the Venn Diagram– Pr(A|B) = 1– Pr(A)– Pr(B)
Are A and B independent? Is Pr(AB)=Pr(A)Pr(B)?
B
A
A B
Disk Drive Failure Example
c (C, C’) x (X, X’)
Brand ‡ X X' TotalCrash C 0.6 0.1 0.7C' 0.2 0.1 0.3Total 0.8 0.2 1.0
Pr(C) =Pr(CI)
= Pr(CX) + Pr(CX’)
= 0.6 + 0.1 = 0.7
Pr(X) = Pr (XI)
=Pr(XC) + Pr(XC’)
0.6 + 0.2 = 0.8
Pr(C|X) =Pr(CX)/Pr(X)
= 0.6/0.8 = 0.75
Outcome Sample Spaces
It is useful in performing probability manipulations to define the random events "Ei" as unions of elementary mutually exclusive and collectively exhaustive events.
Outcome sample spaces are analogous to Venn diagrams where the universe I, is made up of these elementary mutually exclusive and collectively exhaustive events.
The probability of any event Ei is then the sum of the probabilities (due to the mutual exclusivity) of the elementary events which comprise Ei.
Talus Bone Example
Assume that we are to roll two "fair" talus bones; one is the heel bone of a sheep, the other is a heelbone of a deer. Each one has four sides, and if it is "fair" the likelihood of any one side coming up on a roll is equally likely.
Let us further assume that each side of the talus bone has dots on it one dot, two dots, three dots and four dots (similar to a conventional die but with four sides instead of six).
Talus Bone Events
Assume both the sheep and deer talus bones are thrown randomly on the ground. Define the events:– Si = the number of dots on the up side of the sheep talus
bone.– Dj = the number of dots on the up side of the deer talus bone.
Assume the throw of the sheep bone is independent of the throw of the deer bone, – Pr(SiDj ) = Pr(Si )Pr(Dj )
Sample Space for Joint Events
The joint events DiSj (i,j = 1, 2, 3, 4) represent the 16 possible outcomes independent rolls for each talus bone.
The probability of rolling any side is equal to 1/4 for each bone.
Example: Find probability of E4 = the event that the sum of the two numbers is four.
1
2
3
4
Deer Talus
1 2 3 4
2 3 4 5
3 4 5 6
4 5 6 7
5 6 7 8
Sheep Talus
Because the events are mutually exclusive, the probability of event E4 is 1/16 times the number of ways a sum of four can occur.
There are three events that sum to four and thus:
Pr(E4) = 3(1/16) = 3/16
Deer Talus
She
ep T
alus
Example: F<10 = the event that the product of the two numbers is less than ten
Because the events are mutually exclusive, the probability of event F<10 is 1/16 times the number of ways a product less than ten can occur.
There are thirteen events with a product less than ten and thus:
Pr( F<10 ) = 13(1/16) = 13/16
Deer Talus
She
ep T
alus
What if the Talus Bones are Biased?
Pr(D )
Pr(S )
i
i
Suppose the probability of getting four dots is higher than getting three and so on as defined by the probability distribution:
2i-1
Pr(Si) = Pr(Di) = 16
Probability Mass Functionon Si and Di
Biased Talus Bones - Probability Mass Function?
Pr(D )
Pr(S )
i
i
Because these events are mutually exclusive and collectively exhaustive, the sum of the probabilities of all events Si or Di equal one. Consider the sheep talus:
∑i=1,4 Pr(Si) = 1 =
1 + 3 + 5 + 7
16Probability Mass Function
on Si and Di
Biased Talus Bones - Joint Probabilities
Pr(D )
Pr(S )
i
i
Probability Mass Functionon Si and Di
Deer Talus
She
ep T
alus
Joint Probabilities, Pr(Si, Di) x 256
Conditional Probability Expansions of Talus Bone Example
How would one represent events E4 and F<10 as a conditional probability expansion? There are many possible and consistent representations.
If you find it natural to think of the sum of the outcomes of the tosses to be influenced by the outcome of each individual toss, state E would be expressed mathematically by the following expansion: Pr(E ) = Pr(Sum(Di and Sj) ) =Pr( Sum(Di and Sj) | Di ,Sj ) Pr( Di | Sj) Pr(Sj )
Because Di and Sj are assumed to be independent, this reduces to: Pr(E) =Pr(Sum(Di and Sj)) =Pr( Sum(Di and Sj) | Di ,Sj ) Pr( Di) Pr(Sj )
Influence Diagrams & Talus Bone Example: E4
Because Di and Sj are assumed to be independent, this reduces to:
Pr(E) =Pr(Sum(Di and Sj)) =Pr( Sum(Di and Sj) | Di ,Sj ) Pr( Di) Pr(Sj )
Sum
D
S
E
Influence Diagrams & F<10 Example:
Pr(F) = Pr(Product(Di and Sj) )
=Pr( Product(Di and Sj) | Di ,Sj) Pr( Di | Sj) Pr(Sij)
Because Di and Sj are assumed to be independent, this reduces to:
Pr( Product(Di and Sj) | Di ,Sj) Pr( Di ) Pr(Sj )
D
S
F
Product
How do we represent both events E and F in the same
influence diagram?
Both diagrams show Di and Sj as being independent, which would remain unchanged for the combined diagram. But should there be an arc between E and F?
The question to be answered is: given we know the outcome of Di and Sj, would knowing the product give us any new information about the sum and vice versa. The answer is NO, and thus there is no arc between E and F. E and F are said to be conditionally independent of each other given Di and Sj.
E and F are said to be conditionally independent of each other given Di and Sj
Sum
D
S
FE
Product
Given we know the outcome of Di and Sj, knowing the product give us any new information about the sum and vice versa.
The lack of an arc reveals this conditional independence graphically.
Are E and F independent if we do not conditional information on D and S?
Given we know the outcome of Di and Sj, knowing the product give us any new information about the sum and vice versa.
The lack of an arc reveals this conditional independence graphically.
ESum
FProduct
Probability Tree Representation
1
Sheep Talus Outcome
Pr(S | H)
Deer Talus Outcome
Pr(D | S, H)
1
2
3
4
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
Sample Point Probability
Pr(S D | H} = Pr(S | H) • Pr(D | H)1 1
S D1 1 1 1
Pr(S D | H} = Pr(S | H) • Pr(D | H)1 2
S D1 2 1 2
Pr(S D | H} = Pr(S | H) • Pr(D | H)1 3
S D1 3 1 3
Pr(S D | H} = Pr(S | H) • Pr(D | H)1 4
S D1 4 1 4
Pr(S D | H} = Pr(S | H) • Pr(D | H)2 1
S D2 1 2 1
Pr(S D | H} = Pr(S | H) • Pr(D | H)2 2
S D2 2 2 2
Pr(S D | H} = Pr(S | H) • Pr(D | H)2 3
S D2 3 2 3
Pr(S D | H} = Pr(S | H) • Pr(D | H)2 4
S D2 4 2 4
Pr(S D | H} = Pr(S | H) • Pr(D | H)3 1
S D3 1 3 1
Pr(S D | H} = Pr(S | H) • Pr(D | H)3 2
S D3 2 3 2
Pr(S D | H} = Pr(S | H) • Pr(D | H)3 3
S D3 3 3 3
Pr(S D | H} = Pr(S | H) • Pr(D | H)3 4
S D3 4 3 4
Pr(S D | H} = Pr(S | H) • Pr(D | H)4 1
S D4 1 4 1
Pr(S D | H} = Pr(S | H) • Pr(D | H)4 2
S D4 2 4 2
Pr(S D | H} = Pr(S | H) • Pr(D | H)4 3
S D4 3 4 3
Pr(S D | H} = Pr(S | H) • Pr(D | H)4 4
S D4 4 4 4
Expectation of a Random Variable
E(x|H) = ∑ xi Pr(xi)
xi
or E(x|H) = <x|H>
Expectation of a Function of a Random Variable
Let g(x) be any single-valued function of the random variable x.
The expected value of g(x) is designated as E[g(x)] and defined as:
E[g(x)] = ∑ g(xi)Pr(x=xi) = <g(x)>
Is E[g(x)] = g[E(x)]?
Probability Mass Function, Pr(x)
The probability that the random variable x takes on the discrete value xo is defined by means of the probability mass function Pr(x) with the notation below:
Pr(x) = Pr(x=xi) for all xi
i
1/16
3/16
5/16
7/16
Pr(S | H)i
S
1
21 3 4
Biased Talus Bone Example
Cumulative Probability Distribution, Pr(x≤y|y)
Pr(x≤y|y)
= ∑ x≤y Pr(x)
= Pr(x≤y|y=yi) for all yi1/16
4/16
iPr(S ≤ z | z, H)
9/16
16/16
z
1
21 3 4
Biased Talus Bone Example
Cumulative Probability is Summation (Integral) of the Mass Function
1/16
4/16
iPr(S ≤ z | z, H)
9/16
16/16
z
1
21 3 4
i
1/16
3/16
5/16
7/16
Pr(S | H)i
S
1
21 3 4
Probability Mass Function Cumulative Probability Distribution
Complementary Probability Distribution, Pr(x>y|y)
Pr(x>y|y)
= 1 - Pr(x≤y|y)
iPr(S > z | z, H)
z
115/16
12/16
7/16
21 3 4
Biased Talus Bone Example
Joint Discrete Mass Functions
1
2
3
4
Di
Pr(D S | H)
x256
i i
49
35
35
25
21
21
15
15
9
7
7
5
5
3
3
1
S
i
21 3 4
1 3 5 7
3 9 15 21
5 25 3515
7 35 4921
D
i
S
i
21 3 4
2
1
3
4
Two Representations for Joint Probability Mass Function for Two Biased Talus Tosses
Bayesian Pair of Bolts
Suppose that a "pick and place" robot with two manipulators has been designed to pick two bolts of the same size from a bin and place each pair in separate containers on a belt for assembly. Suppose also that there are three different kinds of pairs of bolts that are used in the assembly of this product:
Pair 1 two bolts both made of alloy A
Pair 2 two bolts both made of alloy B
Pair 3 one of alloy A and the other of alloy B
Bayesian Pair of Bolts: Bayes Theorem
The alloys look identical and the only way to determine whether one is A or B requires performing a test.
Suppose that one of the pairs of bolts falls on the ground and the operator picks it up. The probability of getting any specific pair is 1/3.
Now suppose she randomly picks one of the bolts from the pair, performs a test on it and determines that it is made of alloy A.
What is the probability that the other bolt is of alloy A?
Bayesian Pair of Bolts: Define Events
A common erroneous answer to this question is 1/2. But this is incorrect! Define the following events:– P1: The event that pair 1 was picked.– P2: The event that pair 2 was picked.– P3: The event that pair 3 was picked.– A1: The event that the first member of the pair is of alloy A.– A2: The event that the second member of the pair is of alloy A.– B1: The event that the first member of the pair is of alloy B.– B2: The event that the second member of the pair is of alloy B.
What we want to know is Pr(A2|A1,S). We know that Pr(P1|S) = Pr(P2|S) = Pr(P3|S) = 1/3. Using Bayes' theorem and conditional probability find Pr(A2|A1,S).