Axioms and Algorithms Axioms and Algorithms for Inferences for Inferences Involving Involving Probabilistic Probabilistic Independence Independence Dan Geiger, Azaria Paz, and Judea Pearl, Information and Computation 91(1), March 1991, 128-141. Presentation by Guy Moses & Omer Weissbrod for the course 236372 - Bayesian Networks Computer Science Faculty, Technion –
33
Embed
Axioms and Algorithms for Inferences Involving Probabilistic Independence
Axioms and Algorithms for Inferences Involving Probabilistic Independence. Dan Geiger, Azaria Paz, and Judea Pearl, Information and Computation 91(1), March 1991, 128-141. Presentation by Guy Moses & Omer Weissbrod - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Axioms and AlgorithmsAxioms and Algorithmsfor Inferencesfor Inferences InvolvingInvolving
A is complete iff for every and every cl() there exists a
distribution P that satisfies cl)( and does not satisfy .
We’ve shown: given a minimal cl(), there exists a
distribution P that obeys:
1. P does not satisfy .
2. P satisfies .
Given a non-minimal cl(), we will derive its minimal statement ’, and devise a distribution P’ that satisfies but does not satisfy ’. Due to soundness of decomposition, P’ cannot satisfy as well.
14
Scope of CompletenessScope of CompletenessThe proof uses P- a binary p.d. (probability distribution
function) therefore:all p.d.’s over
Udiscrete p.d.’s
binary p.d.’s
normalp.d.’s
however,
for normal p.d.’s, the axiom set a1-d1 is not complete.
a stronger axiom is required:
replace:
with:
P P P( , ) ( , ) ( ,1d Mixin : )g I X Y I X Y Z I X Y Z
P P P1d' Composition: ( , ) ( , ) ( , )I X Y I X Z I X Y Z
•P
15
What’s ahead?
IntroductionIntroduction - some definitions, notations and reminders.
Proof of CompletenessProof of Completeness. - “if it’s true – it can be proved”.
Preparations for the Membership AlgorithmPreparations for the Membership Algorithm – more definitions, and some theoretical groundwork.
The Membership AlgorithmThe Membership Algorithm – description, proof of correctness, complexity analysis.
16
Some more Definitions and ToolsSome more Definitions and Tools
Definition: Spanspan(): the set of elements represented in statement .
Example: span(x1x2,x3,x4) = {x1,x2,x3,x4}
span(): the set of elements represented in all statements of .
Example: span({(x1,x2),(x1,x3)}) = {x1,x2,x3}
17
Some more Definitions and ToolsSome more Definitions and Tools
Definition: ProjectionThe projection of on X, denoted (X), is the statement
derived from by removing all elements not in X from .
Example:
if =(x1x2x3, x4x5) and X={x2,x3,x4} then
(X)=(x2x3, x4).
The projection of on X, denoted (X), is {(X) | }.
18
Some more Definitions and ToolsSome more Definitions and Tools
Projection Lemma: iff ‘ , where ’ = (span())
) if ' then clearly because all the statements in ‘ can be derived from the statements in by decomposition.
19
Some more Definitions and ToolsSome more Definitions and Tools
Projection Lemma: iff ’ , where ’ = (span()), s = span()
) if then there is a derivation chain for : 1, 2, … , k.
For each j:
if k j, k<j, (by symmetry or decomposition)
then k(s) j(s) by symmetry or decomposition respectively.
Similarly,
if j is derived from k and l by mixing,
then j(s) is derived from k(s), l(s) by mixing.
20
Some more Definitions and ToolsSome more Definitions and Tools
Projection Lemma: iff ’ , where ’ = (span()), s = span()
Observations from projection lemma:• Variables not in are unnecessary for determining whether
.
• The problem of verifying whether can be simplified to the problem of verifying whether ' , where '= (span()).
• This problem can be solved with a possibly reduced time and space complexity.
21
Conditions for Conditions for Inference of IndependenceInference of Independence
Maim claim: for a given , we have ’ iff:
1. is trivial: =(X,) (up to symmetry)
OR
2. is in ’: ’ (up to symmetry)
OR
3. is derivable from ’:
there exists ’’ s.t. span() = span(’)
and for ’=(AP,BQ) =(AQ,BP) (A,B,Q,P may be empty)
’ (A,P), ’ (B,Q) (up to symmetry)
22
Proof of Main ClaimProof of Main ClaimMaim claim: for a given , we have ’ iff:
1. is trivial*: =(X,) *up to symmetry
2. is in* ’: ’
3. is derivable* from ’: ’’ s.t. span() = span(’)
and for ’=(AP,BQ) =(AQ,BP) : ’ (A,P), ’ (B,Q)
) if 1. is trivial*
OR 2. is in* ’. than the proof is immediate.
otherwise,
3. there exists ’’ s.t. span() = span(’)
and for ’=(AP,BQ) =(AQ,BP) : ’ (A,P), ’ (B,Q)
we will show a constructive proof under these conditions
23
Proof of Main ClaimProof of Main ClaimMaim claim: for a given , we have ’ iff:
1. is trivial*: =(X,) *up to symmetry
2. is in* ’: ’
3. is derivable* from ’: ’’ s.t. span() = span(’)
and for ’=(AP,BQ) =(AQ,BP) : ’ (A,P), ’ (B,Q)
) (contd.) given that ’ (AP,BQ), ’ (A,P), ’ (B,Q).
1. (A,P)(AP,BQ) (A,PBQ)
2. (B,Q)(AP,BQ) (APB,Q) (PB,Q)
3. (PB,Q)(A,PBQ) (AQ,PB) = (AQ, BP) =
We’ve proven this direction.
mix
mix
mix
dec.
24
Proof of Main ClaimProof of Main ClaimMaim claim: for a given , we have ’ iff:
1. is trivial*: =(X,) *up to symmetry
2. is in* ’: ’
3. is derivable* from ’: ’’ s.t. span() = span(’)
and for ’=(AP,BQ) =(AQ,BP) : ’ (A,P), ’ (B,Q)
) Given ’ , if 1. is trivial* OR 2. is in* ’,
than the proof is immediate.
Otherwise, since no axiom can add new variables to a statement, there must exist ’’ s.t. span() =
span(’) in the derivation chain of .
also: = (AQ,BP) (A,P)
= (AQ,BP) (Q,B)
dec.
dec.
25
Conclusions from ClaimConclusions from Claim• We’ve seen that, after discarding unneeded
variables,it is possible to tell whether ’ (when it’s not immediately obvious) by:
a. Finding another statement ’’ for whichspan() = span(’),
b. Verifying that ’ (A,P), ’ (B,Q)when ’=(AP,BQ) =(AQ,BP).
• This suggests using a recursive “divide and
conquer” approach.
26
What’s ahead?
IntroductionIntroduction - some definitions, notations and reminders.
Proof of CompletenessProof of Completeness. - “if it’s true – it can be proved”.
Preparations for the Membership AlgorithmPreparations for the Membership Algorithm – more definitions, and some theoretical groundwork.
The Membership AlgorithmThe Membership Algorithm – description, proof of correctness, complexity analysis.
27
The Membership Algorithm The Membership Algorithm Procedure Find(,):
1. set ’ := (span()).
2. if is trivial, or ’ (up to symmetry)then Find(,) := TRUE.
3. else if for all non-trivial ’’: span() span(’),
then Find(,) := FALSE.
4. else there exists ’’: span() = span(’),
and ’=(AP,BQ) =(AQ,BP),
set 1:= (A,P), 2:= (B,Q).
Find(,) := (Find(’,1) Find(’,2))
28
Algorithm Correctness ProofAlgorithm Correctness ProofWe will prove that Find(,) := TRUE cl() by
induction on k=.
Induction base: if k=1 then is trivial, therefore the algorithm will return TRUE in step 2 and cl().