Probabilistic Graphical Models 輪読会 Chapter5

Probabilistic Graphical Models

Chapter 5 Local Probabilistic Models

CPDtbd

5.1 Tabular CPDsCPD CPTs V al(Pa ) V al(X)

2X252 = 32

A B Px|A,B

1 1 0.2

1 0 0.1

0 1 0.3

0 0 0.4

x5

5.2 Deterministic CPDs

5.2.1 Representation

X CPDf : V al(Pa ) V al(X)CPD

P (xPa ) =

f[2] or / and[] X = Y + Z

X

X {10

x = f(Pa )Xotherwise

5.2.2 Independencies

Example 5.3

2CAB

ABC(D EA,B)

CABdseparation

dseparation


(X Y Z) deterministic CPDPaX (Z ) X Z dseparation

i+

i+


5.1GD,X,Y ,ZZXY deterministically separated P I (G) PP (XPa )X DP (X Y Z)

exercise 5.1

l

X


5.2GD,X,Y ,ZDETSEP(G,D,X,Y ,Z)P I (G) PP (XPa )X DP (X Y Z)

DETSEP

l

X


Example 5.4

CBAXOR

ACB(D EB,C)


Revisit Example 5.3 (Example 5.5)

CABOR

A = a CORB

P (DB, a ) = P (Da )BDA = a CP (XY ,Z) = P (XZ)(X Y Z)

1

1 1

0


Definition 5.1

X,Y ,ZCX Y Zcc V al(C)

P (XY ,Z, c) = P (XZ, c)whereneverP (Y ,Z, c) > 0

Z(X Y Z, c)cXY contextually independent

contextspecific independencies CSI

c


Revisit Example 5.3 with CSI (Example 5.6)

CABOR

A = a BC = c (C Ba )(D Ba )

C = c A = a B = b (A Bc )(D Ec )

C = c B = b A = a (D Eb , c )

1 1

c1

c1

0 0 0

c0

c0

1 0 1

c0 1

5.3 Context-Specific CPDs

5.3.1 Representation

Example 5.7

Job ... Apply ... SATLetterLetterSAT,SATLetterP (J a , s , l ) = P (J a , s , l )1 1 1 1 1 0

5.3.1.1 Tree-CPDs

Example 5.7 with Tree-CPD (Example 5.8)

J""

ex. P (J a , s , l )J : P (j ) = 0.1, andP (j ) = 0.9

1 1 0

0 1

5.3.1.1 Tree-CPDs

Definition 5.2

1 rooted treettP (X)

tZ Pa Ztarc, Z = z forz V al(Z)t2

X

i i

5.3.1.1 Tree-CPDs

Example 5.9

SAT84treeCPD

0

1 1

5.3.1.1 Tree-CPDs


JobChoiceLetter1Letter2CL L 1 2

5.3.1.1 Tree-CPDs


CPD multiplexer CPD

Definition 5.3

V al(A) = {1, ..., k}P (Y a,Z , ...,Z ) = 1 {Y = Z } CPD P (Y A,Z , ...,Z ) multiplexer CPD

multiplexer CPDex.10Choice

1 k a

1 k

5.3.1.1 Tree-CPDs


CL ,L LJL

1 2

5.3.1.1 Tree-CPDs

CPDCPD

5.3.1.2 Rule CPDs

Definition 5.4 rule

c; pcCp [0, 1]CScope[]

5.3.1.1 Tree-CPDs


CPDP (XPa )CPD

X

5.3.1.2 Rule CPDs

Definition 5.5 rulebased CPD

rulebased CPD P (XPa )R

RScope[] {X} Pa

{X} Pa (x,u)c(x,u)c; p RP (XU) P (xu) = 1CPD

Example 5.11

X

X

X

x

5.3.1.2 Rule CPDs

Example 5.12

XPa = {A,B,C}XCPD

CPD1

X

5.3.1.2 Rule CPDs

Proposition 5.1

BBCPD P (XPa )R R R X,P () = p

X

X

XX+

X+

c;pR,c

5.3.1.2 Rule CPDs

Revisit Example 5.12 with tree-CPD (Example 5.13)

Example 5.12

5.3.1.3 Other Representatioins

decision diagram

5.3 Context-Specific CPDs


"a SL"

contextspecificCPDCPDcontextspecific CPD

0


Revisit Example 5.7 with independence (Example 5.14)

a SATLetter(J S,La )

a , s Letter(J La , s )

0

c0

1 1

c1 1


ctreeCPDX(Pa Scope[c])

Pa = {A,S,L}Scope[c] = {A}, (J S,La )

Scope[c] = {A,S}, (J La , s )

X

J

c0

c1 1


Revisit Example 5.10 with independence (Example 5.15)

JobLetterLetter

ex. (J L c )c 2 1


Revisit Example 5.14 with rule (Example 5.16)

s s a

(J Ls )

XYXcY

1

1

c1


Deifinition 5.6 reduced rule

= c ; pC = ccc cc = c Scope[c ] Scope[c]Scope[c ] Scope[c]creduced rule [c] = c ; p R reduced rule R[c] = {[c] : R, c}


Revisit Example 5.12 with reduced rule (Example 5.17)

Example 5.12R[a ]

a a

1

1 1


Proposition 5.2 reduced rule

RXrulebased CPDR RcY Pa Y Scope[c] = X R[c]Y Scope[] = (X Y Pa Y , c)

exercise 5.4

""CSIY reduced rule exercise 5.7

c

X

c X


Revisit Example 5.7 with CSI-sep (Example 5.18)

IntelligenceIntteligence


A = a S J ,L J dseparation

0


Deifinition 5.7 spurious edge

P (XPa )CPDY Pa cP (XPa )(X Y Pa {Y }, c )Y Xc spurious c = cPa Pa c

CPDreduced rule spurious Y reduced ruleR[c]cY X spurious

X X

X c X

X X


CSIseparation

CSIseparation CSIdseparation

dseparation



S J ,L J spurious aaJIJDdseparatedCSISEPa JIa dseparatedba , s

0 0

1 1


CSIseparationcontextspecific

Theorem 5.3

G PP L (G)cX,Y ,ZXcZYCSIseparatedP (X T Z, c)

exercise 5.8

l

c



C = c L JspuriousJL L (L L J , c )C = c (L L J ,C)CSISEPspurious

12

1 2

1 c 21

21 c 2

5.4 Independence of Causal Influence

noisyor model

generalized linear model

5.4.1 The Noisy-Or Model

Letter

Letter Question Final paper


2 causal mechanism

P (l q , f ) = 0.8

20%

P (l q , f ) = 0.9

10%

1 1 0

1 0 1


0.2 0.1 = 0.02

P (l q , f ) = 0.981 1 1


QF

= P ( q ) = 0.8 = P ( f ) = 0.9

leak probabilityLetter = 0.0001

Q q1 1

F f1 1

0


Definition 5.8 noisyor CPD

Yk2X , ...,X 2P (y X , ...,X ) = (1 ) (1 )P (y X , ...,X ) = 1 [(1 ) (1 )]k + 1 , , ..., PCD P (Y X , ...,X )noisyor

x 1, x 0

P (y X , ...,X ) = (1 ) (1 )

1 k0

1 k 0 i:X =xi i1 i1

1 k 0 i:X =xi i1 i0 1 k

1 k

i1

i0

01 k 0

i=1

i xi


noisyorXP (Y )

a: = 0

b: = 0.50

0

5.4.2 Generalized Linear Models

causal influenceGeneralized Linear ModelsYP (Y X , ...X )

1 k


5.4.2.1 2burdentotal burden ...

f(X , ...,X ) = w X

w

f(X , ...,X )f(X , ...,X ) = w + w X

w =

1 k i=1k

i i

i

1 k

1 k 0 i=1k

i i

0


5.4.2.1 2

sigmoid(z) = 1+ezez


5.4.2.1 2

Definition 5.9 logistic CPD

YkX , ...,X 2P (y X , ...,X ) = sigmoid(w + w X )k + 1w ,w , ...,w CPD P (Y X , ...,X )logistic CPD

wY

2y y O(X) = = = e

X

= = e

1 k1

1 k 0 i=1k

i i

0 1 k

1 k

1 0

P (y X ,...,X )0 1 kP (y X ,...,X )1 1 k

1/(1+e )ze /(1+e )z z z

j

O(X ,x )j j0

O(X ,x )j j1

exp(w + w X )0 ij i i

exp(w + w X +w )0 ij i i j wj


5.4.2.1 2

wlogical CPDb: w = 0, c: w = 5, d: ww 10

bnoisyorw

logistic CPDw X Y

0 0 0

i i


5.4.2.2 Yy , ..., y logistic CPDY"winnertakesall"y 10

Definition 5.10 multinomial logistic PCD

YkX , ...,X mj = 1, ...,ml (X , ...,X ) = w + w XP (y X , ...,X ) =

k + 1w ,w , ...,w CPD P (Y X , ...,X )multinomial logistic CPD

1 n

i

1 k

i 1 k j,0 i=1k

j,i ij

1 k exp(l (X ,...,X ))j =1m

j 1 k

exp(l (X ,...,X ))j 1 k

j,0 j,1 j,k

1 k


5.4.2.2

X ,X 3YX 2

X = x , ...,x X = jX = x 2X , ...,X mX2Ym+ 1P (y X) = sigmoid(w + w 1{X = x })

1 2

i

i i1

im

i i,j i,j1

i,1 i,m

10 j=1

mj

j

chapter5_01chapter5_02

Probabilistic Graphical Models 輪読会 Chapter5

Engineering