Factor Graphs 2005. 5. 20 Young Ki Baik Computer Vision Lab. Seoul National University
Dec 30, 2015
Factor Graph
Computer Vision Lab. SNU
Contents
Introduction
Sum product algorithm
Computing a single marginal function
Computing all marginal function
Probabilistic modeling
Conclusion
Factor Graph
Computer Vision Lab. SNU
Introduction
What is a Factor Graph?
A factor graph shows how a function of several variables can be
factored into a product of “smaller” function.
xxyyxg ),(
,),( 21 yfxfyxg 1 , 21 yyfxxf
x y1f 2f
variables
functions
:Example
Factor Graph
Computer Vision Lab. SNU
Introduction
What is a Factor Graph?
zxfzyfyxfzyxg ,,,),,( 321
variables
functions
:Example
1fx y z2f
3f
Factor Graph
Computer Vision Lab. SNU
Introduction
Why are factor graphs useful?
Factor graphs simplify problems.
Many efficient algorithms can be applied to factor graphs.
Special Feature
Factor graph represent not only variables or constant, but
also functions.
Factor Graph
Computer Vision Lab. SNU
Marginal function
“~{x}” notation
Marginal function
Sum product algorithm
ypyxpxpyxpxy
),(,),(
}{~
1 , ,ix
nii xxgxg variablesofnumber :
index:
n
i
1 32
321}{~
321 , ,:, ,x xx
xxxhxxxh
Object : Get marginal function g(x) using factor graph
Factor Graph
Computer Vision Lab. SNU
Example (Simple Factor Graph)
Let be a function of four variables.
Sum product algorithm
4321 ,,, xxxxg
4321 ,,, xxxxg
423221 ,,, xxfxxfxxf CBA
AfBf
Cf1x 2x 3x
4x
Factor Graph
Computer Vision Lab. SNU
Computing a single marginal function
Example (Simple Factor Graph)
Marginal function for
}{~
432111
1
,,,x
xxxxgxg
}{~ }{~42
}{~3221
423221
423221
1 22
2 43
2 3 4
,,,
,,,
,,,
x xC
xBA
x xC
xBA
x x xCBA
xxfxxfxxf
xxfxxfxxf
xxfxxfxxf
11 xg
Factor Graph
Computer Vision Lab. SNU
Computing a single marginal function
Example (Simple Factor Graph)
Marginal function for
}{~
432122
2
,,,x
xxxxgxg
}{~42
}{~32
}{~21
423221
423221
222
431
1 3 4
,,,
,,,
,,,
xC
xB
xA
xC
xB
xA
x x xCBA
xxfxxfxxf
xxfxxfxxf
xxfxxfxxf
22 xg
Factor Graph
Computer Vision Lab. SNU
Computing a single marginal function
Example (Simple Factor Graph)
Marginal function for and
}{~ 2x
11 xg
1x
}{~ 2x
}{~ 1x
2x
3x 4x
Af
Bf Cf
22 xg
1x
2x
3x 4x
Af Bf Cf
11 xg 22 xg
Bottom-up procedure
Factor Graph
Computer Vision Lab. SNU
Computing all marginal functions
Computing all marginal functions problem
In order to compute all marginal functions ,
we need to calculate single marginal function as much as
n times.
Message passing algorithm
Solution for redundancy problem
nn xgxg 11
AfBf
Cf1x 2x
Afxu 2
2xf Au
messageu :
Factor Graph
Computer Vision Lab. SNU
Message passing algorithm
Let denote the message sent from node x to node
f in the operation.
Let denote the message sent from node f to node
x.
Computing all marginal functions
xu fx
xu xf
Factor Graph
Computer Vision Lab. SNU
Message passing algorithm
Variable to local function
Local function to variable
Computing all marginal functions
}\{ fxnhxhfx xuxu
}{~ }\{
)(x xfny
fyxf yufnfxu
Factor Graph
Computer Vision Lab. SNU
A Detail Example (Message passing algorithm)
The message may be generated in 4 steps.
➀ ~ ➃ are step of message passing algorithm
Algorithms start from each leaves.
Computing all marginal functions
AfBf
Cf
1x 2x3x
4x
➀➀
➀
➁➁
➁
➂➃
➃➂➃ ➂
Factor Graph
Computer Vision Lab. SNU
A Detail Example (Message passing algorithm)
Step 1:
Computing all marginal functions
AfBf
Cf
1x 2x3x
4x
➀ ➀
➀
111 xu
Afx
133 xu
Bfx
144 xu
Cfx
11xu
Afx 33xu
Bfx
44xu
Cfx
}\{ fxnhxhfx xuxu
Variable to local function
Factor Graph
Computer Vision Lab. SNU
A Detail Example (Message passing algorithm)
Step 2:
Computing all marginal functions 4321 ,,, xxxxg
AfBf
Cf
1x 2x3x
4x
➁➁
➁
21}{~
12 ,2
12xxfxuxu A
xfxxf AA
32}{~
32 ,2
32xxfxuxu B
xfxxf BB
42}{~
42 ,2
42xxfxuxu C
xfxxf CC
22xu xf A
22xu xfB
22xu xfC
}{~ }\{
)(x xfny
fyxf yufnfxu
Local function to variable
Factor Graph
Computer Vision Lab. SNU
A Detail Example (Message passing algorithm)
Step 3:
Computing all marginal functions
AfBf
Cf
1x 2x3x
4x
➂
➂
222 222xuxuxu xfxffx CBA
➂
222 222xuxuxu xfxffx CAB
222 222xuxuxu xfxffx BAC
22xu
Afx
22xu
Bfx
22xu
Cfx
}\{ fxnhxhfx xuxu
Variable to local function
Factor Graph
Computer Vision Lab. SNU
A Detail Example (Message passing algorithm)
Step 4:
Computing all marginal functions
AfBf
Cf
1x 2x3x
4x
➃
➃➃
21}{~
21 ,1
21xxfxuxu A
xfxxf AA
32}{~
23 ,3
23xxfxuxu B
xfxxf BB
42}{~
24 ,4
24xxfxuxu B
xfxxf CC
11xu xf A
33xu xfB
44xu xfC
}{~ }\{
)(x xfny
fyxf yufnfxu
Local function to variable
Factor Graph
Computer Vision Lab. SNU
A Detail Example (Message passing algorithm)
Termination
Computing all marginal functions
AfBf
Cf
1x 2x3x
4x
➀➀
➀
➁➁
➁
➂➃
➃➂➃ ➂
111 1xuxg xf A
22222 222xuxuxuxg xffxxf CBA
333 3xuxg xfB
444 4xuxg xfC
Factor Graph
Computer Vision Lab. SNU
Probabilistic Modeling
Markov chain
n
iii xxfxxf
1121 |,,
1xf
1x 2x 3x 4x
12 | xxf 23 | xxf 34 | xxf
4n
Factor Graph
Computer Vision Lab. SNU
Probabilistic Modeling
Hidden markov model
ii
n
iii xyfxxfyyxxf ||,,,,,
112121
1xf 12 | xxf 23 | xxf 34 | xxf
4n
11 | xyf 22 | xyf 33 | xyf 44 | xyf
1x 2x 3x 4x
1y 2y 3y 4y