Authors : Ann Becker and Dan Geiger Presented by : Igor Kviatkovsky

Optimization of Pearl’s Method of Conditioning and Greedy-Like

Approximation Algorithm for the Vertex Feedback Set Problem

Authors: Ann Becker and Dan GeigerPresented by: Igor Kviatkovsky

Outline

• Introduction• The Loop Cutset (LC) problem• Weighted Vertex Feedback Set (WVFS) problem• Reduction from LC to WVFS• MGA (Modified Greedy Algorithm)

– 2-approximation for WVFS

Introduction• Pearl’s method of conditioning is one of the known

inference methods for Bayesian networks• Find a set of vertices such that once the corresponding

variables are instantiated, the remaining network is singly–connected– Pearl’s UPDATE-TREE procedure can be applied

A

L

C

I

D

J

B

M

E

K

Fixing value of A & B

L

C

I

J

M

E

K

D

a b b a

Introduction• Fixing value of A & B & L

breaks all loops. But can we choose less variables to break all loops?

• Are there better variables to choose than others?

• Motivation(Geiger&Becker):– choose the vertices which

break more loops– vertices with higher degree are

more likely to break more loops (vertice J with degree 3 breaks 3 loops!)

A

L

C

I

D

J

B

E

K

M

The Loop Cutset Problem

• Definitions:• The underlying graph G of a directed graph D

is the undirected graph formed by ignoring the directions of the edges in D.

• A loop in D is a subgraph of D whose underlying graph is a cycle.

• A vertex v is a sink with respect to a loop Γ if the two edges adjacent to v in Γ are directed into v .

• Every loop must contain at least one vertex that isn’t a sink with respect to that loop.

sink

The Loop Cutset Problem

• Each vertex that isn’t a sink with respect to a loop Γ is called an allowed vertex with respect to Γ.

• A loop cutset of a directed graph D is a set of vertices that contain at least one allowed vertex with respect to each loop in D.

• A minimum loop cutset of a weighted directed graph D is the one for which the weight is minimum.

The Loop Cutset Problem

• Example– L is a sink with respect to the

loop ACILJDA– L is an allowed vertex with

respect to the loop JLMJ– {A,B,L}, {C,J} are valid loop

cutsets of the graph– Suppose equal size of

variables’ domains (=r)• The number of instances

associated with {C,J} is r2

• The number of instances associated with {A,B,L} is r3

• {C,J} is of lower weight than {A,B,L}

A

L

C

I

D

J

B

E

K

M

Weighted Vertex Feedback Set (WVFS)

• Let G(V,E) be an undirected graph.• Let w: V R+ be a weight function on the vertices of

G.• A vertex feedback set of G is a subset of vertices F

in V such that each cycle in G passes through at least one vertex in F.

• A weight of a set of vertices X is w(X)=ΣvЄX w(v).

• A minimum vertex feedback set of weighted graph G is vertex feedback set F* with the minimum weight.

• WVFS problem is finding minimum vertex feedback set of a given weighted graph G having a weight function w.

Reduction from LC to WVFS• Given a weighted directed graph (D,w), splitting

weighted undirected graph Ds with a weight function ws is constructed– Split each vertex v in D into two vertices vin and vout in

Ds , connect vin and vout .– All incoming edges to v become undirected incident

edges with vin .– All outcoming edges from v become undirected

incident edges with vout .– Set ws(vin)=∞ and ws(vout)=w(v).

VoutV Vin

W(v) ∞ W(v)

V Vin

∞

Vout

W(v)

Γ2

Γ1

C1

C2

Algorithm LC• Ψ(X) is a set obtained by replacing each vertex

vin or vout in X by the respective source vertex v in D

• Algorithm LC– Input: A Bayesian network D.– Output: A loop cutset of D.1. Construct the splitting graph Ds with weight function

ws .2. Apply MGA on (Ds , ws) to obtain a vertex feedback

set F.3. Output Ψ(F).

• One-to-one and onto correspondence between loops in D and cycles in Ds

– MGA 2-approximation for WVFS yields LC 2-approximation!

Algorithm GA (Greedy Algorithm)

• Input: A weighted undirected graph G(V,E,w)• Output: A vertex feedback set F.

– F Ø; i 0;– Repeatedly remove all vertices with degree 0 or 1

from V and their adjacent edges from E and insert the resulting graph into G1 .

– While Gi isn’t the empty graph, do:1. Pick a vertex vi for which w(vi)/d(vi) is minimum in Gi

2. F F U {vi}

3. V V \{vi}4. i i + 15. Repeatedly remove all vertices with degree 0 or 1 from

V and their adjacent edges from E and insert the resulting graph into Gi .

Algorithm MGA (Modified GA)– F ' Ø; i 0;– Repeatedly remove all vertices with degree 0 or

1 from V and their adjacent edges from E and insert the resulting graph into G1 .

– While Gi isn’t the empty graph, do:1. Pick a vertex vi for which w(vi)/d(vi) is minimum in Gi

2. F’ F’ U {vi}3. V V \{vi}4. i i + 15. Repeatedly remove all vertices with degree 0 or 1 from

V and their adjacent edges from E and insert the resulting graph into Gi . For every edge e=(u1,u2) removed in this process do:– C(e) w(vi)/d(vi); – w(u1) w(u1) – C(e); – w(u2) w(u2) – C(e);

MGA (Phase 2)• Phase 2

– F F ‘– For i = |F ’| to 1 do

• If every cycle in G that intersects with {vi} also intersects with F \{vi} then,

– F F \{vi}

• After applying phase 2 on the vertex feedback set, all redundant vertices are removed and we get a minimal vertex feedback set

• Before applying phase 2 – 4-approximation

• After applying phase 2 – 2-approximation

MGA - Motivation• In each iteration remove all the isolated nodes

and all the chains– Isolated nodes and chains don’t break singly-

connectedness

• Pick a node with the minimum cost w(vi)/d(vi) to vertex feedback set– Small w(vi) means, it’s worthy taking node vi into the

vertex feedback set since it has a low weight

– Large d(vi) means, it’s worthy taking node vi into the vertex feedback set since it might break a high number of loops (has a high degree), even if it’s weight w(vi) is large

MGA - Motivation• Subtract the effective cost (cost paid per edge)

from the neighboring nodes weights.

• Example:

v2

v4v5

v3

v1

w1=1

c=0.5v2

v4v5

v3

v1

C(e)=0.5

5

v2

v4v5

v3

v1 C(e)=1.25

G1 G2 G3

w1=3

c=1w1=10

c=5

w1=10

c=5

w1=10

c=10/3

w2=9.5

c=9.5/2

w2=2.5

c=2.5/2 w2=10

c=5

w2=10

c=5

F ‘={v1}

F ‘={v1,v2}

F=F ’

C(e)=0.5

C(e)=0.5

C(e)=0.5

C(e)=1.25

C(e)=1.25

C(e)=1.25

F ’=Φ

Performance and Summary

• Theoretical approximation ratio of MGA is 2 (the worst case)

• Average approximation ratio of MGA on randomly generated graphs is 1.22

• Before running the computations of conditioning method, the optimal complexity of its running time can be computed with high precision using MGA

Extra slides – MGA Analysis and approximation ratio proof

MGA - Analysis• F* - a minimum vertex feedback set of G(V,E,w).

• Vertices in F ‘ are {v1,v2,…,vt} when vi are indexed in the order they inserted into F ’.

• wi(v) and di(v) are weight and degree respectively of vertex v in Gi .

• Vi is a set of vertices of Gi .

• Let • Theorem 6 (proved later):

* * and i i i iF F V F F V

*

( ) 2 ( )i i

i iv F v F

d v d v

MGA - Analysis

• Let Γ1(v) be the set of edges in G1 for which at least one endpoint is v.

• From the description of the algorithm for every vertex v

in G1 , , if vЄF

1 ( )

( ) ( )e v

C e w v

1 ( )

( ) ( )e v

C e w v

Theorem 3

• Theorem 3: Algorithm MGA always outputs a vertex feedback set F whose weight is no more than twice the weight of a minimum vertex feedback set F*.

• Proof:– Let i=j+1, then:

( ) ( ) ( ) ( )

( ) ( ) ( ) 1 ( )j j j i i i j i i

j j ij j j i i i i i

w v w v w v c w vc c c

d v d v d v d v

*( ) 2 ( )w F w F

Theorem 3 (Proof)

– By grouping edges according to iterations they are assigned a weight:

– Let

– By definition:

1

1

1( ) 1

( ) ( ) ( ( ) ( ))i

i

i i i j j i j ie v j

C e c d v c d v d v

1 if and 0 if i i i iv F v F

1

1

11 1 ( ) 1 1

( ) ( ) ( ) ( ) ( ( ) ( ))i

t t t i

i i i i i i i j j i j ii i e v i j

w F w v C e c d v c d v d v

1 1 11 2

( ) ( ) ( ) ( )t t t

i i i i j i ji i j i

w F c d v c c d v

( ) ( )i

t

j i j ij i v F

d v d v

*( ) 2 ( )w F w F

1 ( )

( ) ( )i

ie v

C e w v

Theorem 3 (Proof)

1

* *1

* *1

1 1 12

1 1 12

1

11 1

( ) ( ) ( ) ( )

2 ( ) 2( ) ( )

2 ( ) 2 ( )

i

i

i i

t

i i iv F i v F

t

i i iiv F v F

t t

i i i ii iv F v F

w F c d v c c d v

c d v c c d v

c d v c d v

* * * *1 1 1

* * * *1 1

1

11 1 1\

1

11 \

2 ( ) 2 ( ) 2 ( )

2 ( ) ( ( ) ( )) ( )

i i i i

i i i t

t t t

i i i i i ii i iv F F v F v F

t

i i i i i t ti v F F v F v F

c d v c d v c d v

c d v c d v d v c d v

* *1

*

( )

2 ( ) 2 ( ) 2 ( ).e vv F v F

C e w v w F

*( ) 2 ( )w F w F

Theorem 6:

Grouping edges by iterations they are assigned a weight

*

( ) 2 ( )i i

i iv F v F

d v d v

Theorem 6

• Definitions:– Let

– Let dX(v) be the number of edges whose one endpoint is v and the other is a vertex in X.

• Theorem 6: Let G be a weighted graph for which every vertex has a degree strictly greater than 1, F be a minimal vertex feedback set of G and F* be an arbitrary vertex feedback set of G (possibly a minimum weight one), then:

*

( ) 2 ( )v F v F

d v d v

* *\ and \F V F F V F

v XdX(v)=3

Theorem 6 (Proof)• To prove the theorem l.h.s. is divided to 2 terms and

an upper bound for each term is provided.

• Lemma 7: Let G,F and F* be defined as above. Then,

• Proof:– For every set of vertices B in G

– Since d(v) ≥ 2 for each v in G

( ) 2 ( ( ) 2)v F v F

d v F d v

*

( ) 2 ( )v F v F

d v d v

* *2 ( ) 2 2v F

F d v F F F F

* * *

\

( ) 2 ( ) ( ) 2 2 ( ) \v F v F B v F B

d v F F d v d v F F B F F B

*

\

( ) 2 \ 2 ( ) \v F B

d v F B F F B

F

BF*

≥0

Theorem 6 (Lemma 7)

– Since dB(v) ≤ d(v),

– We have to prove that the following hold for some set of vertices B

* * * *2 ( ) 2 2 ( ) 2 2Bv F B v F

F d v F F B F F B d v F F F F

* * * *

* * * *

* *

( ) 2 2 2 2

( ) 2 2

( ) 2 2 ( ( ) 2) 2

Bv F B

Bv F B

B Bv F B v F B

d v F F B F F B F F B F F B

d v F F B F F B F F B F F B

d v F B F B d v F B

*2 ( ( ) 2) 2Bv F B

F d v F B

F

BF*

* *2 ( ) 2 2v F

F d v F F F F

Lemma 7 – Let’s define a set B for which this inequality can be

proven.– F is minimal

• Each vertex in F can be associated with a cycle in G that contains no other vertices of F.

• We define a graph H that consists of the union of these cycles (one cycle per each vertex).

• Definition: A linkpoint is a vertex with degree 2. A branchpoint is a vertex with degree larger than 2.

• Every vertex in F is a linkpoint in H.• Let B the vertices of H.

*2 ( ( ) 2) 2Bv F B

F d v F B

linkpoint branchpoint

F

BF*

Lemma 7

– The proof is constructive• We apply the following procedure on H and showing that

there are terms on the r.h.s. that contribute 2 for each vєF , and weren’t used for other vєF .

– Pick a vertex vєF and follow the two paths p1 and p2 in H from it until the first branchpoint on each path is found.

– There are 3 cases to consider:• Case 1:

– From definition of H:

– dB(b1)-2 ≥ 1, dB(b2)-2 ≥ 1

– There are terms to contribute 2 to r.h.s.

*2 ( ( ) 2) 2Bv F B

F d v F B

1 2,b b Fv

p1 b1

b2

p2

Lemma 7

• Case 2:– If dB(b1)≥ 4

» dB(b1)-2 ≥ 2– If dB(b1)= 3

» dB(b1)-2 = 1» dB(b2)-2 ≥ 1

– There are terms to contribute 2 to r.h.s.• Case 3:

– Isolated cycle– There exists a vertex in F* , that

resides on no other cycle of H.» There are maximum such

cases – There are terms to contribute 2 to

r.h.s.

*2 ( ( ) 2) 2Bv F B

F d v F B

*F B

p3 b2

v b1

p2

p1

v

p1

b1

p2

v

p1

b1

p2

Lemma 7

– Remove the paths p1 and p2 from H obtaining a graph in which still each vertex in F resides on a cycle that contains no other vertices of F.

– Continue till F is exhausted.

*2 ( ( ) 2) 2Bv F B

F d v F B

• Lemma 8: Let G,F and F* be defined as above. Then,

• Proof: Note that

Lemma 7

* *

* * *

( ( ) 2) ( ) ( ( ) 2) ( ( ) 2)F F

v F v F F v F F v F F

d v d v d v d v

* *

* * *

( ( ) 2) ( ) ( ( ) 2) ( ( ) 2)F F

v F v F F v F F v F F

d v d v d v d v

F*

F v2

v4v5

v3

v1

)2-2)+(3-2=(1 =1 -=1=1

Lemma 8

• So if we prove,

then we prove the lemma• The graph induced by is a forest, and since the

number of edges in a forest is smaller than the number of vertices, the following holds

* *

* * *

( ( ) 2) ( ) ( ( ) 2) ( ( ) 2)F F

v F v F F v F F v F F

d v d v d v d v

*F

* *

* *

( ( ) 2) ( ( ) 2) 0F F

v F F v F F

d v d v

*

*

*

*

* *

* *

*( ) / 2

( ( ) 2) 0

( ( ) 2) ( ( ) 2) 0.

Fv F

Fv F

F Fv F F v F F

d v F

d v

d v d v

• To complete the proof , we use bounds obtained from lemmas 7 and 8:

Theorem 6 (Proof)

*

*

*

* *

* *

* * *

*

* * *

* *

( ) 2 ( ( ) 2)

( ) 2 2 ( )

( ( ) 2) ( ( ) 2)

( ) ( ) ( ) ( )

( ) ( ) 2 ( ).

v F v F

Fv F v F F

Fv F F v F F

F Fv Fv F F v F F v F F

Fv F v F v F

d v F d v

d v F F F F d v

d v d v

d v d v d v d v

d v d v d v

*

( ) 2 ( )v F v F

d v d v