CSC2105-Graph Algorithm.ppt

CSC2105: AlgorithmsCSC2105: AlgorithmsGreedy Graph AlgorithmGreedy Graph Algorithm

Mashiour RahmanMashiour [email protected]@aiub.edu

American International University BangladeshAmerican International University Bangladesh

CSC4226: Algorithms Mashiour Rahman Graph Algorithm2

Spanning TreeSpanning Tree

• A A spanning tree spanning tree of of G G is a subgraph whichis a subgraph which– is a treeis a tree– contains all vertices of contains all vertices of GG

How many edges are there in a spanning tree, if there are V vertices?


Minimum Spanning TreesMinimum Spanning Trees

• Undirected, connected graph Undirected, connected graph

GG = ( = (VV,,EE))

• WeightWeight function function WW: : EE

(assigning cost or length or other (assigning cost or length or other

values to edges)values to edges)

Spanning tree: tree that connects all

vertices

Minimum spanning tree (MST):

spanning tree T that minimizes( , )

( ) ( , )u v T

w T w u v


Optimal SubstructureOptimal Substructure

• Rationale:Rationale:– If If G’G’ would have a cheaper would have a cheaper STST T’T’, , then we would get then we would get

a cheaper a cheaper STST of of GG: : T’T’ + (+ (uu, , vv))

MST(G ) = T u

v”u +v”

MST(G’ ) = T – (u,v )


Idea for an AlgorithmIdea for an Algorithm

• We have to make We have to make V–V–11 choices ( choices (edgesedges of the of the MSTMST) to arrive at the ) to arrive at the optimization goal.optimization goal.

• After each choice we have a After each choice we have a sub-problemsub-problem that that is is one vertex smallerone vertex smaller than the original problem. than the original problem.– A dynamic programming algorithm would consider all A dynamic programming algorithm would consider all

possible choices (edges) at each vertex.possible choices (edges) at each vertex.– Goal: at each vertex cheaply determine an edge that Goal: at each vertex cheaply determine an edge that

definitely belongs to an MST.definitely belongs to an MST.


Greedy ChoiceGreedy Choice

• Greedy choice propertyGreedy choice property: locally optimal : locally optimal (greedy) choice yields a globally optimal (greedy) choice yields a globally optimal solution.solution.

• Theorem on Greedy choice for MSTTheorem on Greedy choice for MST– Let Let GG=(=(VV, , EE)) and and SS VV

– SS is a is a cutcut of of GG (it splits (it splits GG into parts into parts SS and and V-SV-S))

– ((uu,,vv) is a ) is a lightlight edge if it is a edge if it is a minmin-weight edge of -weight edge of GG that connects that connects SS and and V-SV-S

– Then (Then (uu,,vv) belongs to) belongs to aa MSTMST TT of of GG..


...Greedy Choice...Greedy Choice• ProofProof

– Suppose (Suppose (uu,,vv) is light but () is light but (uu,,vv) ) any any MSTMST

– look at path from look at path from uu to to vv in some in some MSTMST T T

– Let (Let (xx, , yy) be the first edge on a path from ) be the first edge on a path from uu to to vv in in TT that crosses that crosses

from from SS to to VV––SS. . Swap (Swap (xx, , yy) with () with (uu,,vv) in ) in TT..

– this improves cost of this improves cost of TT contradictioncontradiction ( (TT is supposed to be an is supposed to be an

MSTMST))

uu vv

xxyy

SS V-SV-S


Generic MST AlgorithmGeneric MST Algorithm

Generic-MSTGeneric-MST(G, w)(G, w)11 A := A := // Contains edges that belong to a MST// Contains edges that belong to a MST2 2 while while A does not form a spanning tree A does not form a spanning tree dodo3 Find an edge (u,v) that is safe for A 3 Find an edge (u,v) that is safe for A 44 A := A := A A {(u,v)}{(u,v)}5 5 return return A A

Generic-MSTGeneric-MST(G, w)(G, w)11 A := A := // Contains edges that belong to a MST// Contains edges that belong to a MST2 2 while while A does not form a spanning tree A does not form a spanning tree dodo3 Find an edge (u,v) that is safe for A 3 Find an edge (u,v) that is safe for A 44 A := A := A A {(u,v)}{(u,v)}5 5 return return A A

A safe edge is an edge that does not destroy A’s property.

MoreSpecific-MST(G, w)1 A := // Contains edges that belong to a MST2 while A does not form a spanning tree do3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S

4 A := A {(u,v)}5 return A

MoreSpecific-MST(G, w)1 A := // Contains edges that belong to a MST2 while A does not form a spanning tree do3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S

4 A := A {(u,v)}5 return A


Prim-Jarnik AlgorithmPrim-Jarnik Algorithm

• Vertex based algorithmVertex based algorithm

• Grows a single Grows a single MST MST TT one vertex at a time one vertex at a time

• The set The set AA covers the portion of covers the portion of TT that was that was already already

computedcomputed

• Annotate all vertices Annotate all vertices vv outside of the set outside of the set AA with with vv.key.key the the

minimum weightminimum weight of an edge that connects of an edge that connects vv to a vertex to a vertex

in in AA

((vv.key.key = = if no such edge exists) if no such edge exists)


MST-PrimMST-Prim(G, s)(G, s)0101 forfor eacheach vertex u vertex u G. G.VV02 u.02 u.key key :=:= 03 u.03 u.predpred := NIL := NIL04 s.04 s.keykey := 0 := 005 05 initinit(Q, G.(Q, G.VV) // Q is a ) // Q is a priority queuepriority queue06 06 whilewhile notnot isEmptyisEmpty(Q)(Q)07 u := 07 u := extractMinextractMin(Q) // add u to T(Q) // add u to T08 08 forfor eacheach v v u. u.adjadj dodo09 09 ifif v v Q Q andand ww(u,v) < v.(u,v) < v.keykey thenthen1010 v.v.keykey := := ww(u,v)(u,v)11 11 modifyKeymodifyKey(Q,v)(Q,v)12 v.12 v.predpred := u := u

...Prim-Jarnik Algorithm...Prim-Jarnik Algorithm

updating keys


Prim-Jarnik’s ExamplePrim-Jarnik’s Example

NodeNode PredPred keykey

A A NILNIL 00

B B NILNIL

C C NILNIL

D D NILNIL

E E NILNIL

F F NILNIL

G G NILNIL

H H NILNIL

I I NILNIL

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

MST-Prim(Graph,A)MST-Prim(Graph,A)

AA

QQ

NodeNode

PredPred

KeyKey


...Prim-Jarnik’s Example...Prim-Jarnik’s Example

H

B C

IA

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


BB AA 44

HH AA 88

CC NILNIL

D D NILNIL

EE NILNIL

FF NILNIL

G G NILNIL

I I NILNIL

AA

QQ

NodeNode AA

PredPred NILNIL

KeyKey 00


NodeNode AA BB

PredPred NILNIL AA

KeyKey 00 44

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


HH AA 88

CC BB 88

FF NILNIL

EE NILNIL

F F NILNIL

GG NILNIL

I I NILNIL AA

QQ



NodeNode AA BB HH

PredPred NILNIL AA AA

KeyKey 00 44 88

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


G G HH 11

I I HH 66

C C BB 88

D D NILNIL

E E NILNIL

F F NILNIL AA

QQ



NodeNode AA BB HH GG

PredPred NILNIL AA AA HH

KeyKey 00 44 88 11

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


FF GG 33

I I GG 55

C C BB 88

D D NILNIL

E E NILNIL

AA

QQ



NodeNode AA BB HH GG FF

PredPred NILNIL AA AA HH GG

KeyKey 00 44 88 11 33

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


C C FF 44

II GG 55

E E FF 1010

D D FF 1313

AA

QQ



NodeNode AA BB HH GG FF CC

PredPred NILNIL AA AA HH GG FF

KeyKey 00 44 88 11 33 44

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


II CC 33

D D CC 66

E E FF 1010

AA

QQ



NodeNode AA BB HH GG FF CC II

PredPred NILNIL AA AA HH GG FF CC

KeyKey 00 44 88 11 33 44 33

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


DD CC 66

E E FF 1010

AA

QQ



NodeNode AA BB HH GG FF CC II DD

PredPred NILNIL AA AA HH GG FF CC CC

KeyKey 00 44 88 11 33 44 33 66

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


EE DD 99

AA

QQ



NodeNode AA BB HH GG FF CC II DD EE

PredPred NILNIL AA AA HH GG FF CC CC DD

KeyKey 00 44 88 11 33 44 33 66 99

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10


AA

Q is emptyQ is empty



NodeNode AA BB HH GG FF CC II DD EE Total Total CostCost

PredPred NILNIL AA AA HH GG FF CC CC DD

KeyKey 00 44 88 11 33 44 33 66 99 3838

H

B C

I

A

D

G

F E

1

8

43

6

3

4 9

NodeNode PredPred KeyKey

AA

Q is emptyQ is empty



MST-PrimMST-Prim(G,r)(G,r)0101 forfor eacheach vertex u vertex u G. G.VV02 u.02 u.key key :=:= 03 u.03 u.parentparent := NIL := NIL04 r.04 r.keykey := 0 := 005 05 initinit(Q, G.(Q, G.VV) // Q is a ) // Q is a priority queuepriority queue06 06 whilewhile notnot isEmptyisEmpty(Q)(Q)07 07 u := u := extractMinextractMin(Q) // add u to T(Q) // add u to T08 08 forfor eacheach v v u. u.adjadj dodo09 09 ifif v v Q Q andand ww(u,v) < v.(u,v) < v.keykey thenthen1010 v.v.keykey := := ww(u,v)(u,v)11 11 modifyKeymodifyKey(Q,v)(Q,v)12 v.12 v.parentparent := u := u

Implementation IssuesImplementation Issues


Priority QueuesPriority Queues• A priority queue maintains a set A priority queue maintains a set SS of elements, each with of elements, each with

an associated an associated keykey value value..

• We need We need PQPQ to support the following operations to support the following operations

– initinit( ( QQ: : PriorityQueuePriorityQueue, , SS: : VertexSetVertexSet) )

– extractMinextractMin( ( QQ: : PriorityQueuePriorityQueue): ): VertexVertex

– modifyKeymodifyKey( ( QQ: : PriorityQueuePriorityQueue, , vv: : VertexVertex) )

• To choose how to implement a To choose how to implement a PQPQ, we need to count , we need to count

how many times the operations are performed:how many times the operations are performed:

– initinit is performed just once and runs in is performed just once and runs in OO((nn))


Prim-Jarnik’s Running TimePrim-Jarnik’s Running Time

• TimeTime = | = |VV|*|*TT((extractMinextractMin) + ) + OO((EE)*)*TT((modifyKeymodifyKey))

QQ T(T(extractMinextractMin)) T(T(modifyKeymodifyKey)) TotalTotal

array array OO((VV)) OO(1)(1) OO((VV22))

binary heapbinary heap OO(log(log V V)) OO(log (log VV)) OO((EE log logVV))

• E E V-1, E < V V-1, E < V22, E = O(V, E = O(V22))• Binary heap implementation: Binary heap implementation:

– Time = Time = OO(V(V loglogV V + + E E loglogVV) = ) = OO(V(V22 loglogVV) = ) = OO((E E loglogVV) )


Kruskal's AlgorithmKruskal's Algorithm

• Edge based algorithmEdge based algorithm

• Add edges one at a time in increasing Add edges one at a time in increasing weight order.weight order.

• The algorithm maintains The algorithm maintains AA:: a a forest of forest of treestrees. .

• An edge is accepted if it connects An edge is accepted if it connects vertices of vertices of distinct treesdistinct trees (the cut (the cut respects respects AA).).


...Kruskal's Algorithm...Kruskal's Algorithm

• The algorithm keeps adding the cheapest edge The algorithm keeps adding the cheapest edge that connects two trees of the forestthat connects two trees of the forest

MST-Kruskal(G)01 A := 02 init(S) // Init disjoint-set03 for each vertex v G.V04 addSingletonSet(S,v)05 sort the edges of G.E by non-decreasing w(u,v)06 for each edge (u,v) E in sorted order do07 if findSet(S,u) findSet(S,v) then08 A := A {(u,v)}09 unionSets(S,u,v)10 return A


S = { A, B, C, D, E, F, G, H, I }S = { A, B, C, D, E, F, G, H, I }E’ = { HG-1, CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }E’ = { HG-1, CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

Kruskal’s ExampleKruskal’s Example

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { A, B, C, D, E, F, S = { A, B, C, D, E, F, GG, , HH, I }, I }E’ = { E’ = { HG-1HG-1, CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }


...Kruskal’s Example...Kruskal’s Example

H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = {S = { A, B, A, B, CC, D, E, F,, D, E, F, GHGH,, II } }E’ = { E’ = { CI-3CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }


H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { A, B, S = { A, B, CICI, D, E, , D, E, FF, , GGH }H }E’ = { E’ = { GF-3GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { A, B, S = { A, B, CCI, D, E, I, D, E, FFGHGH } }E’ = { E’ = { CF-4CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { S = { AA, , BB, , CFGHICFGHI, D, E }, D, E }E’ = { E’ = { AB-4AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { S = { ABAB, , CCFGHI, FGHI, DD, E }, E }E’ = { HI-6, E’ = { HI-6, CD-6CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { AS = { ABB, , CCDFGHIDFGHI, E }, E }E’ = { E’ = { BC-8BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }, AH-8, DE-9, EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { S = { ABCABCDDFGHIFGHI,, EE } }E’ = { AH-8, E’ = { AH-8, DE-9DE-9, EF-10, BH-12, DF-13 }, EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

68

4

12

8

3

6

5 3

4 13 9

10

S = { S = { ABCDEFGHIABCDEFGHI } }E’ = { EF-10, BH-12, DF-13 }E’ = { EF-10, BH-12, DF-13 }



H

B C

I

A

D

G

F E

1

48

3

6

3

4 9

S = { S = { ABCDEFGHIABCDEFGHI } }E’ = { E’ = { HG-1, CI-3, GF-3, CF-4, AB-4, CD-6, BC-8, DE-9 HG-1, CI-3, GF-3, CF-4, AB-4, CD-6, BC-8, DE-9 }; }; Total Cost := 38Total Cost := 38



Implementation Factors:Implementation Factors:Disjoint SetsDisjoint Sets

• We need to maintain a disjoint partitioning of a We need to maintain a disjoint partitioning of a set, i.e., a collection set, i.e., a collection SS of disjoint sets. of disjoint sets. Operations:Operations:– addSingeltonSetaddSingeltonSet((SS::SetSet, , xx::VertexVertex))

• SS := := SS {{ {{xx}}}}

– findSetfindSet((SS::SetSet, , xx::VertexVertex): ): SetSet• returns returns XX SS such that such that xx XX

– unionSetsunionSets((SS::SetSet, , xx::VertexVertex, , yy::VertexVertex))• XX := := findSetfindSet((SS::SetSet, , xx))• YY := := findSetfindSet((SS::SetSet, , yy) ) • SS := := SS – {– {XX, , YY} } { {XX YY}}


Implementation Factors:Implementation Factors:Disjoint Sets as ListsDisjoint Sets as Lists

• Each set is a list of elements identified by the first Each set is a list of elements identified by the first element, all elements in the list point to the first elementelement, all elements in the list point to the first element

• UnionSets: add a shorter list to a longer one, UnionSets: add a shorter list to a longer one, OO(min{|(min{|C(u)|, |C(C(u)|, |C(vv)|}) )|})

• MakeSet/FindSet: MakeSet/FindSet: OO(1)(1)

1 2 3 A B C

4

1 2 3 A B C

4


Kruskal Running TimeKruskal Running Time

• Initialization Initialization OO((VV) time) time

• Sorting the edges Sorting the edges ((E E loglog E E) = ) = ((E E loglog V V))

• OO((EE) calls to FindSet ) calls to FindSet

• Union costsUnion costs– Let Let t(v) t(v) be the number of times be the number of times v v is moved to a new clusteris moved to a new cluster

– Each time a vertex is moved to a new cluster the size of the Each time a vertex is moved to a new cluster the size of the cluster containing the vertex at least doubles: cluster containing the vertex at least doubles: t(v) t(v) log log VV

– Total time spent doing UnionTotal time spent doing Union

• Total time: Total time: OO((EE log log VV) )

( ) logv V

t v V V

CSC2105-Graph Algorithm.ppt

Documents