Minimum Spanning Tree Algorithms Prim’s Algorithm Kruskal’s Algorithm Runtime Analysis Correctness Proofs.

Minimum Spanning Tree Algorithms

Prim’s Algorithm

Kruskal’s Algorithm

Runtime Analysis

Correctness Proofs

Copyright 1999 by Cutler/Head

Greedy/Prims 2

What is A Spanning Tree?

u

v

b

a

c

d

e

f

• A spanning tree for an undirected graph G=(V,E) is a subgraph of G that is a tree and contains all the vertices of G

• Can a graph have more than one spanning tree?

• Can an unconnected graph have a spanning tree?


Greedy/Prims 3

Minimal Spanning Tree.

4 4

3

2

9

15

8

1014

3

u

v

b

a

c

d

e

f

Mst T: w( T )= (u,v) T w(u,v ) is minimized

• The weight of a subgraph is the sum of the weights of it edges.

• A minimum spanning tree for a weighted graph is a spanning tree with minimum weight.

• Can a graph have more then one minimum spanning tree?


Greedy/Prims 4

Example of a Problem that Translates into a MST

The Problem• Several pins of an electronic circuit must be

connected using the least amount of wire.

Modeling the Problem • The graph is a complete, undirected graph

G = ( V, E ,W ), where V is the set of pins, E is the set of all possible interconnections between the pairs of pins and w(e) is the length of the wire needed to connect the pair of vertices.

• Find a minimum spanning tree.


Greedy/Prims 5

Greedy Choice

We will show two ways to build a minimum spanning tree.

• A MST can be grown from the current spanning tree by adding the nearest vertex and the edge connecting the nearest vertex to the MST. (Prim's algorithm)

• A MST can be grown from a forest of spanning trees by adding the smallest edge connecting two spanning trees. (Kruskal's algorithm)


Greedy/Prims 6

Notation

• Tree-vertices: in the tree constructed so far• Non-tree vertices: rest of vertices

Prim’s Selection rule

• Select the minimum weight edge between a tree-node and a non-tree node and add to the tree


Greedy/Prims 7

The Prim algorithm Main Idea

Select a vertex to be a tree-node

while (there are non-tree vertices) { if there is no edge connecting a tree node with a non-tree node

return “no spanning tree”

select an edge of minimum weight between a tree node and a non-tree node

add the selected edge and its new vertex to the tree}

return tree

6 4

5

2

158

1014

3

u

v

b

a

c

d

f


Greedy/Prims 8

Implementation Issues

• How is the graph implemented?

– Assume that we just added node u to the tree.

– The distance of the nodes adjacent to u to the tree may now be decreased.

– There must be fast access to all the adjacent vertices.

– So using adjacency lists seems better

• How should the set of non-tree vertices be represented?

– The operations are:

• build set

• delete node closest to tree

• decrease the distance of a non-tree node from the tree

• check whether a node is a non- tree node


Greedy/Prims 9


• How should the set of non-tree vertices be represented?

– A priority queue PQ may be used with the priority D[v] equal to the minimum distance of each non-tree vertex v to the tree.

– Each item in PQ contains: D[v], the vertex v, and the shortest distance edge (v, u) where u is a tree node

• This means:

– build a PQ of non-tree nodes with initial values -

• fast build heap O (V )

• building an unsorted list O(V)

• building a sorted list O(V) (special case)


Greedy/Prims 10


– delete node closest to tree (extractMin)• O(lg V ) if heap and • O(V) if unsorted list • O(1) sorted list

– decrease the distance of a non-tree node to the tree

– We need to find the location i of node v in the priority queue and then execute (decreasePriorityValue(i, p)) where p is the new priority

– decreasePriorityValue(i, p)

• O(lg V) for heap,

• O(1) for unsorted list

• O(V ) for sorted list (too slow)


Greedy/Prims 11


• What is the location i of node v in a priority queue?

– Find in Heaps, and sorted lists O(n)

– Unsorted – if the nodes are numbered 1 to n and we use an array where node v is the v item in the array O(1)

Extended heap

– We will use extended heaps that contain a “handle” to the location of each node in the heap.

– When a node is not in PQ the “handle” will indicate that this is the case

– This means that we can access a node in the extended heap in O(1), and check v PQ in O(1)

– Note that the “handle” must be updated whenever a heap operation is applied


Greedy/Prims 12


2. Unsorted list

– Array implementation where node v can be accesses as PQ[v] in O(1), and the value of PQ[v] indicates when the node is not in PQ.


Greedy/Prims 13

Lines 1-5 initialize the priority queue PQ to contain all Vertices. Ds for all vertices except r, are set to infinity.

r is the starting vertex of the TThe T so far is empty

Add closest vertex and edge to current T

Get all adjacent vertices v of u,update D of each non-tree vertex adjacent to u

Store the current minimum weight edge, and updated distance in the priority queue

Prim’s Algorithm

1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-heap(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u )

// execute relaxation 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v )) 14. return T // T is a mst.


Greedy/Prims 14

Prim’s AlgorithmInitialization

Prim (G )


4. PQ make-heap(D,V, {})//No edges 5. T


Greedy/Prims 15

Building the MST

// solution check

7. while PQ do//Selection and feasibility 8. (u,e ) PQ.extractMin()

// T contains the solution so far . 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v) ) 14. return T


Greedy/Prims 16

Using Extended Heap implementation

Lines 1 -6 run in O (V )

Max Size of PQ is | V |

Count7 =O (V )

Count7(8) = O (V ) O( lg V )

Count7(10) = O(deg(u ) ) = O( E )

Count7(10(11)) = O(1)O( E )

Count7(10(11(12))) = O(1) O( E )

Count7(10(13)) = O( lg V) O( E ) Decrease- Key operation on the extended heap can be implemented in O( lg V)

So total time for Prim's Algorithm is O ( V lg V + E lg V )

What is O(E ) ?Sparse Graph, E =O(V) , O (E lg V)=O(V lg V )

Dense Graph, E=O(V2), O (E lg V)=O(V2 lg V)

Time Analysis1. for each u V2. do D [u ] 3. D[ r ] 0 4. PQ make-PQ(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v)) 15. return T // T is a mst.Assume a node in PQ can be accessed in O(1)

** Decrease key for v requires O(lgV ) provided the node in heap with v’s data can be accessed in O(1)


Greedy/Prims 17

Using unsorted PQ

Lines 1 - 6 run in O (V )

Max Size of PQ is | V |Count7 = O (V )

Count7(8) = O (V ) O(V )

Count7(10) = O(deg(u ) ) = O( E )

Count7(10(11)) = O(1)O( E )

Count7(10(11(12))) = O(1) O( E )

Count7(10(13)) =O( 1) O( E )

So total time for Prim's Algorithm is O (V + V2 + E ) = O (V2 )

For Sparse/Dense graph : O( V2 )Note growth rate unchanged for adjacency

matrix graph representation

Time Analysis


4. PQ make-PQ(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v))

15. return T // T is a mst.


Greedy/Prims 18

handle

ABC

123

Prim - extended HeapAfter Initialization

T PQ

0, (A, {})

, (B, {}) , (C, {})

1

2 3

A

B

C

2

56

G

Prim (G, r)1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-heap(D,V, { })5. T A

BC

G

A 2

A 6

B 2 C 6

C 5

B 5


Greedy/Prims 19

Prim - extended Heap Build tree - after PQ.extractMin

handle

ABC

Null21

T(A, {})

PQ

, (C, {})

, (B, {})

1

2

A

C

2

56

G

7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T

B


Greedy/Prims 20

Update B adjacent to A

handle

ABC

Null12

T(A, {})

PQ

2, (B, {A, B})

, (C, {})

1

2

A

C

2

56

G

10. for each v Adjacent (u ) 11. // relaxation operation

// relaxation11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v))

B


Greedy/Prims 21

Update C adjacent to A

handle

ABC

Null12

T(A, {})

PQ

2, (B, {A, B})

6, (C, {A, C})

1

2A

B

C

2

56

G


Greedy/Prims 22

Build tree - after PQ.extractMin

handle

ABC

NullNull1

T(A, {})(B, {A, B})

PQ

6, (C, {A, C})1

A

B

C

2

56

G



Greedy/Prims 23

Update C adjacent to B

handle

ABC

NullNull1

T(A, {})

PQT(A, {})(B, {A, B})

5, (C, {B, C})1


A

B2

56

GC


Greedy/Prims 24


handle

ABC

NullNullNull

T(A, {})

PQT(A, {})(B, {A, B}) (C, {B, C})


A

B2

56

GC


Greedy/Prims 25

Prim - unsorted listAfter Initialization

T PQ

A BA

B

C

12

54

G

0, (A, {}) , (B, {}) , (C, {})

C

Prim (G, r)1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-PQ(D,V, { })5. T

ABC

G

A 12

A 4

B 12 C 4

C 5

B 5


Greedy/Prims 26


T(A, {})

PQB

, (B, {}) , (C, {})

C

Null

A

B

C

12

54

G

A



Greedy/Prims 27

Update B, C adjacent to A

T(A, {})

PQB

12, (B, {A, B}) 4, (C, {A, C})

C

Null

A

B

C

12

54

G

A



Greedy/Prims 28


T(A, {})(C, {A, C})

PQ

B

Null12, (B, {A, B})

C

Null

AB

C

12

54

G

A



Greedy/Prims 29

Update B adjacent to C

T(A, {})

PQT(A, {})(C, {A, C})

B

Null5, (B, {C, B})

C

Null

A


B

C

12

54

G

A


Greedy/Prims 30


T(A, {})

PQT(A, {})(C, {A, C}) (B, {C, B})

B

Null Null

C

Null

A


B

C

12

54

G

A


Greedy/Prims 31

Prim (G)1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-heap(D,V, { })5. T

6 4

5

2

9

15

8

1014

3

e

f

b

a

c

d

g

h

r

PQ = {( 0,(a,)), (,(b,?)), ...(,(h,?))}

T = { } G =

D = [ 0, , …, ]


Greedy/Prims 32

6 4

5

2

9

15

8

1014

3

e

f

b

a

c

d

g

h

r

PQ = {

T = {

7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T10. for each v Adjacent (u ) 11. // relaxation operation15. return T

G =

// relaxation11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v))

D = [ 0,


Greedy/Prims 33

Lemma 1

Let G = ( V, E) be a connected, weighted undirected graph. Let T be a promising subset of E. Let Y be the set of vertices connected by the edges in T.

If e is a minimum weight edge that connects a vertex in Y to a vertex

in V - Y, then T { e } is promising.

Note: A feasible set is promising if it can be extended to produce not only a solution , but an optimal solution.

In this algorithm: A feasible set of edges is promising if it is a subset of a Minimum Spanning Tree for the connected graph.


Greedy/Prims 34

Outline of Proof of Correctness of Lemma 1

T is the promising subset, and e the minimum cost edge of Lemma 1

Let T ' be the MST such that T T '

We will show that if e T' then there must be another MST T" such that T {e} T".

Proof has 4 stages:

1. Adding e to T', closes a cycle in T' {e}.

2. Cycle contains another edge e' T' but e' T

3. T"=T' {e} - {e’ } is a spanning tree

4. T" is a MST


Greedy/Prims 35

The Promising Set of Edges Selected by Prim

MST T' but e T'

X

X

XX

X

X

X

TX

X

XX

X

X

XeX

Y

V -Y

Lemma 1


Greedy/Prims 36

Since T' is a spanning tree, it is connected. Adding the

edge, e, creates a cycle.

X

X

XX

X

X

XT' {e}

e

In T' there is a path from uY to v V -Y. Therefore the path must include another edge e' with one vertex in Y and the other in V-Y.

X

X

XX

X

X

X

X

Y

V -Y

e'

Lemma 1

u v Stage 1

Stage 2


Greedy/Prims 37

• If we remove e’ from T’ { e } the cycle disappears.• T”=T’ { e } - {e’} is connected. T’ is connected. Every pair of vertices

connected by a path that does not include e’ is still connected in T”. Every pair of vertices connected by a path that included e’ , is still connected in T” because there is a path in T" =T’ { e } - { e’ } connecting the vertices of e’.

X

Y

V -Y

Lemma 1

X

X

XX

X

X

Xe

T "

w( e ) w( e' )By the way Prim picks the next edge

Stage 3


Greedy/Prims 38

• w( e ) w( e' ) by the way Prim picks the next edge.• The weight of T", w(T") = w (T' ) + w( e ) - w( e' ) w(T').• But w (T' ) w(T") because T' is a MST.• So w (T' ) = w(T") and T" is a MST

X

Y

V -Y

Lemma 1

X

X

XX

X

X

Xe

T"

Conclusion T { e } is promising

Stage 4


Greedy/Prims 39

Theorem : Prim's Algorithm always produces a minimum spanning tree.

Proof by induction on the set T of promising edges.Base case: Initially, T = is promising.

Induction hypothesis: The current set of edges T selected by Prim is promising.

Induction step: After Prim adds the edge e, T { e } is promising.

Proof: T { e } is promising by Lemma 1.

Conclusion: When G is connected, Prim terminates with |T | = |V | -1, and T is a MST.

Kruskal’s Algorithm


Greedy/Prims 41

solution = { } while ( more edges in E) do // Selection

select minimum weight edgeremove edge from E // Feasibility

if (edge closes a cycle with solution so far)then reject edgeelse add edge to solution

// Solution check

if |solution| = |V | - 1 return solution

return null // when does this happen?

Kruskal's Algorithm: Main Idea

6 4

5

2

158

1014

3

u

v

b

a

c

d

f


Greedy/Prims 42

6 4

5

2

9

15

8

1014

3

e

f

b

a

c

d

g

h

C = { {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h} }

C is a forest of trees.

Kruskal's Algorithm:

1. Sort the edges E in non-decreasing weight2. T 3. For each v V create a set. 4. repeat 5. Select next {u,v} E, in order 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then8. add edge (u,v) to T 9. union ( ucomp,vcomp )10.until T contains |V | - 1 edges11. return tree T


Greedy/Prims 43

Kruskal - Disjoint setAfter Initialization

TA

B

C

2

56

G

1. Sort the edges E in non-decreasing weight2. T

3. For each v V create a set.

A B 2

Sorted edges

B C 5

A C 6

A B CDisjoint data set for G

D

B D 77

D


Greedy/Prims 44

Kruskal - add minimum weight edge if feasible

A

C

2

56

G

B

5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union( ucomp,vcomp )

TSorted edges

A B CDisjoint data set for G

Find(A) Find(B)

AB

C

After merge(A, B)

(A, B) A B 2

B C 5

A C 6

B D 7

D

D

7

D


Greedy/Prims 45


A

C

2

56

G

B

5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )

T

A

B

C

Find(B) Find(C)

AB C

After merge(A, C)

(A, B)

(B, C)

Sorted edges A B 2

B C 5

A C 6

B D 7

D

D

7

D


Greedy/Prims 46


A

C

2

56

G

B


T

A

B C

Find(A) Find(C)

A and C in same set

(A, B)

(B, C)

Sorted edges A B 2

B C 5

A C 6

B D 7

D

7

D


Greedy/Prims 47


A

C

2

56

G

B


T

A

B C

Find(B) Find(D)

(A, B)

(B, C)

(B, D)

Sorted edges A B 2

B C 5

A C 6

B D 7

D

A

B C D

After merge

7

D


Greedy/Prims 48

Kruskal ( G )1. Sort the edges E in non-decreasing weight2. T 3. For each v V create a set. 4. repeat 5. {u,v} E, in order 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )11.until T contains |V | - 1 edges12. return tree T

Count1 = ( E lg E )

Count2= (1)

Count3= ( V )

Count4 = O( E )

Using Disjoint set-height andpath compression

Count4(6+7+10)= O((E +V) (V))

Sorting dominates the runtime. We get T( E,V ) = ( E lg E), so for a sparse graph we get ( V lg V)for a dense graph we get( V2 lg V2) = ( V2 lg V)

Kruskal's Algorithm: Time Analysis

Minimum Spanning Tree Algorithms Prim’s Algorithm Kruskal’s Algorithm Runtime Analysis Correctness Proofs.

Documents

tree node

tree slide

nontree node

minimal spanning tree

current spanning tree

non tree node slide

nontree vertex v

notation treevertices