L07 - Hong Kong University of Science and Technologydekai/271/notes/L07/L07.pdfTitle: L07.dvi

Lecture 7: Minimum Spanning Trees andPrim’s Algorithm

CLRS Chapter 23

Outline of this Lecture

� Spanning trees and minimum spanning trees.

� The minimum spanning tree (MST) problem.

� The generic algorithm for MST problem.

� Prim’s algorithm for the MST problem.

– The algorithm

– Correctness

– Implementation + Running Time

1

Spanning Trees

Spanning Trees: A subgraph�

of a undirected graph� � �� is a spanning tree of

�if it is a tree and

contains every vertex of�

.

Example:

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

��

� � ��

a b

cd

e

a b

cd

e

a b

cd

e

a b

cd

e

Graph spanning tree 1

spanning tree 2 spanning tree 3

2

Spanning Trees

Theorem: Every connected graph has a spanningtree.

Question: Why is this true?

Question: Given a connected graph�

, how can youfind a spanning tree of

�?

3

Weighted Graphs

Weighted Graphs: A weighted graph is a graph, inwhich each edge has a weight (some real number).

Weight of a Graph: The sum of the weights of alledges.

Example:

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

��

� � ��

a b

cd

e

a b

cd

e

a b

cd

e

a b

cd

e

10

9

7 32

2310 32

239

7 7

9

32

23

3223

10

weighted graph

Tree 2, w=71 Tree 3, w=72

Tree 1. w=74

Minimum spanning tree

4

Minimum Spanning Trees

A Minimum Spanning Tree in an undirected connectedweighted graph is a spanning tree of minimum weight(among all spanning trees).

Example:

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

��

� � ��

a b

cd

e

a b

cd

e

a b

cd

e

a b

cd

e

10

9

7 32

2310 32

239

7 7

9

32

23

3223

10

weighted graph

Tree 2, w=71 Tree 3, w=72

Tree 1. w=74

Minimum spanning tree

5

Minimum Spanning Trees

Remark: The minimum spanning tree may not beunique. However, if the weights of all the edges arepairwise distinct, it is indeed unique (we won’t provethis now).

Example:

1

2

6724

1

2

6724

MST1 MST2weighted graph

1

2 2

100

6724

6

Minimum Spanning Tree Problem

MST Problem: Given a connected weighted undi-rected graph

�, design an algorithm that outputs a

minimum spanning tree (MST) of�

.

Question: What is most intuitive way to solve?

Generic approach: A tree is an acyclic graph.The idea is to start with an empty graph and try to addedges one at a time, always making sure that what isbuilt remains acyclic. And if we are sure every time theresulting graph always is a subset of some minimumspanning tree, we are done.

7

Generic Algorithm for MST problem

Let � be a set of edges such that � � �, where�

is a MST. An edge��

is a safe edge for � , if� � � �� is also a subset of some MST.

If at each step, we can find a safe edge��

, wecan ’grow’ a MST. This leads to the following genericapproach:

Generic-MST(G, w)Let A=EMPTY;while A does not form a spanning treefind an edge (u, v) that is safe for Aadd (u, v) to A

return A

How can we find a safe edge?

8

How to find a safe edge

We first give some definitions. Let� � ��

be aconnected and undirected graph. We define:

Cut A cut��

of G is a partition of V.

Cross An edge��

crosses the cut��

��if one of its endpoints is in

�, and the other is

in� � �

.

Respect A cut respects a set � of edges if no edgein A crosses the cut.

Light edge An edge is a light edge crossing a cutif its weight is the minimum of any edge crossingthe cut.

9

How to find a safe edge

Lemma

Let� � ��

be a connected, undirected graphwith a real-valued weight function � defined on

�. Let

� be a subset of�

that is included in some minimumspanning tree for

�, let

�� be any cut of

�that

respects � , and let��

be a light edge crossing thecut

�� . Then, edge

�� is safe for � .

It means that we can find a safe edge by

1. first finding a cut that respects � ,

2. then finding the light edge crossing that cut.

That light edge is a safe edge.

10

Proof

1. Let � � �, where

�is a MST. Suppose

��

.

2. The trick is to construct another MST� �

that con-tains both � and

�� , thereby showing

�� is

a safe edge for � .

11

3. Since�

, and�

are on opposite sides of the cut�� , there is at least one edge in

�on the

path from�

to�

that crosses the cut. Let��

besuch edge. Since the cut respects � ,

�� .

Since��

is a light edge crossing the cut, wehave �

��

.

a cut respects A

another MST T’

A

v

u

y

xA

MST T

v

u

y

x

12

4. Add��

to�

, it creates a cycle. By removingan edge from the cycle, it becomes a tree again.In particular, we remove

�� (

�� ) to make anew tree

� �.

5. The weight of� �

is

��

��

��

��

��

6. Since�

is a MST, we must have ��

��

,hence

� �is also a MST.

7. Since � � � �� is also a subset of� �

(a MST),�� is safe for � .

13

Prim’s Algorithm

The generic algorithm gives us an idea how to ’grow’a MST.

If you read the theorem and the proof carefully, youwill notice that the choice of a cut (and hence thecorresponding light edge) in each iteration is imma-terial. We can select any cut (that respects the se-lected edges) and find the light edge crossing that cutto proceed.

The Prim’s algorithm makes a nature choice of the cutin each iteration – it grows a single tree and adds alight edge in each iteration.

14

Prim’s Algorithm : How to grow a tree

Grow a Tree

� Start by picking any vertex � to be the root of thetree.

� While the tree does not containall vertices in the graphfind shortest edge leaving the treeand add it to the tree .

Running time is� � ��

.

15

More Details

Step 0: Choose any element � ; set� � �� and

� � �. (Take � as the root of our spanning tree.)

Step 1: Find a lightest edge such that one endpointis in

�and the other is in

� � �. Add this edge to

� and its (other) endpoint to�

.

Step 2: If� � � � �

, then stop & output (minimum)spanning tree

�� . Otherwise go to Step 1.

The idea: expand the current tree by adding thelightest (shortest) edge leaving it and its endpoint.

e

2420

r

ab

c

d

26

f

g ir

ab

c

d e

f

g i

8 8

12

1614

new

242026

1614

12

23 23

new edge

1212

h h

16

Prim’s Algorithm

Worked Example

a

b

c

d

e

f

g

4

8

9

8

2

1

9

7

10

5

6

2

b

c

d

e

f

g

4

8

10

8

2

1

7

95

6

2

S={a}

Step 0

aV \ S = {b,c,d,e,f,g}

9

Connected graph

lightest edge = {a,b}

17

Prim’s Algorithm

Prim’s Example – Continued

b

c

d

e

f

g

4

8

9

8

2

1

9

7

10

5

6

2

c

d

e

f

g

8

9

10

8

2

1

7

95

6

2

a

a

Step 1.1

Step 1.1 after

4

S={a}

S={a,b}b

before

V \ S = {b,c,d,e,f,g}

V \ S = {c,d,e,f,g}

lightest edge = {a,b}

lightest edge = {b,d}, {a,c}

A={}

A={{a,b}}

18

Prim’s Algorithm


c

d

e

f

g

4

8

9

8

2

1

9

7

10

5

6

2

c

e

f

g

8

10

8

2

1

7

95

6

2

a

a

4b

beforeb Step 1.2 S={a,b}

Step 1.2 after

S={a,b,d}

d

V \ S = {c,d,e,f,g}

V \ S = {c,e,f,g}9

lightest edge = {b,d}, {a,c}

lightest edge = {d,c}

A={{a,b}}

A={{a,b},{b,d}}

19

Prim’s Algorithm


c

e

f

g

4

8

9

8

2

1

9

7

10

5

6

2

e

f

g

8

9

10

8

2

1

7

95

6

2

a

a

4b

beforeb

d

Step 1.3

Step 1.3 after

S={a,b,d}d

S={a,b,c,d}

V \ S = {c,e,f,g}

V \ S = {e,f,g}

c

lightest edge = {d,c}

lightest edge = {c,f}

A={{a,b},{b,d}}

A={{a,b},{b,d},{c,d}}

20

Prim’s Algorithm


e

f

g

4

8

9

8

2

1

9

7

10

5

6

2

e

g

8

9

10

8

2

1

7

95

6

2

a

a

4b

beforeb

d

d

c

S={a,b,c,d}

V \ S = {e,f,g}

c

S={a,b,c,d,f}V \ S = {e,g}

Step 1.4

Step 1.4 after

f

lightest edge = {c,f}

lightest edge = {f,g}

A={{a,b},{b,d},{c,d}}

A={{a,b},{b,d},{c,d},{c,f}}

21

Prim’s Algorithm


e

g

4

8

9

8

2

1

9

7

10

5

6

2

e

8

9

10

8

2

1

7

95

6

2

a

a

4b

beforeb

d

d

c

c

f

S={a,b,c,d,f}V \ S = {e,g}

f

Step 1.5

Step 1.5 after

S={a,b,c,d,f,g}V \ S = {e}

{f,g}}

g

lightest edge = {f,g}

lightest edge = {f,e}

A={{a,b},{b,d},{c,d},{c,f}}

A={{a,b},{b,d},{c,d},{c,f},

22

Prim’s Algorithm


e4

8

9

8

2

1

9

7

10

5

6

2

8

9

10

8

2

1

7

95

6

2

a

a

4b

beforeb

d

d

c

c

f

f

g

S={a,b,c,d,f,g}V \ S = {e}

{f,g}}

g

Step 1.6

Step 1.6 after

S={a,b,c,d,e,f,g}V \ S = {}

{f,g},{f,e}}

MST completed

e

lightest edge = {f,e}

A={{a,b},{b,d},{c,d},{c,f},

A={{a,b},{b,d},{c,d},{c,f},

23

Recall Idea of Prim’s Algorithm

Step 0: Choose any element � and set� � � �� and � � �

.(Take � as the root of our spanning tree.)

Step 1: Find a lightest edge such that one endpoint is in�

andthe other is in � � . Add this edge to � and its (other)endpoint to

�.

Step 2: If � � � �, then stop and output the minimum span-

ning tree �� .Otherwise go to Step 1.

Questions:

� Why does this produce a Minimum SpanningTree?

� How does the algorithm find the lightest edge andupdate � efficiently?

� How does the algorithm update�

efficiently?

24

Prim’s Algorithm

Question: How does the algorithm update�

efficiently?

Answer: Color the vertices. Initially all are white.Change the color to black when the vertex is movedto

�. Use color[

�] to store color.

Question: How does the algorithm find the lightestedge and update � efficiently?

Answer:(a) Use a priority queue to find the lightest edge.(b) Use pred[

�] to update � .

25

Reviewing Priority Queues

Priority Queue is a data structure (can be implementedas a heap) which supports the following operations:

insert(� ��

):Insert

�with the key value

�� in � .

u = extractMin():Extract the item with the minimum key value in � .

decreaseKey(� ��

� -��

):Decrease

�’s key value to

�� -

�� .

Remark: Priority Queues can be implemented so thateach operation takes time

� � �� . See CLRS!

26

Using a Priority Queue to Find the Lightest Edge

Each item of the queue is a triple��

��

,��

,where

��

is a vertex in� � �

,�

�� is the weight of the lightest edge

from�

to any vertex in�

, and�

��

is the endpoint of this edge in�

.The array is used to build the MST tree.

r

ab

c

d e

f

g ir

ab

c

d e

f

g i

242026

1614

8

242026

1614

8

1212

23 23

new edgekey[f] = 8, pred[f] = e

1212

key[i] = infinity, pred[i] = nil key[i] = 23, pred[i] = f

After adding the new edgeand vertex f, update the key[v] and pred[v] for each vertex v adjacent to f

key[g] = 16, pred[g] = c

key[h] = 24, pred[h] = b

f has the minimum key

h h

27

Description of Prim’s Algorithm

Remark: � is given by adjacency lists. The vertices in � �are stored in a priority queue with key=value of lightest edge tovertex in

�.

Prim( � �� )�for each � � � initialize� ��

;�� ;��

; start at root� � �� ! ;" �

new PriQueue( � ); put vertices in"

while("

is nonempty) until all vertices in MST�u=

"$#extraxtMin(); lightest edge

for each ( %&� ' �)(*� �� )�if (( �� %! � � �

)&&(� � � � %! ,+ �� %! ))��-.� %/ � � � � � %! ; new lightest edge"$#

decreaseKey( % � ��-.� %! );� � �0�� %! � � ;�� 1 2��

When the algorithm terminates," � �

and the MST is3 � � � % � � � �0�� %! �$4/%&� � � �� #The pred pointers define the MST as an inverted tree

rooted at � .28

Example for Running Prim’s Algorithm

a

b

c

d

e

f

1

2

3

4

5

1

103

4

u

key[u]

pred[u]

a b c d e f

29

Analysis of Prim’s Algorithm

Let� � � � �

and� � ��

. The data structure PriQueuesupports the following two operations: (See CLRS)

��

to extract each vertex from the queue.Done once for each vertex

� � � � ��

��

time to decrease the key value of neigh-boring vertex.Done at most once for each edge

� � � � � � � � ��

Total cost is then

� � � � � � ��

30

Analysis of Prim’s Algorithm – Continued

Prim(G, w, r) { for each (u in V){

key[u] = +infinity; color[u] = white;

}

key[r] = 0; pred[r] = nil; Q = new PriQueue(V);

while (Q. nonempty()){

u = Q.extractMin(); for each (v in adj[u]){

if ((color[v] == white) &(w(u,v) < key[v])

key[v] = w(u, v); {

}}color[u] = black;

}}

1O(log n)1

11

pred[v] = u;

O(log n)

1

O(deg(u) log n)

1

[O(log n) + O(deg(u) log n)]u in V

11

2n

n

Q.decreaseKey(v, key[v]);

31

Analysis of Prim’s Algorithm – Continued

So the overall running time is

� � � ��

� � ��

� � � � � � ��

� � ��

� � � � � � � � � ��

32

L07 - Hong Kong University of Science and Technologydekai/271/notes/L07/L07.pdfTitle: L07.dvi

Documents