Theory of Algorithms I - SJTU · Theory of Algorithms I Graph Algorithms, Revisited Guoqiang Li School of Software, Shanghai Jiao Tong University

Theory of Algorithms IGraph Algorithms, Revisited

Guoqiang Li

School of Software, Shanghai Jiao Tong University

Instructor and Teaching Assistants

• Guoqiang LI

• Homepage: https://basics.sjtu.edu.cn/˜liguoqiang• Course page:

https://basics.sjtu.edu.cn/˜liguoqiang/teaching/SE222/• Email: li.g@outlook.com• Office: Rm. 1212, Building of Software• Phone: 3420-4167

• TA:

• Yuan HANG: amberbabatt (AT) gmail (DOT)com

• Office hour: Tue. 14:00-17:00 @ Software Building 3203• Office hour: Wed. 13:30-15:00 @ Software Building 1212 (need

reserve!)

• Guoqiang LI• Homepage: https://basics.sjtu.edu.cn/˜liguoqiang• Course page:

• TA:

• Yuan HANG: amberbabatt (AT) gmail (DOT)com

reserve!)

• TA:• Yuan HANG: amberbabatt (AT) gmail (DOT)com

reserve!)

• Office hour: Tue. 14:00-17:00 @ Software Building 3203

• Office hour: Wed. 13:30-15:00 @ Software Building 1212 (needreserve!)

reserve!)

Text Book

• Algorithms: Design Techniques and Analysis• M. H. Alsuwalyel• World Scientific Publishing, 1999.

Text Book

• Algorithms• Sanjoy Dasgupta

University of California• San Diego Christos Papadimitriou

University of California at Berkeley• Umesh Vazirani

University of California at Berkeley• McGraw-Hill, 2007.

• Available at:http://www.cs.berkeley.edu/˜vazirani/algorithms.html

Which one comes first, computer or algorithms?

Al Khwarizmi

Al Khwarizmi (780 - 850)

Al Khwarizmi

Al Khwarizmi (780 - 850)

In the 12th century, Latin translationsof his work on the Indian numerals,introduced the decimal system to theWestern world. (Source: Wikipedia)

Algorithms

Al Khwarizmi laid out the basic methods for• adding,• multiplying,• dividing numbers,• extracting square roots,• calculating digits of π.

These procedures were precise, unambiguous, mechanical, efficient,correct.

They were algorithms, a term coined to honor the wise man after thedecimal system was finally adopted in Europe, many centuries later.

Algorithms

DFS in Graphs, Revisited

Exploring Graphs

EXPLORE(G, v)input : G = (V ,E) is a graph; v ∈ Voutput: visited(u) to true for all nodes u reachable from v

visited(v) = true;PREVISIT(v);for each edge (v, u) ∈ E do

if not visited(u) then EXPLORE(G, u);endPOSTVISIT(v);

Types of Edges in Undirected Graphs

Those edges in G that are traversed by EXPLORE are tree edges.

The rest are back edges.

Types of Edges in Undirected Graphs

Those edges in G that are traversed by EXPLORE are tree edges.

The rest are back edges.

Depth-First Search

DFS(G)

for all v ∈ V dovisited(v) = false;

endfor all v ∈ V do

if not visited(v) then Explore(G, v);end

Previsit and Postvisit Orderings

For each node, we will note down the times of two important events:

• the moment of first discovery (corresponding to PREVISIT);• and the moment of final departure (POSTVISIT).

PREVISIT(v)

pre[v] = clock;clock ++;

POSTVISIT(v)

post[v] = clock;clock ++;

LemmaFor any nodes u and v, the two intervals [pre(u), post(u)] and[pre(u), post(u)] are either disjoint or one is contained within the other.

For each node, we will note down the times of two important events:• the moment of first discovery (corresponding to PREVISIT);• and the moment of final departure (POSTVISIT).

PREVISIT(v)

POSTVISIT(v)

PREVISIT(v)

POSTVISIT(v)

PREVISIT(v)

POSTVISIT(v)

A B C D

E F G H

I J K L

11,22 23,24

Types of Edges in Directed Graphs

DFS yields a search tree/forests.

• root.• descendant and ancestor.• parent and child.

• Tree edges are actually part of the DFS forest.• Forward edges lead from a node to a nonchild descendant in the

DFS tree.• Back edges lead to an ancestor in the DFS tree.• Cross edges lead to neither descendant nor ancestor.

DFS yields a search tree/forests.• root.

• descendant and ancestor.• parent and child.

DFS yields a search tree/forests.• root.• descendant and ancestor.

• parent and child.

DFS yields a search tree/forests.• root.• descendant and ancestor.• parent and child.

• Tree edges are actually part of the DFS forest.

• Forward edges lead from a node to a nonchild descendant in theDFS tree.• Back edges lead to an ancestor in the DFS tree.• Cross edges lead to neither descendant nor ancestor.

DFS tree.

• Back edges lead to an ancestor in the DFS tree.• Cross edges lead to neither descendant nor ancestor.

DFS tree.• Back edges lead to an ancestor in the DFS tree.

• Cross edges lead to neither descendant nor ancestor.

Directed Graphs

Types of Edges

pre/post ordering for (u, v) Edge type

[u [v ]v ]u Tree/forward[v [u ]u ]v Back[v ]v [u ]u Cross

Q: Is that all?

Types of Edges

pre/post ordering for (u, v) Edge type

[u [v ]v ]u Tree/forward[v [u ]u ]v Back[v ]v [u ]u Cross

Q: Is that all?

Directed Acyclic Graphs (DAG)

Definition:A cycle in a directed graph is a circular path

v0 → v1 → v2 → . . . vk → v0

Lemma:A directed graph has a cycle if and only if its depth-first search revealsa back edge.

Definition:A cycle in a directed graph is a circular path

v0 → v1 → v2 → . . . vk → v0

Lemma:A directed graph has a cycle if and only if its depth-first search revealsa back edge.

Linearization/Topologically Sort: Order the vertices such that everyedge goes from a earlier vertex to a later one.

Q: What types of dags can be linearized?

A: All of them.

DFS tells us exactly how to do it: perform tasks in decreasing order oftheir post numbers.

The only edges (u, v) in a graph for which post(u) < post(v) are backedges, and we have seen that a DAG cannot have back edges.

A: All of them.

Lemma:In a DAG, every edge leads to a vertex with a lower post number.

There is a linear-time algorithm for ordering the nodes of a DAG.

Acyclicity, linearizability, and the absence of back edges during adepth-first search - are the same thing.

The vertex with the smallest post number comes last in thislinearization, and it must be a sink - no outgoing edges.

Symmetrically, the one with the highest post is a source, a node withno incoming edges.

Lemma:Every DAG has at least one source and at least one sink.

The guaranteed existence of a source suggests an alternative approachto linearization:

1 Find a source, output it, and delete it from the graph.

2 Repeat until the graph is empty.

Strongly Connected Components (SCC)

Defining Connectivity for DirectedGraphs

Definition:Two nodes u and v of a directed graph are connected if there is a pathfrom u to v and a path from v to u.

This relation partitions V into disjoint sets that we call stronglyconnected components (SCC).

Lemma:Every directed graph is a DAG of its SCC.

Strongly Connected Components

A B,E C,F

DJ,K,L

An Efficient Algorithm

Lemma:If the EXPLORE subroutine at node u, then it will terminate preciselywhen all nodes reachable from u have been visited.

If we call explore on a node that lies somewhere in a sink SCC, thenwe will retrieve exactly that component.

We have two problems:

1 How do we find a node that we know for sure lies in a sink SCC?

2 How do we continue once this first component has beendiscovered?

Lemma:The node that receives the highest post number in a depth-first searchmust lie in a source SCC.

Lemma:If C and C′ are SCC, and there is an edge from a node in C to a nodein C′, then the highest post number in C is bigger than the highest postnumber in C′.

Hence the SCCs can be linearized by arranging them in decreasingorder of their highest post numbers.

Solving Problem A

Consider the reverse graph GR, the same as G but with all edgesreversed.

GR has exactly the same SCCs as G.

If we do a depth-first search of GR, the node with the highest postnumber will come from a source SCC in GR.It is a sink SCC in G.

Solving Problem A

Solving Problem B

Once we have found the first SCC and deleted it from the graph, thenode with the highest post number among those remaining will belongto a sink SCC of whatever remains of G.

Therefore we can keep using the post numbering from our initialdepth-first search on GR to successively output the second stronglyconnected component, the third SCC, and so on.

Solving Problem B

Once we have found the first SCC and deleted it from the graph, thenode with the highest post number among those remaining will belongto a sink SCC of whatever remains of G.

Therefore we can keep using the post numbering from our initialdepth-first search on GR to successively output the second stronglyconnected component, the third SCC, and so on.

The Linear-Time Algorithm

1 Run depth-first search on GR.

2 Run the EXPLORE algorithm on G, and during the depth-firstsearch, process the vertices in decreasing order of their postnumbers from step 1.

The Linear-Time Algorithm

1 Run depth-first search on GR.

2 Run the EXPLORE algorithm on G, and during the depth-firstsearch, process the vertices in decreasing order of their postnumbers from step 1.

Thinking About

How the SCC algorithm works when the graph is very, very huge?

An Easter egg: A report or a solution will earn extra scores!

Thinking About

How the SCC algorithm works when the graph is very, very huge?

An Easter egg: A report or a solution will earn extra scores!

A Question to Think About

How about edges instead of paths?

Exercises

Exercises 1

3.16. Suppose a CS curriculum consists of n courses, all of them mandatory. The prerequisite graph G

has a node for each course, and an edge from course v to course w if and only if v is a prerequisite

for w. Find an algorithm that works directly with this graph representation, and computes the

minimum number of semesters necessary to complete the curriculum (assume that a student

can take any number of courses in one semester). The running time of your algorithm should be

linear.

Exercises 2

3.22. Give an efficient algorithm which takes as input a directed graph G = (V, E), and determineswhether or not there is a vertex s ∈ V from which all other vertices are reachable.

BFS in Graphs„ Revisited

Breadth-First SearchBFS(G, v)input : Graph G = (V ,E), directed or undirected; Vertex v ∈ Voutput: For all vertices u reachable from v, dist(u) is the set to the

distance from v to u

for all u ∈ V dodist(u) =∞;

enddist[v] = 0;Q = [v] queue containing just v;while Q is not empty do

u=Eject(Q);for all edge (u, s) ∈ E do

if dist(s) =∞ thenInject(Q,s);dist[s] = dist[u] + 1;

endend

Dijkstra’s Shortest-Path AlgorithmDIJKSTRA(G, l, s)input : Graph G = (V ,E), directed or undirected; positive edge length

{le | e ∈ E}; Vertex s ∈ Voutput: For all vertices u reachable from s, dist(u) is the set to the distance from s

for all u ∈ V dodist(u) = ∞;prev(u) = nil;

enddist(s) = 0;H =makequeue(V)\\ using dist-values as keys;while H is not empty do

u=deletemin(H);for all edge (u, v) ∈ E do

if dist(v) > dist(u) + l(u, v) thendist(v) = dist(u) + l(u, v); prev(v) = u;decreasekey (H ,v);

endend

An Example

A: 0 D:∞

B: 4 E:∞

A: 0 D: 5B: 3 E: 6C: 2

1 3A: 0 D: 6B: 3 E: 7C: 2

4 A: 0 D: 5B: 3 E: 6C: 2

An Example

A: 0 D:∞

B: 4 E:∞

A: 0 D: 5B: 3 E: 6C: 2

1 3A: 0 D: 6B: 3 E: 7C: 2

4 A: 0 D: 5B: 3 E: 6C: 2

An Example

A: 0 D:∞

B: 4 E:∞

A: 0 D: 5B: 3 E: 6C: 2

1 3A: 0 D: 6B: 3 E: 7C: 2

4 A: 0 D: 5B: 3 E: 6C: 2

An Example

A: 0 D:∞

B: 4 E:∞

A: 0 D: 5B: 3 E: 6C: 2

1 3A: 0 D: 6B: 3 E: 7C: 2

4 A: 0 D: 5B: 3 E: 6C: 2

An Example

A: 0 D:∞

B: 4 E:∞

A: 0 D: 5B: 3 E: 6C: 2

1 3A: 0 D: 6B: 3 E: 7C: 2

4 A: 0 D: 5B: 3 E: 6C: 2

An Example

A: 0 D:∞

B: 4 E:∞

A: 0 D: 5B: 3 E: 6C: 2

1 3A: 0 D: 6B: 3 E: 7C: 2

4 A: 0 D: 5B: 3 E: 6C: 2

Shortest Paths in the Presence of Negative Edges

Negative EdgesDijkstra’s algorithm works in part because the shortest path from thestarting point s to any node v must pass exclusively through nodes thatare closer than v.

This no longer holds when edge lengths can be negative.

Q: What needs to be changed in order to accommodate this newcomplication?

A crucial invariant of Dijkstra’s algorithm is that the dist values itmaintains are always either overestimates or exactly correct.

They start off at∞, and the only way they ever change is by updatingalong an edge:

UPDATE ((u, v) ∈ E)

dist(v) = min{dist(v), dist(u) + l(u, v)};

Update

This UPDATE operation is simply an expression of the fact that thedistance to v cannot possibly be more than the distance to u, plusl(u, v). It has the following properties,

1 It gives the correct distance to v in the particular case where u isthe second-last node in the shortest path to v, and dist(u) iscorrectly set.

2 It will never make dist(v) too small, and in this sense it is safe.For instance, a slew of extraneous update’s can’t hurt.

Update

Lets→ u1 → u2 → u3 → . . .→ uk → k

be a shortest path from s to t.

This path can have at most |V | − 1 edges (why?).

Update

Lets→ u1 → u2 → u3 → . . .→ uk → k

Update

Lets→ u1 → u2 → u3 → . . .→ uk → k

Update

Lets→ u1 → u2 → u3 → . . .→ uk → k

Bellman-Ford Algorithm

If we don’t know all the shortest paths beforehand, how can we besure to update the right edges in the right order?

We simply update all the edges, |V | − 1 times!

If we don’t know all the shortest paths beforehand, how can we besure to update the right edges in the right order?

We simply update all the edges, |V | − 1 times!

SHORTEST-PATHS(G, l, s)input : Graph G = (V ,E), edge length{le | e ∈ E}; Vertex s ∈ Voutput: For all vertices u reachable from s, dist(u) is the set to the

distance from s to u

for all u ∈ V dodist(u) =∞;prev(u) = nil;

enddist[s] = 0;repeat |V | − 1 times: for e ∈ E do

UPDATE(e);end

Running time:O(|V | · |E|)

SHORTEST-PATHS(G, l, s)input : Graph G = (V ,E), edge length{le | e ∈ E}; Vertex s ∈ Voutput: For all vertices u reachable from s, dist(u) is the set to the

enddist[s] = 0;repeat |V | − 1 times: for e ∈ E do

UPDATE(e);end

Running time:O(|V | · |E|)

8 Iteration

Node 0 1 2 3 4 5 6 7

S 0 0 0 0 0 0 0 0A ∞ 10 10 5 5 5 5 5B ∞ ∞ ∞ 10 6 5 5 5C ∞ ∞ ∞ ∞ 11 7 6 6D ∞ ∞ ∞ ∞ ∞ 14 10 9E ∞ ∞ 12 8 7 7 7 7F ∞ ∞ 9 9 9 9 9 9G ∞ 8 8 8 8 8 8 8

Negative Cycles

If the graph has a negative cycle, then it doesn’t make sense to evenask about shortest path.

Q: How to detect the existence of negative cycles:

Instead of stopping after |V | − 1, iterations, perform one extra round.

There is a negative cycle if and only if some dist value is reducedduring this final round.

Negative Cycles

Shortest Paths in Dags

Graphs without Negative EdgesThere are two subclasses of graphs that automatically exclude thepossibility of negative cycles:

• graphs without negative edges,• and graphs without cycles.

We already know how to efficiently handle the former.

We will now see how the single-source shortest-path problem can besolved in just linear time on directed acyclic graphs.

As before, we need to perform a sequence of updates that includesevery shortest path as a subsequence.

In any path of a DAG, the vertices appear in increasing linearizedorder.

Graphs without Negative EdgesThere are two subclasses of graphs that automatically exclude thepossibility of negative cycles:• graphs without negative edges,• and graphs without cycles.

A Shortest-Path Algorithm for DAG

DAG-SHORTEST-PATHS(G, l, s)input : Graph G = (V ,E), edge length{le | e ∈ E}; Vertex s ∈ Voutput: For all vertices u reachable from s, dist(u) is the set to the

enddists = 0;linearize G;for each u ∈ V in linearized order do

for all e ∈ E doUPDATE(e);

endend

Note that the scheme doesn’t require edges to be positive.

In particular, we can find longest paths in a DAG by the samealgorithm: just negate all edge lengths.

Note that the scheme doesn’t require edges to be positive.

In particular, we can find longest paths in a DAG by the samealgorithm: just negate all edge lengths.

Exercises

Exercises 3

Professor Fake suggests the following algorithm for finding theshortest path from node s to node t in a directed graph with somenegative edges: add a large constant to each edge weight so that all theweights become positive, then run Dijkstra’s algorithm starting atnode s, and return the shortest path found to node t.

Exercises 4

4.14. You are given a strongly connected directed graph G = (V, E) with positive edge weights alongwith a particular node v0 ∈ V . Give an efficient algorithm for finding shortest paths between all

pairs of nodes, with the one restriction that these paths must all pass through v0.

Homework

[DPV07]. 3.7, 3.11, 3.28, 4.11, 4.12, 4.16

[Als99]. 9.18, 9.33,

Theory of Algorithms I - SJTU · Theory of Algorithms I Graph Algorithms, Revisited Guoqiang Li School of Software, Shanghai Jiao Tong University

Documents

Basic Algorithms in Number Theory

Parallel Algorithms: Theory and Practice

COT 6405 Introduction to Theory of Algorithms -...

Design and Analysis of Algorithms (III) -...

Design and Analysis of Algorithms...

Theory of Evolutionary Algorithms for Combinatorial ...

Current Trends in Algorithms, Complexity Theory, and...

Blind Ptychography: Theory & Algorithms - Mathematics Home

Computer Vision: Theory, Algorithms, Applications, Learning

TGV: theory, principles and algorithms · TGV: theory,...

GUOQIANG TIAN - people.tamu.edu

Online Learning: Theory, Algorithms, and...

Design and Analysis of Algorithms (I) -...

Some Theory Behind Algorithms for Stochastic...

Algorithms in Number Theory - Infoscience

Algorithms in Graph Theory - TU Wien · Algorithms in Graph...