Joint work with:

Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

1

Parallel Computationand Algorithms

by

Raymond GreenlawArmstrong Atlantic State University

Joint work with:

Larry Ruzzo Department of Computer Science

University of Washington

Pilar de la TorreDepartment of Computer Science

University of New Hampshire

Magnus Halldorsson Decode Genetics

Teresa Przytycka Department of Biophysics

John Hopkins University Medical School

Alex SchafferNational Institute of Health

Jim HooverComputer Science Department

University of Alberta

Rossella Petreschi Department of Computer ScienceUniversity of Rome La Sapienza


2

Part I: An Introduction toParallel Computation

and Ρ-Completeness Theory


3

Part I: Outline

• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems


4

Part I: Outline



5

Introduction

• Sequential computation:Feasible ~ n O(1) time

(polynomial time).• Parallel computation:

Feasible ~ n O(1) time and ~ n O(1) processors.


6

Introduction

• Goal of parallel computation:Develop extremely fast ((log n)O(1) ) time algorithms using a reasonable number of processors.

• Speedup equation: (best sequential time)(number of processors)

Allow a polynomial number of processors.

(parallel time).


7

Introduction

Roughly,• A problem is feasible if it can be solved by a

parallel algorithm with worst case time and processor complexity n O(1).

• A problem is feasible highly parallel if it can be solved by an algorithm with worst case time complexity (log n) O(1) and processor complexity n O(1).

• A problem is inherently sequential if it is feasible but has no feasible highly parallel algorithm for its solution.


8

Part I: Outline



9

Parallel Models of Computation

• Parallel Random Access Machine Model• Boolean Circuit Model• Circuits and PRAMs


10

Parallel Random Access Machine

RAM Processors

Global Memory Cells

P0 P1 P2

C0 C1 C2


11

Boolean Circuit Model

1 0 1 1 0

AND OR

AND

ANDOR

OR

NOT


12

Circuits and PRAMS

Theorem:A function f from {0,1}* to {0,1}* can be computed by a logarithmic space uniform Boolean circuit family {n } with depth (n) є (logn)O(1) and size (n ) є n

O(1)

if and only if

f can be computed by a CREW-PRAM M on inputs of length n in time t(n) є (logn)O(1) using p(n) є n

O(1).


13

Part I: Outline



14

Basic Complexity

• Decision, Function, and Search Problems• Complexity Classes• Reducibility• Completeness


15

Decision, Function, and Search Problems4 4

4 4

3 3 366

Spanning Tree-DGiven: An undirected graph G = (V,E) with weights from N labeling edges in E and a natural number k.Problem: Is there a spanning tree of G with cost less than or equal to k ?

Spanning Tree-FGiven: Same (no k ).Problem: Compute the weight of a minimum cost spanning tree.

Spanning Tree-SGiven: Same.Problem: Find a minimum cost spanning tree.


16

Complexity Classes

Definitions:P is the set of all languages L that are decidable in sequential time n O(1).NC is the set of all languages L that are decidable in parallel time (logn)O(1) and processors n O(1). FP is the set of all functions from {0,1}* to {0,1}* that are computable in sequential time n O(1).FNC is the set of all functions from {0,1}* to {0,1}* that are computable in parallel time (logn)O(1) and processors n O(1).NC k, k 1, is the set of all languages L such that L is recognized by a uniform Boolean circuit family {n } with size(n ) = n O(1) and depth (n ) = O((logn)k ).


17

Definitions:A language L is many-one reducible to a language L’, written L P L’ , if there is a function f such that x є L if and only if f(x) є L’.

L is P many-one reducible to L’, written L P L’ , if the function f is in FP.

For k 1, L is NC k many-one reducible to L’, written L NC kL’, if the function f is in FNC k.

L is NC many-one reducible to L’, written L NC L’, if the function f is in FNC.

Many-One Reducibility

m

m

m

m


18

Many-One Reducibility

m

Lemma:

The reducibilities m, P, NC k(k 1), and

NC are transitive.

– If L’ є P and L P L’, L

NC k L’ (k > 1), or L NC L’ then L є P.

– If L’ є NC k (k > 1), and L NC kL’ then L є NC k.

– If L’ є NC and L NC L’ then L є NC .

m m m

mm m

m


19

Turing Reducibility

Definition:L is NC Turing reducible to L’, written L

NC L’, if and only if the L’-oracle PRAM on inputs of length n uses time (logn)O(1) and processors n O(1).

Lemma:

The reducibility NC is transitive.

– If L NC L’ then L NC L’.

– If L NC L’ and L’ є NC then L є NC.

– If L NC L’ and L’ є P then L є P.

T

T

m T

T

T


20

Completeness

Definitions:A language L is P-hard under NC reducibility if L’ NC L for every L’ є P.

A language L is P-complete under NC reducibility if L є P and L is P-hard.

Theorem:If any P-complete problem is in NC then NC equals P.

T


21

Part I: Outline



22

Evidence That NC P

• General Simulations Are Not Fast• Fast Simulations Are Not General• Natural Approaches Provably Fail


23

Evidence That NC P

Gaps in SimulationsModel Resource

1Resource

2Max R2C NC

Min R2C P

DTMDTMATMPDAUCPRAM

Time = n O(1)

Time = n O(1)

2Space = n O(1)

2Space = n O(1)

Size = n O(1)

Procs = n O(1)

SpaceReversalslog(Treesize)log(Time)DepthTime

log n(log n)O(1)

(log n)O(1)

(log n)O(1)

(log n)O(1)

(log n)O(1)

n O(1)

n O(1)

n O(1)

n O(1)

n O(1)

n O(1)


24

Part I: Outline



25

P-Complete Problems

There are approximately 175 P-complete problems (500 with variations).

Categories:– Circuit complexity– Graph theory– Searching graphs– Combinatorial

optimization and flow– Local optimality– Logic

– Formal languages– Algebra– Geometry– Real analysis– Games– Miscellaneous


26

Circuit Value Problem

Given: An encoding of a Boolean circuit , inputs x1,…,xn, and a designated output y. Problem:Is output y of TRUE on input x1,…,xn?

Theorem: [Ladner 75]The Circuit Value Problem is P-complete under NC

1 reductions.

m


27

P-Complete Variations of CVP

– Topologically Ordered [Folklore]– Monotone [Goldschlager 77]– Alternating Monotone Fanin 2, Fanout 2

[Folklore]– NAND [Folklore]– Topologically Ordered NOR [Folklore]– Synchronous Alternating Monotone Fanout

2 CVP [Greenlaw, Hoover, and Ruzzo 87]– Planar [Goldschlager 77]


28

NAND Circuit Value Problem

Given: An encoding of a Boolean circuit that consists solely of NAND gates, inputs x1,…,xn, and a designated output y.

Problem: Is output y of TRUE on input x1,…,xn?

Theorem: The NAND Circuit Value Problem is P-complete.


29

NAND Circuit Value Problem

Proof:Reduce AM2CVP to NAND CVP. Complement all inputs. Relabel all gates as NAND.

01 0 0 1

1

1

OR

AND

OROR1

1

10 1 1 0

1

1

NAND

NAND

NANDNAND1

0


30

Graph Theory

– Lexicographically First Maximal Independent Set [Cook 85]

– Lexicographically First ( + 1)-Vertex Coloring [Luby 84]

– High Degree Subgraph [Anderson and Mayr 84]

– Nearest Neighbor Traveling Salesman Heuristic [Kindervater, Lenstra, and Shmoys 89]


31

Lexicographically First Maximal Independent Set

Theorem: [Cook 85]LFMIS is P-complete.

Proof:Reduce TopNOR CVP to LFMIS. Add new vertex 0. Connect to all false inputs.

1 2 3 4 5

NOR9

NOR8

NOR7

NOR6

1 1 0 0 0

01

1 2 3 4 5

6 7

0

8

9

0

0

X X

X

X


32

Searching Graphs

– Lexicographically First Depth-First Search Ordering [Reif 85]

– Stack Breadth-First Search [Greenlaw 92]

– Breadth-Depth Search [Greenlaw 93]


33

Context-Free Grammar Empty

Given: A context-free grammar G = (N,T,P,S).Problem: Is L(G) empty?

Theorem: [Jones and Laaser 76], [Goldschlager 81], [Tompa 91]CFGempty is P-complete.

Proof: Reduce Monotone CVP to CFGempty. Given construct G = (N,T,P,S) with N, T, S, and P as follows:


34

Context-Free Grammar Empty

N = {i | vi is a vertex in }T = {a }S = n, where vn is the output of .P as follows:

1. For input vi, i a if value of vi is 1,2. i jk if vi vj Λ vk, and3. i j | k if vi vj V vk .

Then the value of vi is 1 if and only if i , where є {a }+.*


35

CFGempty Example

x1 = 0, x2 = 0, x3 = 1, and x4 = 1.

G = (N, T, S, P ), whereN = {1, 2, 3, 4, 5, 6, 7 }T = {a }S = 7P = {3 a, 4 a, 5 1 | 2, 6 34, 7 56}

1 2 3 4

5 6

7

x3 x4x1 x2

ORAND

AND


36

Part I: Outline



37

Comparator Circuit Class

Definition: [Cook 82]Comparator Circuit Value Problem (CCVP)

Given: An encoding of a circuit composed of comparator gates, plus inputs x1,…,xn, and a designated output y.Problem: Is output y of TRUE on input x1,…,xn?

Definition: CC is the class of languages NC reducible to CCVP.


38

CC-Complete Problems

– Lexicographically First Maximal Matching [Cook 82]

– Telephone Connection [Ramachandran and Wang 91]

– Stable Marriage [Mayr and Subramanian 92]


39

Part I: Outline



40

Open Problems

Find an NC algorithm or classify as P-complete.

– Edge Ranking– Edge-Weighted Matching– Restricted Lexicographically First Maximal

Independent Set– Integer Greatest Common Divisor– Modular Powering


41

Part II:Sequential and Parallel Algorithms

for Some Problems on Trees

Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree ProblemsRaymond Greenlaw


42

Part II: Outline

• Introduction• Parallel Preliminaries• Node Ranking• Edge Ranking• Prüfer Codes• Open Problems



43

Part II: Outline




44

Trees



45

Part II: Outline




46

Parallel Random Access Machine


RAM Processors

Global Memory Cells

P0 P1 P2

C0 C1 C2


47

Preliminary Parallel Algorithms

• Brent’s Scheduling Principle• Parallel Prefix Computation• Euler Tour• List Ranking• Parallel Tree Contraction



48

Brent’s Scheduling Principle

(Brent 1974) If processor allocation is not a problemthen a t(n) time parallel algorithm that requires w(n) computational operations

canbe simulated using w(n)/p(n) + t(n) time

andp(n) processors. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree ProblemsRaymond Greenlaw


49

Example of Brent’s



50

Parallel Prefix Computation

(Ladner & Fisher 1980)The Parallel Prefix Problem can besolved in O(log n) time using n/log nprocessors on an EREW-PRAM.



51

Example of Parallel Prefix



52

Euler Tour

(Tarjan & Vishkin 1985)An Euler tour of an n-node tree can becomputed in O(log n) time using n/log nprocessors on an EREW-PRAM.



53

Example of Euler Tour



54

List Ranking

(Anderson & Miller 1988)Given a list with n nodes, the List Ranking Problem can be solved in O(log

n)time using n/log n processors on an EREW-PRAM.



55

Example of List Ranking



56

Parallel Tree Contraction

(He 1986; Miller & Teng 1987; Abrahamson,Dadoun, Kirkpatrick & Przytycka 1989)Let T be an n-leaf regular binary expressiontree. Then all of the algebraic expressionsassociated with the internal nodes (one pernode) of T can be evaluated in O(log n) time using n/log n processors on an EREW-PRAM.



57

Example of Tree Contraction

Scrunched tree



58

Part II: Outline




59

Node Ranking• A node ranking is a labeling of the nodes of

a tree with natural numbers such that if nodes u and v have the same label then there exists another node with a greater label on the path between them.

• An optimal node ranking is a node ranking in which the largest label assigned to any node is as small as possible among all node rankings.

• The node ranking problem is to compute an optimal node ranking.



60

Node Ranking Example



61

Node Ranking: Sequential Results

• Node Ranking Problem• O(n log n) time (Iyer, Ratliff & Vijayan 1988)• O(n) time (Schaffer 1989)



62

Example of Sequential Node Ranking Algorithm

• Critical list at * is {3,4}• Label 3 at ** covers values 1 and 2



63

Example of Sequential Node Ranking Algorithm

• Critical list at * is {3,4}• Label 3 at ** covers values 1 and 2



64

Node Ranking: Parallel Results• Approximate Optimal Tree Ranking• O(log2 n) time, n processors EREW-PRAM (Liang, Dhall & Lakshmivarahan 1990)• Optimal Tree Ranking• O(log n) time, n2/log n procs CREW-PRAM (de la Torre & Greenlaw 1991) (Przytycka 1991)• Super Critical Tree Numbering O(log n) time, n2/log n procs CREW-PRAM (de la Torre, Greenlaw & Przytycka 1992)



65

Part II: Outline




66

Edge Ranking• An edge ranking is a labeling of the edges of

a tree with natural numbers such that if edges u and v have the same label then there exists another edge with a greater label on the path between them.

• An optimal edge ranking is an edge ranking in which the largest label assigned to any edge is as small as possible among all edge rankings.

• The edge ranking problem is to compute an optimal edge ranking.



67

Edge Ranking Example



68

Edge Ranking: Sequential Results• Approximate Edge Ranking• O(n log n) time (Iyer, Ratliff & Vijayan 1991)• Optimal Edge Ranking• O(n3 log n) time (de la Torre, Greenlaw & Schaffer 1995)• O(n2 log n) time (Zhou, Kashem & Nishizeki 1996)• O(n log n) time, O(n) time (Lam & Ling 1996, 1997)



69

From Local to Global Optimality



70

Greedy Cover Labeling

• Lc > Lc-1 > … > L1

is a greedy cover labeling if and only if cover(Li, crit(vj)) lex cover(Li, crit(vi)) for all i and for all j < i.



71

Example of Edge Ranking Algorithm



72




73




74

Edge Ranking: Parallel Results

• Approximate Edge Ranking• O(log2 n) time, n2/log n processors CREW-PRAM (de la Torre, Greenlaw & Schaffer 1995)• Optimal Edge Ranking of Constant Degree Trees NC (de la Torre, Greenlaw & Schaffer 1995)



75

Motivation for Studying Rankings



76

Motivation for Studying Rankings



77

Part II: Outline




78

Definition of Prüfer CodeA Prüfer code of a labeled free tree with n

nodes is a sequence of length n – 2 constructed by the following sequential process:

for i ranging from 1 to n – 2 do insert the label of the neighbor of the smallest remaining leaf into the i-th

position of the sequence; delete the leaf;



79

Example of a Prüfer Code

(9,6,5,6,1,6,1,1)Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree ProblemsRaymond Greenlaw


80

Related Work

• Prüfer code to tree EREW-PRAM O(log n) time, n processors

proposed as an open problem the reverse direction

(Wang, Chen & Liu 1997)• Tree to Prüfer code EREW-PRAM O(log n) time, n processors (Chen & Wang 1998, Greenlaw &

Petreschi 1998)



81

Related Work Continued

• Chain to Prüfer code EREW-PRAM O(log n) time, n/log n (Greenlaw & Petreschi 1998)• Used Prüfer codes for random tree generation (Kumar, Deo & Kumar 1998)



82

Can Sequential AlgorithmPrüfer Code Be Parallelized?

• Initial intuition suggests no.• What type of running time do we want?• How do we proceed?



83

Parallel Algorithm Prüfer Chain

The Prüfer code of this chain is(2,8,3,7,5,6)



84

Parallel Algorithm Prüfer ChainStep 1:

/* Compute the position of each node in the chain. */1. Use parallel list ranking to construct the

array Position such that Position[i] = v, 1 i n, where node v has a ranking of i in T

1286573487654321



85


/* Compute the maximum nodes encountered thus far in left-to-right and right-to-left traversals over the chain. */

2. Use parallel prefix computation to construct the arrays LRMax and RLMax

LRMax[i] = max{Position[j] | 1 j i} RLMax[i] = max{Position[j] | n – i + 1 j n}

0 4 4 7 7 7 8 8 8 90 1 2 8 8 8 8 8 8 9



86


/* Compute when a node becomes a maximum (if it does) for both left-to-right and right-to-left traversals. */

3. For 1 i n in parallel do if LRMax[i-1] LRMax[i] then LRStart[LRMax[i]] = i; if RLMax[i-1] RLMax[i] then RLStart[RLMax[i]] = i;

0 0 0 1 0 0 3 61 2 0 0 0 0 0 3



87


/* Compute when a node is no longer a maximum (if it was) for both left-to-right and right-to-left traversals. */

4. For 1 i n in parallel do if LRMax[i] LRMax[i+1] then LREnd[LRMax[i]] = i; if RLMax[i] RLMax[i+1] then RLEnd[RLMax[i]] = i;

0 0 0 2 0 0 5 81 2 0 0 0 0 0 8



88


/* Compute how many positions a node was maximum for. */5. For 1 i n in parallel do if LRStart[i] 0 then LRSpan[i] = LREnd[i] - LRStart[i] + 1; if RLStart[i] 0 then RLSpan[i] = RLEnd[i] - RLStart[i] + 1;

0 0 0 0 2 0 0 3 30 1 1 0 0 0 0 0 6



89


/* Compute how many nodes a given node is greater than from the left. Similiarly for the right. */

6. Use parallel prefix to construct the array LeftClear and RightClear, where 1 i n LeftClear[i] = LRSpan[0] + … + LRSpan[i-1]; RightClear[i] = RLSpan[0] + … + RLSpan[i-1];

0 0 0 0 2 2 2 50 1 2 2 2 2 2 2



90


/* Removal[i] denotes when the node in Position[i] is removed. */7. For 1 i n in parallel do if RLMax[n-i] > LRMax[i-1] then k = LRMax[i]; Removal[i] = i + RightClear[k]; else k = RLMax[n-i+1]; Removal[i] = (n – i) + 1 + LeftClear[k];

3 4 5 6 7 8 2 1Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree ProblemsRaymond Greenlaw


91

Theorem on Chains

(Greenlaw & Petreschi 1998)

The Prüfer code of an n-node labeled chain can be computed in O(log n) time using n/log n processors on an

EREW-PRAM.



92

Parallel Algorithm Prüfer Tree



93

Parallel Algorithm Prüfer Tree



94

Theorem on Trees (Greenlaw, Halldorsson & Petreschi 1999) The Prüfer code of an n-node labeled free tree can be computed in O(log n) time

using n/log n processors on an EREW-PRAM. In the other direction we show Given the Prüfer code of an n-node labeled

chain we can output the corresponding chain in O(log n) time using n/log n processors on an EREW-PRAM.



95

Prüfer Code SummaryTree to P-code P-code to Tree

sequential

O(n) folklore O(n) (K 78)

parallel O(log n) time, n processorsEREW-PRAM (CW 98, GP 98)

O(log n) time, n processorsEREW-PRAM (WCL 97)

For a chain O(log n) time,n/log n processors EREW-PRAM (GHP 99)

For a chainO(log n) time, n/log n processorsEREW-PRAM (GHP 99)

For a treeO(log n) time, n/log n processorsEREW-PRAM (GHP 99)Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems

Raymond Greenlaw


96

Part II: Outline




97

Open Problems• Can an optimal parallel algorithm be

developed for building a tree given its Prüfer code?

• Can the Element Distinctness Problem be solved on a CREW-PRAM in O(log n) time using n/log n processors?

• Is the edge ranking problem on trees P-complete?• Can all n-node trees be labeled with the

values 1 through n such that neighors receive pairwise relatively prime labels?



98

References

See www.cs.armstrong.edu/greenlaw


Joint work with:

Documents

processors n o1

n o1 processors

size n n o1

sequential time n o1

n o1 time polynomial

pn n o1

processor complexity

parallel time logno1