PuLP Complex Objective Partitioning of Small-World Networks Using Label Propagation George M. Slota 1,2 Kamesh Madduri 2 Sivasankaran Rajamanickam 1 1 Sandia National Laboratories, 2 The Pennsylvania State University [email protected], [email protected], [email protected]SIAM CSE15 17 March 2015 1 / 18
28
Embed
PuLP - Complex Objective Partitioning of Small-World ...kxm85/papers/PULP_CSE15_slides.pdfPuLP Complex Objective Partitioning of Small-World Networks Using Label Propagation George
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
PuLPComplex Objective Partitioning of Small-World Networks
We present PuLP, a multi-constraint multi-objectivepartitioner
Shared-memory parallelism (OpenMP)
PuLP demonstrates an average speedup of 14.5×relative to state-of-the-art partitioners on small-worldgraph test suite
PuLP requires 8-39× less memory than state-of-the-artpartitioners
PuLP produces partitions with comparable or betterquality than state-of-the-art partitioners for small-worldgraphs
Exploits the label propagation clustering algorithm
2 / 18
Label PropagationAlgorithm progression
Randomly label with n labels
Iteratively update each v with max per-label count over neighbors, ties broken randomly
Algorithm completes when no new updates possible
3 / 18
Label PropagationAlgorithm progression
Randomly label with n labels
Iteratively update each v with max per-label count over neighbors, ties broken randomly
Algorithm completes when no new updates possible
3 / 18
Label PropagationAlgorithm progression
Randomly label with n labels
Iteratively update each v with max per-label count over neighbors, ties broken randomly
Algorithm completes when no new updates possible
3 / 18
Label PropagationAlgorithm progression
Randomly label with n labels
Iteratively update each v with max per-label count over neighbors, ties broken randomly
Algorithm completes when no new updates possible
3 / 18
Label PropagationAlgorithm progression
Randomly label with n labels
Iteratively update each v with max per-label count over neighbors, ties broken randomly
Algorithm completes when no new updates possible
3 / 18
Label PropagationOverview and observations
Label propagation: initialize a graph with n labels,iteratively assign to each vertex the maximal per-labelcount over all neighbors to generate clusters, ties brokenrandomly (Raghavan et al. 2007)
Clustering algorithm - dense clusters hold same labelFast - each iteration in O(n+m)Naıvely parallel - only per-vertex label updatesObservation: Possible applications for large-scalesmall-world graph partitioning
4 / 18
PartitioningProblem description, objectives and constraints
Goal: minimize execution time for small-world graphanalytics
Constraints:
Vertex and edge balance: per-task memory andcomputation requirements
Objectives:
Edge cut and max per-part cut: total and per-taskcommunication requirements
Single level vs. multi level partitioners
Multi level produces high quality at cost of computingSingle level produces sub-optimal quality
Until now...
5 / 18
PartitioningProblem description, objectives and constraints
Goal: minimize execution time for small-world graphanalytics
Constraints:
Vertex and edge balance: per-task memory andcomputation requirements
Objectives:
Edge cut and max per-part cut: total and per-taskcommunication requirements
Single level vs. multi level partitioners
Multi level produces high quality at cost of computingSingle level produces sub-optimal quality
Until now...
5 / 18
PuLPOverview and description
PuLP : Partitioning using Label Propagation
Utilize label propagation for:
Vertex balanced partitions, minimize edge cut (PuLP)Vertex and edge balanced partitions, minimize edge cut(PuLP-M)Vertex and edge balanced partitions, minimize edge cutand maximal per-part edge cut (PuLP-MM)Any combination of the above - multi objective, multiconstraint
6 / 18
PuLP-MMAlgorithm overview
Randomly initialize p partitionsCreate partitions with degree-weighted label propagationfor Some number of iterations do
for Some number of iterations doBalance partitions to satisfy vertex constraintRefine partitions to minimize edge cut
for Some number of iterations doBalance partitions to satisfy edge constraint and minimize
per-part edge cutRefine partitions to minimize edge cut
7 / 18
PuLP-MMPuLP-MM algorithm
Randomly initialize partition labels for p parts
Run degree-weighted label prop to create initial partsIteratively balance for vertices, minimize edge cutBalance for edges, minimize per-part edge cut
Network shown is the Infectious network dataset from KONECT (http://konect.uni-koblenz.de/) 8 / 18
PuLP-MMPuLP-MM algorithm
Randomly initialize partition labels for p partsRun degree-weighted label prop to create initial parts
Iteratively balance for vertices, minimize edge cutBalance for edges, minimize per-part edge cut
Network shown is the Infectious network dataset from KONECT (http://konect.uni-koblenz.de/) 8 / 18
PuLP-MMPuLP-MM algorithm
Randomly initialize partition labels for p partsRun degree-weighted label prop to create initial parts
Iteratively balance for vertices, minimize edge cutBalance for edges, minimize per-part edge cut
Network shown is the Infectious network dataset from KONECT (http://konect.uni-koblenz.de/) 8 / 18
PuLP-MMPuLP-MM algorithm
Randomly initialize partition labels for p partsRun degree-weighted label prop to create initial partsIteratively balance for vertices, minimize edge cut
Balance for edges, minimize per-part edge cut
Network shown is the Infectious network dataset from KONECT (http://konect.uni-koblenz.de/) 8 / 18
PuLP-MMPuLP-MM algorithm
Randomly initialize partition labels for p partsRun degree-weighted label prop to create initial partsIteratively balance for vertices, minimize edge cut
Balance for edges, minimize per-part edge cut
Network shown is the Infectious network dataset from KONECT (http://konect.uni-koblenz.de/) 8 / 18
PuLP-MMPuLP-MM algorithm
Randomly initialize partition labels for p partsRun degree-weighted label prop to create initial partsIteratively balance for vertices, minimize edge cutBalance for edges, minimize per-part edge cut
Network shown is the Infectious network dataset from KONECT (http://konect.uni-koblenz.de/) 8 / 18
PuLP-MMLabel propagation step
P ← PULP-lp(G(V,E), p, P,N, Ip)Minv ← (n/p)× (1− εl)i← 0, r ← 1while i < Ip and r 6= 0 do
r ← 0for all v ∈ V do
C(1 · · · p)← 0for u ∈ E(v) do
C(P (u))← C(P (u)) + |E(u)|x← Max(C(1 · · · p))if x 6= P (v) and N(P (v))− 1 > Minv then
P (v)← xr ← r + 1
i← i+ 1return P
9 / 18
PuLP-MMVertex Balancing Step
P ← PULP-vp(G(V,E), P, p,N, Ib)i← 0, r ← 1Maxv ← (n/p)× (1 + εu)Wv(1 · · · p)← Max(Maxv/N(1 · · · p)− 1,0)while i < Ib and r 6= 0 do
r ← 0for all v ∈ V do
C(1 · · · p)← 0for all u ∈ E(v) do
C(P (u))← C(P (u)) + |E(u)|for j = 1 · · · p do
if Moving v to Pj violates Maxv thenC(j)← 0
elseC(j)← C(j)×Wv(j)
x← Max(C(1 · · · p))if x 6= P (v) then
Update(N(P (v)), N(x))Update(Wv(P (v)),Wv(x))P (v)← xr ← r + 1
i← i+ 110 / 18
PuLP-MMEdge balancing and max per-part cut minimization step
P ← PULP-cp(G(V,E), P, p,N,M, T, U, Ib)i← 0, r ← 1Maxv ← (n/p)× (1 + εu), Maxe ← (m/p)× (1 + ηu)CurMaxe ← Max(M(1 · · · p)), CurMaxc ← Max(T (1 · · · p))We(1 · · · p)← CurMaxe/M(1 · · · p)− 1, Wc(1 · · · p)← CurMaxc/T (1 · · · p)− 1Re ← 1, Rc ← 1while i < Ip and r 6= 0 dor ← 0for all v ∈ V doC(1 · · · p)← 0for all u ∈ E(v) doC(P (u))← C(P (u)) + 1
for j = 1 · · · p doif Moving v to Pj violates Maxv , CurMaxe , CurMaxc then
C(j)← 0elseC(j)← C(j)× (We(j)× Re +Wv(j)× Rc)
x← Max(C(1 · · · p))if x 6= P (v) thenP (v)← xr ← r + 1Update all variables for x and P (v)
Test graphs:LAW graphs from UF Sparse Matrix, SNAP, MPI, KoblenzReal (one R-MAT), small-world, 60 K–70 M vertices,275 K–2 B edges
Test Algorithms:METIS - single constraint single objectiveMETIS-M - multi constraint single objectiveParMETIS - METIS-M running in parallelKaFFPa - single constraint single objectivePuLP - single constraint single objectivePuLP-M - multi constraint single objectivePuLP-MM - multi constraint multi objective
Metrics: 2–128 partitions, serial and parallel running times, memoryutilization, edge cut, max per-partition edge cut
12 / 18
ResultsPuLP Running Times - Serial (top), Parallel (bottom)
In serial, PuLP-MM runs 1.7× faster (geometric mean) than nextfastest of METIS and KaFFPa
uk-2005 graph from LAW, METIS-M (left) vs. PuLP-MM (right)Blue: low comm; White: avg comm; Red: High commPuLP reduces max inter-part communication requirements andbalances total communication load through all tasks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Part Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Par
t Num
ber
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Part Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Par
t Num
ber
16 / 18
ResultsRe-balancing for multiple constraints and objectives
PuLP can balance a quality but unbalanced (single objective and constraint)partition
We observe up to 11% further improvement in edge cut produced byPuLP-MM when starting with a single constraint partition from KaFFPa
No cost to second objective (max per-part cut), our approaches show up to400% improvement relative to KaFFPa
●
●
●
●
●
●
●
0.1
0.2
0.3
0.4
2 4 8 16 32 64 128Number of Parts
Edg
e C
ut R
atio
Partitioner ● PULP−MM KaFFPa KaFFPa+PULP−MM
●
●
●
●
●
●
●
0.025
0.050
0.075
2 4 8 16 32 64 128Number of Parts
Max
Per
−pa
rt E
dge
Cut
Rat
io
Partitioner ● PULP−MM KaFFPa KaFFPa+PULP−MM
17 / 18
Conclusionsand future work
Average of 14.5× faster, 23× less memory, better edgecut or better per-part edge cut than next best testedpartitioner on small-world graph test suite
Future work:
Implementation in Zoltan2Explore techniques for avoiding local minima, such assimulated annealing, etc.Further parallelization in distributed environment formassive-scale graphsExplore tradeoff and interactions in various parametersand iteration countsNew initialization procedures for various graph types(meshes, road nets, etc.)