Effective-resistance Preserving Spectral Reduction of Graphs Zhiqiang Zhao Zhuo Feng Design Automation Group Department of Electrical & Computer Engineering Michigan Technological University Acknowledgements: NSF CCF grants #1350206, #1318694 & #1618364, KeySight Technologies.
18
Embed
Design Automation Group Effective-resistance Preserving ...zhuofeng/MTU_VLSI_DA_files/... · – Key to spectral graph partitioning & data clustering [1-2] [1] J. Lee, et al. Multiway
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Effective-resistance Preserving Spectral Reduction of Graphs
Zhiqiang Zhao Zhuo Feng
Design Automation Group
Department of Electrical & Computer EngineeringMichigan Technological University
– Key to development of faster numerical and graph-related algorithms:• Solving large PDEs & sparse matrices, graph partitioning & data clustering, semi-supervised learning
(SSL), graph convolution neural networks, maximum flows of undirected graphs, …
– Allow for efficient handling of big (data) graphs• Big graph computing on personal computers, or even mobile devices
Spectral Reduction
Applications in VLSI CAD Spectrally-reduced circuits preserve effective resistances [1]
– Opportunity: development of much faster VLSI CAD algorithms
2
4 6
8 11
13 14
1 18
1621 17 22
25 27
2
4 6
8 11
13 14
1 18
1621 17 22
25 27
Ci1
VinCs1
clk1 clk2
clk2 clk1
Ci2
Cs2
clk1 clk2
clk2 clk1
y1y2
Cf1
clk2 clk1
clk1 clk2
Cf2
clk2 clk1
clk1 clk2
Comparator
Q
_Q
D flip-flop
CLK1
d
d1 d2
0 50 100 150 2000
50
100
150
200
2
4
6
8
10
12
x 10-3
Spectral Graph Approximation
Spectral Circuit Sparsification & Reduction
Nearly-Linear Time VLSI CAD Algorithms
M2M1
R7
M5
L1
L0C0
Vlo+ M3 M4
M6
R1
R3
R8
L2
R10
L3C1
R2
Vrf+ R5 Vrf-R6
Vlo-R4
VDD
[1] [8]
[21 ] [16]
[25] [27]
[20] [7]
[15]
[26 ]
[13] [14][11] [18]
[22][17]
[4] [6]
[2]
[21]
[2]
[1] [8]
[16]
[25] [27]
[20] [7]
[15]
[13] [14][11] [18]
[22][17]
[4] [6]
[1] Z. Zhao, and Z. Feng. A spectral graph sparsification approach to scalable vectorless power grid integrity verification. DAC’17.
reduced graph original graph reduced graph
Applications in Machine Learning & Data Mining Graph Convolutional Neural Networks deal with unstructured data [1]
– Spectrally-reduced graphs preserve key features & reduce computation cost
[1] M. Defferrard, et al. Convolutional neural networks on graphs with fast localized spectral filtering. NIPS’16[2] T. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. ICLR’17[3] Hamilton, Will, et al. "Inductive representation learning on large graphs." In NIPS’17
Figure modified from [1]
Overview of Our Approach
Graph Density?
Spectral Graph Reduction
Original Graph
C. Effective-ResistancePreserving Post Scaling
Spectrally Reduced Graph
B. Spectral Graph Sparsification & Scaling
A. Spectrum-Preserving Node Reduction
High
B. Spectral Graph Sparsification & Scaling
A. Spectrum-Preserving Node Reduction
Low
Original Graph
Reduced Graph
Sparsified Reduced Graph
A
B & C
Graph Laplacian Matrix Graph Laplacian matrix (admittance matrix of a resistor network)
– Graph 𝐺𝐺 = 𝑉𝑉,𝐸𝐸,𝑤𝑤 and its Laplacian matrix 𝐿𝐿𝐺𝐺:• 𝑳𝑳𝑮𝑮 is symmetric diagonally dominant (SDD)• Eigenvalues of 𝑳𝑳𝑮𝑮 are nonnegative
( , )
( , ) if ( , )( , ) ( , ) if
otherwise0G u v E
w u v u v EL u v w u v u v
∈
− ∈
= ==
∑
1 2
4
3
5
1.5
22
1.51
0.5
A Weighted Graph 𝑮𝑮
3.5 1.5 21.5 4 2 0.5
2 3 10.5 1 3 1.5
2 1.5 3.5
− − − − − − − − − − − −
Graph Laplacian 𝑳𝑳𝑮𝑮
Laplacian eigenvectors vs signals in Fourier analysis– Higher “frequencies” in Fourier analysis higher eigenvalues [1]
– “Low-frequency” eigenvectors graph global properties
[1] D. Shuman, et al. "The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains." IEEE Signal Processing Magazine (2013)
Global graph embedding using the first few Laplacian eigenvectors– First k eigenvectors for k-dimensional spectral graph embedding
– Key to spectral graph partitioning & data clustering [1-2]
[1] J. Lee, et al. Multiway spectral partitioning and higher-order Cheeger inequalities. JACM’14[2] R. Peng, et al. Partitioning well-clustered graphs: Spectral clustering works!." COLT’15.
𝒘𝒘𝟑𝟑
𝒘𝒘𝟐𝟐
𝒘𝒘𝟒𝟒
A naïve spectral node reduction scheme: aggregate nodes that are close to each other in the embedding spaceChallenge: too costly eigenvector computations
Spectral Aggregation: A Local Embedding Approach
Smoothers in multigrid for approximating low eigenvectors
Aggregation by spectral embedding w/ smoothed vectors:
low to high freq. components of a random vector(combination of all eigenvectors)
low freq. components after smoothing
smooth
(combination of the first few eigenvectors)
𝑋𝑋(1),𝑋𝑋(2),⋯ ,𝑋𝑋 𝐾𝐾 : K smoothed vectors obtained by running a few Gauss-Seidel (GS) relaxations w/ initial K random vectors [1]
[1] O. Livne and A. Brandt. Lean algebraic multigrid (LAMG): Fast graph Laplacian linear solver, SIAM Journal on Scientific Computing, 34(4):B499–B522, 2012.
Step B: Spectral Graph Sparsification & Scaling Find a graph proxy (sparsifier) to mimic the original graph
– w/ the same set of nodes but much fewer edges
– Key to designing fast numerical & graph algorithms
Spectral graph sparsifiers– Spectral sparsifiers preserve (Spielman & Teng. SIAM J. Comp.’11):
• Eigenvalues & eigenvectors of graph Laplacian matrices• Pair-wise distances, commute times, cuts between nodes, …• Effective resistances of resistor networks
The original graph The sparsified graph
Laplacian quadratic form measures the boundary size
Edges going out S:
1( )
0x u
=
if node u is in S
otherwise
( , )TGx L x cut S S=
S
( , , )G V E w=
0
0 0
1
1
max 1
min 11 0
max max ( , )
min min
Courant Fischer theorem:
( , )T
TGx
TGx
x
x L x cut S S
x L x cut S S
τ
τ=
==
= ⇒
−
= ⇒
S
( , , )G V E w=x1 x2
x3
x4
x5
Fiedler vector: the eigenvector for minτ
Revisiting Graph Laplacian Eigenvectors
Generalized Eigenvectors for Embedding & Sparsification [1]
max 1
Courant Fischer theorem for generalized eigenvalues
max cuts in Gmin c
maxuts in P
TG
TxP
x L xx L x
λ=
−
= ≥
Graph G Subgraph P
Max cut mismatch is bounded by 𝜆𝜆𝐦𝐦𝐦𝐦𝐦𝐦 !
[1] Z. Feng. Spectral Graph Sparsification in Nearly-Linear Time Leveraging Efficient Spectral Perturbation Analysis. DAC’16
An Optimization Framework for Edge Scaling Subgraph scaling via constrained optimizations
– Minimize 𝜆𝜆𝐦𝐦𝐦𝐦𝐦𝐦 by scaling up edge weights iteratively
– Control the decrease of 𝜆𝜆min for improving spectral similarity