MAT 280: Harmonic Analysis on Graphs & Networks …saito/courses/HarmGraph/lecture16.pdf · MAT 280: Harmonic Analysis on Graphs & Networks Lecture 16: Wavelets on Graphs Naoki Saito

MAT 280: Harmonic Analysis on Graphs & NetworksLecture 16: Wavelets on Graphs

Naoki Saito

Department of MathematicsUniversity of California, Davis

May 30, 2012

[email protected] (UC Davis) Wavelets on Graphs May 30, 2012 1 / 54

Outline

1 Wavelets using Graph Laplacians

2 Haar-Like Wavelets on Graphs

3 Summary


Outline



3 Summary


Work of Hammond-Vandergheynst-Gribonval

D. K. Hammond, P. Vandergheynst, R. Gribonval: “Wavelets ongraphs via spectral graph theory,” Applied and ComputationalHarmonic Analysis, vol. 30, no. 2, pp. 129-150, 2011.Conceptually, it is an adaptation of the continuous wavelet transformfor graphs.Let G (V ,E ) be a weighted graph with |V | = n.The graph Fourier transform of f ∈ L2(V ) is defined as

f (`) = 〈f , φ`〉 =n∑

k=1

f (k)φ∗` (k), ` = 0, 1, . . . , n − 1

where φ` := (φ`(1), . . . , φ`(n))T ∈ Rn is the `th graph Laplacianeigenvector corresponding to the eigenvalue λ`. The original vector fcan be reconstructed by

f (k) =n−1∑`=0

f (`)φ`(k), k = 1, 2, . . . , n.


Work of Hammond-Vandergheynst-Gribonval . . .

Let Tg = g(L) : L2(V )→ L2(V ) be defined as a Fourier multiplier as

Tg f (`) = g(λ`)f (`),

where g is a wavelet generating kernel, also called the spectral graphwavelet kernel (SGWT kernel). We now have:

(Tg f )(k) =n−1∑`=0

g(λ`)f (`)φ`(k).

The spectral graph wavelet operator at scale s > 0 is defined byT sg = g(sL). Hence, the spectral wavelet function at scale s and

vertex vm is realized as ψs,m(k) := T sg δm(k) where δm is an impulse

located at vm.

ψs,m(k) =n−1∑`=0

g(sλ`)φ∗` (m)φ`(k).



The wavelet coefficient of a given function f ∈ L2(V ) is computed by

Wf (s,m) := 〈f , ψs,m〉 =(T sg f)

(m) =n−1∑`=0

g(sλ`)f (`)φ`(m).

Lemma (H-V-G)

If the SGWT kernel g satisfies the admissibility condition:∫ ∞0

g2(x)

xdx =: Cg <∞, and g(0) = 0, then

1

Cg

n∑m=1

∫ ∞0

Wf (s,m)ψs,m(k)ds

s=: f ](k)

where f ] = f − 〈f , φ0〉φ0. In other words, f can be reconstructed fromthe information {Wf (s,m)} and the DC component.



An example of g(x):

g(x) =

(x/x1)α for 0 ≤ x < x1;

s(x) for x1 ≤ x ≤ x2;

(x2/x)β for x > x2.

H-V-G used x1 = 1; x2 = 2; α = β = 2; and s(x) = −5 + 11x − 6x2 + x3.



The scaling function ϕ can be defined as:

ϕm = ϕ1,m := Thδm = h(L)δm,

where h : R+ → R+ acts as a low pass filter with h(0) > 0 andh(x)→ 0 as x →∞. An example is: h(x) = γ exp

(−(x/0.6λmin)4

)where γ is set such that h(0) = maxx≥0 g(x).The scaling coefficient of a given function f ∈ L2(V ) is computed by

Sf (m) := 〈f , ϕm〉 .



For a discrete transform, sample the scale parameter s in alogarithmically equispaced manner between sJ = x2/λmax ands1 = x2/λmin where λmax ≥ λn−1, λmin = λmax/K for some K > 0.

Theorem (H-V-G)

Given a set of scales {sj}1≤j≤J , the setF := {ϕm}1≤m≤n ∪ {ψsj ,m}1≤j≤J;1≤m≤n constitutes a frame with boundsA, B given by

A = minλ∈[0,λn−1]

G (λ); B = maxλ∈[0,λn−1]

G (λ)

where G (λ) := h2(λ) +∑J

j=1 g2(sjλ).

Hence, for any f ∈ L2(V ), we have:

A‖f ‖22 ≤n∑

m=1

|Sf (m)|2 +J∑

j=1

|Wf (sj ,m)|2 ≤ B‖f ‖22.



H-V-G proposed a fast transform instead of computing all of thegraph Laplacian eigenvalues and eigenvectors that would requireO(n3) operations.

The fast algorithm requires O(C · |E |+ C ′ · Jn), where C ,C ′ > 0 aresome constants.

The fast algorithm is based on the Chebyshev polynomialapproximation p(sjx) to the function g(sjx) and fully utilizes theChebyshev recurrence relation.

Hence Wf (sj ,m) ≈ δ∗m p(sjL) f , i.e., done by matrix-vector products.

It is especially effective if the graph (hence L) is sparse.

As for the inverse transform, a stable algorithm exists because F

forms a frame. As the frame theory indicates, it involves the pseudoinverse. Hence, it is not super fast even if one uses the conjugategradient method.

Software demo now!


My Reaction to SGWT

The eigenvalue axis is not the same as the frequency axes particularlyif a given graph comes from data from a topologically skewed shape ora narrow strip. In those cases, the eigenvalue orders are not intuitive.

The inverse transform is still slow even if one uses the CG method.


Outline



3 Summary


Work of Coifman-Gavish-Nadler

R. R. Coifman and M. Gavish: “Harmonic analysis of digital databases,” in Wavelets and Multiscale Analysis: Theory and Applications(J. Cohen and A. Zayed, eds.), Birkhauser, Chap. 9, pp.161–197,2011.

R. R. Coifman, M. Gavish, B. Nadler: “Multiscale wavelets on trees,graphs and high dimensional data: Theory and applications to semisupervised learning,” Proc. 27th Intern. Conf. Machine Learning,2010.

The following slides are through the courtesy of R. R. Coifman, M.Gavish, and B. Nadler.



Haar-like bases

(Coifman, Gavish) Harmonic Analysis of Data Bases January 14, 2010 21 / [email protected] (UC Davis) Wavelets on Graphs May 30, 2012 14 / 54


Haar-like bases



Haar-like bases



Haar-like bases



Haar-like basesWith B. Nadler, Weizmann






















































Tensor product of Haar-like bases













































Tensor Haar-like basis function


My Reaction to C-G-N’s Work

Is there any automatic algorithm to build a multiscale tree of graphs(i.e., multiscale graph coarsening) of a given original graph?

Can one develop smoother wavelets instead of Haar-like basis on agraph?

If so, when such smoother wavelets become critically useful? =⇒Maybe some network flow data, e.g.:

flow measurements in river systems or drainage systems;biological system (e.g., pulse propagation on nervous systems). . .


Outline



3 Summary


Summary

Another very interesting and recent idea is to adapt Donoho’s averageinterpolating wavelets to graph setting proposed by R. M. Rustamov:ArXiv:1110.2227v1 [math.FA], which should be studied very closely.

Very important to transfer harmonic and wavelet analysis techniques originallydeveloped on the usual Euclidean spaces to graphs

Discrete harmonic, analytic, Green’s functions on graphs have been developed (L.Lovasz, F. Chung, S.-T. Yau, . . . )

Cannot avoid tight interactions with mathematicians in different disciplines(discrete math, graph theory, optimization, numerical linear algebra, geometry &topology, PDEs, probability and statistics, . . . ) and with domain experts (biology,sociology, electrical engineering, computer science, geology, geophysics, . . . )

Many interesting mathematics will come out from this endeavor!

There are many more proposals to construct wavelets on graphs that I could notcover today, e.g., diffusion wavelets of Coifman & Maggioni, an application of thelifting scheme to graphs by Jansen, Nason, & Silverman, . . .

Analysis of directed graphs needs more attention.

How about models based on quantum graphs?


MAT 280: Harmonic Analysis on Graphs & Networks …saito/courses/HarmGraph/lecture16.pdf · MAT 280: Harmonic Analysis on Graphs & Networks Lecture 16: Wavelets on Graphs Naoki Saito

Documents