Spatiotemporal Pattern Extraction by Spectral Analysis of ... · Spatiotemporal Pattern Extraction by Spectral Analysis of Vector-valued Observables Dimitris Giannakis Center for
Post on 15-Jul-2020
2 Views
Preview:
Transcript
Spatiotemporal Pattern Extractionby Spectral Analysis of Vector-valued Observables
Dimitris Giannakis
Center for Atmosphere Ocean ScienceCourant Institute of Mathematical Sciences
New York University
Geometry and Topology of DataICERM, 12/11/2017
Collaborators: Abbas Ourmazd, Joanna Slawinska, Jane Zhao
10−4 m 10−1 m 102 m
105 m 107 m 1025 m
Setting
~F (xn)
~F
AX
x0
xn = Φnτ x0
~F (A)
HY
Y
~F (x0)
• Dynamical flow Φt : X 7→ X on a manifold with an ergodic invariant measureµ, supported on a compact set A ⊆ X
• Compact spatial domain Y , equipped with a finite measure ρ
• Continuous, vector-valued observation map ~F : X 7→ C(Y )
Objective. Given time-ordered measurements ~F (x0), ~F (x1), . . ., with
xn = Φnτ (x0), decompose ~F into spatiotemporal patterns ~φj : X 7→ C(Y ),
~F =∑j
cj ~φj , cj ∈ R
Separable space-time patterns
A widely used approach is to recover temporal patterns through theeigenfunctions of an operator T on HA = L2(A, µ),
Tϕk = λkϕk , ϕk ∈ HA
Many choices for T , including:
• Covariance operators (POD, PCA, SSA, . . . )• Heat operators (Laplacian eigenmaps, diffusion maps, . . . )• Koopman operators (DMD, EDMD, . . . )
Spatial patterns ψk in HY = L2(Y , ν) can then be obtained by pointwiseprojection of the observation map onto the temporal patterns:
ψk(y) = 〈ϕk ,Fy 〉HA , Fy : x ∈ A 7→ ~F (x)(y)
This is equivalent to treating ~F as a function in the tensor product spaceHA ⊗ HY , and performing the decomposition
~F ≈ Fl =l∑
k=0
ϕk ⊗ ψk
• In the presence of symmetries and/or spatiotemporal intermittency, puretensor product patterns, ϕk ⊗ ψk , may suffer from poor descriptive efficiencyand physical interpretability (Aubry et al. 1993)
Hilbert spaces of observables
We have the Hilbert space isomorphisms
H ' HA ⊗ HY ' HM ,
HA = L2(A, µ), HY = L2(Y , ν), H = L2(A, µ;HY ), HM = L2(M, ρ),
with M = A× Y ⊆ Ω = X × Y , ρ = µ× ν
As a result, the observation map ~F can be equivalently thought of as:
1 A vector-valued observable ~F : A 7→ HY in H
2 An element of the tensor product space HX ⊗ HY , i.e., ~F =∑
jk cjkeAj ⊗ eYk
for bases eAj of HA and eYk of HY
3 A scalar-valued observable F : M 7→ R in HM , s.t. F (x , y) = ~F (x)(y)
Given x ∈ A, the function t 7→ ~F (Φt(x)) corresponds to a spatiotemporalpattern
Vector-valued spectral analysis (VSA) framework
We decompose ~F using the eigenfunctions of a compact operator PQ : H 7→ H,
PQ~φj = λj
~φj , ~F ≈ ~Fl =l∑
j=0
cj ~φj , cj ∈ R
This operator is associated with an operator-valued kernel (Micchelli & Pontil
2005, Caponnetto et al. 2008,Carmeli et al. 2010), constructed from delay-coordinatemapped data with Q delays
PQ~f =
∫A
LQ(·, x)~f (x) dµ(x), LQ : X × X 7→ L(HY )
Desirable properties include:
• Ability to recover patterns without a tensor product structure
• Symmetry group actions are naturally factored out
• Asymptotic commutation property with Koopman operators allows to identifyintrinsic dynamical timescales
Operator-valued kernel construction
For the purposes of this work:
1 A scalar-valued kernel on Ω = X × Y will be a continuous function
k : Ω ×Ω 7→ R+,
bounded above and away from zero on compact sets
2 An operator-valued kernel on X will be a continuous function
κ : X × X 7→ L(HY )
Associated with k and κ are kernel integral operators K : HM 7→ HM andK : H 7→ H, respectively, where
Kf =
∫M
k(·, ω)f (ω) dρ(ω), K~f =
∫A
κ(·, x)~f (x) dµ(x)
We can assign k to the operator-valued kernel κ, where
κ(x , x ′) = Kxx′ , Kxx′g(y) =
∫Y
k((x , y), (x ′, y ′))g(y ′) dν(y ′)
Kernels from delay-coordinate maps (G. & Majda 2012; Berry et al. 2013; G. 2017; Das & G. 2017)
1 Start from a pseudometric dQ : Ω ×Ω 7→ R0, s.t.,
d2Q((x , y), (x ′, y ′)) =
1
Q
Q−1∑q=0
|F (Φ−qτ (x), y)− F (Φ−qτ (x ′), y ′)|2.
2 Choose a continuous shape function h : R0 7→ [0, 1], and define the kernel
kQ : Ω ×Ω 7→ R+, kQ(ω, ω′) = h(dQ(ω, ω′));
here, h(s) = e−s2/ε, with ε > 0
3 Normalize kQ to obtain a continuous Markov kernel pQ : Ω ×Ω 7→ R+ usingthe procedure introduced in the diffusion maps algorithm (Coifman & Lafon 2006):
pQ(ω, ω′) =kQ(ω, ω′)
lQ(ω)rQ(ω′), rQ =
∫M
kQ(·, ω) dρ(ω), lQ =
∫M
kQ(·, ω)
rQ(ω)dρ(ω)
The kernel pQ induces the compact operators PQ : HM 7→ HM andPQ : H 7→ H, s.t.
PQ f =
∫M
pQ(·, ω)f (ω) dρ(ω), PQ~f =
∫A
LQ(·, x)~f (x) dµ(x)
where LQ : X × X 7→ L(HY ) is the operator-valued kernel associated with pQ
Vector-valued eigenfunctions
• Identify spatiotemporal patterns, t 7→ ~φj(Φt(x)), through the eigenfunctions of
PQ :PQ
~φj = λj~φj , ~φj ∈ H, 1 = λ0 > λ1 ≥ λ2 ≥ · · ·
• Expand the observation map ~F in the ~φj eigenbasis of H, i.e.,
~F =∞∑j=0
cj ~φj , cj = 〈~φ′j , ~F 〉H ,
where ~φ′j are eigenfunctions of P∗Q , satisfying 〈~φ′j , ~φk〉H = δjk
• Operationally, we obtain (λj , ~φj) through the eigenvalue problem for PQ ,
PQφj = λjφj , φj ∈ H, ~φj(x)(y) = φj((x , y))
Remark. The ~φj are not restricted to a pure tensor product form, ϕj ⊗ ψj , withϕj ∈ HA and ψj ∈ HY
Bundle structure of spatiotemporal data
• The kernel kQ can be expressed as a pullback of a kernel κQ on RQ , the spaceof delay-coordinate sequences with Q delays,
kQ(ω, ω′) = kQ(FQ(ω),FQ(ω′)),
FQ(ω) = (F (ω),F (ω−1), . . . ,F (ω−Q+1)), ω = (x , y), ωq = (Φqτ (x), y)
• Defining BQ = FQ(Ω) and πQ : Ω 7→ BQ s.t. πQ(ω) = FQ(ω), the triplet(Ω,BQ , πQ) is a topological bundle, with total space Ω, base space BQ , andprojection map πQ
• This partitions Ω into equivalence classes, [·]Q , s.t. ω′ ∈ [ω]Q ifπQ(ω) = πQ(ω′)
• Every function in the closed subspace
HQ = ranPQ = spanφj : λj > 0 ⊆ HM ,
is a pullback of a function in L2(JQ , αQ), with JQ = πQ(M) and αQ = πQ(ρQ),i.e., it is ρ-a.e. constant on the [·]Q equivalence classes
• HQ is not necessarily expressible as a tensor product of HA and HY subspaces.
Limit of no delays
If no delays are performed (Q = 1), and M is connected, then J1 = π1(M) is aclosed interval
• The eigenfunctions φj are pullbacks of orthogonal functions ηj on J with respectto the L2 inner product associated with the pushforward measure α1 = π1∗ρ,
φj(ω) = ηj(π1(ω)) = ηj(F (ω))
• In particular, the φj are constant on the level sets of the obsevation map F
In a number of cases (e.g., α1 has a C 2 density wrt. Lebesgue measure, and thekernel bandwidth ε is small), η1 will be monotonic
• In such cases, even the one-term expansion F ≈ F1 = c1φ1 recovers thequalitative features of the input signal
Spatial symmetries
An important example with nontrivial [·]Q equivalence classes is that of PDEmodels with equivariant dynamics under the action of a group G on thespatial domain Y
• Suppose that X is a subset of HY (e.g., an inertial manifold of a dissipativePDE system), and there is a group action Γ g
Y : Y 7→ Y , g ∈ G , satisfying
Φt Γ gX = Γ g
X Φt , Γ g
X (x) = x Γ g−1Y
• Then, defining Γ gΩ = Γ g
X ⊗ ΓgY , the following diagram commutes:
Ω Ω
BQ
ΓgΩ
πQπQ
Spatial symmetries
Under the previous assumptions:
1 For every ω ∈ Ω, the G -orbit ΓΩ(ω) = Γ gΩ(ω) | g ∈ G lies in [ω]Q
2 Moreover, the pseudometric dQ has the invariance property
dQ(Γ gΩ(ω), Γ g′
Ω (ω′)) = dQ(ω, ω′),
for all ω, ω′ ∈ Ω and g , g ′ ∈ G
If, in addition, Γ gΩ preserves null sets with respect to ρ, then it induces a
representation of G on HM , with representatives
RgM : HM 7→ HM , Rg
M f = f Γ gΩ
Theorem. The operators PQ and RgM satisfy [PQ ,R
gM ] = 0 and PQR
gM = PQ for
all g ∈ G . As a result, every eigenspace Wj of PQ at nonzero eigenvalue is afinite-dimensional (by compactness of PQ), trivial representation space of G ,i.e., Rg
M f = f for every f ∈Wj .
Remark. In PCA-type decompositions, ϕj ⊗ ψj , the spatial (ψj) and temporal(ϕj) patterns also lie in G representation spaces, but the representations arenot necessarily trivial
Correspondence with Koopman operators
• Consider the unitary group of Koopman operators U t : HA 7→ HA, t ∈ R,acting on scalar-valued observables in HA by composition with the flow map,
U t f = f Φt
• A distinguished class of observables in HA is that of Koopman eigenfunctions,
U tzj = e iωj tzj , 〈zj , zk〉HA = δjk , ωj ∈ R
• This leads to the U t-invariant decomposition
HA = D ⊕D⊥, D = spanzj
• Because the system is ergodic, the eigenspaces of U t are all one-dimensional
• Similarly, we can define a group of unitary Koopman operators U t : HM 7→ HM ,with
U t f = (U t ⊗ IHY )f = f (Φt ⊗ IY )
• In this case, we have the U t-invariant decomposition
HM = D ⊕ D⊥, D = D ⊗ HY ,
but the eigenspaces of U t , spanzj ⊗ HY , are infinite-dimensional
Correspondence with Koopman operators
In the limit of infinitely many delays, the following hold (Das & G., 2017):
1 d∞ = limQ→∞ dQ is well-defined as a function in HM ⊗ HM
2 d∞ lies in D ⊗ D3 d∞ is invariant under U t ⊗ U t
Therefore, we can define a compact operator P∞ : HM 7→ HM , and ranP∞ ⊆ D
Theorem. The operators P∞ and U t commute for all t ∈ R. As a result, theyare simultaneously diagonalizable on the finite-dimensional eigenspaces of P∞.
Corollary. The eigenspaces Wj of P∞ corresponding to nonzero eigenvalue λj
have the form Wj = spanzj ⊗ Vj , where zj ∈ HA is an eigenfunction of U t
and Vj a finite-dimensional subspace of HY
Note also the following:
1 In the presence of symmetries, P∞, U t , and RgM are mutually commuting
operators
2 In Dynamic Mode Decomposition (DMD) (Schmidt & Henningson 2008, Rowley et al.
2009) and related techniques, one assigns a single spatial patternψj(y) = 〈zj ,Fy 〉HA (Koopman mode) to a given Koopman eigenfunction; here,the number of such patterns is equal to dimVj
Data-driven approximation
• In many cases of interest, the invariant set A is a non-smooth subset of X ofzero Lebesgue measure (e.g., a fractal attractor)
• Moreover, in realistic experimental environments, the sampled dynamical statesdo not lie exacty on A
Assumptions for data-driven approximation
1 The measure µ is physical; that is, there exists a set Bµ ⊆ X , of positiveLebesgue measure, such that for every f ∈ C(X ) and x ∈ Bµ,
limNX→∞
1
NX
NX∑n=0
f (Φnτ (x)) =
∫A
f dµ (1)
2 There exists a compact, forward invariant set U ⊆ X , of positive Lebesguemeasure, s.t. A ⊆ U ⊆ Bµ
3 We make measurements on Y at a sequence of points y0, y1, . . . such that ananalog of (1) holds for f ∈ C(Y )
4 Measurements F (xn, yr ), xn = Φnτ (x0), are available along an (unknown) orbitstarting at x0 ∈ Bµ
Data-driven approximation
• As a data-driven analog of HM = L2(M, ρ), we employ the N-dimensional,N = NXNY , Hilbert space HΩ,N = L2(M, ρN) associated with the samplingmeasure,
ρN =1
N
N−1∑j=0
δωj , ωj = (xnj , yrj ), 0 ≤ nj ≤ NX − 1, 0 ≤ rj ≤ NY − 1
• On this space, PQ is approximated by PQ,N : HΩ,N 7→ HΩ,N , where
PQ,N f =
∫Ω
pQ,N(·, ω)f (ω) dρN(ω) =1
N
N−1∑j=0
pQ,N(·, ωj)f (ωj),
pQ,N(ω, ω′) =kQ(ω, ω′)
lQ,N(ω′)rQ,N(ω),
rQ,N =
∫Ω
kQ(·, ω) dρN(ω), lQ,N =
∫Ω
kQ(·, ω)
rQ,N(ω)dρN(ω)
• The eigenvalue problem for PQ,N is equivalent to an N × N matrix eigenvalueproblem
Spectral convergence
• One issue with establishing convergence (pointwise, in operator norm, etc) ofPQ,N to PQ is that, as defined, these operators act on different spaces (HΩ,Nand HM , respectively)
• We thus examine the analogous operators PQ,N and PQ on C(V),V = U × Y ⊆ Ω, defined using the same (continuous) kernels as PQ,N and PQ ,respectively.
Theorem. For every nonzero eigenvalue λj of PQ and correspondingeigenfunction φj ∈ HM :
1 The sequence of eigenvalues λj,N of PQ,N , N ≥ j − 1, converges to λj
2 There exist eigenfunctions φj,N ∈ HΩ,N such that φj,N ∈ C(V) convergesuniformly to φj ∈ C(V), where
φj,N =1
λj,N
∫Ω
pQ,N(·, ω)φj,N(ω) dρN(ω), φj =
∫M
pQ(·, ω)φj(ω) dρ(ω)
Proof. Establish that (i) PQ is compact, and (ii) PQ,N converges compactly toPQ . The claim then follows from spectral approximation results for compactoperators (Von Luxburg et al 2008; Chatelin 2011).
Application to the Kuramoto-Sivashinksy model
• The Kuramoto-Sivashinsky (KS) model is a prototype dissipative PDE modelexhibiting complex spatiotemporal dynamics, while having a number of usefulknown properties such as inertial manifolds (Foias et al. 1986) and symmetries(Kevrekidis et al. 1990, Cvitanovic et al. 2009)
• The governing equation for the real-valued scalar field u(t, ·) : Y 7→ R, t ≥ 0,Y = [0, L], is given by
u = −u∇u + ∆u −∆2u, u ∈ HY = L2(Y , Leb),
subject to periodic boundary conditions
• The domain size parameter L controls the dynamical complexity of the system;here, we apply VSA to data generated by the KS model at the chaotic regimesL = 22 and L = 94
• For our purposes, the state space manifold X ⊆ HY will be an inertial manifoldof the KS system
KS patterns, L = 22
At a small number of delays, Q = 15, the recovered eigenfunctions areapproximately constant on the level sets of the input signal
KS patterns, L = 22
• At a larger number of delays Q = 500, the leading vector-valued eigenfunctionscapture O(2) families of unstable equilibria (wavenumber L/2 structures), andsmaller-scale traveling waves embedded in those structures
• In contrast, NLSA (G. & Majda 2012), a scalar-valued kernel technique alsoutilizing delays, requires several modes to capture these families
KS patterns, L = 94
Conclusions
Kernel algorithms operating on spaces of vector-valued observables have anumber of useful properties for spatiotemporal pattern extraction, including theability to:
• Recover patterns with a non-separable structure in the spatial and temporaldegrees of freedom
• Quotient out symmetries
• Recover intrinsic timescales associated with the point spectrum of theKoopman operator of the dynamical systems
Physical measures allow spectral convergence of data-driven approximations ofsuch operators for ergodic dynamical systems with non-smooth invariant sets
Ongoing and future work includes applications in atmosphere ocean scienceand extensions of VSA to skew-product dynamical systems
References
• Giannakis, D., J. Slawinska, A. Ourmazd, Z. Zhao (2017). Vector-ValuedSpectral Analysis of Space-Time Data. Proceedings of the NIPS 2017 TimeSeries Workshop.
• Giannakis, D., A. Ourmazd, J. Slawinska, Z. Zhao (2017). Spatiotemporalpattern extraction by spectral analysis of vector-valued observables. Submitted.arXiv: 1711.02798.
• Das, S., and D. Giannakis (2017). Delay-coordinate maps and the spectra ofKoopman operators. arXiv:1706.08544
• Giannakis, D. (2017). Data-driven spectral decomposition and forecasting ofergodic dynamical systems. Applied and Computational Harmonic Analysis,doi:10.1016/j.acha.2017.09.001
Research supported by DARPA grant HR0011-16-C-0116, NSF grantDMS-1521775, ONR grant N00014-14-1-0150, and ONR YIP grantN00014-16-1-2649
top related