-
Singular Value Decomposition (SVD)
It is a fundamental matrix operation. A generalization of
diagonalization of a matrix.
The singular value decomposition for a matrix A writes A as a
product (hanger)(stretcher)(aligner)
Example: For an 5 x 4 matrix A, the singular value decomposition
of A is given by:
h and a are orthogonal and s is a 5 by 4 diagonal matrix with
real, nonnegative diagonal elements, si such that
si = 0 for i > min(5,4)
1
=
→
→
→
→
→→→→→
4
3
2
1
4
3
2
1
54321
a
a
a
a
0000s0000s0000s0000s
)h|h|h|h|h(A
-
If we use this operation on a matrix that is the
cross-covariance matrix of two data sets Ynt,ny and Znt,nz where nt
is time steps and ny and nz space/variable, just like CCA, SVD
isolates linear combinations of variables within the two fields
that tend to be linearly related.
If Ynt,ny and Znt,nz are equal, SVD provides the principal
componets.
SVD provides spatial patterns from the two fields that explains
most of the covariance between them.
Also called Maximum Covariance Analysis.
2
-
TheoryGiven two data sets Ynt,ny and Znt,nz
nt time steps. Same for Y and Z. ny nz space points/variables do
not need to be the same.
The mean per column of Y and Z must be zero.
Find spatial patterns G and H that are a linear combination from
Y and Z, respectively, explains most of the total covariance
U=YG; (1)V=ZH; (2)
The cross-covariance matrix of Y and Z is:
Z'Y1nt
1CYZ −=
The patterns G and H are chosen such, as the covariance between
U and V is maximized.
maxV'U1nt
1)V,Ucov( =−
= (3)
3
-
Using (1) and (2) in (3) comes:
maxHC'GZH'Y'G1nt
1)V,Ucov( YZ ==−=
We have now a maximization problem. Constraints: G'G=H'H=1, i.e.
the total variance of each of the fields equals one.
Now we have a maximization problem with its constrains.
Introduce the Lagrange multipliers µ and σ. Our function to be
maximized become:
F(G,H)=G'CYZH-σ(G'G-1)-µ(H'H-1)
Now, setting the derivatives 0'G
F =∂∂
and 0HF =
∂∂
we can find the
maximum values of F.
0GHC'G
FYZ =σ−=∂
∂(4)
0'HC'GHF
YZ =µ−=∂∂
(5) ***
Solution:
σ= GCH ZY
(CYZCZY-σ2I)G=0
Where σ2 are the eigenvalues of CYZCZY and G the
eigenvectors.
4
-
The SVD (matrix operation) of any matrix Mnt,nz is given by
M=GkσH'k
where Gk and Hk are orthogonal and σ is a nt by nz diagonal
matrix with real, nonnegative diagonal elements, σi, such that
σ1≥σ2≥…≥σmin(nt,nz) ≥ 0
Comparing with a eigenvalue problem:
σi are the eigenvalues (singular values) of M, Gk is the left
eigenvector (singular vector), Hk is the right eigenvector
(singular vector), k is the mode of the singular vector.
σ has as many nonzero elements as the minimum between nt and nz,
the remaining ones are zero.
G and H are composed by a number of vectors equal to the minimum
between nt and nz.
If SVD is applied to the CYZ cross-covariance matrix between two
fields - Ynt,ny the predictor (left field) and Znt,nz the
predictand (right field) - we can identify the pairs of spatial
patterns that explains most of the temporal covariance between the
two fields. Gny,k Hnz,k
The new time series are:
Unt,k = Ynt,ny*Gny,k and Vnt,k = Znt,nz*Hnz,k
5
-
For easier interpretation of the spatial patterns (eigenvectors
G and H), we look at them as
homogeneous correlation maps (g):
- vector of correlations between the values of one (Y or Z)
field and the kth mode of the singular vector of the same field (G
for Y and H for Z).
- indicator of the geographic localization of the covarying part
of the field.
the heterogeneous correlation maps (h):
- vector of correlations between the values of one field (Y or
Z) and the kth mode of the singular vector of the other field (H
for Y and G for Z).
- indicates how well the data points of one field can be
predicted from the kth expansion coefficient of the other
field.
You need to calculate the significance of these
correlations.
6
-
The maximum covariance is equal to the largest singular value
σ1.
The total squared covariance explained by a single pair of
patters (Gk,Hk) is σ2k.
The squared covariance fraction (SCF), i.e., the percentage of
the squared covariance explained by the pair of patterns, is:
∑=
σ
σ=
)nz,ntmin(
1l
2l
2k
kSCF
SCF is useful for comparing the relative importance of modes in
one given expansion.
For different SVD expansions it is used the normalized squared
covariance defined as:
2/1
i jji
2k
k varvarNSC
σ
=∑ ∑
vari and varj are the variances at the ith variable in the left
field and the jth variable in the right field respectively. For
normalized
data, 1varvari j
ji =∑ ∑ .
0 >= NSCk
-
Caveats
SVD can only be applied to data sets under specific
conditions.Solution: compare the heterogeneous maps with maps of
simple correlation.
Newman and Sardeshmukh, 1995, J. Climate, 8, 352-360.
Modeling
If Y and Z denote the predictor and predictand fields,
respectively, the model is generated by first calculating a matrix
of regression coefficients (A) which relates the values of
predictor time series (U) to the individual points in the
predictand field (Z).
and
but U=YG, so that
In practice, - choose only the significant modes of U to compose
the model so that the model will not fit to noise,- reserve part of
the data set for validation so that
8
U'UZ'UA =
YGAZ =∧
YvalGAvalZ =∧
UAZ =∧
-
-
Examples of SVD use:
SVD working as EOF Data: precipitation (Uvo and Berndtsson,
1996)
Stn 1 2 3 4 … ntime 1234
t9
-
10
-
11
-
SVD Relating SST to precipitation (Uvo et al. 1998)
12
-
13
-
14
NSC = 14.8%
NSC= 20.6%
-
Suggested references: Bretherton et al 1992 JCWallace et al.
1992 JCCherry 1996 JCUvo Berndtsson 1996 GJR- AtmosphereUvo et al.
1998 JC
15
-
Report
You are expected to read through the references above before
doing your report on SVD.
The report should contain at least:Description of the
data.Application of SVD to your data set. Physical interpretation
of the results.
How is the explained variance distributed among the modes?How
many modes you consider important, why?Can you explain physically
the modes you consider important? What do they mean?
If you are using the same data you developed CCA with, compare
CCA and SVD results.
Develop a SVD model.
Which one performed better? CCA or SVD model?
Other analysis or comments.
16