Page 1
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
82nd GAMM Annual Scientific ConferenceGraz, April 19, 2011
Preconditioned Inverse Iteration forHierarchical Matrices
Peter Benner and Thomas Mach
Max Planck Institute for Dynamics of Complex Technical SystemsComputational Methods in Systems and Control Theory
Magdeburg
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 1/28
Page 2
Hierarchical (H-)Matrices PINVIT Numerical Results
Outline
1 Hierarchical (H-)Matrices
2 PINVIT
3 Numerical Results
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 2/28
Page 3
Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Some dense matrices, e.g. BEM or FEM, can be approximated byH-matrices in a data-sparse manner.
hierarchical tree TI block H-tree TI× I
I = {1, 2, 3, 4, 5, 6, 7, 8}
{1, 2, 3, 4} {5, 6, 7, 8}
{1, 2} {3, 4} {5, 6} {7, 8}
{1}{2}{3}{4}{5}{6}{7}{8}
1 1 1 12 2 2 23 3 3 34 4 4 45 5 5 56 6 6 67 7 7 78 8 8 8
1 1 1 12 2 2 23 3 3 34 4 4 45 5 5 56 6 6 67 7 7 78 8 8 8
dense matrices, rank-k-matrices
rank-k-matrix: Ms×t = ABT , A ∈ Rn×k ,B ∈ Rm×k (k � n,m)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 3/28
Page 4
Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Hierarchical matrices
H(TI× I, k) ={M ∈ RI× I
∣∣ rank (Ma×b) ≤ k ∀a× b admissible}
22
3 3
7 10
3 7
3 10
19 10
10 31
14 8
11 11
14 118 11
19 10
10 31 11
11 31
11 9
9 16 12
11 1611 8
9 16 11
11 16
619 73 3
8
11 11
9 3
7 3
11
8 11
25 10
10 19 11
11 31
8 5
11 8
8 15 8 12
116 515 6
13
13
7 5
13 8
8 11 8 11
116 5
15 6
8
11 8
8 15
5 8 11
126 15
5 6 13
137
13 8
8 11
5 8 11
116 15
5 6
6110 103
6 14
10 3 6
10 14
25 10 6
10
6
19 10
10 31
15 9
9 11 10
10 1615 9
8 11 10
10 16
5110 107 9 7
3 3
10 7
109 3
7 3
25 11
1125 10
10 19
11 811 8
8 15 9
8 15 12 13
1310 7
13 8
8 11 9
9 15 1119
2010 7
13 8
9
11 8
8 15 11 12
139 7
13 8
9
13 8
8 1111
1011 8
8 15 8
9 15
7 12 13
1310
13 8
8 11 8
9 15
7 1120
199
13 9
8
11 8
8 15
7 11 12
129
13 9
813 8
8 11
7 11
39 10
10 25
3 7
3 10 107 10 6
3 3
7 10 7
1010
6
22
3 3
7 10
3 7
3 10
19 10
10 31
15 10
11
11 9
9 16
15 11
1011 8
9 16
34 10
10 25
13 10
7 1113 7
10 11 61
6 513 6 11
12
8 5
11 8
8 15 812
136 5
13 6 11
11 236 13
5 6 12
118
11 8
8 15
5 813
12
6 13
5 6 10
11 23
20 9
9 39
9 7
103 7
3 10
9 107
3 3
7 10 61
15 10
1015 9
9 11
15 10
1015 9
8 11
20 9
9 34
9 7
10 13
9 107 13 51
adaptive rank k(ε)
storage NSt,H(T , k) = O(n log n k(ε))
complexity of approximate arithmetic
MHv O(n log n k(ε))+H,−H O(n log n k(ε)2)
∗H,HLU(·), (·)−1H O(n (log n)2 k(ε)2)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 4/28
Page 5
Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Hierarchical matrices
H(TI× I, k) ={M ∈ RI× I
∣∣ rank (Ma×b) ≤ k ∀a× b admissible}
22
3 3
7 10
3 7
3 10
19 10
10 31
14 8
11 11
14 118 11
19 10
10 31 11
11 31
11 9
9 16 12
11 1611 8
9 16 11
11 16
619 73 3
8
11 11
9 3
7 3
11
8 11
25 10
10 19 11
11 31
8 5
11 8
8 15 8 12
116 515 6
13
13
7 5
13 8
8 11 8 11
116 5
15 6
8
11 8
8 15
5 8 11
126 15
5 6 13
137
13 8
8 11
5 8 11
116 15
5 6
6110 103
6 14
10 3 6
10 14
25 10 6
10
6
19 10
10 31
15 9
9 11 10
10 1615 9
8 11 10
10 16
5110 107 9 7
3 3
10 7
109 3
7 3
25 11
1125 10
10 19
11 811 8
8 15 9
8 15 12 13
1310 7
13 8
8 11 9
9 15 1119
2010 7
13 8
9
11 8
8 15 11 12
139 7
13 8
9
13 8
8 1111
1011 8
8 15 8
9 15
7 12 13
1310
13 8
8 11 8
9 15
7 1120
199
13 9
8
11 8
8 15
7 11 12
129
13 9
813 8
8 11
7 11
39 10
10 25
3 7
3 10 107 10 6
3 3
7 10 7
1010
6
22
3 3
7 10
3 7
3 10
19 10
10 31
15 10
11
11 9
9 16
15 11
1011 8
9 16
34 10
10 25
13 10
7 1113 7
10 11 61
6 513 6 11
12
8 5
11 8
8 15 812
136 5
13 6 11
11 236 13
5 6 12
118
11 8
8 15
5 813
12
6 13
5 6 10
11 23
20 9
9 39
9 7
103 7
3 10
9 107
3 3
7 10 61
15 10
1015 9
9 11
15 10
1015 9
8 11
20 9
9 34
9 7
10 13
9 107 13 51
adaptive rank k(ε)
storage NSt,H(T , k) = O(n log n k(ε))
complexity of approximate arithmetic
MHv O(n log n k(ε))+H,−H O(n log n k(ε)2)
∗H,HLU(·), (·)−1H O(n (log n)2 k(ε)2)
A1
BT1
+ A2
BT2
= A1 A2
BT1
BT2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 4/28
Page 6
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λi in O(n (log n)α kβ)?
X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 5/28
Page 7
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)?
X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 5/28
Page 8
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Page 9
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Page 10
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Page 11
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Page 12
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
Definition
The function
µ(x) = µ(x ,M) =xTMx
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
xi+1 := xi − α∇µ(xi ), ∇µ(x) =2
xT x(Mx − xµ(x)) ,
+ preconditioning ⇒ update equation:
xi+1 := xi − B−1 (Mxi − xiµ(xi )) .
Residual r(x) = Mx − xµ(x).
Preconditioned residual B−1r(x) = B−1 (Mx − xµ(x)).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28
Page 13
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr 2009]
xi+1 := xi − B−1 (Mxi − xiµ(xi ))
If
M ∈ Rn×n symmetric positive definite and
B−1 approximates the inverse of M, so that∥∥I − B−1M∥∥M≤ c < 1,
then Preconditioned INVerse ITeration (PINVIT) converges andthe number of iterations is independent of n.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 7/28
Page 14
Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration[Knyazev, Neymeyr, et al.]
The residual
ri = Mxi − xiµ(xi )
converges to 0, so that
‖ri‖2 < ε
is a useful termination criterion.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 8/28
Page 15
Hierarchical (H-)Matrices PINVIT Numerical Results
Variants of PINVIT[Neymeyr 2001: A Hierarchy of Precond. Eigens. for Ellipt. Diff. Op.]
Classification by Neymeyr:
PINVIT(1): xi+1 := xi − B−1ri .
PINVIT(2): xi+1 := arg minv∈span{xi ,B−1ri} µ(v).
PINVIT(3): xi+1 := arg minv∈span{xi−1,xi ,B−1ri} µ(v).
PINVIT(n): Analogously.
PINVIT(·,d): Replacing x by a rectangular full rank matrixX ∈ Rn×d one gets the subspace version of PINVIT(·).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 9/28
Page 16
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H
O(n (log n)2 k (c)2)
R := MX0 − X0µ, µ = XT0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)
O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi
O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB® eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 10/28
Page 17
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 10/28
Page 18
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)?
X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 11/28
Page 19
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 11/28
Page 20
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 12/28
Page 21
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d < O(log n),
use subspace version PINVIT(·,d).
0λn λn−1λn−2. . .
λ1
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Page 22
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d � log n?
shift with σ near λi .
0λn. . .
λi+1 λi λi−1. . .
λ1
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Page 23
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d � log n,
shift with σ near λi .
0λn. . .
λi+1 λi λi−1. . .
λ1σ
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Page 24
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α kβ)?
If i = n − d with d � log n,
shift with σ near λi .
0λn. . .
λi+1 λi λi−1. . .
λ1σ
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28
Page 25
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
Folded Spectrum Method [Wang, Zunger 1994]
Mσ = (M − σI)2
Mσ is s.p.d., if M is s.p.d. and σ 6= λi .
Assume all eigenvalues of Mσ are simple.
Mv = λv ⇔ Mσv = (M − σI)2 v
= M2v − 2σMv + σ2v
= λ2v − 2σλv + σ2v
= (λ− σ)2v
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 14/28
Page 26
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Page 27
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Page 28
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Page 29
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Page 30
Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.2 Compute
a) Mσ = (M − σI)2 andb) M−1
σ or LLT = Mσ.
3 Use PINVIT to find the smallest eigenpair (µσ, v) of Mσ.4 Compute µ = vTMv/vTv .
(µ, v) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting isprohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28
Page 31
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)?
X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 16/28
Page 32
Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = MT ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn},λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓How to find λn in O(n (log n)α kβ)? X
How to find λi in O(n (log n)α kβ)? X
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 16/28
Page 33
Hierarchical (H-)Matrices PINVIT Numerical Results
Numerical Results
Numerical Results
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 17/28
Page 34
Hierarchical (H-)Matrices PINVIT Numerical Results
Hlib
Hlib [Borm, Grasedyck, et al.]
We use the Hlib1.3 (www.hlib.org) for theH-arithmetic operations and some examples out ofthe library for testing the eigenvalue algorithm.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 18/28
Page 35
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 5.6146E-010 0.01
FEM16 256 4.5918E-010 0.02 2.00 106.67
FEM32 1 024 3.7550E-010 0.12 6.17 27.08
FEM64 4 096 3.8009E-010 0.82 6.68 8.06
FEM128 16 384 4.4099E-010 5.84 7.09 5.44
FEM256 65 536 3.9651E-010 34.47 5.91 5.22
FEM512 262 144 3.7877E-010 194.00 5.63 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
∥∥∥∥∥∥∥∥∥
λ1 − λ1λ2 − λ2λ3 − λ3λ4 − λ4
∥∥∥∥∥∥∥∥∥2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 19/28
Page 36
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 5.6146E-010 0.01
FEM16 256 4.5918E-010 0.02 2.00 106.67
FEM32 1 024 3.7550E-010 0.12 6.17 27.08
FEM64 4 096 3.8009E-010 0.82 6.68 8.06
FEM128 16 384 4.4099E-010 5.84 7.09 5.44
FEM256 65 536 3.9651E-010 34.47 5.91 5.22
FEM512 262 144 3.7877E-010 194.00 5.63 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
∥∥∥∥∥∥∥∥∥
λ1 − λ1λ2 − λ2λ3 − λ3λ4 − λ4
∥∥∥∥∥∥∥∥∥2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 19/28
Page 37
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-inversion
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni ))
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 20/28
Page 38
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-inversion
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni )) MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 20/28
Page 39
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28
Page 40
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28
Page 41
Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n, X0 ∈ Rn×d (XT0 X0 = I , e.g. randomly chosen)
Output: Xp ∈ Rn×d , µ ∈ Rd×d , with ‖MXp − Xpµ‖ ≤ εB−1 = (M)−1H or B−1 = L−TH L−1H O(n (log n)2 k (c)2)R := MX0 − X0µ, µ = XT
0 MX0
for (i := 1;‖R‖F > ε; i + +) do
Xi := Orthogonalize(Xi−1 − B−1R
)O(n (log n) k (c)2)
R := MXi − Xiµ, µ = XTi MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrixinversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2).
Competitive to MATLAB eigs.
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28
Page 42
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-Cholesky decomp.
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 4.6920E-010 0.01
FEM16 256 4.7963E-010 0.02 2.00 106.67
FEM32 1 024 3.4696E-010 0.08 4.00 27.08
FEM64 4 096 4.6414E-010 0.48 6.00 8.06
FEM128 16 384 3.3206E-010 3.20 6.67 5.44
FEM256 65 536 3.8468E-010 13.90 4.34 5.22
FEM512 262 144 3.1353E-010 62.40 4.49 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-Cholesky decomposition)
∥∥∥∥∥∥∥∥∥
λ1 − λ1λ2 − λ2λ3 − λ3λ4 − λ4
∥∥∥∥∥∥∥∥∥2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 22/28
Page 43
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-Chol. decomp.
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni ))
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 23/28
Page 44
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
FEM16FEM32
FEM64FEM128
FEM256FEM512
10−3
10−2
10−1
100
101
102
103
104
CP
Uti
me
ins
H-PINVIT(1,4) H-Chol. decomp.
H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni )) MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 23/28
Page 45
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Page 46
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−1, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Page 47
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−2, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Page 48
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−3, FEM64
−1 −0.5 00.5 1−1
0
1−5
0
5
·10−2
−6
−4
−2
0
2
4
6·10−2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28
Page 49
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, inner eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 1.7219E-013 <0.01
FEM16 256 7.2388E-012 0.03 106.67
FEM32 1 024 6.0045E-013 0.12 3.57 27.08
FEM64 4 096 1.0915E-012 0.93 7.48 8.06
FEM128 16 384 1.5655E-012 6.08 6.50 5.44
FEM256 65 536 1.1976E-011 52.40 8.62 5.22
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 25/28
Page 50
Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, inner eigenvalues
2D Laplace over [−1, 1]× [−1, 1], precond.: H-Cholesky decomp.
Name ni error titi
ti−1
N(ni )N(ni−1)
FEM8 64 1.5306E-013 <0.01
FEM16 256 9.8071E-012 0.02 106.67
FEM32 1 024 1.0577E-012 0.05 2.50 27.08
FEM64 4 096 1.5776E-012 0.31 6.20 8.06
FEM128 16 384 2.1416E-012 1.64 5.29 5.44
FEM256 65 536 6.7400E-012 9.73 5.93 5.22
FEM512 262 144 1.5002E-010 545.00 56.01 6.51
d = 4, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-Cholesky decomposition)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 25/28
Page 51
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
dense matrix, but approximable by an H-matrix ⇒ MATLAB eigsis expensive, precond.: H-inversion
Name ni error titi
ti−1
N(ni )N(ni−1)
BEM8 258 7.5498E-08 0.02
BEM16 1 026 2.0455E-07 0.16 8.00 24.18
BEM32 4 098 5.4044E-07 1.33 8.31 24.22
BEM64 16 386 1.8159E-06 11.90 8.95 23.99
BEM128 65 538 — 146.00 12.27 14.66
d = 6, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 26/28
Page 52
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
dense matrix, but approximable by an H-matrix ⇒ MATLAB eigsis expensive, precond.: H-Cholesky decomp.
Name ni error titi
ti−1
N(ni )N(ni−1)
BEM8 258 8.2758E-08 0.02
BEM16 1 026 1.9546E-07 0.18 9.00 24.18
BEM32 4 098 5.2752E-07 0.95 5.28 24.22
BEM64 16 386 2.1935E-06 7.66 8.06 23.99
BEM128 65 538 — 33.20 4.33 14.66
d = 6, c = 0.2, εeig = 10−4, N(ni ) = ni (log2 ni )CspCid
Time t only H-PINVIT (without H-Cholesky decomposition)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 26/28
Page 53
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-inversion O(N(ni ) log ni )
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Page 54
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-inversion O(N(ni ) log ni )
MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Page 55
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-Chol. decomp. O(N(ni ) log ni )
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Page 56
Hierarchical (H-)Matrices PINVIT Numerical Results
BEM example, smallest eigenvalues
BEM8BEM16
BEM32BEM64
BEM12810−3
10−2
10−1
100
101
102
103
104
105
CP
Uti
me
ins
H-PINVIT(1,6) O(N(ni ))H-Chol. decomp. O(N(ni ) log ni )
MATLAB eigs
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Page 57
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Page 58
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Page 59
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Page 60
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Page 61
Hierarchical (H-)Matrices PINVIT Numerical Results
Conclusions
Once we have inverted M ∈ H(T , k) finding the eigenvaluesby H-PINVIT is cheap and storage efficient.
For dense, data-sparse matrices, H-PINVIT can solveproblems where the MATLAB function eigs (implicitly,restarted Arnoldi/Lanczos iteration) is not applicable.
The folded spectrum method enables us to compute innereigenvalues, too.
Further details in our preprint MPIMD/11-01:http://www.mpi-magdeburg.mpg.de/preprints/.
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28
Page 62
Appendix
Adaptive H-Inversion [Grasedyck 2001]
Adaptive H-Inversion
Input: M ∈ H (TI×I ), c = c√‖M‖2
∈ R
Output: M−1H , with∥∥I −M−1H M
∥∥2< c
Compute M−1H with εlocal = c/(Csp(M) ‖M‖H2 )
δ0M :=∥∥I −M−1H M
∥∥H2/c
while δiM > 1 doCompute M−1H with εlocal = εlocal/δ
iM
δiM :=∥∥I −M−1H M
∥∥H2/c
end∥∥I −M−1H M∥∥2< c = c√
‖M‖2⇔
∥∥I −M−1H M∥∥M≤ c
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Page 63
Appendix
Adaptive H-Cholesky Decomposition
Adaptive H-Cholesky Decomposition
Input: M ∈ H (TI×I ), c = c√‖M‖2
∈ R
Output: LHLTH, with
∥∥∥I − L−TH L−1H M∥∥∥2< c
Compute LHLTH = M with εlocal = c/(Csp(M) ‖M‖H2 )
δ0M :=∥∥∥I − L−TH L−1H M
∥∥∥H2/c
while δiM > 1 doCompute LHL
TH = M with εlocal = εlocal/δ
iM
δiM :=∥∥∥I − L−TH L−1H M
∥∥∥H2/c
end∥∥∥I − L−TH L−1H M∥∥∥2< c = c√
‖M‖2⇔
∥∥∥I − L−TH L−1H M∥∥∥M≤ c
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 27/28
Page 64
Appendix
Why is it called Preconditioned Inverse Iteration?
Inverse Iteration:
xi+1 = µ(xi )A−1xi ,
xi+1 = xi − A−1Axi︸ ︷︷ ︸=0
+µ(xi )A−1xi ,
xi+1 = xi − A−1 (Axi + µ(xi )xi ) .
Replace A−1 with the inexact solution by the preconditioner B:
xi+1 = xi − B−1 (Axi + µ(xi )xi ) .
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 28/28