Iterative solution of saddle point problems Miroslav Rozloˇ zn´ ık, Pavel Jir´ anek Institute of Computer Science, Czech Academy of Sciences, Prague and Faculty of Mechatronics and Interdisciplinary Engineering Studies, Technical University of Liberec SNA 2007, Ostrava, 22. – 26. 1. 2007 M. Rozloˇ zn´ ık, P. Jir´ anek Iterative solution of saddle point problems 1
121
Embed
Iterative solution of saddle point problems - cs.cas.cz · Iterative solution of saddle point problems ... Chua, Desoer, Kuh, Strans, ... Primal vs. dual variable approach for
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Iterative solution of saddle point problems
Miroslav Rozloznık, Pavel Jiranek
Institute of Computer Science,Czech Academy of Sciences, Prague
and
Faculty of Mechatronics and Interdisciplinary Engineering Studies,Technical University of Liberec
SNA 2007, Ostrava, 22. – 26. 1. 2007
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 1
Outline
1 Applications leading to saddle point problems
2 Saddle point systems – equivalent definitions and properties
3 Basic solution approaches
4 Iterative methods for linear systems
5 Preconditioning of saddle point problems
6 Implementation and numerical stability
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 2
Problem statement – the most frequent definition:
Au ≡„
A BBT 0
«„xy
«=
„f0
«≡ b
A ∈ Rn×n is a square matrix of order n,
B ∈ Rn×m is an overdetermined matrix with n ≥ m,
f ∈ Rn is an n-dimensional right-hand side vector.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 3
Problem statement – generalizations and generalized saddle point problems:
„A B
BT −C
«„xy
«=
„fg
«
C ∈ Rm×m is a square matrix of order m,
g ∈ Rm is an m-dimensional right-hand side vector.
„A B
DT −C
«„xy
«=
„fg
«
D ∈ Rn×m is an overdetermined matrix with n ≥ m.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 4
Basic reference
M. Benzi, G. H. Golub, J. Liesen. Numerical solutionof saddle point problems. Acta Numerica, 2005, pp. 1–137.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 5
Applications leading to saddle point problems
Applications leading to saddle point problems
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 6
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 7
Applications leading to saddle point problems
Saddle point problems in Czech republic:
L. Luksan, J. Vlcek. Indefinitely preconditioned inexact Newton methodfor large sparse equality constrained non-linear programming problems.Numerical Linear Algebra with Applications 5, 1998, pp. 219–247.
Z. Dostal, D. Lukas. Multigrid preconditioned augmented Lagrangians forthe Stokes problem. Proceedings of SNA’06, ICS AS CR, 2006, pp. 25–28.
J. Kruis. Reinforcement-matrix interaction modelled by FETI method.Proceedings of SNA’06, ICS AS CR, 2006, pp. 51–54.
R. Kucera, J. Haslinger, T. Kozubek. An algorithm for solvingnonsymmetric saddle-point linear systems arising in FDM. Proceedings ofPANM 13, MI AS CR, 2006.
M. Hokr. Modelling of flow and transport problems in geological media.Proceedings of SNA’07, UGN AS CR, 2007.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 8
Applications leading to saddle point problems Application example
Uranium deposit Straz – geographical location and hydrogeological situation:
classical deep mining: 1966 – 1993,
underground acidic leaching: 1968 – 1996,
produced 14000 tons of uranium,
4 · 106 tons of H2SO4 injected in sandstone area,
190 · 106 m3 of contaminated water in cretaceous collectors,
hydrological barrier (injection of clean water),
drainage channels (pumping out the solution), mine drainage.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 9
Applications leading to saddle point problems Application example
Application fields solved in Diamo s.e.:
modelling of the underground water flow and transport of contaminants,
modelling of remediation scenarios,
modelling of flooding of the deep uranium mines,
modelling of chemical leakage from the waste pond,
fractured rock flow, modelling of radioactive waste deposit.
Mathematical models used in Diamo s.e.:
structural and situation models: describe the structure and state ofobjects, provide input data for other models,
models of flow and transport: computational models based on the FEMmethod, space and time discretization,
thermodynamical and kinetical models: modelling of chemical processesand reactions,
economical and optimization models: making decisions support and tools.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 10
Applications leading to saddle point problems Application example
Particular application – porous media flow:
impermeable (generally nonparallel) bottom and top layers,
modelled area covers approximately 120 km,
vertical thickness up to 200 m,
trilateral prismatic discretization,
3D meshes of the order from 2 · 105 to 8 · 105 elements,
systems of order 106 at every time step,
implementation and GWS software developed at Diamo s.e.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 11
Applications leading to saddle point problems Application example
References:
M. Rozloznık, V. Simoncini. Krylov subspace method for saddle pointproblems with indefinite preconditioning. SIAM J. Mat. Anal. Appl. 24,2002, pp. 368–391.
J. Maryska, M. Rozloznık, M. Tuma. The potential fluid flow problem andthe convergence rate of the minimal residual method. Numer. Lin. Alg.Appl. 3, pp. 525–542.
M. Arioli, J. Maryska, M. Rozloznık, M. Tuma. Dual variable methods formixed-hybrid finite element approximation of the potential fluid flowproblem in porous media. ETNA 22, 2006, pp. 17–40.
J. Maryska, M. Rozloznık, M. Tuma. Schur complement systems in themixed-hybrid finite element approximation of the potential fluid flowproblem. SIAM J. Sci. Comp. 22, 2000, pp. 704–723.
J. Maryska, M. Rozloznık, M. Tuma. Mixed hybrid finite elementapproximation of the potential fluid flow problem. J. Comp. Appl. Math.63, pp. 383–392.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 12
Applications leading to saddle point problems Application example
References:
J. Maryska, M. Rozloznık, M. Tuma. Primal vs. dual variable approach formixed-hybrid finite element approximation of the potential fluid flowproblem in porous media. Proceedings of the 3rd International Conferenceon “Large-Scale Scientific Computations”, Lecture Notes in ComputerScience 2179, Sv. Margenov, J. Wasniewski, P. Yalamov (eds.), June6-10, 2001, pp. 417–424.
P. Jiranek, M. Rozloznık. Maximum attainable accuracy of inexact saddlepoint solvers. submitted to SIMAX, 2006.
P. Jiranek, M. Rozloznık. Limiting accuracy of segregated solutionmethods for nonsymmetric saddle point problems. submitted to JCAM,2006.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 13
Saddle point systems – equivalent definitions and properties
Saddle point systems – equivalent definitions and properties
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 14
Saddle point systems – equivalent definitions and properties Solvability conditions
A is nonsingular ⇒
A is nonsingular ⇔ BTA−1B is nonsingular,„A B
BT 0
«=
„I 0
BTA−1 I
«„A B0 −BTA−1B
«.
A symmetric positive definite and B of full column rank⇒ BTA−1B is symmetric positive definite⇒ A is nonsingular.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 15
Saddle point systems – equivalent definitions and properties Solvability conditions
A symmetric positive semidefinite and B of full column rank ⇒
N(A) ∩ N(BT ) = 0 ⇔ A is nonsingular.
A nonnegative real ( 12(A + AT ) symmetric positive semidefinite)
and B of full rank ⇒
N( 12(A + A)T ) ∩ N(BT ) = 0 ⇒ A is nonsingular.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 16
Saddle point systems – equivalent definitions and properties Constrained optimization and least squares
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 27
Basic solution approaches
Basic solution approaches
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 28
Basic solution approaches
Basic solution schemes for saddle point problems:
1 segregated methods:reduce the whole problem to a smaller one, compute the component x or yas a solution of the reduced problem,back-substitution into the original system to obtain the remainingcomponent.
2 coupled methods:do not explicitly use the block structure of the problem,compute the components x and y at once.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 29
Basic solution approaches
Schur complement reduction:
Block LU factorization of the saddle point matrix:„I 0
BTA−1 −I
«„A B
BT 0
«„xy
«=
„I 0
BTA−1 −I
«„fg
«⇓„
A B0 BTA−1B
«„xy
«=
„f
BTA−1f − g
«
1 Solve the system with the Schur complement matrix
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 37
Basic solution approaches Application example
Null-space projection:
B is an incomplete incidence matrix of certain graph,
fixed geometry of the domain, iterative change of material (physical)properties (solving inverse problems or sequences of time-dependent ornonlinear problems),
use divergence-free finite elements (null-space approach embedded informulation) vs. fully algebraic mixed of mixed-hybrid approach.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 38
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 44
Basic solution approaches Application example
Choice of the null-space basis:
Fundamental cycle null-space basis based on incidence vectors of cyclesin the mesh: find a shortest path spanning tree; form cycles using non-treeedges;
σ(Z) ⊂ [1, c5h−2], λ(ZTAZ) ⊂ [c1, c2c
25h−4].
Orthogonal null-space basis based on the QR decomposition of B (MA49from HSL); projected system with ZT AZ does not depend on the meshsize h.
Partial null-space approach: Z has orthogonal columns and can beexplicitly constructed without any computation,
σ(ZTB) ⊂ [c7h, c8].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 45
Iterative methods for linear systems
Iterative methods for linear systems
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 46
Iterative methods for linear systems
Continuous problem
↓
Discretization
↓
Linear algebraic system
Dense or sparse direct solver?If we can solve the system directly, let’s do it!
If not, use iterative method with some preconditioner.
Use as much information from the problem as possible!
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 47
Iterative methods for linear systems
Linear system, iterative methods:
Au = b, A ∈ RN×N , b ∈ RN .
u0, r0 = b − Au0,
uk = . . . , rk = b − Auk ,
uk → u, rk → 0.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 48
Iterative methods for linear systems
Iterative methods:
stationary methods:
→ solvers,
→ preconditioners,
→ smoothers.
Uzawa method, augmented Lagrangian methods, other splittings,...
Krylov subspace methods:
→ solvers,
← need preconditioners.
CG, MINRES, GMRES,...
Algebraic multigrid, aggregation and multilevel methods
← need smoothers,
← need solvers,
← need preconditioners.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 49
Iterative methods for linear systems
References:
Overviews:
J. Stoer, Solution of large linear systems of conjugate gradient typemethods, In Mathematical Programming, Springer, Berlin, 1983,pp. 540–565.
R. Freund, G. H. Golub, N. Nachtigal, Iterative solution of linear systems,Acta Numerica 1, 1992, pp. 1–44.
Theory:
A. Greenbaum, Iterative Methods for Solving Linear Systems, SIAM,Philadelphia, 1997.
Practical issues:
Y. Saad, Iterative Methods for Sparse Linear Systems, PWS Pub. Co.,Boston, 1996 (2nd edition: SIAM, Philadelphia, 2003).
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 50
Iterative methods for linear systems Stationary methods
Uzawa iteration [Uzawa, 1958]:
1 choose y0
2 for k = 0, 1, 2, . . . until convergence
3 given yk compute xk+1 such that L(xk+1, yk) is minimized:xk+1 = A−1(f − Byk)
4 perform the Richardson update:yk+1 = yk + βk(B
T xk+1 − g)
5 end
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 51
Iterative methods for linear systems Stationary methods
Uzawa method as the fixed point iteration:
Muk+1 = Nuk + b,
M =
„A 0
BT −β−1I
«, N =
„0 −B0 −β−1I
«.
The iteration matrix
T =M−1N =
„0 −A−1B0 I − βBTA−1B.
«
Richardson method applied to the Schur complement system
Sy ≡ BTA−1By = BTA−1f .
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 52
Iterative methods for linear systems Stationary methods
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 60
Iterative methods for linear systems Krylov subspace methods
Minimal residual methods – convergence analysis:
‖rk‖ = minpk∈Pkpk (0)=1
‖pk(A)r0‖
A = VDV−1 is diagonalizable ⇒
‖rk‖‖r0‖
≤ minpk∈Pkpk (0)=1
maxλ∈λ(A)
‖Vpk(D)V−1‖
≤ κ(V) minpk∈Pkpk (0)=1
maxλ∈λ(A)
|pk(λ)|.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 61
Iterative methods for linear systems Krylov subspace methods
Normal systems – eigenvalues play an important role:
‖rk‖‖r0‖
≤ minpk∈Pkpk (0)=1
maxλ∈λ(A)
|pk(λ)|.
Diagonalizable systems – κ(V) can be large;
General nonnormal (nondiagonalizable) systems – we can get any convergencebehaviour independently on the spectrum [Greenbaum, Ptak, Strakos, 1996].
Another approaches: field of values and pseudospectra [Ernst, 2000], [Starke,1997], [Nachtigal, Reddy, Trefethen, 1992], polynomial numerical hull[Greenbaum, 2002].
The initial residual should be included in the analysis [Liesen, Strakos, 2005].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 62
Iterative methods for linear systems Krylov subspace methods
Symmetric case:
The convergence of symmetric iterative methods is essentially determined bythe eigenvalue distribution.
Positive definite case:
λ(A) ⊂ [a, b], 0 < b < a ⇒
minpk∈Pkpk (0)=1
maxλ∈λ(A)
|pk(λ)| ≤ 2
√a−√
b√
a +√
b
!k
,
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 63
Iterative methods for linear systems Krylov subspace methods
Indefinite case:
λ(A) ⊂ [−a,−b] ∪ [c, d ], 0 < b < a, 0 < c < d , a− b = d − c ⇒
minpk∈Pkpk (0)=1
maxλ∈λ(A)
|pk(λ)| ≤ 2
√ad −
√bc√
ad +√
bc
!»k2
–,
The asymptotic convergence rate can be estimated
limk→∞
„‖rk‖‖r0‖
« 1k≤ lim
k→∞
0@ minpk∈Pkpk (0)=1
maxλ∈I−∪I+
|pk(λ)|
1A1k
.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 64
Iterative methods for linear systems Application example
Unpreconditioned MINRES on the whole saddle point system:
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 72
Preconditioning of saddle point problems
Preconditioning of saddle point problems
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 73
Preconditioning of saddle point problems
Preconditioning = transformation of Au = b into another system.
left: M−1Au =M−1b,
right: AM−1v = b, u =M−1v ,
two-sided: M =M1M2,M−11 AM
−12 v =M−1
1 b, u =M−12 v .
Better convergence properties on the preconditioned system (M should bea “good” approximation to A),
M (or M−1) should be easily computed and the system withM shouldbe easily solved.
For symmetric systems, the convergence of iterative methods depends on thedistribution of eigenvalues of the system matrix → the cluster of eigenvaluesand/or reduced conditioning ensures fast convergence.
For nonsymmetric systems, the cluster of eigenvalues may be not enough (butit is in a practice) – reduction of minimal polynomial degree.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 74
Application dependent preconditioners – the information about theunderlying continuous problem is needed.
The preconditioner quality depends on how much information from theoriginal problem we use.
The range of problems, that can be treated by a particual preconditioner,is limited.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 75
Preconditioning of saddle point problems
Iterative solution – the method of choice:
Symmetric positive definite case + positive preconditioner:→ CG.
Symmetric indefinite case + positive definite preconditioner:→ CG, MINRES, SYMMLQ.
Symmetric indefinite case + indefinite preconditioner:→ GMRES; Simplified BiCG and QMR.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 76
Preconditioning of saddle point problems
Symmetric indefinite system + symmetric positive definite preconditioner:
Au = b
A symmetric indefinite, M symmetric positive definite.
M− 12AM− 1
2 v =M− 12 b, u =M− 1
2 v ,
Au = b, A is symmetric, but indefinite!
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 77
Preconditioning of saddle point problems
Symmetric indefinite system + indefinite preconditioner:
Au = b
A symmetric indefinite,M =M1M2 symmetric indefinite.
M1 andM2 can be nonsymmetric.
M−11 AM
−12 v =M−1
1 b, u =M−12 v ,
Au = b, A is nonsymmetric!
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 78
Preconditioning of saddle point problems
Iterative solution of indefinitely preconditioned nonsymmetric system:
A =M−11 AM
−12 , J =MT
1M2
⇓
ATJ = JA
Simplified J -symmetric Lanczos process
AVk = Vk+1Tk+1,k , ATWk = Wk+1Tk+1,k ,
W Tk Vk = I ⇒ Wn = JVn.
J -symmetric variant of Bi-CG and QMR [Freund, Nachtigal, 1995].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 79
Preconditioning of saddle point problems
Iterative solution of preconditioned system with simplified Lanczos process:
J -symmetric Bi-CG algorithm is nothing but classical CG algorithmpreconditioned with indefinite matrix J !
Preconditioned conjugate gradients method applied to indefinite system withindefinite preconditioning is in fact conjugate gradients method applied tononsymmetric (and often nonnormal) preconditioned system with AM−1.
Nevertheless, it frequently works in practice [R, Simoncini, 2002].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 80
Preconditioning of saddle point problems
Preconditioners for saddle point problems:
Preconditioners for saddle point problems exploit their block structure and weneed the information about the problem – (good) saddle point preconditionersare application dependent.
Basic preconditioning schemes for saddle point problems (overview [Zulehner,2002], [Axelsson, Neytcheva, 2003]):
block preconditioners,
constraint preconditioners,
incomplete factorizations for symmetric indefinite systems.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 81
Preconditioning of saddle point problems Block preconditioners
Block preconditioners rely on the availability of (approximate) solution ofsystems with A and S = BTA−1B.
Block factorization of A:
A =
„I 0
BTA−1 −I
« „A 00 S
«| z block diagonalpreconditioner
„I A−1B0 I
«
| z block triangularpreconditioner
.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 82
Preconditioning of saddle point problems Block preconditioners
Block diagonal preconditioners:
M =
„A 00 S
«.
The preconditioned saddle point matrix is diagonalizable and
λ(M−1A) = 1, 12(1±
√5)
[Murphy, Golub, Wathen, 2000] – GMRES terminates in at most three steps(also true for C 6= 0 [Ipsen, 2001]).
Application of the exact preconditioner is expensive – inexact preconditioning
M =
„A 0
0 S
«, A ≈ A, S ≈ S .
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 83
Preconditioning of saddle point problems Block preconditioners
Block diagonal preconditioners:
In [Silvester, Wathen, 1993, 1994], the case of C 6= 0 is considered. Theygive the spectral bounds for various choices of A and S assuming thespectral equivalence of A and A and BTA−1B and S .
In [Fisher, Ramage, Silvester, Wathen, 1998] (A spd, C = 0), thepreconditioner has the form A = αA, S = βS with α > 0, β ∈ −1, 1and S ≈ BTA−1B. The spectrum of for β = −1 and β = 1 is different butthe convergence of MINRES and GMRES is the same for both choices ofβ (for a fixed α) for the initial residual of the form r0 = (0, ∗)T .
In [de Sturler, Liesen, 2005], the authors gave bounds for the spectrum ofM with respect to the spectrum of M with A and S = BT A−1B (for thegeneralized saddle point problem).
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 84
Preconditioning of saddle point problems Block preconditioners
Block diagonal preconditioners:
Stokes problem with and without stabilization – using diagonal scaling[Wathen, Silvester, 1993]:
λ(M−1A) ⊂ (−a,−bh) ∪ (ch2, d).
Estimates for the asymptotic convergence rate [Wathen, Fisher, Silvester,1995].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 85
Preconditioning of saddle point problems Block preconditioners
Block triangular preconditioners:
M =
„A B0 S
«.
The preconditioning matrix is diagonalizable and its spectrum is
λ(M−1A) = ±1
M =
„A B0 −S
«.
The preconditioned saddle point matrix has the spectrum
λ(M−1A) = 1
but is not diagonalizable.
In both cases, the minimal polynomial degree is equal to 2 [Murphy, Golub,Wathen, 2000], [Ipsen, 2001] – GMRES terminates in at most two steps.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 86
Preconditioning of saddle point problems Block preconditioners
Block triangular preconditioners:
Inexact preconditioning
M =
„A B
0 S
«, A ≈ A, S ≈ S .
The application of block triangular preconditioner:
M−1 =
„A−1 00 I
«„I −B0 I
«„I 0
0 S−1
«.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 87
Preconditioning of saddle point problems Block preconditioners
Block triangular preconditioners:
M−1A is nonnormal – field of values analysis [Klawonn, Starke, 1999],[Loghin, Wathen, 2004].
For symmetric problems, the symmetry is destroyed (symmetrization[Bramble, Pasciak, 1988] seldom necessary) – compensated by the fastconvergence of GMRES.
The solution of preconditioned system (with r0 = (r(x)0 , 0)T ) is equivalent to the
null-space projection method, i.e. to the solution of
(I − Π)A(I − Π)x = (I − Π)f .
⇒ PCG can be applied (with an appropriate safeguarding strategy) [Luksan,Vlcek, 1998], [R, Simoncini, 2002], [Gould, Hribar, Nocedal, 2001].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 91
Preconditioning of saddle point problems Constraint preconditioners
Constraint preconditioner:
[Golub, Wathen, 1998] – A nonsymmetric but positive real (Oseenequation), M = 1
2(A + AT ); effective for sufficiently large viscosities;
inexact solves [Baggag, Sameh, 2004].
[Botchev, Golub, 2004] – small viscosity flows with M incorporating theskew-symmetric part of A.
[Axelsson, Neytcheva, 2003], [Bergamashi, Gondzio, Zilli, 2004], [Dollar,2005], [Durazzi, Ruggiero, 2001], [Perugia, Simoncini, 2000], [Toh, Phoon,Chan, 2004], [Zulehner, 2002] – A = AT , C 6= 0; systems with C 6= 0 areoften easier to solve – iterative methods converge faster – regularizedpreconditioning (for problems with C = 0) [Axelsson, 1979] with
M =
„A B
BT −εI
«, ε > 0.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 92
Implementation and numerical stability
Implementation and numerical stability
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 93
Implementation and numerical stability
Effects of rounding errors:
Delay of convergence:
rounding errors slow down the real rate of convergence,
rounding errors lead to loss of numerical rank of computed basis.
Limiting accuracy:
there is a limit in the accuracy of computed iterates,
Look for better preconditioning or better methods which perhaps mitigatethese effects.
Try more stable (but also more expensive) implementations.
Stopping criterion: level of termination cannot be arbitrarily small – it shouldbe above the maximum attainable accuracy level, stopping criteria based onthe backward error or related to the problem.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 94
Implementation and numerical stability
0 10 20 30 40 50 60 70 80 90 10010
−30
10−25
10−20
10−15
10−10
10−5
100
iteration number
A−
norm
of t
he e
rror
exact and finite precision CG on MINIJ(50)
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 95
Implementation and numerical stability Schur complement reduction
The Schur complement reduction:
A =
0BB@A B1 B2 B3
BT1 0 0 0
BT2 0 0 0
BT3 0 0 0
1CCA0BB@
upλ1
λ2
1CCA =
0BB@q1
q2
q3
q4
1CCA
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 96
Implementation and numerical stability Schur complement reduction
Subsequent reduction to the Schur complement systems withoutadditional fill-in: [Kaaschieter, Huijben, 1992], [Maryska, R, Tuma, 1996]
A → −A/A→ (A/A)/A11 − ((A/A)/A11)/B22.
Numerical stability of the block LU decomposition [Demmel, Higham,1995], [Higham, 1996], [Wilkinson, Reinsch, 1971], [Golub, Van Loan,1989].
Iterative solution of the final (symmetric positive definite) Schurcomplement system with the matrix −((A/A)/A11)/B22 for the unknownvector λ1 by CG with the prescribed backward error tol.
Block back-substitution process for the unknown vectors λ2, p and u usingthe factors from the Schur complement reduction.
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 97
Implementation and numerical stability Schur complement reduction
Maximum attainable accuracy of the computed approximate solutions:
Au = b + ∆b, A ∈ RN×N ,
‖∆b‖ ≤ O(maxtol, N32 ε)
ׄ‖A‖+ ‖B‖+ (1 + ‖B‖)‖A−1‖
×max‖A‖, ‖B‖(1 + ‖B‖)‖A−1‖p
κ(A)κ(B)
«‖u‖.
(A+ E)u = b,
‖E‖ ≤ O(maxtol, N32 εN− 1
3 ).
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 98
Implementation and numerical stability Schur complement reduction
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 99
Implementation and numerical stability Schur complement reduction
0 100 200 300 400 500 60010
−14
10−12
10−10
10−8
10−6
10−4
10−2
100
102
104
iteration number
recu
rsiv
e an
d tr
ue r
esid
ual n
orm
s
potential fluid flow problem − model problem
discretization parameter h=1/15
tol=1.0D−6
tol=1.0D−8
tol=1.0D−10
tol=1.0D−12
tol=1.0D−14
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 100
Implementation and numerical stability Maximum attainable accuracy of segregated methods
Maximum attainable accuracy of segregated methods:
Segregated methods compute x or y as the solution of a reduced system,compute the remaining component by the back-substitution into theoriginal system (this can be done using various back-substitution formulas).
The limiting accuracy level of computed approximate solutions xk and yk
measured by their (true) residuals f − Axk − Byk and −BT xk depends on:
the maximum over the norms of the iterates beginning from the initial stepup to the current iteration step [Greenbaum, 1994, 1997],
the backward error associated with the approximate solutions of innersystems which are solved inexactly [Simoncini, Szyld, 2003], [van den Eshof,Sleijpen, 2004],
the back-substitution formula [J, R, 2006].
M. Rozloznık, P. Jiranek Iterative solution of saddle point problems 101
Implementation and numerical stability Maximum attainable accuracy of segregated methods
Schur complement reduction:
Solution of inner systems Au = b with the backward error τ :
(A + ∆A)u = b + ∆b,‖∆A‖‖A‖ ,
‖∆b‖‖b‖ ≤ τ, τκ(A) 1.
Choose y0, x0 = A−1(f − By0), r(y)0 = −BT x0.
Update the approximation yk and the residual r(y)k :