1 Carol S. Woodward and Radu Serban Center for Applied Scientific Computing, LLNL SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers Work performed under the auspices of the U. S. Department of Energy by Lawrence Livermore National Laboratory under Contract W-7405-Eng-48 Outline SUNDIALS Overview SUNDIALS History The SUNDIALS suite — CVODE, IDA, KINSOL Preconditioning Data Structures Usage Fortran interface Application examples Sensitivity analysis Definition and motivation Approaches — Forward (FSA), Adjoint (ASA) FSA in SUNDIALS — Usage, Methods ASA in SUNDIALS — Usage, Implementation Application examples Future work
22
Embed
SUNDIALS: Suite of Nonlinear and Differential/Algebraic ... · 1 Carol S. Woodward and Radu Serban Center for Applied Scientific Computing, LLNL SUNDIALS: Suite of Nonlinear and...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Carol S. Woodward and Radu Serban
Center for Applied Scientific Computing, LLNL
SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers
Work performed under the auspices of the U. S. Department of Energy by Lawrence Livermore National Laboratory under Contract W-7405-Eng-48
Outline
SUNDIALS Overview
SUNDIALS History
The SUNDIALS suite
— CVODE, IDA, KINSOL
Preconditioning
Data Structures
Usage
Fortran interface
Application examples
Sensitivity analysis
Definition and motivation
Approaches
— Forward (FSA), Adjoint (ASA)
FSA in SUNDIALS
— Usage, Methods
ASA in SUNDIALS
— Usage, Implementation
Application examples
Future work
2
LLNL has a long history of R&D in ODE/DAE methods and software
Solver routines interface to user’s problem-defining routine and preconditioning routines
For portability, all user routines have fixed names.
Examples are provided.
Plan a move to the Babel language interoperability tool for access to other languages as well
11
Some Applications
CVODE is used in a 3D parallel tokamak turbulence model in LLNL’s Magnetic Fusion Energy Division. Typical run: 7unknowns on a 64x64x40 mesh, with 60 processors
KINSOL with a hypre multigrid preconditioner is used in LLNL’s Geosciences Division for an unsaturated porousmedia flow model. Fully scalable performance has beenobtained on up to 225 processors on ASCI Blue.
All solvers are being used to solve 3D neutral particle transport problems in CASC. Scalable performance obtained on up to 5800 processors on ASCI Red.
Other applications: disease detection, astrophysics, magnetohydrodynamics
Many more...
Sensitivity analysis in SUNDIALS
Definition and motivation
Approaches
— FSA
— ASA
FSA in SUNDIALS
— Usage
— Methods
ASA in SUNDIALS
— Usage
— Implementation
Application examples
12
Sensitivity analysis
Sensitivity Analysis (SA) is the study of how the variation in the output of a model (numerical or otherwise) can be apportioned, qualitatively or quantitatively, to different sources of variation.
Applications:
— Model evaluation (most and/or least influential parameters), Model reduction, Data assimilation, Uncertainty quantification, Optimization (parameter estimation, design optimization, optimal control, …)
Approaches:
— Forward sensitivity analysis
— Adjoint sensitivity analysis
Sensitivity analysis approaches
Computational cost:
(1+Np)Nx increases with Np
==)()0(
0),,,(
0 pxx
ptxxF &
pii
pixix Nidpdxs
FsFsFi ,,1,
)0(
0
0
K&& =
==++
px gsgdpdg
pxtg
+=
),,(
( )TTpxpp
T
xFdtFgdpdG
dtpxtgpxG
00**
0
)(
),,(),(
∫
∫
−−=
=
&λλ
==−=−′
TtxF
gFF
px
xxx
at...
)(*
**
&
&
λλλ
Parameter dependent system
FSA ASA
Computational cost:
(1+NG)Nx increases with Ng
13
Forward Sensitivity Analysis
For a parameter dependent system
find si=dx/dpi by simultaneously solving the original system with the Np sensitivity systems obtained by differentiating the original system with respect to each parameter in turn:
Gradient of a derived function
Obtain gradients with respect to p for any derived function
Computational cost - (1+Np)Nx - increases with Np
px gsgdpdgpxtg +=⇒),,(
==)()0(
0),,,(
0 pxx
ptxxF &
pii
pixix Nidpdxs
FsFsFi ,,1,
)0(
0
0
K&& =
==++
Adjoint Sensitivity Analysis
1
**
***
)(,,,
),(0),,(
−∃∂∂=
∂∂=
∂∂=
−=−=++→
==
CBxf
Cyf
Bxf
A
gBgCA
pxfpyxfx
add
y
xa
d
ληλλ&&
TtCT
== ** )( ξλ
0* ==TtxF&λ ptx
Tpp xFdtFg
dpdG
00
*0
* )()(=
+−= ∫ &λλ
Tt
apyp
T ap
dpp fCBgxdtffg
dpdG
=−−+++= ∫ 1
0*
0** )()0()( ληλ
Tty CCBgT=
−−= 1* )()(λ
app
app
ayyy
fxfCxpxf
CBggCBgB
Tt
**
1***
0),(
)(
: At
ξλξξλ
−=⇒−=⇒=−=⇒−=⇒−=
=−
TTpxpp
T
xFdtFg
dtpxtg
pG
pxG
00**
0
|)()(
),,(
dd
),(
∫ −−∫
=
=
&λλ
==−=−′
TtxFxgFF
px
xx
at...
)(*
**
&
&
λλλ
impose final conditions of the form
index-0 and index-1 DAE
Hessenberg index-2 DAE
14
Adjoint Sensitivity Analysis - Sensitivity of g(x,T,p)
( ) ( )Tt
px
tpxT
pTtppTt dT
xFdxFdtFFg
dpdG
dTd
dpdg
===
=
−−+−== ∫
)( *
0
*0
** &&
λµµλ
===−′
TtFF xx
at0)(
*
**
K&
µµµ
Impl
icit
OD
ES
emi-e
xplic
itin
dex-
1 D
AE
Hes
senb
erg
inde
x-2
DA
E
1,,
0),(
−∃∂∂=
∂∂=
=
AxF
BxF
A
xxF
&
&
1)(,,,
)(0),(
−∃∂∂=
∂∂=
∂∂=
==
CBxf
Cyf
Bxf
A
xfyxfx
add
a
d&
1,,,,
),(0),(
−∃∂∂=
∂∂=
∂∂=
∂∂=
==
Dyf
Dxf
Cyf
Bxf
A
yxfyxfx
aadd
a
d&
===−′
TtatgABA
x**
** 0)(µ
µµ
=−=+=
−−=
− TtatgDCgDB
CA
yx*1***
**
**
)(0µ
νµνµµ&
[ ]CCBBIP
TtatgCBCgCBCAgPB
CA
yyx
1
*1****1******
*
**
)(
)()(0
−
−−
−=
=−−==
−−=
&
&
µµ
νµµ
Stability of the adjoint system
Explicit ODE: proof using Green’s function;
Semi-explicit index-1 and Hessenberg index-2 DAE: the EUODE of the adjoint system is the adjoint of the EUODE of the original system;Example: Semi-explicit index-1 DAE
+=+=
ad
add
DxCx
BxAxx
0
&
+=−−=
ad
add
DB
CA
µµµµµ
**
**
0
&
ddd CxDBAxx 1)( −−=& ddd BDCA µµµ *1*** )( −+−=&
Axx =& µµ *A−=&
15
Stability of the adjoint system (contd.)
Implicit ODE and index-1 DAE: use bounded transformation
Lemma (Campbell, Bichols, Terrel)Given the time dependent linear DAE system
and nonsingular time dependent differentiable matrices P(t) multiplying the equations of the DAE and Q(t) transforming the variables, the adjoint system of the transformed DAE is the transformed system of the adjoint DAE.
TheoremFor general index-0 and index-1 DAE systems, if the original DAE system is stable then the augmented DAE system is stable.
)()()( tfxtBxtA =+&
=−−=−0*
**
λλλλ
x
xx
F
gF
&
&
User main routineSpecification of problem parametersActivation of sensitivity computationUser problem-defining functionUser preconditioner function
User main routineSpecification of problem parametersActivation of sensitivity computationUser problem-defining functionUser preconditioner function
Options- sensitivity approach (simultaneous or staggered)- sensitivity residuals: analytical, FD(DQ), AD, CS- error control on sensitivity variables- user-defined tolerances for sensitivity variables
Options- sensitivity approach (simultaneous or staggered)- sensitivity residuals: analytical, FD(DQ), AD, CS- error control on sensitivity variables- user-defined tolerances for sensitivity variables
Staggered Direct Method: On each time step, converge Newton iteration for state variables, then solve linear sensitivity system
— Requires formation and storage of Jacobian matrices, Not matrix-
free, Errors in finite-difference Jacobians lead to errors in
sensitivities
Simultaneous Corrector Method: On each time step, solve the nonlinear system simultaneously for solution and sensitivity variables
— Block-diagonal approximation of the combined system Jacobian,
Requires formation of sensitivity R.H.S. at every iteration
Staggered Corrector Method: On each time step, converge Newton for state variables, then iterate to solve sensitivity system
— With SPGMR, sensitivity systems solved (theoretically) in 1 iteration
FSA – Generation of the sensitivity system
Analytical
Automatic differentiation
— ADIFOR, ADIC, ADOLC
— complex-step derivatives
Directional derivative approximation
),min(2
),,(),,(
),1max(1
),max(
2),,(),,(
2),,(),,(
xiiiii
ii
iWRMSiix
ii
i
iiii
i
x
ixixi
epsxtfepsxtfpf
sxf
or
ps
rtolp
epxtfepxtfpf
psxtfpsxtfs
xf
σσσσ
σσσσ
σσ
εσ
σσσ
σσσ
=−−−++≈∂∂+
∂∂
=
=
−−+≈∂∂
−−+≈∂∂
iii p
fs
xf
s
pxtfx
∂∂+
∂∂=
=
&
& ),,(
CVODES case
17
Adjoint Sensitivity Analysis in SUNDIALS
User main routineActivation of sensitivity computationUser problem-defining functionUser reverse functionUser preconditioner functionUser reverse preconditioner function
User main routineActivation of sensitivity computationUser problem-defining functionUser reverse functionUser preconditioner functionUser reverse preconditioner function
(Modified)VectorKernels
(Modified)VectorKernels
Implementation- check point approach; total cost is 2 forward solutions + 1 backward solution - integrate any system backwards in time- may require modifications to some user-defined vector kernels
Implementation- check point approach; total cost is 2 forward solutions + 1 backward solution - integrate any system backwards in time- may require modifications to some user-defined vector kernels