C++14 Generic Programming as a Domain-Specific Language ...roystgnr/2014_roystgnr-talk.pdf · Demands C++1y (gcc 4.8.2+, clang 3.4+) return type deduction Opaque ContextType for forward

PECOSPredictive Engineering and Computational Sciences

C++14 Generic Programming as a Domain-SpecificLanguage for PDEs

Roy H. Stogner1

1Institute for Computational Engineering and SciencesThe University of Texas at Austin

May 12, 2014ICES

R. H. Stogner C++14 PDEs May 12, 2014 1 / 14

Goals of Multiphysics PackageWhat we would like?• Leverage preexisting kernels for various “physics” with minimal code

rewritingI Not as fine-grained as “dial-an-operator”I Trustworthy common code for different applications

• Still want to retain generality and extensibility

• Don’t sacrifice performance (too much)

• Interfaces to benefits of libMesh andunderlying libraries

I Hybrid meshingI Adaptive meshing, time steppingI Adjoint analysisI SolversI Hybrid parallelism


Object-Oriented Path to Multiphysics

(One) Object-Oriented Approach• Still cling to loop over elements, then quadrature, then DoF paradigm

I Can parallelize (both distributed and threaded) element loopindependent of physics kernel

• Use polymorphic functions within loop for “element” routineI vtable cost amortized by cost of residual/Jacobian calculationI May not be true if polymorphic at the quadrature level

Another Object-Oriented Approach∗

• Build graph of physics operators

• Each operator works on large blocks of quadrature points

• Fewer vtable lookups, but more loops over quadrature points

∗P. K. Notz, R. P. Pawlowski, J. C. Sutherland, “Graph-Based Software Design for Managing Complexity and Enabling Concurrency in

Multiphysics PDE Software”, ACM TOMS, 39(1), 2012


libMesh and FEMSystem

Highlights• Designed using first OO approach discussed

• User subclasses FEMSystem (or FEMPhysics, DifferentiableQoI)to implement (multi-)physics element routine

• Automatically access existing libMesh infrastructure without requiringuser code

I MPI+threaded parallelismI Interfaces to important packages, e.g. PETSc, Trilinos, etc.I AMR

• Lots of flexibility, but no physics repository→ no reuse of physicskernels

• Used in several Ph.D. projects


GRINS

Highlights• https://github.com/grinsfem

• Built around FEMSystem in libMesh

• Runtime selectable (input file) repository ofI Physics modulesI Solvers (steady vs. unsteady, adaptive)

• Easy extensibility to add or create standalone applicationI E.g. proprietary or export controlled modelsI Parameter continuation solver

• Can parse complex boundary conditions from input file (fparser)

• Adjoint analysis ready• Interface to statistical calibration packages

I QUESO, https://github.com/libqueso

• Adaptive modeling in-the-works


https://github.com/grinsfem

https://github.com/libqueso

GRINS Reacting low-Mach with surface catalysis

Variable-density cavity Vacuum Arc Remelting


Pros and Cons of Object-Oriented Design

Flexibility on CPU, at element levels• Multithreading, distributed partitioning from library

• No GPU, MIC, etc. support

Virtual Function Calls• Total runtime flexibility

• In inner loops: inefficient

• Outside inner loops: inflexible

• Hard to upgrade fixed type hierarchy APIs “under the hood”


Beginning Generic Programming: “Duck Typing”

• Template algorithms aroundunspecified data types T1, T2, ...

• Expect overloaded operators, functions

I T walks like a duck, quacks like a duck• Simple data type examples:

I T = float, double, complex<Tin>

• Templated MMS:https://github.com/manufactured-solutions

• Templated aerothermochemistry:https://github.com/libantioch

• Advanced examples:I T = Eigen::Array<Tin,N,1>I T = vex::vector<Tin>I T = MetaPhysicL::DualExpression

<Tin,Din>I Nested combinations!


https://github.com/manufactured-solutions

https://github.com/libantioch



• Expect overloaded operators, functionsI T walks like a duck, quacks like a duck

• Simple data type examples:I T = float, double, complex<Tin>









































Duck Impressions

Is your Numeric Class a “Duck”?

• Constructor issues for resizeablevectors, naive expression templates

• “Give me a compatible int/boolean”is non-standard

• Some operators (ternary!) may notbe universally implemented

• Ambiguous nesting behavior

Partially Specializable Interfaces• Antioch::zero clone() and kin

• Antioch::if else()

• Simple default implementations

• Per-package overrides


Duck Impressions

Is your Numeric Class a “Duck”?• Constructor issues for resizeable

vectors, naive expression templates









Duck Impressions

Is your Numeric Class a “Duck”?• Constructor issues for resizeable

vectors, naive expression templates









Getting our Ducks in a RowVectorizing Types

• CPU VectorizationI std::valarrayI Eigen::ArrayI MetaPhysicL::NumberArray

• GPU VectorizationI VexCL vex::vector

https://github.com/ddemidov/vexcl

Mathematical Types• MetaPhysicL::NumberVector

I Adds dot products, outer products, etc. to NumberArray

• MetaPhysicL::SparseNumberVectorI Compile-time metaprogramming for efficient sparse operations

• MetaPhysicL::NamedIndexArrayI A smarter version of Matlab/Octave bsxfun

• MetaPhysicL::DualNumber, DualExpressionI Automatic Differentiation for data, expression templates


https://github.com/ddemidov/vexcl

ExamplesAssembly Code

// node_xyz: N_e*N_n*N_DP

// dpsi_dxi: N_n*N_q*N_DM

auto dx_dxi = partial_sum<Indices::Local_Node_ID>

(node_xyz * dpsi_dxi);

// dx_dxi: N_e*N_q*N_DP*N_DM, expression template

Features• Correct permutation function is autogenerated by operator* onNamedIndexArray

• An underlying expression-template-based data vector (e.g.vex::vector) is required for efficiency

• Mapping jacobian dx dxi will also be an expression template withdeferred evaluation

• Reusable data (e.g. det J, dphi dx) gets evaluated and cached


ExamplesAssembly Code

// node_xyz: N_e*N_n*N_DP

// dpsi_dxi: N_n*N_q*N_DM

auto dx_dxi = partial_sum<Indices::Local_Node_ID>

(node_xyz * dpsi_dxi);

// dx_dxi: N_e*N_q*N_DP*N_DM, expression template

Features• Correct permutation function is autogenerated by operator* onNamedIndexArray

• An underlying expression-template-based data vector (e.g.vex::vector) is required for efficiency

• Mapping jacobian dx dxi will also be an expression template withdeferred evaluation

• Reusable data (e.g. det J, dphi dx) gets evaluated and cached


Examples

Physics Class Methods

auto // C++1Y!

weak_interior_residual(const ContextType& context,

const CacheType&) const {

auto& du_dx = std::get<1>(context.u);

auto& dv_dx = std::get<1>(context.v);

return -k*dot_product(du_dx,dv_dx);

}

Features• Demands C++1y (gcc 4.8.2+, clang 3.4+) return type deduction

• Opaque ContextType for forward compatibility

• std::tuple for arbitrary order PDEs

• DualExpression (u,phi)→ u gives AD jacobians


Examples

Physics Class Methods

auto // C++1Y!

weak_interior_residual(const ContextType& context,

const CacheType&) const {

auto& du_dx = std::get<1>(context.u);

auto& dv_dx = std::get<1>(context.v);

return -k*dot_product(du_dx,dv_dx);

}

Features• Demands C++1y (gcc 4.8.2+, clang 3.4+) return type deduction

• Opaque ContextType for forward compatibility

• std::tuple for arbitrary order PDEs

• DualExpression (u,phi)→ u gives AD jacobians


Cons of Generic-Programming Design

NamedIndexArray Fixed Loop Sizes• Hybrid meshes require one pass per element type

• CSR Assembly with hanging node constraints?

Lovecraftian Debugging• Huge template expression errors

• gcc 4.8.1 ICE from some C++1Y code

• Physics development: double→ NumberArray→ vex::vector

Run-time vs. Compile-time physics selection• Factory methods proliferate factorially!

• Run-time selection required for chemistry


Thank you!

Questions?


C++14 Generic Programming as a Domain-Specific Language ...roystgnr/2014_roystgnr-talk.pdf · Demands C++1y (gcc 4.8.2+, clang 3.4+) return type deduction Opaque ContextType for forward

Documents