1 Email: [email protected] www.hpc-escape.eu The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 800897 Andreas Müller 1 , Giovanni Tumolo, Willem Deconinck, Nils Wedi, Peter Bauer European Centre for Medium-Range Weather Forecasts, Reading, UK ESCAPE 2: Energy-efficient SCalable Algorithms for weather and climate Prediction at Exascale Introduction • Numerical Weather Prediction (NWP) and Climate models contain decades of algorithmic developments for conventional CPU hardware architectures • Paradigm shift towards more parallel and energy efficient many-core hardware architectures due to breakdown of Dennard scaling • Large impact on programming models expected in the near future • Rethink of design choices for future software frameworks: ‣ Scalability ‣ Energy efficiency ‣ Flexibility in algorithmic choices ‣ Maintainability Figure 1: ESCAPE 2 Partners • Combine scientific and computer- science expertise • Define and co-design necessary steps towards affordable exascale HPC simulations of weather and climate ESCAPE 2 (EU Horizon 2020) Weather and Climate benchmarks • HPWC: a hierarchy of benchmarking components representing key elements in the workflow of weather and climate systems Weather Figure 6: Strong scaling study on the supercomputer Summit for the spectral transform dwarf ported to GPUs in ESCAPE1. Each node uses 6 GPUs resulting in a maximum total number of 11520 GPUs. TCO3999 (2.5km) runtime per timestep in seconds 0.01 1 100 number of nodes 240 480 960 1920 perfect scaling IBM Power9 CPUs NVIDIA V100 GPUs 12.5x 23.7x This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE office of Science User Facility supported under contract DE-AC05-00OR22725. Weather and Climate Dwarfs Weather and Climate Dwarfs are self- contained algorithms representing key functional blocks of a NWP & Climate model. They must be verifiable and possible to integrate back in the model. Figure 2: Overview of dwarfs used in ESCAPE 2 and models from which they originate. FVM and DG are two alternative dynamical cores for the IFS. ICON and NEMO represent ocean models. 55% 5% 4% 13% 23% Dynamics — SH spectral transform Dynamics — semi-Lagrangian advection Physics: radiation Physics: cloud microphysics Non-ESCAPE dwarf now tomorrow time-step spectral transform semi-Lagrangian discontinuous Galerkin options: ACRANEB2 radiation time-stepping advection, gradient computation physics components: elliptic solver, multigrid HEVI MPDATA finite volume CLOUDSC microphysics IFS FVM ALARO DG ICON NEMO MUSCL finite difference AI radiation fault-tolerant GMRES Combining advanced algorithms with High-Level IR Weather Figure 4: Aiming at the combination of performance, accuracy, resilience and portability for weather and climate prediction through novel mathematical and algorithmic methodologies, programming and uncertainty estimation. Spatial resolution Model complexity Benet beyond the state-of-the-art Uncertainty estimate Time&energy-to-solution Portability Resilience Mathematical methods and algorithms Semi-implicit, semi-Lagrangian CG/DG Hierarchical multigrid tools Fault resilient solver Articial neural networks DSL toolchain Ensemble based URANIE and: State-of-the-art Figure 5: Design of a modular domain specific language (DSL) toolchain for Weather & Climate dwarfs by using a High-Level intermediate representation (IR). DSL Frontends Domain Specic Checkers Optimizers Code Generator Python DSL Frontend Read before write Missing Update Boundary Data dependency race conditions Naive C/Fortran Generator Optimized GridTools Generator Out of Bounds Stencil Access Software Managed Caches Full vertical parallelization Stage Fusion Data Locality Exploit Data Locality Exploit clang DSL (C++) High-Level IR CLAW DSL (Fortran) Energy-Efficient Scalable Algorithms for Weather and Climate Prediction at Exascale