Best Practices Supporting Science at CSCS Michele De Lorenzi (CSCS)
Best Practices Supporting Science at CSCS Michele De Lorenzi (CSCS)
Case 1 – Simplify the usage of HPC resources GPU Accelerated In-Situ Visualization
§ Peter Messmer (NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich)
Best Practices Supporting Science at CSCS 3
HPC Workflow
Workstation
Viz Cluster Supercomputer
File System
Setup
Dump, Checkpointing Visualization,
Analysis
Analysis, Visualization
Best Practices Supporting Science at CSCS 4
Traditional Workflow: Challenges
Workstation
Viz Cluster Supercomputer
File System
Setup
Dump, Checkpointing Visualization,
Analysis
Analysis, Visualization
Lack of interactivity prevents “intuition”
I/O becomes main simulation bottleneck
Viz resources need to scale with simulation
High-end viz neglected due to workflow complexity
Eliminating the IO Bottleneck: In-Situ ViZ
Workstation
Viz Cluster Supercomputer
Setup Analysis, Visualization
Best Practices Supporting Science at CSCS 5
Eliminating the IO Bottleneck: In-Situ ViZ
Workstation
Viz Cluster Supercomputer
Setup Analysis, Visualization
Best Practices Supporting Science at CSCS 6
Different Visualization Scenarios
§ 1) Legacy workflow § Separate compute & Viz system § Communication via filesystem
§ 2) In-situ partitioned or integrated § Different nodes for viz & compute § Communication via High Performance Network
§ 3) In-situ co-processing
§ Compute and visualization on same node
HPC System
Compute
GPU
Viz System
Viz GPU
Filesystem
HPC System
Compute
GPU
Viz GPU Network
HPC System
Compute+Viz nodes
Best Practices Supporting Science at CSCS 7
Typical Scientific Visualization system
§ Visualization is more than Rendering
§ Simulation: Data as needed in numerical algorithm
§ Filtering: Conversion of simulation data into data ready for rendering § Typical operations: binning, down/up-sampling, iso-surface extraction, interpolation, coordinate transformation, sub-
selection, .. § Sometimes embedded in simulation
§ Rendering: Conversion of shapes to pixels (Fragment processing)
§ Compositing: Combination of independently generated pixels into final frame
Visualization
Filtering Rendering Compositing Simulation
Best Practices Supporting Science at CSCS 8
Visualization Dataflow
CPU
GPU
Simulation
Filtering Rendering
Best Practices Supporting Science at CSCS 9
Fully GPU accelerated Visualization
CPU
GPU
Simulation Rendering
Filtering
Best Practices Supporting Science at CSCS 10
FuLLy GPU accelerated Visualization
CPU
GPU
Simulation Rendering
Filtering Speedup
Best Practices Supporting Science at CSCS 11
Bonsai With In-Situ Viz On Daint
Best Practices Supporting Science at CSCS 12
Acknowledgement
§ Bonsai development: Evghenii Gaburov (SURFsara), Jeroen Bédorf (CWI)
§ OpenGL on Daint: Gilles Fourestey (CSCS), Nina Suvanphim (Cray)
§ OpenGL on Titan: Don Maxwell (ORNL)
Thomas Schulthess (CSCS) and Jack Wells (ORNL) for providing access to Daint and Titan for these experiments
Best Practices Supporting Science at CSCS 13
Case 2: Software as a service Supporting the MARVEL community
14
MATERIALS’ REVOLUTION: COMPUTATIONAL DESIGN AND DISCOVERY OF NOVEL MATERIALS
EPFL-ETHZ-UNIBAS-UNIFR-UNIGE-USI-UZH-IBM-CSCS-EMPA-PSI MARVEL
Best Practices Supporting Science at CSCS 15
Access Pattern
Piz Dora
External Login Access (ELA) CSCS
Best Practices Supporting Science at CSCS 16
EPFL
AiiDA Server
/store
Repository access
MARVEL Researcher
Research Community
Traditional access (batch jobs)
Access through AiiDa
Scientific Community access
Case 3: Supporting scientific communities The Platform for Advanced Scientific Computing (PASC)
17
Swiss Platform for Advanced Scientific Computing
§ Promote joint effort to address key scientific issues in different domain sciences exploiting the potential of the next generation of HPC architectures
§ Interdisciplinary collaboration between § domain scientists,
§ computational scientists,
§ software developers,
§ computing centres
§ hardware developers.
§ Builds on the principle of co-design involving all these actors
18 Best Practices Supporting Science at CSCS
Best Practices Supporting Science at CSCS 19
PASC Organization
PASC instruments
Organized in four pillars:
§ Application Support Network (ASN)
§ Co-design projects
§ Institutional computing system pillar supporting the procurement of institutional computing systems
§ The training pillar supporting the Swiss Graduate Program Foundations in Mathematics and Informatics for Computer Simulations in Science and Engineering (FOMICS).
20 Best Practices Supporting Science at CSCS
Application Support Network (ASN)
§ ASN are intended to coordinate and support the effort in different domain science areas to pursue the objectives of the project
§ The following Domain Science Networks are active today: § Climate - Climate and Atmospheric Modelling
§ Earth - Solid Earth Dynamics
§ Life Sciences - Life Sciences Across Scales
§ Materials - Materials Simulations
§ Physics - Plasma Physics, Astrophysics and Multiscale Fluid Dynamics
21 Best Practices Supporting Science at CSCS
Co-design projects
§ Interdisciplinary projects involving several (or all) actors § address the following priorities for optimal exploitation of new and
emerging supercomputing platforms: § Refactoring of key application codes and re-engineering of their algorithms.
§ Development of numerical libraries and their integration into application codes.
§ Incorporations of innovative sub-systems (e.g. I/O, data streams, etc.) into existing simulation codes
§ Development of programming environments or components, performance tuning and analysis/monitoring tools.
22 Best Practices Supporting Science at CSCS
Supported projects
Physics and Astrophysics: § Particles and fields; PI: Laurent Villard (EPFL)
§ DIAPHANE Radiative Transport in astrophysical simulations; PI: Lucio Mayer (University of Zurich)
Material Science
§ ANSWERS: nano-device simulations; PI: Mathieu Luisier (ETH Zurich)
§ Electronic Structure Calculations: electronic structure calculations; PI: Jürg Hutter (University of Zurich)
§ ENVIRON A Library for Electronic-structure Simulations; PI: Stefan Goedecker (University of Basel)
23 Best Practices Supporting Science at CSCS
Supported projects (continued)
Life Science § Angiogenesis in Health and Disease: In-vivo and in-silico; PI:
Petros Koumoutsakos (ETH Zurich)
§ Coupled Cardiac Simulations: HPC Framework for Coupled Cardiac Simulations; PI: Rolf Krause (University of Lugano), Alfio Quarteroni (EPFL)
§ Genomic Data Processing; PI: Ioannis Xenarios (University of Lausanne)
§ HPC-ABGEM: agent-based general ecosystems models; PI: Christoph Zollikofer (University of Zurich)
24 Best Practices Supporting Science at CSCS
Supported projects (continued)
Geo-physics and Climate
§ GeoPC: hybrid parallel smoothers for multigrid preconditioners; PI: Paul Tackley (ETH Zurich)
§ GeoScale: A framework for multi-scale seismic modelling and inversion; PI: Andreas Fichtner (ETH Zurich)
§ Grid Tools: Towards a library for hardware oblivious implementation of stencil based codes; PI: Oliver Fuhrer (Meteosuisse)
Computer Science
§ Heterogeneous Compiler Platform for Advanced Scientific Codes; PI: Torsten Hoefler (ETH Zurich)
§ Multiscale applications: Optimal deployment of multiscale applications on a HPC infrastructure; PI: Bastien Chopard (UNIGE)
25 Best Practices Supporting Science at CSCS
PASC conference
§ PASC organizes yearly a Platform of Advanced Scientific Computing Conference.
§ The goal is to bring together research groups within diverse scientific domains to foster interdisciplinary collaborations and to strengthen (HPC) knowledge exchange.
§ PASC15 – June 1-2, 2015 hosted by ETH Zurich: http://www.pasc15.org/
Best Practices Supporting Science at CSCS 26
Best Practices Supporting Science at CSCS 27
Thank you for your attention.