Research towards Exascale at HLRSevents.science-japon.org/hpc17/slides/Joseph... · Center, Founded 1996 • Central unit of the University of Stuttgart • Open to Eurepean users
Post on 17-Jul-2020
0 Views
Preview:
Transcript
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Research towards Exascale at HLRS
Joseph Schuchart – Scalable Programming Models and Tools, HLRS
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
OVERVIEW - HLRS
Research at HLRS 2
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
High Performance Computing Center Stuttgart – HLRS
Research at HLRS 3
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
High Performance Computing Center Stuttgart – HLRS
Research at HLRS 4
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Bundeshöchstleistungsrechenzentrum Stuttgart – HLRS
• 1st German national Supercomputing Center, Founded 1996
• Central unit of the University of Stuttgart
• Open to Eurepean users since 2004 through PRACE (Partnership for Advanced
Computing in Europe)
• Funding per year:
– 3,9 M€ State funding
– 5,0 M€ Project funding
• Staff:
– ~ 25 permanent Staff
– ~ 60 Project Staff
– ~ 20 Research Assistants
• > 7500 TFlops installed computing power in three main systems
• Comprehensive user training
Research at HLRS 5
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Education & Training• Education
• Training (20 courses – 600 participants)
• 4-5 International Tutorials at conferences
• 4-5 own conferences and workshops at HLRS
• PRACE Advanced Training Center together with JSC and LRZ
• PRACE Industrial Training Center
• New Training Center (2017)
Research at HLRS 6
04.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Research Departments• Applications and Visualisation
– Visualisation
– Scalable Programming Models & Tools
– Scalable Computing & Optimisation Methods
– Service Management & Business Processes
• Software & Systems
– Numerical Methods & Libraries
– Project & User Management, Accounting
• Staff Unit
– Philosophy of Science & Technology of Computer Simulation
Research at HLRS 7
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
SYSTEMS
Research at HLRS 8
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
HLRS - Systems
2015 Cray XC40 “Hazel Hen”• 41 Cabinets• 7712 Compute nodes
– 2 sockets– 12 Cores per socket– 128 GB Memory
• 185,088 Cores, Intel Haswell• 7.420 PF/s Peak performance• ~ 5.640 PF/s Sustained performance• ~ 10 PB Disk Space• Power consumption: ~ 3.2 MW
• Cray Aries Network • Cray-Linux-Environment • Cluster Compatibility Mode (For ISV Applications)
Research at HLRS 9
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
HLRS - Systems
NEC PC-Cluster• Mainly industrial usage with ISV codes• Scientific Linux 6.6• Redundant System in two computing rooms• Nobelstraße 19
– 650 Nodes with various architectures• Sandy Bridge, Haswell• AMD Interlagos
– 12 GB – 1TB Main memory per node– ~ 500 TB Workspace Filesystem– Infiniband Network
• Almandring 30– 192 Nodes with Intel Sandy Bridge Architecture– ~ 128 TB Workspace Filesystem– Infiniband Network
Research at HLRS
Nobelstraße19
Almandring 30
10
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
HLRS - Systeme
NEC SX-ACE
• 64 Nodes
– 4 Cores per Node
– 64 GB Main memory per Node
– Compact and power efficient construction
• 16.384 TFLOPS Peak Performance
• 250 TB NEC Scalable Technology File System (ScaTeFS)
HPSS Datamanagement System
• Hierarchical Storage Management System (HSM)
• >12 PB Storage capacity on tape (max. 30PB)
• 10,000 Tapes (handled by tape robot)
• ~500 TB HDD Cache
Research at HLRS
SX-ACE Node Card
NEC SX-ACE Single Rack
HPSS
11
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
RESEARCH AREAS
Research at HLRS 12
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
HLRS RESEARCH
Research at HLRS 14
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Mont Blanc: Model-centric debugging with
• Traditional debuggers aim to cover a wide range of programming languages and models and thus present information using the lowest common denominator, i.e. in terms of low-level concepts
• Model-centric debugging allows using the same language and semantics for debugging and programming.
• Temanejo is a graphical debugger for task-based programming models as e.g. OpenMP or OmpSs
Temanejo supports direct interaction with tasks (step into/over, pause, continue) and dependencies (adding, removing, inspecting).
Research at HLRS 15
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
CoE POP: Performance Optimization and Productivity
Goal: understanding common performance issues of real codes
Approach: analyze as many codes as possible
Incentive for users: offer performance analysis and optimization services
Service levels:
• Performance audit and analysis
• Proof-of-concept implementation of optimizations
• Training on performance tools Ateles CFD code50% performance improvementhttps://pop-coe.eu/blog/pop-proof-of-concept-study-leads-to-nearly-50-higher-performance
Research at HLRS 16
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Simulation software & Application Areas
• Topics
– Energy
– Climate Change & Environment
– Health & The Aging Society
– Mobility in the 21st Century
Research at HLRS 18
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
CONVECTION PERMITTING SEASONAL LATITUDE-BELT SIMULATION
Applications in High Performance Computing – Climate Simulations
Research at HLRS 19
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Convection permitting seasonal latitude-belt simulation
• Application by:– Intitute of Physics and Meteorology – University of
HohenheimT. Schwitalla, K. Warrach-Sagi, and V. Wulfmeyer
• Purpose– Improve long range forecasts with Limited Area Models
(LAM)– Reduce model assumptions– Overcome Resolution Problems
• Model setup– Full Belt between
20° North and 65° North– Resolution of 0.03° (3.3km)– Simulation period
July and August 2013
Research at HLRS 20
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Convection permitting seasonal latitude-belt simulation
• Simulation with community code WRF
• 84,000 compute cores
• Hybrid OpenMP / MPI execution
• 84 h (3,5 days) Runtime
• 450 TB Result Data
• I/O
– 3D data every 30min
– Diags every 15min
• 36 h (1,5 days) I/O Runtime
Research at HLRS 21
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Results
Research at HLRS
Low pressure system development over the North Atlantic
22
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Results - Typhoon Soulik
Research at HLRS
• Emerged July 2013
• Made landfall late on July 12th in northern Taiwan
• Time of emergence and landfallwas calculated correctly
• Landfall was calculatedwith 100 km error
• Feature is not visible / reproduciblewith global climate models at 50 kmor 100 km resolution.
23
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
AXIAL FAN TURBULANCESIMULATION
Applications in High Performance Computing – Mechniacs Simulations
Research at HLRS 24
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Prediction of the Turbulent Flow Field Around a Ducted Axial Fan
• Cooperation with the Institute for Aerodynamicsat the RWTH Aachen University (AIA)
• Funded through industrial research partnership
Goals:
• Development of a reliable prediction tool for the noisegeneration of ducted axial fans
• Development of reliable turbulence models forindustrial applications
Research at HLRS
1) Copyright: Institute of Aerodynamics, RWTH Aachen University2) „Luefter“ Das Original wurde von Huha in der Wikipedia auf Deutsch Lizenziert unter CC BY-SA 3.0 über Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Luefter.jpg#/media/File:Luefter.jpg
1)1)
2)
25
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Prediction of the Turbulent Flow Field Around a Ducted Axial Fan
• Noise created through
– Interaction of fan and turbulances
– Streaming phenomenae at Tip Gap
Probleme:
• Complex geometries
• Turbulent field with
– Unfavorable pressure gradients
– Heavily bound streamlines
• Results of Computational Aeroacoustics (CAA) depend heavily on quality of CFD results
Research at HLRS
1) Copyright: Institute of Aerodynamics, RWTH Aachen University2) „Luefter“ Das Original wurde von Huha in der Wikipedia auf Deutsch Lizenziert unter CC BY-SA 3.0 über Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Luefter.jpg#/media/File:Luefter.jpg
2)
Tip Gap
Berechnungsgitter imBlatt-Mantel Spalt
Berechnungsgeometrie
1)
1)
26
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Prediction of the Turbulent Flow Field Around a Ducted Axial Fan
• Results • RANS: Reynolds Averaged NavierStokes Method
– Commercial ISV codes
– „Workstation result“
• LES: Large Eddy Simulation
– Simulated using ZFS
– 92,000 compute cores
– 110 walltime
– 80 TB result data
– 1 Billion grid cells
– 320,000 time steps for 4 rotations
Research at HLRS
All figures: Copyright: Institute of Aerodynamics, RWTH Aachen University
27
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Challenges on the path to Exascale (at HLRS)
Research at HLRS
Exascale
UsablePerformance
Infra-structure
ApplicationScalability
Productivity
I/O
Training
28
09.04.2017:: ::
::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :::::
::
Thank you for your attention!
High Performance Computing Center Stuttgart
top related