Scientific visualization of HPC simulation data – introduction and overview on MPG projects – Elena Erastova, Markus Rampp, Klaus Reuter [email protected]Max Planck Computing and Data Facility (MPCDF) Interdisciplinary Cluster Workshop on Visualization, Excellence Cluster Universe 2015/11/04, Garching
29
Embed
Scientific visualization of HPC simulation data ... · Scientific visualization of HPC simulation data ... ParaView's graphical user interface. ... (low-level), HDF5 (library, tools),
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Scientific visualization of HPC simulation data– introduction and overview on MPG projects –
● visualization: visual representation of data e.g. chemical structures: create a ball-and-stick model fromatom positions (and compute an image or do a 3D print)
● rendering: generation of a 2D image from a 1, 2, 3D modele.g. create a 2D projection of a 3D “balls and sticks” model
Popular techniques for 3D scalar fields
● volume rendering
● ray casting: follow straight lines starting from a camera model (view point),intersecting an image plane (2D grid), and hitting objects (3D model)
● splatting: “throw” voxels at 2D image plane from back to front (like “snowballs”)
● transfer function and color table map from data space to color space,e.g. we may intuitively mimick opacity and emissivity of a gas
● provides qualitative and quantitative information
● pseudocolor plots
● color table maps from data space to color space
● 2D is straightforward, 3D requires clipping
● provides mostly quantitative information
● iso surfaces (surfaces of constant scalar value)
[wikipedia.org]
[visitusers.org]
[MPCDF]
[MPCDF]
K. Reuter, MPCDF Garching, 2015/11/04
ray casting
View Ray
Image
Camera
[adapted from wikipedia.org]
[MPCDF]
K. Reuter, MPCDF Garching, 2015/11/04
ray tracing
Light Source
Scene Object
Shadow RayView Ray
Image
Camera
[adapted from wikipedia.org]
[blender.org]
[K. Reuter]
K. Reuter, MPCDF Garching, 2015/11/04
some basics on scientific visualization (cont'd)
Popular techniques for 3D vector fields
● colored arrow plots
● streamlines, streaklines, pathlines
● contraction to 3D scalar field
● vector magnitude
● projection
[visitusers.org]
streamlines
[visitusers.org]
colored arrow plot
[MPCDF]
streamwise vorticity
[by courtesy of V. Avsarkisov (TU Darmstadt)]
[MPCDF]
velocity magnitude
[MPCDF]
K. Reuter, MPCDF Garching, 2015/11/04
some basics on scientific visualization (cont'd)
Gridded vs ungridded data
● depending on the simulation approach, data values may be availableat the vertices of a grid or at (independent) points in space
● gridded data (vertices connected by edges)
● connectivity information greatly simplifies interpolation and parallelization
● many of the aforementioned algorithms require gridded data
● comprehensive open-source toolbox for scientific visualization
● scalable parallel architecture, implementation based on VTK, MPI and TCP/IP
● mature code base, but still actively developed
*2000 LANL & Kitware, since 2005 Sandia & Kitware
● extensible via plugins (eg. data readers, data filters, in-situ connectors)
● controllable via Python scripting
● widely used
● well documented → www.paraview.org
[www.paraview.org]
ParaView's graphical user interface
K. Reuter, MPCDF Garching, 2015/11/04
● comprehensive open-source toolbox for scientific visualization
● scalable parallel architecture, implementation based on VTK, MPI and TCP/IP
● mature code base, but still actively developed
*2000 LLNL
● extensible via plugins (eg. data readers, data filters, in-situ connectors)
● controllable via Python scripting
● widely used
● well documented → visit.llnl.gov
[MPCDF]
K. Reuter, MPCDF Garching, 2015/11/04
Workflow (VitIt example)
[MPCDF]
K. Reuter, MPCDF Garching, 2015/11/04
Data management
How to get data from an HPC code into your favourite visualization tool?
Criteria for selecting an IO library (and data format):
● performance: a parallel code requires parallel IO
● portability: code maturity, availability at HPC centers
● usability: code changes, debug options
For small data sets (time series of aggregate variables) it may work to
● write text or binary files in a proprietary format during a code run
● apply a data conversion pipeline as a second step (e.g. using Python)
However, for larger data sets (~ few hundred MB) this procedure is inefficient and will finally become the bottleneck.
VTK (www.vtk.org) provides – among others – an extensive framework to handle griddedand ungridded data of various kinds, including file formats that are compatible withParaView and VisIt.Pro: Python bindings are helpful to implement postprocessing including format conversion.Caveat: VTK's (complex!) C++ code may be hard to integrate with existing HPC code.
Simulations by L. Shi, M. Avila, B. Hof (MPI f. Dynamics and Self Organization, FAU Erlangen, IST Austria)
● DNS of fluids (pipe flows, Taylor-Couette flows)
● Code NSCOUETTE: Solves incompressibleNavier Stokes equations using a pseudospectral method(Shi, Rampp, Hof, Avila, Computers and Fluids, 2015)
● Basic research in turbulence: lab experiments, numerical simulations, astrophysis: accretion in cold discs(Hof et al., Science, 2010, Avila et al., Science, 2011)
● very limited support by standard software, special-purpose software does not cover full spectrum of features: interactivity, slicing, quantitative analysis
?
visualization as particles
visualization as asmooth density field
K. Reuter, MPCDF Garching, 2015/11/04
SPH visualization (cont'd)
Visualisation approach (C. Simion, MPE and K. Reuter, MPCDF)
● mapping to unstructured grids which can be handled by VisIt and Paraview
● approach: 3D Delaunay triangulation to create a tetrahedral mesh
● preserves resolution, avoids interpolation to a regular grid
● initially used inefficient VTK library implementation
● CPU time scales as N2
● huge memory requirements
● implemented custom code using the faster Qhull library
[MPCDF]
visualization as asmooth density field
Synergies with a materials science project (FHI)
● principle/code was reused to analyze materials science data[MPCDF]
K. Reuter, MPCDF Garching, 2015/11/04
SPH visualization (cont'd)
[MPCDF]
● Splotch is a special ray tracer to visualize SPH simulation data (without conversion).→ Talk by K. Dolag (LMU) this afternoon will give you all the technical details.→ K. Dolag, et al., New Journal of Physics, Volume 10, Issue 12, pp. 125006 (2008)
● Code extension to tackle time-dependent visualizations of large simulation data sets (~1010 particles) (K. Reuter, MPCDF, 2011).→ Implemented MPI parallelization of particle handling and frame interpolation.
● Parallelization enabled visualization support for users at the MPE and MPA,movie rendering utilized the complete memory of the VIZ cluster at the MPCDF.
K. Reuter, MPCDF Garching, 2015/11/04
Geospatial data and bird migration
Visualisation approach (K. Reuter, MPCDF & K. Safi, MPI-Orn., 2012)
● visualization with ParaView, lots of Python scripting
● tedious generation and optimization of camera movement → better use artist's tools such as Blender?
Data by M. Wikelski (MPI for Ornithology)
● observational data
● a bird’s (gull) GPS track correlated with wind data
● topography data, earth's magnetic field data, …
● time-dependent data
Visualization adapted to wall-projection at the “hennhouse”, ie. thevisitors and media center MPI-Orn at Radolfzell/Bodensee.
[MPCDF]
movie presented by M. Wikelski at the general assembly of the MPG, 2012
K. Reuter, MPCDF Garching, 2015/11/04
Summary and Conclusions
● ParaView and VisIt provide (partly parallel) implementationsof many state-of-the-art visualization algorithms
● spohisticated and mature open-source tools
● end-user ready
● strong in visualizing gridded data at HPC scale
● Use HDF5 (& XDMF) to implement IO, if possible
● Check out our visualization gallery at www.mpcdf.mpg.de