Presented by Scientific Data Management Center Nagiza F. Samatova Oak Ridge National Laboratory Arie Shoshani (PI) Lawrence Berkeley National Laboratory DOE Laboratories ANL: Rob Ross LBNL: Doron Rotem LLNL: Chandrika Kamath ORNL: Nagiza Samatova PNNL: Terence Critchlow Jarek Nieplocha Universities NCSU: Mladen Vouk NWU: Alok Choudhary UCD: Bertram Ludaescher SDSC: Ilkay Altintas UUtah: Steve Parker Co-Principal Investigators
12
Embed
Scientific Data Management Center...Automating scientific workflow in SPA Enables scientists to focus on science not process Scientific discovery is a multi-step process. SPA-Kepler
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Presented by
Scientific Data Management Center
Nagiza F. Samatova
Oak Ridge National Laboratory
Arie Shoshani (PI)
Lawrence Berkeley National Laboratory
DOE LaboratoriesANL: Rob Ross
LBNL: Doron Rotem
LLNL: Chandrika Kamath
ORNL: Nagiza Samatova
PNNL: Terence Critchlow
Jarek Nieplocha
UniversitiesNCSU: Mladen Vouk
NWU: Alok Choudhary
UCD: Bertram Ludaescher
SDSC: Ilkay Altintas
UUtah: Steve Parker
Co-Principal Investigators
2 Samatova_SDMC_SC07
Scientific Data Management Center
SciDAC Review, Issue 2, Fall 2006 Illustration: A. Tovey
Lead Institution: LBNL
PI: Arie Shoshani
Laboratories:
ANL, ORNL, LBNL, LLNL, PNNL
Universities:
NCSU, NWU, SDSC, UCD, U. Utah
Established 5 years ago (SciDAC-1)
Successfully re-competed for next5 years (SciDAC-2)
Featured in Fall 2006 issue ofSciDAC Review magazine
3 Samatova_SDMC_SC07
SDM infrastructureUses three-layer organization of technologies
Supports Parallel-netCDF library builton top of MPI-IO implementation calledROMIO, built in turn on top of AbstractDevice Interface for I/O system, usedto access parallel storage system
Early performance testing showedPnetCDF outperformed HDF5 for somecritical access patterns.
The HDF5 team has responded byimproving its code for these patterns,and now these teams activelycollaborate to better understandapplication needs and systemcharacteristics, leading to I/Operformance gains in both libraries.
11 Samatova_SDMC_SC07
Active storage
• Modern filesystems such as GPFS,Lustre, PVFS2 use general purposeservers with substantial CPU andmemory resources.
• Active Storage moves I/O-intensivetasks from the compute nodes to thestorage nodes.
• Main benefits:
local I/O operations,
very low network traffic (mainlymetadata-related),
better overall system performance.
• Active Storage has been ported toLustre and PVFS2.