VAPoR: A Discovery Environment for Terascale Scientific Data Sets Alan Norton & John Clyne National Center for Atmospheric Research Scientific Computing.
Post on 14-Jan-2016
214 Views
Preview:
Transcript
Supercomputing • Communications • Data
NCAR Scientific Computing Division
VAPoR: VAPoR: A Discovery Environment forA Discovery Environment forTerascale Scientific Data SetsTerascale Scientific Data Sets
Alan Norton & John Clyne Alan Norton & John Clyne National Center for Atmospheric ResearchNational Center for Atmospheric Research
Scientific Computing DivisionScientific Computing Division
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
VAPoR project overviewVAPoR project overview
GOALGOAL: Improve ability of earth sciences researchers to : Improve ability of earth sciences researchers to analyze and explore complex dynamics found in vast data sets analyze and explore complex dynamics found in vast data sets resulting from high-resolution, 4D (3D space x time) numerical resulting from high-resolution, 4D (3D space x time) numerical simulations of fluid flow in the oceans, atmosphere and sunsimulations of fluid flow in the oceans, atmosphere and sun
Open Source software development effort aimed at bridging Open Source software development effort aimed at bridging gap between visualization research and application tool gap between visualization research and application tool availabilityavailability
Funded by the NSF ITR program, VAPoR is a collaborative Funded by the NSF ITR program, VAPoR is a collaborative effort between NCAR’s Visualization and Enabling Technology effort between NCAR’s Visualization and Enabling Technology Section, U.C. Davis’ Institute for Data Analysis and Section, U.C. Davis’ Institute for Data Analysis and Visualization, and Ohio State University’s, Department of Visualization, and Ohio State University’s, Department of Computer and Information SciencesComputer and Information Sciences
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
The numbers – an example from solar compressible The numbers – an example from solar compressible convectionconvection 512x512x2048512x512x2048 5 variables 5 variables
(u,v,w,rho,temp)(u,v,w,rho,temp) ~500 time steps saved~500 time steps saved 9TBs storage9TBs storage Six months compute time Six months compute time
required on 112 IBM SP required on 112 IBM SP RS/6000 processors RS/6000 processors (blackforest)(blackforest)
Three months for post-Three months for post-processingprocessing
Data may be analyzed for Data may be analyzed for several yearsseveral years
Image courtesy of Joseph Mendoza, NCAR/SCD
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
Our approachOur approach
Exploit power of multiresolution data representation Exploit power of multiresolution data representation combined with progressive data access and efficient combined with progressive data access and efficient region sub-setting to enable user to make region sub-setting to enable user to make speed/quality tradeoffs speed/quality tradeoffs [[Pascucci and Frank, SC2001; Clyne, VIIP2003]Pascucci and Frank, SC2001; Clyne, VIIP2003]
Tightly couple traditional, non-visual (MatLab, IDL) Tightly couple traditional, non-visual (MatLab, IDL) analysis with highly interactive, advanced visualization analysis with highly interactive, advanced visualization [Clyne and Rast, VDA2005][Clyne and Rast, VDA2005]
Develop domain-specific application for numerical Develop domain-specific application for numerical simulation of fluid flowsimulation of fluid flow
Emphasize Emphasize usability usability for scientistsfor scientists
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
A visual comparison between coarsened and original dataA visual comparison between coarsened and original data
1283 5123
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
Combining visual and non-visual analysis toolsCombining visual and non-visual analysis tools
When visualization and quantitative tools are When visualization and quantitative tools are appropriately combined, result is a powerful appropriately combined, result is a powerful environment for scientific discoveryenvironment for scientific discovery
Couple existing quantitative tools (e.g. IDL, Matlab, Couple existing quantitative tools (e.g. IDL, Matlab, NCL) with VAPoR’s advanced visualization capabilitiesNCL) with VAPoR’s advanced visualization capabilities
Benefits of integration with existing analysis toolsBenefits of integration with existing analysis tools– Avoids duplication of effortAvoids duplication of effort– Lower development timeLower development time– Leverage what scientists already knowLeverage what scientists already know
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
Integrated analysis and visualization systemIntegrated analysis and visualization system
VtkHigh quality rendering
VAPoRInteractive visual browsing
IDLData manipulation &
analysis
Multiresolution data access
Disk Array
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
Software Design PrioritiesSoftware Design Priorities
Functional and usability requirements prioritized by scientific usersFunctional and usability requirements prioritized by scientific users Cross-platform (Linux, Irix, Windows, Mac,…)Cross-platform (Linux, Irix, Windows, Mac,…) Enable side-by-side comparison of multiple time-varying Enable side-by-side comparison of multiple time-varying
visualizationsvisualizations– Unique approach to management of multiple parameter setsUnique approach to management of multiple parameter sets
Exploit high-performance graphics cardsExploit high-performance graphics cards– GPU programmingGPU programming
Incorporate recent advances in data visualizationIncorporate recent advances in data visualization– Volume renderingVolume rendering– Flow visualizationFlow visualization– Isosurface generationIsosurface generation– Color/transparency mappingColor/transparency mapping
XML data descriptionXML data description– Encourage wider usage of multi-resolution data representationEncourage wider usage of multi-resolution data representation
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
DemonstrationDemonstration
Load multiresolution data into VAPoR applicationLoad multiresolution data into VAPoR application Combine global and local parameter settingsCombine global and local parameter settings Edit transfer function(s)Edit transfer function(s) Interactive region selectionInteractive region selection AnimationAnimation Export to IDLExport to IDL Use IDL for analysisUse IDL for analysis Visualize analysis resultsVisualize analysis results
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
Future PlansFuture Plans
Incorporate visualization techniques based on scientists’ Incorporate visualization techniques based on scientists’ needsneeds– Nonuniform gridsNonuniform grids– Vector field visualizationVector field visualization– IsosurfacesIsosurfaces– Contour planesContour planes
Understand effect of lossy data compressionUnderstand effect of lossy data compression– Error analysis and error visualization Error analysis and error visualization – Obtain bounds on degradation of analysis resultsObtain bounds on degradation of analysis results– Measure the effect of performing analysis on lower-resolution Measure the effect of performing analysis on lower-resolution
datadata Improve access to terabyte datasetsImprove access to terabyte datasets
– Multiresolution data output as a byproduct of the simulationMultiresolution data output as a byproduct of the simulation
Supercomputing • Communications • Data
NCAR Scientific Computing Division Alan Nortonalan@ncar.ucar.edu
SummarySummary
NSF funded collaboration with Davis and Ohio stateNSF funded collaboration with Davis and Ohio state Domain specific application aimed at improving productivity Domain specific application aimed at improving productivity
of turbulence researchers (though other groups may benefit)of turbulence researchers (though other groups may benefit) Scientists are target user group (not visualization specialists)Scientists are target user group (not visualization specialists) Multiresolution data representation exploited for handling Multiresolution data representation exploited for handling
large datalarge data Quantitative and advanced visualization methods combinedQuantitative and advanced visualization methods combined Usability is key to adoption by scientistsUsability is key to adoption by scientists
I can directly testify to the crucial importance of this [effort] to our I can directly testify to the crucial importance of this [effort] to our numerical work... to develop interactive tools that can access and numerical work... to develop interactive tools that can access and analyze the flow in its entirety, and visualize it in complex ways analyze the flow in its entirety, and visualize it in complex ways including parallelized algorithms of perspective volume rendering, including parallelized algorithms of perspective volume rendering, superposition of scalar and vector fields, and multi-resolution access superposition of scalar and vector fields, and multi-resolution access to the flows.to the flows.
Annick Pouquet, Director of ESSLAnnick Pouquet, Director of ESSL
top related