C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Data Management Requirements: Computational Combustion and Astrophysics John Bell Center for.

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Data Management Requirements:

Computational Combustion and Astrophysics

John BellCenter for Computational Sciences and Engineering

Lawrence Berkeley National Laboratory

[email protected]

http://seesar.lbl.gov

December 11, 2006

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

What we are trying to do

Combustion Detailed analysis of premixed turbulent combustion

• Lean premixed systems have potentially high-efficiency and low emissions

• Design issues because premixed flames are inherently unstable

Astrophysics Simulate white dwarf from convection through explosion Type Ia supernovae are play a key role in modern

cosmology but the explosion mechanism is not understood

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Computational approach

Components of a computational model

Mathematical model: describing the science in a way that is amenable to representation in a computer simulation

Approximation / discretization: approximating an infinite number of degrees of freedom with a finite number

Solvers and software: developing algorithms for solving the discrete approximation efficiently on high-end architecture

We attempt to exploit the special structure of the problems we are considering to compute more efficiently

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Adaptive Mesh Refinement

Spatial discretization should exploit locality

Structured adaptive mesh refinement Hierarchical patches of data Dynamically created and destroyed

Combination of new numerical methodologies reduces computational effort by several orders of magnitude

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

V-flame

Simulate turbulent V-flame Strategy – Independently characterize nozzle and

specify boundary conditions at nozzle exit 12 £ 12 £ 12 cm domain Methane at = 0.7

• DRM 19, 20 species, 84 reactions• Mixture model for species diffusion

Mean inflow of 3 m/s Turbulent inflow

• lt = 3.5mm, u' = 0.18 m/sec• Estimated = 220 m

No flow condition to model rod Weak co-flow of air

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Experimental comparisons

Simulation Experiment

Instantaneous flame surface animation

Flame brush comparisonsJoint with M. Day, J. Grcar, M. Lijewski, R. Cheng, M. Johnson and I. Shepherd, PNAS, 2005

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Thermo-diffusive Effects

Low swirl burner flames for different fuels

Experiments focused on effect of different fuels on flame behavior Identical fueling rate and turbulence Nearly the same stabilization nearly the same

turbulent burning speed

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Local flame speed analysis

Construct local coordinate system around flame and integrate reaction data

Other mathematical analysis paradigms Stochastic particles Pathline analysis

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Diffusion flames

Study behavior of fuel bound nitrogen characteristic of biomass fuels

What do experimentalists measure Exhaust gas composition Planar laser-induced fluorescence

• Temperature• NO concentration

NO measurements Illuminate flame with a tuned laser sheet

• NO absorbs a photon• Measure emission

Problem – NO can lose photon in a collision before it is emitted -- Quenching

Joint with P. Glarborg, A. Jensen, W. Bessler, C. Schulz

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

NO measurement

Quenching requires knowledge of local composition and temperature

fB,i – Boltzmann population term

g,i – Linear shape profile

Qk(p,T,X) – Electronic quenching

Experimentalists typically guess the composition for quenching correction

Generate synthetic PLIF images from simulation

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

NO measurement – cont’d

NO-NO A-X(0,0)

Excitation

NO A-X(0,2) Excitation

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

NO Cont’d

Can use simulation data to compute quenching correction to experimental data

Simulation also provides a more detailed picture of nitrogen chemistry Reaction path gives quantitative picture chemical

behavior of the system

225

192

160

128

96

64

32

0

0 ppm 590 ppm 790 ppm 1420 ppm ppm

Sim

ulat

ion

Exp

erim

ent

420 ppm260 ppm

Proc. Comb. Inst., 2002

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Type Ia Supernovae

Thermonuclear explosion of C/O white dwarf.

Brightness rivals that of host galaxy, L ¼ 1043 erg / s

Large amounts of Radioactivity powers the

light curve Light curve is robust

Standard candle in determining the expansion of the universe

56Ni

SN 1994D

Computational Astrophysics Consortium: Adaptive

Methods

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Astrophysics issues

. . . Are about the same

Specialized treatment of fluid mechanics Chemistry -> Nuclear physics Complex diffusive transport -> radiation

Simulations Common software framework AMR

Turbulent Spectrum

Astrophysical Journal, 2006

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Workflow

How do we extract “science” from the simulation data

We typically don’t do visual analysis of the raw data

Our analyses typically start with some “mathematical” transformation of the data but , . . . to leading order, we can’t a priori define what this means I can’t define requirements Typically, we will do some prototype as part of defining what we

want to look at Data analysis tool needs to be able to ingest application specific

information Data analysis tool requirements

Work with hierarchical data Incorporate problem physics Hardened version of prototype tools Integrated with visualization

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Data management

How does the data flow through this process Run simulation Dump data in plotfiles (reasonable I/O) Tar plotfiles and archive in mass storage

• We only store data at end of coarse steps, and maybe not each coarse time step

Analyze data• Pull data from mass storage• Move data to analysis platform

— Analysis must be done in parallel— Machine for computation may not be good for analysis

• Untar plotfile data• Run analysis program (reasonable I/O; demand driven)

This fundamentally does not scale Have to move everything from mass storage to rotating disk Need to shift data to appropriate platforms

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

I/O

How do we do I/O? AMR Plotfiles

Model used to be “each processor writes to disk” Level x Processors files

Now,

Identify number of desired channels Tell processors when it is their turn to write Level x desired channels files

Is this scalable? Will something work better?

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Visualization / Analytics

AMRVIS?D Reads plotfiles directly Main visualization is slices through data Limited functionality of contouring, vector fields, volume

rendering Supports a data spreadsheet capability

Fancier visualization done with TECPLOT Not particularly scalable

Adopt VISIT as principal vis tool Relation to AMRVIS – replace? need spreadsheet

capability What functionality is missing

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N

Requirements

Data management cartoon Simulate at ORNL – make large tar files Move data to NERSC archive (manually) Iterate on

• Pull data from archive on DaVinci (manually)• Run analysis programs (manually)

We need to develop a schema to facilitate analysis without so much manual data movement Automate data transfer Archive data so we only read what we need and

can stage retrieving data from storage Automate processing large amount of data once

prototypes are operational