Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. Optimization and Model Insight Research Directions at Sandia National Laboratories Scott A. Mitchell INFORMS Chicago Chapter CUSTOM Managing Risk in an Uncertain World
70
Embed
Optimization and Model Insight Research Directions at Sandia National Laboratories
Optimization and Model Insight Research Directions at Sandia National Laboratories. Scott A. Mitchell INFORMS Chicago Chapter CUSTOM Managing Risk in an Uncertain World. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Optimization and Model InsightResearch Directions at
Sandia National Laboratories
Scott A. Mitchell
INFORMS Chicago ChapterCUSTOM
Managing Risk in an Uncertain World
Intro
• Take home messages
– Why we’re doing what we’re doing, not much of the how• Pose the questions, not all the answers
– Sandia environment• Breadth of SNL mission• Unique applications
– Physics variety
– Extreme computations and simulations
– Risks to manage, uncertainties to assess
– Our response• Current R&D activities• New research directions
• ( My “not-to-do” list
– Policy and political issues )
Acknowledgements
• Optimization and Uncertainty Estimation dept. – http://www.cs.sandia.gov/departments/9211/index.htm
• Thanks to Urmila for invitation• Thanks to department staff…
– Tim Trucano– Tony Giunta– Mike Eldred– Bart van Bloemen Waanders– Roscoe Bartlett
• … and others throughout Sandia, from whose viewgraphs I have borrowed liberally – Marty Pilch– John Aidun
• Every year, Labs’ director signs off on stockpile– Says testing not needed (so far)
• Standards< 1 in 10^6 accidents result in “any” nuclear reaction
• Conservatism in design– Almost no performance data
• Very few controlled tests. Systems rarely actually used or in accidents (that’s good, but makes data scarce!)
– Tests designed to demonstrate performance, not to test limits of failure. Rarely designed for parameter studies. (Aim for mid-points of parameter ranges, not extremes.)
• Contrast to automotive industry– Imagine building the Ford Expedition fleet and meeting
individual and aggregate specs, if last car fleet actually driven on roads to failure were 10 model T’s.
Pulsed Power & Inertial Confinement Fusion (ICF)
• ICF is a goal at Sandia National Labs
• Pulsed Power Technique using Z-machine
•Wire arrays explode,creating a plasma sheath, which implodes and stagnates.
•X-rays hit capsule, generating fusion
•Capsule performance very sensitive to variations!
To fulfill our National Security mission, we develop systems and components designed to perform
Sandia: The Extreme Engineering Lab
extreme applications, under adverse conditions.
The duty cycle can include damage, crush and failure;
Extremely Small & Efficient:• Microsystems & Nanotechnology• Solid State Lighting• Next Generation NG
Extremely Reliable:• Electrical Connectors• Bonds
– Solder Joints– Adhesives Joints– Brazes and Welds
World’s smallest linear accelerator
StructuralDynamics
Incompressible Fluid flow
Heat transfer
Shock Physics
Compressiblefluid flow
Geophysics
Solid mechanics
Fire
Electrical
Design Tools:optimizationsensitivity
uncertainty analysis
Motivation: Target Simulation Codes
Spin the wheel:our example du jour
Large Structural Dynamics Sandia Calculations
• Example Large Sandia Calculations– Gordon Bell Prize won by ‘Salinas’ Structural
Dynamics Code team in “special” category at SuperComputing 2002
• Sandia has had many wins in the past• Only non Earth Simulator winner in 2002• Sandia Director of Engineering Sciences
Tom Bickel: first time a “production code” won– C++– Math and solver libraries– Systems of equations like
• Production eigen solver based on ARPACK, Ax=Bx• Linear statics, dynamics, Ax=b• Nonlinear statics, dynamics, A1x1=b1, A2x2=b2,… Anxn=bn
Northrop Grumman Newport News Full Ship Model Description
•Large-scale detailed model–2 million equations–Shell & beam elements– Interior structure
•60 processors (Baby Q)•20 Modes/1.5 hours
Sway
Blast
Increasing Levels of Model Complexity
Explosion in computer hardware and software technologies allows higher levels of structural dynamics modeling sophistication.
15 years ago:Shellshock 2.5D
200 equations, Pre-Cray
Recent Past:NASTRAN
Electronics package30,000 equations,
Cray, Vector Supercomputer
Electronics package model 2 yrs ago
400,000 equations,Parallel
Supercomputer
Increasing Levels of Model Complexity
Now, 10+ million equation (modeling at circuit board level)
Electronics package model 1 yr ago: 800,000 equations
Cubit: Sandia Meshing Tool
Gordon Bell SC2002: 0.5M equations, 18 minutes on 128 processors of ASCI Red
Sandia Capabilities
• Sandia considers itself (needing to be an) expert in– Advanced Manufacturing– Biosciences– Chemical and Earth Sciences– Computer Information Sciences– Electronics– Engineering– Homeland Security– Materials and Process– Microsystems– Nanotechnology– Pulsed Power
• Modeling and Simulation of the above!• My dept’s role: tools for design optimization,
reliability assessment, of the above in uncertain environments.
Computational Focus
• Limitations of testing(I’m talking about non-nuclear engineering testing here… Even so, for some things, we wouldn’t test even if we could…)
– Too many possible designs / scenarios• 20d param space
– Not going to do 2^20 tests, but could explore via sampling, SAND (later)
– Could you instrument an experiment adequately?
– Worst case scenarios only identifiable by computational tools
• paradigm shift from “engineering intuition”
Limits of Computation
• In the past, • Calibrate computational models
to observed experiments
– Codes used for interpolating between tested points
– “Verification” criteria is approx. curve fit up to “view graph norm” after calibration
– Underlying physics unimportant, any function that curve-fits is ok
• In the future, want • Predictive capability
– Codes used for extrapolating to unknown designs / environments
– “Verification” criteria is error bars in computed answer matching error bars in experimental answer
– Confidence in underlying physics, “validation” that important phenomena are modeled
Key Challenges of this Approach
• How much credibility is sufficient?– Variabilities and uncertainties must be
acknowledged, and their impact in the decision context quantified
Technical themes to meet these challenges1. Validation & Verification
• To trust the simulations
2. Sensitivity-based (intrusive) methods• Expensive simulations and/or large design spaces• Seven levels of intrusion• Large scale problem motivation• Inversion and homeland security projects
3. DAKOTA• Noisy and/or expensive simulations• Levels of parallelization• Strategies and methods• SBO, OUU -> SBOUU
1. Validation & Verification Theme
• Hamming – “The purpose of computing is insight…” (?)
• ASCI – the purpose of computing is to provide “high-performance, full-system, high-fidelity-physics predictive codes to support weapon assessments, renewal process analyses, accident analyses, and certification.” (DOE/DP-99-000010592)
• Philip Holmes – “…a huge simulation of the ‘exact’ equations…may be no more enlightening than the experiments that led to those equations…Solving the equations leads to a deeper understanding of the model itself. Solving is not the same as simulating.” (SIAM News, June, 2002)
Useful quotes to keep in mind.
ASCI Program
• ASCI Applications developing suite of simulation codes– Milestones like “simulate phenomena X”– Major supporter of DAKOTA / our dept’s activities
• ASCI Validation & Verification program– To verify those codes, and validate the model
applicationso we have confidence in our answer
– Uncertainty Quantification seen as a key technology for validating codes, by overlapping computational and experimental error bars.
– DAKOTA is the tri-labs delivery vehicle for UQ technology
• Validation definition– Is the computational model an accurate
representation of reality (the reality I care about)?• Depends on phenomena of interest• Depends on decision you need to make
• Verification definition– Given the computational model, does the code
produce the right answer?• Depends on SQE, accurate solvers, allowing code to
run to convergence, etc.
Validation and Verification
• Hint: Validation is a “physics problem.”
• Hint: Verification is a “math problem.”
Consider the following “validation” exercise:
1
2
3
5
6
4
Incident Angle
p
r/
pin
c
35 40 45 5030
Experiment + Error Bar
Analytic
ALEGRA Calculation
This is physics.
This is math.
• So, fundamentally, what does this comparison mean?• Please note that the calculation is not converged.• Stringently, “verification” for numerical PDE codes
basically means:
Demonstrate convergence to the correct answer.
• …if not for the “code” at least for the particular calculation(s).
• Since it is unlikely that we will establish convergence and since we don’t know what the correct answer is this is quite a problem.
There is at least one essential problem with the previous comparison – there are no numerical error bars.
• Uncertainty in verification arises from:– (Recall complexity of simulations = functions we hope to
• Code crashes are the least of our problems.• Mutually reinforcing errors are also “easily” detectable.• Mutually canceling errors are of greater concern.
– Inadequate algorithms• No amount of resolution will solve the problem.
– Inadequate resolution• Resolution “solves” the problem but is probably
unavailable.
• The issue of “verifying” ASCI Level 1 milestones is becoming prominent.
“Uncertainty In Verification”
Is Probabilistic Software Reliability (PSR) useful for computational science software?
• We are test fixated in building software, properly so:
“Based on the software developer and user surveys, the national annual costs of an inadequate infrastructure for software testing is estimated to range from $22.2 to $59.5 billion.” (“The Economic Impacts of Inadequate Infrastructure for Software Testing,” NIST report, 2002.)
• If we can’t test software perfectly, then testing alone does not solve the verification problem.
A view of software “reliability” is decreasing # of “failures” and increasing # of “users” and they are correlated.
A notional “reliability” diagram for a PDE code thus looks something like the following:
Development and test Capability I Capability II Capability etc
Fai
lure
Rat
e
Application DecisionsApplication Decisions
# of Users
1st use / Validation
(WH
AT
IS
A F
AIL
UR
E?
)
• I don’t know what the error is with certainty.
• I still need to apply the calculation.
• The alternative is analysis paralysis (in particular, NUMERICAL analysis paralysis).
– “Global warming” is an example; keep it in mind.
The bottom line in the previous example can be generalized: numerical accuracy is an important uncertainty.
Qualitative: “I’m uncertain what the accuracy of this calculation is.”
Qualitative: “I’m uncertain what the accuracy of this calculation is.”
Quantitative leap: “I need to apply probabilistic language to describe my understanding of the
accuracy of this calculation”
Quantitative leap: “I need to apply probabilistic language to describe my understanding of the
accuracy of this calculation”
Are Probabilistic Error Models (PEM) useful for computational science software?
• Suppose that we can neither “verify codes” nor “verify calculations.”
– “When quantifying uncertainty, one cannot make errors small and then neglect them, as is the goal of classical numerical analysis; rather we must of necessity study and model these errors.”
– “…most simulations of key problems will continue to be under resolved, and consequently useful models of solution errors must be applicable in such circumstances.”
– “…an uncertain input parameter will lead not only to an uncertain solution but to an uncertain solution error as well.”
• These quotes reflect a new view of “numerical error” expressed in B. DeVolder, J. Glimm, et al. (2001), “Uncertainty Quantification for Multiscale Simulations,” Los Alamos National Laboratory, LAUR-01-4022.
– “All models are wrong-but some models are useful,” statistician George P. E. Box.
2. Sensitivities Theme
Sensitivities Project
• Sensitivities (Derivatives, Jacobians, Hessians, etc.) can dramatically speed up optimization over large PDE-based codes
• NAND• SAND approach names
– PDE-Constrained Optimization, orSimultaneous Analysis and Design – SAND, orall-at-once-approach
Large Scale PDE Constrained Optimization
Optimizer
PDE simulationInput Output
PDE simulationInput Outputoptimizer
BlackBox
SANDMOOCHO
Idea: add PDE equations as constraints to optimization problem
PDE equations as constraints
• PDE-constrained optimization formulation
u = design vars, nu moderatey = PDE FEM state vars, ny huge
– NAND: Nested Analysis and Design• Eliminate or solve PDE equations, • Then optimize keeping them eliminated / solved• Bonus: sensitivities speed up NAND
– SAND: Simultaneous Analysis and Design• Start infeasible• Then solve as you converge• Alternative gradient formulations, SAND / NAND
– Alternative sensitivity formulations - Next slidesPremo solution time using sensitivities
Gradients
• NAND Solve
– By eliminating / solving c, get
• Use gradient based optimization
T
NAND Solve ----------
• Grad based opt, 3 choices, keep c solvedLevel-1: Finite differences
– Exact gradients, solve either
Level-2
Level-3 or
Pre-compute adjoint sensitivity matrix, independent of u
compute direct sensitivity matrix,depends on u, so compute product each step
How to get sensitivity matrices? Paper and pencil? AD?
SAND Solve -----------
• Truly solve simultaneously
– Level-4, use direct sensitivities
– Level-5, use adjoint sensitivities• Transpose of Jacobians
– Level-6, assemble & solve full KKT system (or QP subproblem)• Second derivatives
Problem Size: solving for velocities (x,y,z), temperature, pressure, 3 species for a total of 31,992 unknowns
Payoff: run-time approximately independent of number of design variables (PDE solution dominates, done about once).
3. DAKOTA Theme
Surrogate Based Optimization
4 Levels of Parallel ComputationExample: MINLP
• Sophisticated parallel efficiency
1. Strategy: concurrent exploration of design space– Parallel branching cases due to MINLP branching
2. Different simulation code runs– Objective or constraints require different physics codes, e.g.
constrain vibration and temperature
3. Multiple simulation runs at nearby values– For finite differences or pattern search
4. Parallel simulation codes– E.g. Salinas, PRONTO inherent parallelism
Background: Surrogate-Based Optimization
• To meet Sandia’s challenge: surrogate based optimization– algorithms that perform optimization on a low fidelity “surrogate model” with
periodic corrections from a high fidelity model.• Two motivations
– Low fidelity model can be computationally inexpensive– Low fidelity model can be more smooth
• Surrogate model types:– Multifidelity simulation models having different mesh densities and/or physical
accuracy. (Inexpensive)– Multidimensional surface fitting methods that smooth out “noise” in the
simulation data. (Inexpensive & more smooth)• polynomial regression, spline interpolation, radial basis functions, etc.
• Ad hoc SBO algorithms have been used by engineers for decades, but these often failed without explanation.
• Provably-convergent and heuristic SBO algorithms in DAKOTA.– Provably convergent to a local minimum when gradients are available
(journal article in review)
– Insight into methods for proving convergence when gradients are unavailable(collaboration with Prof. Luis Vicente – Univ. Coimbra, Portugal)
– Recent 2nd order correction results for TR-SBOUU
1.0
0.3
0.41.0
x1
f(x)
x2
1.0
0.0
Surface Fitting Functions as Surrogate Models
• Inexpensive, more smooth
• Many smoothing function types:– polynomial regression– radial basis functions– kriging interpolation– neural networks
• Usually employ data sampling or statistical design of experiments methods.
• Practical limit of O(101-102) independent variables.– sampling becomes inefficient for
high dimensional problems– but, many engineering design
problems are low dimensional
Area smoothing
function is applied.
From SBO to SBOUU
From SBO to SBOUU:• SBO is provably convergent with trust region
globalization– 1st order consistency (e.g., beta correction)– verification of approx. steps
• Extensions to SBOUU– 1st order consistency, assuming a worthwhile
stoch. gradient– verification of stats. in relative sense:
nonoverlapping confidence intervals (rigorous or stoch. approx.)
Sequence of trust regions
Optimization under Uncertainty with Surrogates
Opt
UQ
Sim
{d} {Su}
{u} {Ru}
Opt
UQ
Sim
{d} {Su}
{u} {Ru}
Data Fit
{d} {Su}
Opt
UQ
{d} {Su}Data Fit
{d} {Su}
Sim
{u} {Ru}
Data Fit/Hier
{d} {Ru}{u}
Opt
UQ
Sim
{d} {Su}
{u} {Ru}
Data Fit/Hier
{d} {Ru}{u}
Formulations 2 & 4 amenable to trust-region approachesGoals: maintain quality of results, provable convergence (for a selected confidence level)
DakotaModel
Single Layered Nested
Data Fit Hierarchical
Nested model: internal iterators/models execute a complete iterative study as part of every evaluation.
Layered model: internal iterators/models used for periodic update and verification of data fit (global/local/multipoint) or hierarchical (variable fidelity) surrogates.