23 April 2003 DOE “Science Case for Ultrasim” Pre-Workshop Pre-workshop panel on the science case for ultrascale computing in the Office of Science Washington,

23 April 2003 DOE “Science Case for Ultrasim” Pre-Workshop

Pre-workshop panel on the science case for ultrascale computing

in the Office of Science

Washington, DC

23-24 April 2003

This slide set is based on the original pacing presentation, modified by discussion that took place in committee.


Happy Planck’s Birthday! Born: 23 April 1858, Kiel Nobel Prize in Physics, 1918 Fellow, Royal Society, 1926 President, Kaiser Wilhelm

Gesselschaft (the “German NSF”), 1930-37 & 1945-46; Max Planck Institutes in various fields in Germany today

Died: 4 Oct 1947, Göttingen

“An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarised with the ideas from the beginning”. - Planck


Charge (April 2003, Polansky) “Identify rich and fruitful directions for the computational sciences

from the perspective of scientific and engineering applications” Build a “strong science case for an ultra-scale computing

capability for the Office of Science” “Address major opportunities and challenges facing computational

sciences in areas of strategic importance to the Office of Science” “Report by July 30, 2003” “Foster additional workshops, meetings and discussions on

specific topics that can be identified and analyzed over the course of the next year”

Though our formal charge is Office of Science-centric, the report may ultimately be combined with others in a multiagency thrust, and should be prepared with a broader science case and need not be confined scientifically to Office of Science interests alone


Who we areASCR BER BES FES HENP

ANL Rosner

BNL Glimm

GA Dalhburg

JLAB Whitney

LANL Malone

LBNL Colella, Meza

LLNL Brown Colvin

NREL Hammond

ORNL Harrison Mezzacappa

PNL Windus Windus

PPPL Jardin

SLAC Ko

SNL FreitagPlus ex officio members from DOE HQ, etc.


Choice of strategies The “zero sum” strategy:

Try to look better than the other guys in the same program Improve the appearance of your science and/or denigrate the

prospects for theirs Insist that nothing that now exists can be “on the table”

The growth strategy: Create a united and confident appearance for the program Find common ground (or, even better, synergism), as an

organizing principle for growth in the program Allow any necessary specialization for each constituent (but

explain it well, so that it extends the program rather than distracting from it)


Goals for Wednesday Not to help SC decide directly what to ask for in FY

2005; rather: To outline a report for the science-based case for an

ultrascale simulation capability in the Office of Science To plan the structure, schedule, core invitee list, and

advance homework assignments of a workshop in June that will provide content for the report

To begin to assign responsibilities for report sections and corresponding workshop components

To delegate to a small core group the detailed subsequent coordination of the workshop (weekly telecons)


Goals for Thursday To digest the minutes of Wednesday in a smaller group

with DOE HQ staff To write up a short summary that can be appended to

workshop invitations


Envisioning the report Dozens of authors from labs and universities Editorial oversight by small board, including expertise from

each Office Written hierarchically, with easy-to-read summaries for

Congressional staffers and in-depth sections to impress non-computational experts from scientific community

Broad and as comprehensive as possible, as to areas touched Deep in a number of illustrative places, through case studies Written with strong awareness of antecedents and distinct

mission Laden with good charts and good (but not gimmicky)

scientific eye-candy


Possible topics for report (not in appearance or priority order)

Discussion of relevant previous studies, their impact, and the particular contribution of this report, in context

Discussion of related initiatives that are on-going in other agencies, and in other countries, and perhaps discussion of impacts in the commercial and industrial IT worlds, beyond fundamental research

Discussion of historical outcomes of heavy investment in large-scale computing hardware (e.g., NSF NPACI, ASCI, NERSC, Earth Simulator)

Estimation of the “knees” in the curves of scientific results as a function of increasing power and resolution, for a variety of important applications (climate, astrophysics, QCD, combustion, magnetically confined fusion, proteomics, etc.)


Possible topics for report (not in appearance or priority order), cont.

Demonstration of scaling through quantitative performance models, based on code structure, architecture, and programming model

Discussion of algorithmic and software bottlenecks in the motivating applications for the next 5 years and 10 years

Discussion of what makes a balanced program to produce apps that scale as needed and enlarge the computational community to exploit and interpret them (e.g., software, networking, algorithmic research, staffing, training)

Discussion of balance in the hardware architecture and evaluation of positive and negative trends in the marketplace (e.g., number of functional units, number of processors per node, memory BW per processor, caches and/or vectors, network topology/latency/BW, I/O, archival storage)


Possible topics for report (not in appearance or priority order), cont.

Discussion of meaningful scientific benchmarks for real apps of all important types (e.g., Eulerian PDEs, Lagrangian PDEs, lattice computations/cellular automata, mixed particle/continuum methods, short- and long-range force MD and N-body problems, bio-informatics string-matching algorithms)

Estimation of community needs, distinguishing between development, capability, and capacity purposes

Discussion of actions that we may need to take to encourage vendors to play with us in building scientific instruments rather than just networked webservers


Workshop planning Selected plenaries on context and structure Selected plenaries on scientific content and scaling

analyses Parallel sessions of breakout groups organized by:

Disciplinary areas (one session, w/approx. 10 breakouts) Cross-cutting discussions (two sessions)

Parallel sessions scribed by invited and prepared graduate students; overall meeting and written products professionally scribed with DOE HQ support

Web supported (with a priori and a posteriori postings of position papers and presentations)


Beyond the report Book-length treatment? Dynamic hypertext document?


Antecedents 1982: “Large Scale Computing in Science and

Engineering” (convened by NSB) Four major recommendations:

Increase access to regularly upgraded supercomputing facilities via high bandwidth networks

Increase research in computational mathematics, software, and algorithms

Train people in scientific computing Invest in research on new supercomputer systems

Led to establishment of NSF supercomputer centers interconnected by high speed network, in 1984


Antecedents, cont. 1992: “Grand Challenges: High Performance

Computing and Communications” (OSTP) Proposed 30% increase in federal support of HPCC (to

$638M/yr) Four major components:

High performance computing systems Advanced Software Technology and Algorithms National Research and Education Network Basic Research and Human Resources

Two famous charts (Fig 2 on Grand Challenge requirements and expected year of achievement, and Fig 4 on the joint contributions of algorithms and hardware)

Eight one-page case studies (weather, aero design, oncogenes, etc.)


Antecedents, cont. 1993: “From Desktop to Teraflop: Exploiting the U.S.

Lead in High Performance Computing” (NSF) Introduced the computing pyramid Fourteen recommendations for NSF Four major challenges:

Remove barriers to rapid evolution of HPC technology Provide scalable access to all levels of HPC capability Create incentives to promote broadening of base of participation in HPC Create intellectual and managerial leadership for the future of HPC

Several appendices, including usage statistics and demographics

Statements by individual panel members


Antecedents, cont. 1995: “Report of the Task Force on the Future of the NSF

Supercomputer Centers Program” (NSF) Ten-year review of the NSF supercomputer center program Led to the creation of the PACIs Seven recommendations for NSF

Continue strong, viable centers program Provide national lead-edge sites with balanced resources for rapid advancement of

computational science & engineering Assure partnering with other major research centers Announce new competition, on five-year cycles with favoritism for existing sites Provide research support related to center missions, but not for independent research Increase involvement of NSF directorates in service unit allocations Provide leadership in interagency plans for apex systems

Extensive usage data and testimonials by prominent scientists


Antecedents, cont. 1999: “Information Technology Research: Investing in

Our Future” (PITAC) Issued at peak of economic boom and assigned large share of

credit for the new economy to past federal investments in IT, citing “spectacular return”

Findings and recommendation Federal IT R&D investment is inadequate Federal IT R&D is too heavily focused on near-term problems Recommendation: Create a strategic initiative in long-term information

technology R&D

Put strong focus on software Emphasized essential role of government in IT, though more

than the half the panel was from industry Recommended $1.3B/yr increase in federal IT R&D by 2004


Antecedents, cont. 2000: “Scientific Discovery through Advanced

Computing” (DOE SC) Successor to “Scientific Simulation Initiative” based on “National

Workshop on Advanced Scientific Computing” (NAS, 1998) Identified major challenges in each of the five Offices that can

only be addressed through advanced in scientific computing Recommended three research foci:

Scientific challenge codes Computing systems and mathematical software Collaboratory software infrastructure

Recommended scientific computing hardware infrastructure Flagship computing facility Topical computing facilities

Consciously addressed relationship to ASCI and other federal supercomputing programs


Antecedents, cont. 2003: “Revolutionizing Science and Engineering

through Cyberinfrastructure” (NSF) Presented cyberinfrastructure (CI) requirements of a knowledge

economy in analogy to infrastructure requirements of an industrial economy

Overarching finding IT has crossed thresholds that make a comprehensive cyber infrastructure on which to build new

types of scientific and engineering knowledge environments and organizations possible

Overarching recommendation for NSF Establish and lead interagency Advanced CI Program (ACP) to create, deploy, and apply CI in

ways that radically empower all scientific and engineering research and allied education, with new NSF investment of $1B/yr and coordinated co-investment from other agencies

PACIs to be replaced, following two-year extension to preserve human assets of these centers

Presented archetypal balanced 60 Tflop/s machine costing $180M in 2004; need “sufficient number”of these (~5)


Lessons from antecedents Workshops such as this can really make a difference,

motivating federal investments for years to come Not all of these antecedent reports had a similar charter

as ours; some were official federal advisory committees There are many choices of presentation modes within

these reports: formal text, side bars, charts, case studies, portfolio profiles, detailed appendices, personal statements, etc., that we can consider using, with the aim of featuring strong cases, while being inclusive of science that is not as ripe at the time of writing, but has important potential


Web resources at DOE SC Ultrascale Simulation for Science (www.ulstrasim.info/doe_docs/)

12 “final release” documents (8/02-4/03) 16 “working” documents (8/03-4/03)

Scientific Discovery through Advanced Computing 76 “2-pagers” prepared for the 2003 SciDAC PI meeting (3/03) (

http://www.osti.gov/scidac/updates2003.html) Proceedings (http://www.nersc.gov/conferences/SciDAC2003)

Recent whitepapers precusory to new initiatives throughout the Offices Theory and Modeling in Nanoscience Genomes to Life Fusion Simulation Project etc.

The “applications” matrix (www.appsmatrix.info)


Other resources Topical computing center whitepapers Blue planet and Cray X1 proposal and evaluation

documents Strategic plans of the individual Offices Advisory committee reports


Related initiatives in progress OASCR strategic plan (Dan Hitchcock) “Mission Computing” workshop (Gary Johnson) Hardware taskforce (Rick Stevens) ASCAC HPC roadmap taskforce (Greg McRae) NITRD “High End Computing Revitalization Task

Force” (HECRTF) (www.itrd.gov/hecrtf-outreach/); see also HPCWire from 4 April 2003

NAS study on the Future of Supercomputing Others …


How is our task different? Let’s leverage our SciDAC and other high-end

computational science experience to produce a report that:

Identifies “knees in the curve” of scientific knowledge plotted against computer resources in a variety of applications

Describes the implications of important applications for hardware architecture (without delving into a deep discussion of architecture)

Recalibrates how researchers evaluate the merit of a computational attack (hardware and software), with scientific ends in view (not misleading metrics like teraflop/s)

Anticipates algorithmic and software bottlenecks lurking behind new hardware

Describes a well-balanced program to take us past the “knees” (not just hardware, but methods research, software, training, etc.)


How is our task different?, cont. Let’s not major in recommending dollar amounts or

retargeting of existing programs [not the charter of our type of committee]

Let’s wax quantitative rather than waxing philosophical [there are plenty of reports that motivate and advocate our transition to an information or knowledge economy in general terms, but nothing readable at a non-expert level that takes on the science case for high-end resources as thoroughly as we are capable of doing]


June calendar1 2 3 4 5 6 7

8 9 10 11 12 13 14

15 16 17 18 19 20 21

22 23 24 25 26 27 28

29 30

NSF meeting (DK)

SIAM meeting

OFES review meeting

DOE Mission Computing meeting

Tentative dates

23 April 2003 DOE “Science Case for Ultrasim” Pre-Workshop Pre-workshop panel on the science case for ultrascale computing in the Office of Science Washington,

Documents

doe science case

sciencebased case

strong science case

broader science case

office of science report

science andor

office of science washington

office of science interests