Date: 20151207
UMAC Review Highlights atNCEP Production Suite Review (NPSR)
UMAC committee
Richard Rood and Frederick Carr, co-chairs
December 7, 2015
UMAC Membership: https://www.earthsystemcog.org/projects/umac_model_advisory/members
UMAC Website: https://www.earthsystemcog.org/projects/umac_model_advisory/
Date: 20151207
NPSR Review December 7, 2015
Committee
• Frederick Carr, co-chair• Richard Rood, co-chair• Alan Blumberg• Chris Bretherton• Andy Brown• Eric Chassignet• Brian Colle• James Doyle• Tom Hamill• Anke Kamrath
• Jim Kinter• Ben Kirtman• Cliff Mass• Peter Neilley• Christa Peters-Lidard
• Ex Officio• Gilbert Brunet• William Kuo• Tsengdar Lee
Date: 20151207
NPSR Review December 7, 2015
Report Structure
• Executive Summary• Introduction• Overarching Findings and Recommendations• Findings and Recommendations for Specific Modeling
Themes – Global– Regional– Water-related– Ensembles– Reanalysis, Reforecast, Post-processing
• NCEP as an End-to-End system• Appendices
IntroductionOverarching Findings and RecommendationsSpecific Recommendations
Date: 20151207
NPSR Review December 7, 2015
Timing
• On November 20, 2015 a draft was released to NCEP for fact checking
• Official release: December 7, 2015
Date: 20151207
NPSR Review December 7, 2015
from Overarching Findings and Recommendations
UMAC notes that NOAA's organization of NEWP is unusual in many respects. No other peer environmental prediction service has chosen to separate its basic research and development from its applied, operational model implementation. No other national prediction service has produced such a diversity of prediction systems, most without the critical mass of resources to make them world-best. No other prediction service lacks top-level oversight spanning
the research to the applied development. …
Date: 20151207
NPSR Review December 7, 2015
from Overarching Findings and Recommendations
The key finding is an optimistic one: U.S. Environmental Prediction has the potential to rapidly progress to world leadership. This requires a new level of organization and bold, evidence-driven decision-making.
Date: 20151207
NPSR Review December 7, 2015
from Overarching Finding and Recommendations
• Reduce the complexity of the NCEP Production Suite.
• The NOAA environmental modeling community requires a rational, evidence- driven approach towards decision-making and end-to-end modeling system development.
• A unified, collaborative strategy for model development across NOAA is needed.
Date: 20151207
NPSR Review December 7, 2015
from Overarching Finding and Recommendations
• Essential to effective planning and execution is the creation of a Chief Scientist position for Numerical Environmental and Weather Prediction (NEWP).
• NOAA needs to better leverage the capabilities of the external community
• NOAA must continue to enhance High Performance Computing (HPC) capabilities.
Date: 20151207
NPSR Review December 7, 2015
from Overarching Finding and Recommendations
• NOAA must develop a comprehensive and detailed vision document and strategic plan that lay out future development of national environmental prediction capabilities.
• Execute strategic and implementation plans based on stakeholder requirements.
Date: 20151207
NPSR Review December 7, 2015
The future will require
• Moving to more unified systems• Addressing increasing scientific and data
complexity• Relying more broadly and more effectively
on community research and assets
Date: 20151207
NPSR Review December 7, 2015
Balanced, evidence-based decisions across products and systems
• Stakeholder requirements• Scientific excellence • Cost
Date: 20151207
NPSR Review December 7, 2015
Highlighting Global (Rood Take 1)
• The NGGPS and NEMS are very encouraging as scientific, software, and management advances.– Organizational focus needs to unite on these
efforts – Data assimilation needs to be incorporated into
these efforts– GFS, existing global systems, need to have a phase
out plan to support NGGPS becoming operational
Date: 20151207
NPSR Review December 7, 2015
Highlighting Regional (Rood Take 1)
• Daunting complexity that is not scientifically justified.
• Needs to eliminate systems • Needs to integrate better with global
modeling• Needs a vision
– Unified systems– Convective-resolving U.S. ensembles– Software management
Date: 20151207
NPSR Review December 7, 2015
Air Quality, Dispersion, Space
• Panel felt we are not properly staffed to address technical issues.– Endorse Space Weather integrated into NGGPS– Air Quality and Dispersion requirements /
interface should be integrated with NGGPS and atmospheric systems
– We note the community design and review of Air Quality (and Dispersion?) as a successful model of practice
Date: 20151207
NPSR Review December 7, 2015
Highlighting ‘Water” (Rood Take 1)
• Multi-organizational demands more management attention
• Requirements and resource demands for the evolving National Water Center need more attention and integration with EMC and NCEP Central Operations
• WAVEWATCH a successful model of community engagement
Date: 20151207
NPSR Review December 7, 2015
Highlighting Ensembles (Rood Take 1)
• NCEP should consolidate its ensemble prediction under unified dynamical cores using physically based stochastic parameterizations to treat model uncertainty in the ensembles
• Need to better integrate with global• Scientific progress more likely to be made
through robust investigation in a well managed system in a controlled environment than through algorithm and model diversity
Date: 20151207
NPSR Review December 7, 2015
Highlighting ‘Reanalysis” (Rood Take 1)
• Decisions on what computational and storage resources are allocated to R/R should be done in a systematic manner, based on how they help NOAA meet its requirements, as recommended elsewhere by UMAC
• Nonetheless, given the demonstrated value of carefully constructed reforecast data sets, post processing and the production of supporting data sets (reanalysis/reforecast, R/R) should be considered an integral part of NCEP’s future production suite and resourced accordingly
• Need more management, design and uniformity in grids used as stakeholder interfaces
Date: 20151207
NPSR Review December 7, 2015
Highlight systems (Rood take 1)
• UMAC calls for a “new-level of organization,” and “evidence-based decision making”
• This section is an analysis and description of the possible meaning of organizational recommendations– Project management– Software management– Organizational governance
Date: 20151207
NPSR Review December 7, 2015
Highlight systems (Rood take 1)
• Short-term to gain control over complexity– Document and maintain complete range of products and
systems– Identify and publicize leads for all products and systems– Hire or identify software leads, with proven expertise in
scientific software– Develop Change Review Boards for all products and systems– Develop a software release schedule for major systems on
the order of 12-24 months– Replace all the code that EMC uses with code developed
with formalized software management
Date: 20151207
NPSR Review December 7, 2015
Final Remarks
• “a transformation to a science-based, service organization directed toward building a weather-ready nation”– Requires a new type of organization:
• More unified• Better able to participate as member of community• Recognize and manage complexity• Requires commitment and continuity from leadership• Requires buy in from entire organization• Will be a transition of years, not simply responding to
reports such as this one
Date: 20151207
NPSR Review December 7, 2015
Acronyms
• NOAA – National Oceanographic and Atmospheric Administration
• NWS – National Weather Service• NOS – National Ocean Service• OAR – Oceanic and Atmospheric Research• NCEP – National Centers for Environmental
Prediction• NWC – National Water Center
• UCACN - University Corporation for Atmospheric Research Community Advisory Committee for NCEP
• UMAC – UCACN Model Advisory Committee
• NGGPS: Next Generation Global Prediction Center
• NEMS: NOAA Environmental Modeling System
• CMC: Canadian Meteorological Center• ECMWF: European Center for Medium-
range Weather Forecasts• NCAR: National Center for Atmospheric
Research• NRL: Naval Research Laboratory
• S2S: Sub-seasonal to seasonal• S2A: Seasonal to Annual