1 CEOS WGISS-21 12 February 2014 NOAAs Global Earth Observation Integrated Data Environment (GEO-IDE) CEOS WGISS-21 Budapest, Hungary 2006 David Clark.

1 CEOS WGISS-21

April 10, 2023

NOAA’s Global Earth Observation

Integrated Data Environment (GEO-IDE)

CEOS WGISS-21Budapest, Hungary 2006

David Clark NOAA for

Glenn Rutledge, GEO-IDE Co-Chair

2 CEOS WGISS-21

April 10, 2023

NOAA’s GEO-IDE

Scope – NOAA-wide architecture development to integrate legacy systems and guide development of future NOAA environmental data management systems in support of US-GEO and GEO

Vision – NOAA’s GEO-IDE is envisioned as a “system of systems” – a framework that provides effective and efficient integration of NOAA’s many quasi-independent systems

Foundation – built upon agreed standards, principles and guidelines

Approach – evolution of existing systems into a service-oriented architecture

Result – a single system of systems (user perspective) to access the data sets needed to address significant societal questions

3 CEOS WGISS-21

April 10, 2023

*"Things should be made as simple as possible, but no simpler.“

Albert Einstein

Service-Oriented Architecture*

4 CEOS WGISS-21

April 10, 2023

Why is GEO-IDE Needed?Important societal issues require data from many observing systems

Atmospheric Observations

Land Surface Observation

Ocean Observations

Space Observations

Data Systems Coordinated, efficient,integrated, interoperable

Discipline Specific View Whole System View

Current systems are program specific, focused, individually efficient.But incompatible, not integrated, isolated from one another and from wider environmental community

4

5 CEOS WGISS-21

April 10, 2023

Today’s Challenges Incompatible syntax (formats) and semantics (terminology) among

science disciplines within NOAA. Thousands exist. Several examples:

• Naming standards – Surface Air Temperature• Meteorology (WMO) named “Temperature/dry bulb temperature• Meteorology (air pollution) named “Boundary layer temperature”• Oceanography named “Air Temperature”

• Location standards (latitude, longitude, elevation)• Lat/Lon can be degrees/minutes/seconds or degrees to tenths and hundredths • Latitude E/W, 0-180 positive and negative, or 0-360 running east or west• Z used to designate elevation in both atmosphere and ocean but positive is up in the

atmosphere and down in the ocean• Formats (>50 formats used within NOAA; translators and standards needed)

• GRIB, NetCDF, HDF and others used for gridded data• BUFR, NetCDF, and many others used for observations

Potential for no answer or the “wrong” answer to important societal questions due to separate NOAA data management systems

6 CEOS WGISS-21

April 10, 2023

Goals

Through GEO-IDE NOAA will Identify and address integration gaps in data management systems Create interoperability across existing data management systems Develop and adopt data standards for formats and terminology Integrate measurements, data, and products Examine the need for future data management requirements

and will achieve Cost avoidances in NOAA business through improved efficiency

and reduced duplication Reduced risks for US IEOS and GEOSS

7 CEOS WGISS-21

April 10, 2023

GEO-IDE Bridging the gaps between stove-pipe systems

Weather Climate Oceanography Biology Geophysics

Standard procedures, protocols, metadata,

formats, terminology.Web Services

• Integration of data across disciplines• Improved data stewardship

• Increased efficiency• Leverage industry and community initiatives

• Integration of data across disciplines• Improved data stewardship

• Increased efficiency• Leverage industry and community initiatives

HydrologyGeospatialFramework

8 CEOS WGISS-21

April 10, 2023

GEO-IDE - an essential component ofenvironmental information management for NOAA

Integrated observing, data processing and information management systems

Connected by NOAA’s Integrated Data Environment

Contributes to U.S. Global Earth Observation System (USGEO) andInternational Global Earth Observing System of Systems (GEOSS).

9 CEOS WGISS-21

April 10, 20239

Why Now?

Critical to USGEO– six near term opportunities

Uncoordinated development leads to inefficiencies, incompatibilities, and duplication of effort.

Integration of data among systems is needed to answer questions that address diverse societal benefits

Increased efficiency is needed to handle the expected exponential increase in data volumes that will occur over the next decade

Societal Benefits

• Improve weather forecasting

• Reduce Loss of Life and Property from Disasters

• Protect and Monitor our Ocean Resources

• Understand, Assess, Predict, Mitigate and Adapt to Climate Variability and Change

• Support Sustainable Agriculture and Combat Land Degradation

• Understand the Effect of Environmental Factors on Human Health & Well Being

• Develop the Capacity to Make Ecological Forecasts

• Protect and Monitor Water Resources

• Monitor and Manage Energy Resources

10 CEOS WGISS-21

April 10, 2023

Vision

“System of systems” – a framework to effectively and efficiently integrate NOAA’s many systems

Minimize impact on legacy systems

Utilize standards: – Adopt, adapt and only

reluctantly create– Emphasis on flexibility

Work towards a service-oriented architecture

11 CEOS WGISS-21

April 10, 2023

Approach

Standards– Adopt, adapt and only as a last resort, create– Open, inclusive process for adoption– Inclusive not exclusive use of standards

Service Oriented ArchitectureReference: Federal CIO Council, Jan ’06 "Services and Components Based Architectures: A Strategic Guide for Implementing Distributed and Reusable Components and Services in the Federal Government"

12 CEOS WGISS-21

April 10, 2023

Standards

Standard names and terminology Metadata standards

– FGDC and ISO 19115 w/ remote sensing extensions, Format standards

– XML Schemas, Spatial Databases (SQL), data formats (WMO, NetCDF, HDF, etc.)

Open Geospatial Consortium (OGC) standards:– Features, Coverage (data), Geographic Markup

LanguageWeb Services Standards (World Wide Web

Consortium)

13 CEOS WGISS-21

April 10, 2023

Service-Oriented Architecture

Under an SOA, capabilities are built one at a time to create “Web Services”

The fabric of the SOA is built upon standards for:– discovery (e.g. CF, FGDC, OGC, ISO)– transport (e.g. HTTP, FTP, OPeNDAP, Grid)– use (e.g. netCDF, HTML, OGC, etc.)

Can be tightly coupled (SOAP) or loosely coupled (REST)

14 CEOS WGISS-21

April 10, 2023

SOA Concept

15 CEOS WGISS-21

April 10, 2023

Key Development Strategies

Maintain and minimize impact on legacy systems

Evolutionary development through pilot projects

Coordinate activities through “Structural Data Types”– Grids, time-series, moving-sensor multi-dimensional,

profiles, trajectories, geospatial framework, point data and metadata

16 CEOS WGISS-21

April 10, 2023

Project Management

Undersecretary for Atmosphere and Oceans

DMITData Management Integration Team

NOSCNOAA Observing System

Council (NOAA CIO Member)

DMCNOAA Data Management

Committee (CIO Council Co-Chair)

NOAA Goal Theme:

Weather & Water

NOAA Goal Theme:

Ecosystems

NOAA Goal Theme:

Commerce & Transportation

NOAA Goal Theme:

Climate

17 CEOS WGISS-21

April 10, 2023

Future Direction - Priorities FY07 Work with scientists/data system managers to

assess requirements and systems– Develop enterprise architecture and GEO IDE

Implementation Plan– Implement standards process– Active out-reach activities- inclusive (CEOS WGISS)

FY08/09 Incrementally execute work packages– Develop data standards and interoperability

mechanisms, e.g., translators and directory services– Direct, test and evaluate changes being made to

data management systems

FY10/11 Re-evaluate architecture related to new data systems (across NOAA & with national /international partners)

18 CEOS WGISS-21

April 10, 2023

Conclusions

NOAA faces daunting challenges of vastly increasing data volumes and an increasing need for interdisciplinary use of data

NOAA is committed to enhancing access and ensuring data and products of enduring value are preserved for future generations

NOAA has initiated several activities to actively respond to these challenges. GEO-IDE is a key component