1 Web Services at NOAAs National Climatic Data Center Steve Ansari, Rich Baldwin, Stephen Del Greco, Neal Lott, Glenn Rutledge NOAAs National Climatic.

1

Web Services at NOAA’s National Climatic Data Center

Steve Ansari, Rich Baldwin, Stephen Del Greco, Neal Lott, Glenn Rutledge NOAA’s National Climatic Data Center

AGU Fall Meeting 2007

2

Background

• National Climatic Data Center (NCDC) currently archives over 1.5 petabytes of data.

• Sources include in-situ networks, numerical models, radar, satellite and more.

• Manual access through web pages

3

Implementing SOA

Different approaches for different datasets

• In-situ data:

REST, WaterML, OGC GIS Services

• Severe Weather Data Inventory: REST

• Model/Radar/Satellite:

REST via THREDDS Data Server

4

In-Situ Data

• Network examples:

NCDC-ISD, NCDC-COOP, NCDC-15Precip, NCDC-ASOS• Metadata (station locations) in ArcSDE / Oracle Spatial• Time-series data stored in Oracle

• Metadata: OGC GIS Web Services (WMS and WFS)

http://gis.ncdc.noaa.gov/

• Data: Implementation of CUAHSI Web Services

Web Services will be available soon!

http://cdo.ncdc.noaa.gov/ - interactive web pages

5

In-Situ Data

• Consortium of Universities for Advancement of Hydrologic Sciences (CUAHSI)

• Partners with state and federal agencies

• Defines WaterML – a specialized schema definition for water resources

6

In-Situ Data

• REST Web Services– Simple access to archive data in early 2008 – Provides access to data at a station or within a

country, state, county, climate division, watershed– Easily extendable– Uses common names or FIPS identifiers– Authentication token– http…/servlet/network/type/location/variable/

startdate/enddate/output

• Output types (waterML, xml, text, csv)

7

In-Situ Data

8

Severe Weather Data Inventory

• Geospatial database of weather phenomena represented as point, line or polygon

Current layers: Prelim. Local Storm Reports, NEXRAD Level-III Storm Attributes (TVS, Meso, Hail, Structure)

• REST Web Service Access:

Search based on location, data range, within area, etc…

Statistics – count, summarize by hour, day, month

Output types (csv, xml, kmz, Shapefile, GeoRSS)

• Basis for SWDI interactive web pages / applications

http://www.ncdc.noaa.gov/swdi

9

Severe Weather Data Inventory

Example SWDI REST URLs:

(Almost a Virtual File System – hopefully intuitive!)

http://www.ncdc.noaa.gov/swdiws1. /kmz/plsr/20071101:20071115

2. /csv/nx3tvs/20070501:20070601?stat=count

3. /xml/nx3tvs/20070501:20070601

?radius=5.0&center=-96.0,35.0

4. /shp/nx3hail/20070501:20070601?state=ca

10

Severe Weather Data Inventory

11

Severe Weather Data Inventory

12

Severe Weather Data Inventory

13

Model / Radar / Satellite

• The NOMADS project• Utilize NetCDF and Common Data Model for

interoperability, decoders, etc…• THREDDS Data Server – access to data via OPeNDAP,

NetCDF Server and OGC Web Coverage Service (WCS)• GrADS Data Server and the Live Access Server

Current datasets in THREDDS:

NARR, RUC, GFS, NAM, NDFD, Stage-IV NEXRAD MPE, HIRS, VTPR, SST

14

• Foster research within the geo-sciencecommunities (ocean, weather, and climate)to study multiple earth systems using collections of distributed data

• Promote model evaluation and product development

• Develop institutional partnerships via distributed open technologies

• Provide distributed access to models and associated data

NOMADS GoalsNOMADS Goals

The NOAA Operational Model Archive and Distribution System

15

Jan-Nov 2007

NOMADS StatsNOMADS Stats

The NOAA Operational Model Archive and Distribution System

Full File Downloads Subset Downloads

17,161,778 downloads 9,083,422 downloads

46.46 TB 0.85 TB

2.84 MB / download 100.5 KB / download

16

Goals

• Ease of use

• Interoperability with existing tools

• Flexible, distributed architecture

• Data stewardship to multiple user communities

• Efficient use of hardware and bandwidth

17

Data Usage - A Few Samples

• Engineering design:  Ice loads for towers, cables, wires, etc Wind loads for buildings, etc Heating/cooling requirements Drainage/runoff extremes (pipes, culverts)

• Aircraft operations: Crosswinds (runway design), instrument landing systems, etc

• Space shuttle accident investigation: Nexrad data• Ship routing and oil rig placement• Global re-analyses for climate trends assessment, etc• HAZMAT operations and studies: oil spills, toxic release, etc• Weather risk management industry (now > $40 billion per year)• Insurance investigations and verification• Court cases and criminal investigations• Aircraft accident investigations• Wind energy studies: Wind farms, US and overseas• Commercial innovation and design: Typical and extreme

conditions for a new market

18

Contact Info

Steve Ansari – [email protected]

NCDC Web Services website

http://www.ncdc.noaa.gov/ws