1 CLASS Simple NOAA Archive Access Portal SNAAP Eric Kihn and Rob Prentice NGDC CLASS Developers Meeting July 14th, 2008 Simple NOAA Archive Access Portal.

1CLASS –Simple NOAA Archive Access Portal

SNAAP Eric Kihn and Rob Prentice

NGDC CLASS Developers Meeting

July 14th, 2008

Simple NOAA Archive Access Portal

2CLASS –Simple NOAA Archive Access Portal

SNAAP Goals

Integrate diverse systems via a standards based, user focused WS interface

Demonstrate the concept of “fundamental separation” of archive and storage from access

Create a tool that shows the benefit of API’s for users Discover new technology and evaluate cutting edge tools

for handling integration and presentation Document integration of multiple data types and sources

through an single API (time-series, grid, swath, etc..)

3CLASS –Simple NOAA Archive Access Portal

The Data Environment Now

CLASS ADS

Web Interface Subscriptions

Human Machine

DataProviders

Visualization Inventory

Order

NM

MR

/Metadata

NGDC/SPIDR

Web Interface

Web Interface

Metadata

SPIDR WS

SNAAP

Web Interface

Metadata

Data Access

4CLASS –Simple NOAA Archive Access Portal

Three Simple Items Search for appropriate data sets

Discover data by time, type, location, abstract, etc. Support for multiple metadata schema Standards based This is in CLASS 4.3.3 – awaiting policy

Order Data Needs a standard way to describe ordering Ordering extensions + Standard metadata Needs some framework for service integration (i.e. transforms)

Integration Layer We are using OGSA-DAI for the demo but this is not a final choice Should not affect the consumer at any rate.

5CLASS –Simple NOAA Archive Access Portal

So a new provider wouldhave to …

Create a standard metadata record for the data set. (They should be doing this anyway).

Create an Ordering extension document. If they are offering web distribution this is trivial.

Write an adapter to the middle layer, unless they have a supported service (like JDBC, FTP, etc..) in which case adapt an existing one.

6CLASS –Simple NOAA Archive Access Portal

Advantages

This approach mirrors that of the highly successful OGC specification for map services (what, make, retrieve)

It minimizes entry burden for new providers Its simplicity makes it scalable It doesn’t preclude or interfere with on-going activity, just

opens up new integration possibilities.

7CLASS –Simple NOAA Archive Access Portal

Current snapshot of the CLASS API architecture

CLASS

CLASS products

Visualization service

SPIDRESG - INE - IDEAS – ESSE

NCEP , ERA-40 , ...

DB cluster

Inventory service

Order service

WebServices

OPeNDAP

OGSA-DAI for CLASS (Globus Toolkit 4)

Plugin PluginPluginPlugin

NOMADS

NARR

NetCDFNetCDF

NetCDF

THREDDS(OPeNDAP)

WCS(OGC)

SABR

OLS DMPS

Previewimage

Plugin

Inventory

GranuleGranuleSatellite

Granule

CLASS web interfaceCLASS

metadata

REST API for CLASS

DB cluster

OGSA-DAI client toolkitGRID client toolkit

8CLASS –Simple NOAA Archive Access Portal

Data Order Flow

OE Web Form Servlet

XSLT get

Client OGSA-DAI

XML Result

Get DataProcess

Mine

SQLXML

GranuleTime

Series

12

Adapter

Storage

ErrorMessage

Data

9CLASS –Simple NOAA Archive Access Portal

CLASS Ordering Extensions (OE) schema upper level

10CLASS –Simple NOAA Archive Access Portal

CLASS OE schema: parameters and time

11CLASS –Simple NOAA Archive Access Portal

CLASS OE schema: map and ROI

12CLASS –Simple NOAA Archive Access Portal

CLASS OE schema: station map

13CLASS –Simple NOAA Archive Access Portal

Why OGSA-DAI?

Standard tool in the Grid community Includes a “process” for extraction Built in support for asynchronous transactions Includes support for domain login Looked at alternatives like OpenDap, WCS, Custom

applications some documentation forth coming Problem: Its very complex and not well accepted in NOAA

14CLASS –Simple NOAA Archive Access Portal

Visualization screens – 1time series and map server

15CLASS –Simple NOAA Archive Access Portal

Visualization screens – 2NASA World Wind plugins

16CLASS –Simple NOAA Archive Access Portal

MS Excel and NASA World Windinteroperability for trajectory data

Excel with track points

Excel with data along tracksDrag & Drop controls

NASA World Wind

17CLASS –Simple NOAA Archive Access Portal

Asynchronous Ordering

CLASS API Demonstration

18CLASS –Simple NOAA Archive Access Portal

Orbit Inventory List

19CLASS –Simple NOAA Archive Access Portal

User Basket (Order & Check-Status)

20CLASS –Simple NOAA Archive Access Portal

Order System Entities

•Granule List now using ECHO XML in the return•New entity “Data Basket” – handle to collection of order•Demo currently orders all data centrally will need to be adapted

21CLASS –Simple NOAA Archive Access Portal

Order Sequence Diagram

•Data basket ID is the API handle•OGSA-DAI activities handle the extraction and packaging•Status check is currently by basket only (XML return)•Finished granules return a URL handle to the entity

22CLASS –Simple NOAA Archive Access Portal

Future Plans

Plan to deliver two documents A discussion of the OGSA-DAI choice and its

implications A detailed discussion of the use of baskets

Would like to identify a CLASS build and date for OE inclusion This would require some review process (including

outside) and a versioning of OE Plan to include AVHRR in in the asynchronous demo Plan to include non-granule based asynchronus (i.e.

models, transforms) data extractions Enhance the connection with GEO-IDE

23CLASS –Simple NOAA Archive Access Portal

Future plans

Extend the OE documentation to the CLASS development team and provide support

Extensive work on the connector layer Expand proto-type to cover more NOAA and non-NOAA

data types and services and document lessons learned Release the work to the public

http://sourceforge.net/projects/vxoware/ Seek outside review