1 CLASS Simple NOAA Archive Access Portal SNAAP Eric Kihn and Rob Prentice NGDC CLASS Developers Meeting July 14th, 2008 Simple NOAA Archive Access Portal.

1CLASS –Simple NOAA Archive Access Portal

SNAAP Eric Kihn and Rob Prentice

NGDC CLASS Developers Meeting

July 14th, 2008

Simple NOAA Archive Access Portal


SNAAP Goals

Integrate diverse systems via a standards based, user focused WS interface

Demonstrate the concept of “fundamental separation” of archive and storage from access

Create a tool that shows the benefit of API’s for users Discover new technology and evaluate cutting edge tools

for handling integration and presentation Document integration of multiple data types and sources

through an single API (time-series, grid, swath, etc..)


The Data Environment Now

CLASS ADS

Web Interface Subscriptions

Human Machine

DataProviders

Visualization Inventory

Order

NM

MR

/Metadata

NGDC/SPIDR

Web Interface

Web Interface

Metadata

SPIDR WS

SNAAP

Web Interface

Metadata

Data Access


Three Simple Items Search for appropriate data sets

Discover data by time, type, location, abstract, etc. Support for multiple metadata schema Standards based This is in CLASS 4.3.3 – awaiting policy

Order Data Needs a standard way to describe ordering Ordering extensions + Standard metadata Needs some framework for service integration (i.e. transforms)

Integration Layer We are using OGSA-DAI for the demo but this is not a final choice Should not affect the consumer at any rate.


So a new provider wouldhave to …

Create a standard metadata record for the data set. (They should be doing this anyway).

Create an Ordering extension document. If they are offering web distribution this is trivial.

Write an adapter to the middle layer, unless they have a supported service (like JDBC, FTP, etc..) in which case adapt an existing one.


Advantages

This approach mirrors that of the highly successful OGC specification for map services (what, make, retrieve)

It minimizes entry burden for new providers Its simplicity makes it scalable It doesn’t preclude or interfere with on-going activity, just

opens up new integration possibilities.


Current snapshot of the CLASS API architecture

CLASS

CLASS products

Visualization service

SPIDRESG - INE - IDEAS – ESSE

NCEP , ERA-40 , ...

DB cluster

Inventory service

Order service

WebServices

OPeNDAP

OGSA-DAI for CLASS (Globus Toolkit 4)

Plugin PluginPluginPlugin

NOMADS

NARR

NetCDFNetCDF

NetCDF

THREDDS(OPeNDAP)

WCS(OGC)

SABR

OLS DMPS

Previewimage

Plugin

Inventory

GranuleGranuleSatellite

Granule

CLASS web interfaceCLASS

metadata

REST API for CLASS

DB cluster

OGSA-DAI client toolkitGRID client toolkit


Data Order Flow

OE Web Form Servlet

XSLT get

Client OGSA-DAI

XML Result

Get DataProcess

Mine

SQLXML

GranuleTime

Series

12

Adapter

Storage

ErrorMessage

Data


CLASS Ordering Extensions (OE) schema upper level


CLASS OE schema: parameters and time


CLASS OE schema: map and ROI


CLASS OE schema: station map


Why OGSA-DAI?

Standard tool in the Grid community Includes a “process” for extraction Built in support for asynchronous transactions Includes support for domain login Looked at alternatives like OpenDap, WCS, Custom

applications some documentation forth coming Problem: Its very complex and not well accepted in NOAA


Visualization screens – 1time series and map server


Visualization screens – 2NASA World Wind plugins


MS Excel and NASA World Windinteroperability for trajectory data

Excel with track points

Excel with data along tracksDrag & Drop controls

NASA World Wind


Asynchronous Ordering

CLASS API Demonstration


Orbit Inventory List


User Basket (Order & Check-Status)


Order System Entities

•Granule List now using ECHO XML in the return•New entity “Data Basket” – handle to collection of order•Demo currently orders all data centrally will need to be adapted


Order Sequence Diagram

•Data basket ID is the API handle•OGSA-DAI activities handle the extraction and packaging•Status check is currently by basket only (XML return)•Finished granules return a URL handle to the entity


Future Plans

Plan to deliver two documents A discussion of the OGSA-DAI choice and its

implications A detailed discussion of the use of baskets

Would like to identify a CLASS build and date for OE inclusion This would require some review process (including

outside) and a versioning of OE Plan to include AVHRR in in the asynchronous demo Plan to include non-granule based asynchronus (i.e.

models, transforms) data extractions Enhance the connection with GEO-IDE


Future plans

Extend the OE documentation to the CLASS development team and provide support

Extensive work on the connector layer Expand proto-type to cover more NOAA and non-NOAA

data types and services and document lessons learned Release the work to the public

http://sourceforge.net/projects/vxoware/ Seek outside review

http://sourceforge.net/projects/vxoware/

1 CLASS Simple NOAA Archive Access Portal SNAAP Eric Kihn and Rob Prentice NGDC CLASS Developers Meeting July 14th, 2008 Simple NOAA Archive Access Portal.

Documents