ECHO For International Partners March 8, 2005 Robin Pfister Yonsook Enloe.

ECHO For International Partners

March 8, 2005

Robin PfisterYonsook Enloe

EDG to WIST Transition Plan

V0 system first went operational in 1994. NASA ESDIS moving to next generation system – ECHO.

EDG client currently provides search and order capabilities to V0 servers and to the ECS servers through the V0 interface

Multiple CEOS partners host V0 servers or a translator/bridge that speaks V0 to/from their agency inventory systems

NASA DAACs currently analyzing whether to support both ECHO and V0 interface or support just the ECHO system. Each DAAC will make an independent decision based on resources. Decisions will come late 2005 or early 2006.

WIST (Warehouse Inventory Search Tool) client which provides comprehensive inventory search for ECHO will start operations in 2005.

US EDG clients will operate through 2006. End date for US EDG clients is Jan 31, 2007.

After US EDG clients are disabled, a final V0 Valids run for international sites who still use EDG will be made but with the US V0 Servers removed. No further maintenance will be performed to update the V0 valids file.

What Is ECHO?

Functionally Registry of spatial, temporal, numeric and textual

metadata Order broker User and provider account service Services registry and broker (future)

Objectives ECHO is a portal to Earth Science data and services. It allows providers of data to share their metadata

and offload some of their search responsibilities. It also brokers orders from clients to the appropriate

providers, providing tracking services for both the client and the provider.

ECHO presents a messaging interface based on XML, but does not currently provide a search GUI.

System Drivers

Present an API for organizations to connect their own user interfaces and programs to

Make it easy for providers of Earth Science data and services to participate in the system

Provide searches that respond quickly Broker orders for both data and services Minimize operational costs Build upon advances in industry and use e-

commerce systems as a model Build a system that can be scaled up to handle

large numbers of requests

Architecture Overview:

Context

Community Tailored View of ECHO holdings

Application

Metadata and Service Registries

Service

Service with a UI

Data Inventory

KEY:

Community

Community

Non-ECHO

Registry

ECHO-based

Registry

ECHO

Registry XML Messages

XML Messages

XML Message via Adaptors

Anyone can participate

From any level

Provider-set access controls

Approach: Layered Architecture

BMGT Proxy

Ingest (XML via FTP)

Full RecordInsert, Update, Delete

Partial RecordInsert, Update, Delete

Clie

ntC

lient

OG

CC

lient

Clie

ntC

lient

OG

CA

dapt

erS

OA

PA

dapt

er

Clie

nt In

terf

ace

(RM

I)(E

CH

O A

PI S

ervi

ces)

Par

tner

Mgm

tC

atal

ogS

earc

hP

lace

Ord

erU

ser

Acc

tsA

dmin

AP

IC

onfig

Info

ECHO Kernel

Business Objects

Browse ImageRepository

Data

Registry

Service

RegistryInteroperable Quote

Request

Cancel

Request

Subm

itR

equest

Partner Interface (S

OA

P)

(Quote, S

ubmit, C

ancel)

V0 G

W P

roxyO

RN

L Proxy

Approach: Interoperable Registries

Service RegistryService offerings leveraging Web Service Standards

Publish Capability Advertisements Service Interfaces Service Implementations Service GUIs Linkage to Data Registry

Discovery Capability UDDI Standard Based Data Registry View

• What services are appropriate for this collection/granules?

Future Service Brokering (ECHO 8.0) Orchestration of Service Chains Access Controls

Data RegistryRepresenting data resources through metadata

Publish Capability• Supports Collection, Granules and

Browse publication• Product Specific Attributes extend

the standard data model Discovery Capability

• Collection/Dataset• Granule/Inventory• Based on Z39.50 Standard• Full Results Management

Access Capability• Online Access• Legacy Order Mechanisms• Access Controls

• Visibility of data resources• Access to data resources

Metadata Subscriptions

In the future will enable orchestration of Data and Service brokering and chaining

Partner Status

Data Partners: 5 Operational Data Providers 5 Development/Test Nearly 42 Million data granules (including those in

ECS’ Data Pools) and over 7 Million Browse Images available from ASF, GSFC-ECS, LPDAAC-ECS, NSIDC-ECS, ORNL DAAC, SEDAC, Stennis (other providers in test or planning).

ECHO holdings report available from the echo website (eos.nasa.gov/echo), click on “status”, click on “operations reporting”.

Client Partners: 3 Operational 11 Development/Test

ECHO Client Partners

Operational Mercury EOS – ORNL client Power User Interface – script for bulk ordering SIMECC (Simple MODIS ECHO Client) – MODIS data search and order

In Test WIST – EOS Data Gateway using ECHO ASF ECHO Client - client built by the Alaska SAR Facility WISRD (Web Interface for Searching, Subsetting, Stitching, Resampling,

Regriding, and Reformating Data) – NSIDC Client to search for swath, scene, and gridded data

MODIS Website – client via existing MODIS website and ORNL’s shopping cart Data Validation User Interface – client for MODIS Land Data Validation Team

Under development Annoterra Version2 – link between EOS newsfeeds, GCMD, and ECHO

granules SNOWI-E – NSIDC Client NEO Invasive Species Data Service

There are plans for other clients (MODIS rapid response, OGC client)

Availability

ECHO uses a cluster and load balancing approach to address availability

We are in the process of maturing to a completely redundant system planned by the end of 2005

The system is monitored by automated systems 24X7, with pager notification of current, and in some cases pending, faults

ECHO Website Tour

http://eos.nasa.gov/echo

ECHO Data Partner Participation

International Participation in ECHO

Active data provider Inactive data provider by allowing ECHO to

harvest metadata and store it in the ECHO Registry (ECHO needs to complete development of a harvester)

Become a distributed search provider where ECHO issues a distributed search to the external system (ECHO future plans): Distributed search to a separate ECHO

Registry Distributed search to a separate non-ECHO

system

Active Data Provider Options

Download, validate, and correct metadata periodically to ECHO

Search & Order options : Keep own agency data system unchanged

and do not provide access to NASA data Or keep own agency data system and

provide access to NASA data through ECHO Client APIs.

Or discontinue agency data system for user access but develop own agency client to search desired subset of ECHO metadata & services, using ECHO Client APIs and some reusable software and software libraries

Inactive Data Provider Options

ECHO project planning on developing metadata harvester in the near future (1 to 2 years?)

Data Provider provides information for ECHO harvester

ECHO harvests metadata periodically Search & Order Options :

Data provider keeps own agency system If access to NASA data is desired, agency

develops capability of agency client to access ECHO through established ECHO Client APIs.

ECHO Distributed Search Data Provider

Use open source ECHO software to install and populate ECHO Registry at agency. The content of the agency ECHO Registry will be distinct from the content of the NASA ECHO Registry

Become ECHO distributed search data provider using non-ECHO system

Will need to agree on search distribution interface and common metadata search & present attributes with ECHO team. Can be bi-directional interface.

NASA ECHO will also respond to search requests through already established ECHO Client APIs.

Can CEOS international community desiring this option agree on one comment interface?

Future ICS Happenings with ECHO

ECHO data model mapping to CIP and to ISO 19115

ECHO Client APIs comparison to CIP and to OGC Catalog 2.0 (general model & http profile)

The details of these analyses will be discussed at future ICS telecons. We promise to focus the ECHO discussion at the ICS telecons by breaking up topics into manageable chunks. No marathon telecons!

Schedule bi-lateral telecons with agency partners desiring ECHO information

Provider Account

New ECHO Data Partners fill out an application to become a Data Partner

Data Partners “Getting Started” on the website, http://eos.nasa.gov/echo

ECHO Operations (Ops) at [email protected]

An ECHO Provider Account is created Can associate multiple ECHO Registered User Accounts to

manage the Provider Account Maintains a list of named contacts (i.e. Order Manager,

User Services, etc.) Providers can establish lists of users and give them

privileged access to provider metadata and ordering privileges

Next Steps: ECHO Development

Highly-available, redundant back-up facility Service Brokering Asynchronous Searching Reference client Event Subscriptions Orchestration/Service Chaining Interagency and International Interoperability

ECHO Operations

The ECHO Operations Team (ECHO Ops) is the point of contact for direct interaction between ECHO, its Partners, and end users

ECHO Ops is responsible for the operation and maintenance of the ECHO operational and partner test systems

ECHO Ops general support for Partners includes: Assisting Partners in understanding the

ECHO API and DTDs Ingest management and accounting Problem tracking and resolution Advertising Data Partner holdings and

availability of new datasets Promoting and engaging new Client Partners

Support:Tools, Training, Documentation

ECHO Ops Staff Availability: Sysadmin 24X7, Ops Staff 11X5 [email protected]

ECHO website http://eos.nasa.gov/echo User’s Guide, Holdings Summary, Upcoming

Functionality, APIs and DTDs, Training Materials organized by partner type

ECHO mailing lists echo-all, echo-status, echo-client, echo-data

ECHO Technical Committee (ETC) Meetings All partners can participate in weekly telecons to

share and discuss approaches, issues, solutions. Contact ECHO Ops for dial-in information