GES DISC Services

GES DISC ServicesPush Harder?Be Careful?Change Direction?What about adding ______?

Discovery Services Mirador

Development scaled back to sustaining engineering level

External Search (in Test mode TS1) Technically successful, but... Usability-challenged

Start and stop date/time Total number of hits Uniform sort order Duplicates

Usability: Simplicity vs. Features (esp. Services) Mirador Usability Sounding Board?

mail list for queries on usability quandaries

Data Services

Number of Users* - March 2011

NativeGiovanniOGCnetCDFSubsettingOPeNDAPDQSS

*OK, not really. It’s the number of distinct IP addresses

Number of Users*: Sep 2010 – Apr 2011

201

201

201

201

201

201

201

201

0100020003000400050006000700080009000

DQSSGiovanninetCDFSubsettingOPeNDAPOGCNative

Data Quality Screening Service

The quality of AIRS data varies considerably

AIRS Parameter Best (%)

Good (%)

Do Not Use (%)

Total Precipitable Water

38 38 24

Carbon Monoxide 64 7 29Surface Temperature

5 44 51Version 5 Level 2 Standard Retrieval Statistics

Quality Schemes can be complicated

Hurricane Ike, viewed by the Atmospheric Infrared Sounder (AIRS)

PBest : Maximum pressure for which

quality value is “Best” in temperature profiles

Air Temperatureat 300 mbar

Current user scenarios...

Nominal scenario Search for and download data Locate documentation on handling quality Read & understand documentation on

quality Write custom routine to filter out bad pixels

Equally likely scenario (especially in user communities not familiar with satellite data) Search for and download data Assume that quality has a negligible effect

Repeat for

each user

The effect of bad qualitydata is often not

negligible

Total Column Precipitable

WaterQuality

Best Good Do Not Usekg/m2

Hurricane Ike, 9/10/2008

DQSS replaces bad-quality pixels with fill values

Mask based on user criteria(Quality level

< 2)

Good quality data pixels

retained

Output file has the same format and structure as the input file (except for extra mask and original_data fields)

Original data array(Total column precipitable water)

DQSS Demo

DQSS Status + Plans

Operational for AIRS L2 Standard Retrieval Nearly operational for MODIS Water Vapor Next: MODIS Aerosols, MLS Water Vapor Next: ??? Also, OPeNDAP Gateway nearly reader to

front-end DQSS Allow OPeNDAP access to DQSS-served data.

OPeNDAP*

Remote access to data: no need to download Access at fine granularity

Variable Array regions Stride

Present HDF data as netCDF/CF Enhances Tool Usability

Reformatting: ASCII, netCDF

*OPeNDAP = OpenSource Project for a Network Data Access Protocol

Who Uses OPeNDAP?

Industrial-strength scripters looking for subsets

Thick client users GrADS, Panoply, IDV, McIDAS-V, Ferret

Internal Systems Giovanni MapServer Simple Subset Wizard

OPeNDAP Demo

OGC* Standards - WMS

Web Map Service (WMS) URL request: returns map image Implemented with open-source MapServer

Giovanni also supports WMS Consumers:

AIRS NRT page Google Earth GIS programs IDV Giovanni

*OGC = Open Geospatial Consortium

OGC - WCS

Returns “coverages”: data variables in NetCDF/CF1

Used by other systems DataFed Giovanni Atmospheric Composition Portal Simple Subset Wizard

Subsetting

Semi-custom tools for some products Reuse HSE libraries from UAH Reuse Lats4D from A. DaSilva

Usually HDF in -> HDF out Implemented as REST* URLs

Subsetting at time of download Subsets are implemented as user requests come in Areas where we should proactively develop

subsetters?

~100 Subsettable Datasets

AIRS Radiances (channel), L2 Retrievals (variable), L3 (spatial+variable via SSW)

MLS L2 (spatial+variable) TOMS L3, OMI L2-L3 (spatial+variable), OMI L2 TRMM L3 (spatial+variable) Models (spatial+variable) Did we miss any (that shouldn’t be missed)?

Should all SSW subsets be offered in Mirador?

Format Conversion

Custom code for some L3 and L2 datasets HDF -> netCDF/CF Improves usability in tools Moving toward external tools where possible

OPeNDAP Lats4d: based on GrADS

Simple Subset Wizard

Desired: “Just give me the data from time 1 to time 2 for this spatial box”.

Current: “search for granules, view granules, select granules, select subset option, re-enter spatial box...”

ESDIS-funded technology infusion effort DEMO

Giovanni Evolution

G3 Evolution to Agile Giovanni (G4)

Factors driving evolution G3 architecture was never completed

No workflow engine Cost of adding significant features is too high

Architecture is too brittle

Key G4 Goals

Reduce cost and time to add new features Improve performance over G3 Support external maintenance of external

data

Evolution Plan

Implement new projects in Agile Giovanni (G4) Aerostat ACCESS project

Point data in database, bias corrections Year of Tropical Convection (YOTC)

Level 2 data Community-based Giovanni

Externally maintained portals and data

Implement G4 features to meet existing G3 functionality

Migrate G3 instances to G4 portals

Roads Not Taken

Giovanni 3 enhancements

ISO 19115 Metadata Document

architecture Mirador features and

usability revamp Persistent locators Unique identifiers

Not Giovanni Evolution DQSS Atmospheric Composition

Portal Simple Subset Wizard Community-based

Initiatives Mirador External Search Expanding data services

Taken

Backup Slides

Agile Giovanni Architectural Features

Model-view-controller Semantic Web underpinnings Variable-centric, not dataset-centric Code reuse: Kepler, YUI, JCache, MapServer