Data Grid: GRASP

Data Grid: GRASP

Mike Smorul<[email protected]>

Grid Retrieval and Search Platform

Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the UMIACS GLCF.

Provides a graphical interface into data grid holdings.

Access to entire GLCF holdings through the Storage Resource Broker(SRB)

Earth Science Data Interface

ESDI Overview

Designed to allow for intuitive browsing and searching of large geospatial data sets.

Tightly integrated set of web, ftp, and file servers. (customized to GLCF)

Distributes over 7Tb of data per month Over 27,000 Landsat scenes and 13Tb of

data available for download

SRB Grid Testbed (original)

Modified the SRB to hold spatial data Contributed Informix port of the SRB (v3.2) Linked three ESIP sites Tested replication between

GMU and UMD. Remote registration at

UNH

UMD MCAT enabled srbmaster

GMU srbmaster UNH srbmaster

Informix

MODISTM

MSS(1)MSS(2)

UMD srbmasterwith dfs access

Lessons Learned

The SRB can easily handle textual metadata.

Spatial data could be stored into extended Informix attributes, but querying was available only through the DAILimited SRB MCAT to Informix based

systems

GRASP Architecture

I/O Abstraction Layer

Data Grid

Clients

Data download

Query Abstraction

Browse / Display

Spatial Information

Textual Information

Data Discovery

GRASP Architecture

GRASP uses a data grid as an abstract storage repository.

Metadata in the grid is mined from the grid itself or from external sources and published into a browsable form.Data grids may allow for platform independent

metadata, but may not be optimal for access

GRASP Screenshot

Grid Holdings

Registered GLCF holdingsOver 338,000 registered files4.4Tb total sizeGranular permissions on registered holdings

No need to move all data into grid, registered pre-existing holdings in place.

Current data grid

Designed using newer SRB software that allows for Federated grid model.

Created using standard SRB software configuration (postgreSQL)

Large data sites can maintain independent MCATs Administratively independent Ability to customize data grid to site security/data

requirements while maintaining compatibility across federation.

Smaller peers can register as clients of UMD

Data grid overview

UMD MCAT

SRB Master

DFS Gateway

SRB Master

Local Peer

Remote MCAT

SRB Master SRB Master

SRB Master

Zone: esip-umiacsZone: esip-remote

GRASP Interface

Growing data grid

Additional MCATs can be federated Additional SRB aware clients can be

added Remote data supplying sites can

contribute data sets from their resources Sites needing quick local access to data

can replicate to local SRB setup.

Questions?