Page 1 LAITS Laboratory for Advanced Information Technology and Standards 9/6/04 Progresses on GMU Grid Project Laboratory for Advanced Information Technology.

Page 1

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Progresses on GMU Grid Project

Laboratory for Advanced Information Technology and Standards (LAITS)George Mason University (GMU)

[email protected]://laits.gmu.edu

Presented to 19th CEOS WGISS meeting

March 8, 2005

Liping Di

Page 2

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Content of the presentation

• Introduction

• Status of Software Development and Integration.– Data and service catalogs

• Grid-based catalog service for web (GWRS)

– Metadata– Services– BPEL engine– Web service client

• Status of Testbed deployment– Development environment at GMU/LAITS– Creditable testbed to be deployed.– Data at the testbed– Services available at the testbed.

Page 3

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Introduction

• GMU’s OGC Grid project is to provide Geospatial web services, which are compliant with OGC specifications, at the Grid environment– Enable interoperable data access to large data archives.– Provide standard-based chainable geospatial analysis services– Provide geospatial modeling and virtual data product generation.

• Partners in the project– NASA Ames Research Center (NASA Information Power Grid).– DOE Lawrence Livermore National Lab (LLNL) (Earth System Grid).

• This project is also a part of CEOS Grid testbed.• Funded by NASA Earth Science Technology Office (ESTO)

– Three-year project from May 2003-April 2006• This presentation gives an update on recent progress of this

project since last WGISS meeting in Beijing in September 2004.

Page 4

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Software Development--The Grid-enable OGC services

• The following Grid-enabled OGC services have been developed and deployed:

– WCS– Compliant with WCS 1.0– CS/W – Compliant with CAT 2.0 Web profile.– WMS – Compliant with WMS 1.1.1 --– Services work at both Web and Grid environment.

• The services are wrapped as a Grid catalog service from the original web services.

• Many Web Geospatial Processing services are developed and are being wrapped into Grid.

– WCTS – Image coordinate transformation services.• Support the re-projection of imagery data in all GCTP supported projections.• GMU was one of major contributors of WCTS IPR.

– WICS – Image Classification Service• Provide unsupervised classification services• LAITS was the editor of WICS IPR.

– IMCS -- Web Image Cutting Service• Cutting image based on provided boundary

Page 5

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Software DevelopmentGRASS Web/Grid Services

• GRASS is a open-source GIS package. It contains more than 200 geospatial processing/analysis functions

• We are wrapping GRASS functions into Chainable Web Services.– Provide WSDL descriptions– Enable SOAP binding.

• The Grass Web Services will be wrapped into Grid services.

Page 6

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

BPELPower

• LAITS has developed a BPEL management and execution engine called BPELPower.– Based on Open Source software such as BPWS4J, WSIF,

WSDL4J, AXIS and SOAP which only provide the function library.

– Running on the top of application servers, such as Tomcat, J2EE, JBoss, Weblogic, and WebSphere.

• Integrates services into collaborative and transactional business processes.

• Supports the various invocation of web services, e.g., EJB, JMS, JCA, SOAP, MIME, HTTP GET / POST and Java classes.

• BPELPower has been tested in many different situations and is reliable.

Page 7

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Web Service Client

• Our Grid system will provide OGC Interfaces.– OGC client will be able to access the Grid the same as OGC

Web services.

• Our OGC compliant client--Multi-protocol Geospatial Client (MPGC) has been updated to support– OWS 2 WCS enhancement (WCS 1.0 & WCS 1.0.2.0)– Support the CSW specifications– Dynamic construction of geospatial process models (Service

chaining and virtual data generation)

• Cleaning up the code & bug fixing for alpha release.

Page 8

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Software Development Currently Underway

• Geospatial Process Model Representation

• Logical and Physical Instantiation Services for creating executable workflow from geospatial process model and user inputs.

• Auto-construction of executable workflows by decomposing user requests.

Page 9

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Security and VO for the Testbed

GMU (Solaris) (laits.gmu.edu)Globus 3.2, 3.9 with CEOS Certs.

GMU (Linux)(llinux.laits.gmu.edu)Globus 2.2 with Laits

Certs.

GMU CA center

Ames ipg05 (Linux)(ipg05.ipg.nasa.gov)Globus 3.0 with IPG

Certs.GMU LAITS VO NASA IPG VO

GMU (Linux)(data.laits.gmu.edu)

Globus 3.2, 3.9 with Laits Certs.

GMU (Mac)(geobrain.laits.gmu.edu)

Globus 3.2

IPG CA center

NASA SGT (Linux)(arao2.sgt-inc.com)Globus 3.2 with CEOS

Certs.

NASA (Linux)(former.intl-interfaces.net)Globus 3.0 with CEOS Certs.

CEOS VO

Authentication among different VO

Page 10

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

GMU GeoBrain Server Cluster

• A)    One Apple 64-bit Xserver G5 head node with dual 2.0GHz G5 processors, 4GB of DDR SDRAM and 2 * 250GB hard drive.

• B)     Five Apple 64-bit Xserver G5 cluster nodes, two with dual 2.0GHz G5 processors, and other three with 2.3 GHz G5 processors. Each node has 2GB of DDR DSRAM and an 80GB hard drive.

• C)    Five Xserver RAID systems with total capacity of 21.7TB.• D)    2Gbps Fiber Switch for exchanging data between cluster nodes

and the RAID system.• E)     8 Port Gigabit Ethernet Switch for system administration online.• F)     3000VA UPS for power protection and supply.• G)    25U Enclosure for racking cluster nodes and RAID system.• H)    PowerBook G4 for system administration.• Network: 1Gb/s Internet II Abilene Network, 100 Mb/s Internet I

connection.• Total system: 12 G5 CPUs, 14 Gb memory, 22.6 Tb disk.

Page 11

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Software On the GeoBrain Cluster

Pre-installed Apple Mac OS X server V10.3 Unlimited-Client and 10-Client on head node and cluster nodes.

Pre-installed Apple Mac OS X Panther V10.3 on PowerBook G4.

Installed Apple Remote Desktop 2 on cluster nodes and PowerBook.

Installed Apple Xcode Tools. Installed other GNU tools. Installed HDF, HDF-EOS tools and development library OGC Web Services (CS/W, WMS, WCS, WCTS, WICS). More

services will be added Installed Globus 3.2.1 from the source code. Globus 3.9.3

installed and is under test. Grid-enabled OGC services: GCSW, GWCS, GWMS are

available. More Grid-enabled services will be included.

Page 12

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Data in GeoBrain Server

Name Year Space Description

MSS 1975 North East Europe, West Russia

304 scenes, 4 bands in each scene, total size is 19.22GB

TM 1990 Global 8473 scenes, 7 bands in each scene, total size is 4.07TB

ETM+ 2000 Global 7935 scenes, 9 bands in each scene, total size is 13.52TB

• Landsat Data

• 200 GB of sample HDF-EOS data.

•Global 90 meter DEM data to be added soon.

•All of those data have been cataloged in CS/W

Page 13

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Access to EOSDIS data for Virtual Product Generation

• User requests a virtual product which may request inputs from EOSDIS data storages.

• Working on provide live and transparent access to EOSDIS data storages for the virtual product generation.– Using ECHO to find the location of needed data– On-line access to EOS data pools. First one to be the pool at

Goddard DAAC.– On-line access to NASA RDS data through GridFTP.– Machine to Machine gateway to access near-real-time data

at NASA DAACs.

Page 14

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Services at the GeoBrain Server Cluster

• OGC-compliant data access and process services will be available at the system– Data access: WCS, WMS, CS/W-available now– Process services: WCTS, WICS, Reformating,…

• Porting and testing in progress.

• All services are available both to CEOS Grid testbed as Grid services and as general web services.

• Virtual geospatial products will be provided through geospatial modeling and workflow executions– All services available at the system are chainable and can be bond

with OGC-compliant data sources dynamically.

• Currently data services is available to all Internet users in the form of web services and to all users in the Grid virtual organization through Grid services.– The services are in final testing stage and will be openly available

to user community at the end of March.

Page 15

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Some problems in WCS implementation

• The current way of using XML capability for data cataloging is not scalable for large data archives– The capability XML document for Landsat scenes is 30 Mb

in size. It is not feasible to use the getCapabilities interfaces.– Currently we use CS/W to search the data catalogs and then

returns the capability XML for only those data that meets user’s requirement.

– This approach requires an integrated client that talks both CS/W and WCS protocols. Our MPGC can do that. However, it requires users to download and install the client.

• We are developing a web portal that allows users to directly access the data and services without to install the client.

Page 16

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

URLs for GeoBrain Services

• WCS- no XML capability document• WCS 1.0.20 – http://geobrain.laits.gmu.edu/cgi-bin/Landsat/WCS1020• WCS 1.0.0 – http://geobrain.laits.gmu.edu/cgi-bin/Landsat/WCS100

• WMS 1.1.1 -- http://geobrain.laits.gmu.edu/Landsat/cgi-bin/WMS111• WCTS --- http://data.laits.gmu.edu/cgi-bin/WCTS/ • WICS -- http://data.laits.gmu.edu:8099/wics/classifiers/ • IMCS -- http://data.laits.gmu.edu/cgi-bin/IMCS/ • GRASS Web Service:

http://geobrain.laits.gmu.edu:8099/webservices/grass/Grass_Webservices.html

• CS/W– Description, sample of uses:

http://geobrain.laits.gmu.edu/CSW/discovery/

– Data catalog search:

http://geobrain.laits.gmu.edu:8099/CSWDiscovery/Query

• BPELPower: http://bpel.laits.gmu.edu

• General information and software downloading– http://laits.gmu.edu

Page 17

LAITS

Laboratory for Advanced Information Technology and Standards

9/6/04

Further Work on Landsat data access

• Obtain data based on geographic name (I.e., counties or states)– The cutting service will cut the data into the shape described

by the boundary G-ring of the counties or the states.– Boundary data is from U.S. Census Bureau.

• Mosaic service– Current GeoBrain does not have the mosaic services.– If a request region falls into more than one Landsat scenes,

parts of the scenes that covered by the request region are returned, results in multiple files.

– A mosaic service, as an option, will merge the parts together to form a single image to cover the request region.

– This effectively provides a single Landsat image that covers of whole globe.