Integrating CyberGIS Gateway with Windows Azure: A Case ... · Windows Azure Cloud platform provides computation and hosting services for general purposes PaaS ! ... – Exploits

1

Babak Behzad1,2, Anand Padmanabhan1,3, Yong Liu3, Yan Liu1,3, Shaowen Wang1,3

1CyberInfrastructure and Geospatial Information Laboratory (CIGI) Department of Geography

School of Earth, Society, and Environment 2Department of Computer Science

3National Center for Supercomputing Applications (NCSA) University of Illinois at Urbana-Champaign

HPDGIS’11 Workshop - November 1, 2011

Integrating CyberGIS Gateway with Windows Azure: A Case Study on MODFLOW Groundwater Simulation

Motivations

n  Data Deluge –  The volume of spatial data across numerous domains has

dramatically increased –  The trend likely continues unabated in future

n  Different computing modalities needed to infer knowledge from massive amounts of spatial data –  Researchers and users need to be empowered to manage

and analyze vast quantity of spatial data in an efficient and timely fashion

2

Why Cloud Computing?

n  Cloud computing aims to address on-demand elastic computing needs

n  For geospatial applications, the integration with Cloud computing environments adds new analysis and modeling capabilities –  Example: We are interested to support analyses and models

that are data-intensive and require large ensemble runs of applications that also desire fast turnaround

3

Why Windows Azure?

n  Large number of existing spatial analyses and models are developed in Microsoft Windows environments. –  Cannot be easily ported to other operating

systems

n  Provides a typical cloud computing model (Platform-as-a-service (PaaS)) –  Experiences and interfaces developed can be

leveraged for other cloud computing integration platforms

4

Windows Azure Background

n  Windows Azure Cloud platform provides computation and hosting services for general purposes PaaS

n  Azure applications have different kinds of roles: –  Web roles –  Worker roles

n  The storage services provided by Azure are categorized as follows: –  Blob: For storing file objects and raw data –  Table: For semi-structured data –  Queue: For the messages in distributed applications

5

CyberGIS Gateway – A Primer

n  The CyberGIS Gateway is a Web-based collaborative high-performance and distributed GIS software environment

n  Enables a large number of users to simultaneously work on computationally intensive spatial analysis and modeling based on friendly access to cyberinfrastructure resources

n  Current cyberinfrastructure resources: –  National Science Foundation (NSF) XSEDE –  Open Science Grid (OSG)

6

CyberGIS Architecture

7

n  Hybrid Cyberinfrastructure

n  Geospatial Middleware

n  User Environment

Integrating Cloud Computing into CyberGIS – Our Approach

n  In order to add Cloud computing platforms as a new resource to CyberGIS Gateway we need: –  Data transfer: We developed a scalable mechanism for

transferring data between user clients and the application on the Cloud through a newly developed GISolve module.

–  Computation management: The job management service employed in CyberGIS is based on generically designed GISolve middleware. To add support for the Windows Azure platform, modules to submit and monitor computation jobs to the Azure platform were developed.

8

How does it work?

n  OpenServiceAPI-Client n  Job Submission n  Job Monitor n  Job Results

9

Implementation

10

Application Use Case: MODFLOW

n  USGS MODular finite-difference FLOW model application –  Simulate groundwater dynamics –  Typical MODFLOW simulations require ensemble runs of

different scenarios

n  This integration provides the groundwater modeling community easy and transparent access to on-demand and elastic computational resources

11

Case Study: ModflowOnAzure

n  We started with a version of MODFLOW application based on the Windows Azure called ModflowOnAzure

n  To enable integration with the CyberGIS Gateway the ModflowOnAzure implementation was modified. In particular this study –  Implemented a new data fetcher Worker role –  Defined new message types –  Added actions in Master Worker role to process tasks coming

from the CyberGIS gateway

12

Case Study: ModflowOnAzure

n  CyberGIS Gateway manages interactions with the ModflowOnAzure on users’ behalf –  Exploits the CyberGIS Open Services API –  Makes this application as a software service that can be accessed

simultaneously by a large number of users

n  Integration tested using Arizona water resources management use case which runs the ADWR (Arizona Depart of Water Resources) groundwater model periodically –  Each of these runs takes a few minutes –  Hundreds of stochastic runs take several hours –  Using our platform can reduce the time to completion significantly

13

Summary

n  This research developed a mechanism for integrating Cloud computing platforms with CyberGIS Gateway –  Microsoft Windows Azure

n  We enable the use of hybrid cyberinfrastructure in CyberGIS Gateway –  Achieved by adding a modular adapter in GISolve

middleware

14

Future Work

n  Visualization of ModflowOnAzure on CyberGIS n  Integrating other Cloud Computing platforms

–  Amazon EC2

n  Scalability and reliability testing using large scale experiments

n  Comparing cloud computing with other computational modalities for CyberGIS applications

n  Better use of the CyberGIS data management component to track application input and output files in the Cloud

15

Acknowledgment

n  This material is based in part on work supported by the National Science Foundation grant: OCI-1047916 and a Microsoft Research grant.

n  We also thank the Arizona Department of Water Resources for their assistance in the preparation of data and models for the case study.

16

Thank you

Babak Behzad bbehza2@uiuc.edu

17