OGCE Workflow Suite GopiKandaswamy Suresh Marru SrinathPerera ChathuraHerath Marlon Pierce TeraGrid 2008
Dec 29, 2015
OGCE Workflow Suite
GopiKandaswamySuresh MarruSrinathPerera
ChathuraHerathMarlon Pierce
TeraGrid 2008
Workflow Suite
• Generic Service Toolkit– Tool to wrap command-line applications as web services– Handles file staging&job submissions– Extensible runtime for security, resource brokering& urgent computing– Generic Factory service for on-demand creation of application services
• XRegistry– Information repository for the OGCE workflow suite– Register, search, retrieve&share XML documents– User & hierarchical group based authorization
• XBaya– GUI based tool to compose&monitor workflows– Extensible support for compiler plug-ins like BPEL &Jython
2OGCE TeraGrid08 Tutorial
Features
• Security– Authentication and authosrization– Secure invocations between services– Support for gateway community accounts– Support for multiple user accounts
• Reliability– Retry job submissions and file staging– Fault Tolerance and Recovery service
• Over-provisioning and migration
• Compatibility– Taverna, Kepler and Trianna
3OGCE TeraGrid08 Tutorial
Applications
• LEAD– Lower entry barrier to using weather analysis tools– Improve detection, analysis & prediction of mesoscale weather
• Motif-Network– Transformation of sequenced genomes to “domain-space”
• Cyber-Infrastructure Evaluation– Performance evaluation of future supercomputer architectures
• ADAM– Algorithms for feature extraction, data normalization,
classification and normalization
4OGCE TeraGrid08 Tutorial
88D Radar Re-
mapper
Satellite Data Re-mapper
NIDS Radar Re-mapper
Radar data (Level II)
Surface data, upper air mesonet data and
wind profiler data
Radar data (Level III)
Satellite data
ADAS
Terrain Preprocessor
3D Model Data
Interpolator (Initial
Boundary Conditions)
3D Model Data
Interpolator(lateral Boundary
Conditions)
Terrain data files NAM, RUC, GFS data
WRF Static Preprocessor
ARPS to WRF Data
Interpolator
ARPS Plotting Program
IDV
Surface, terrestrial data files
1
WRF to ARPS DataInterpolator
2
3
4
5
6
7
9
11
12
13
Run once per forecast region
Repeat periodically
for new data
Triggered if a storm is detected
Visualization on users request
ADAM
Data mining:look for storm
signature
WRFWRF
WRFWRF
14
ARPS Ensembl
e Generato
r
15
Static data Initialization Forecast VisualizationReal time data Analysis Data Mining
10
8
LEAD: A Weather Forecasting Workflow (1/2)
5
Motif-Network: Whole Genome workflow
• Domain webs of large genomes– Input list of amino acid sequences– Identify all known domains– Construct webs
Parallel processingCapability-type computing
Ensemble-type processing(minimal network reqs)
Capacity-type computing
7Jeff Tilson, RENCIOGCE TeraGrid08 Tutorial
CI: Execute Sub-Workflow
• Input a campaign step filename• Execute GAMESS per step
specification
8Jeff Tilson, RENCIOGCE TeraGrid08 Tutorial
More Information
OGCE TeraGrid08 Tutorial 9
Tuesday 1:30 pm Engaging with the LEAD Science Gateway Project: Lessons Learned in Successfully Deploying Complex System Solutions on TeraGrid
Tuesday 3:30 pm Reliable and Scalable Job Submission: LEAD Science Gateway's Testing and Experiences with WS GRAM on TeraGrid Resources
Tuesday 3:30 pm The LEAD-WxChallenge Pilot Project: The Potential of Grid-Enabled Learning
Wednesday 10:30 am Workflow Infrastructure for Multi-scale Science Gateways
Wednesday 4:00 pm On-Demand Severe Weather Forecasts Using TeraGrid via the LEAD Portal