http://www.dorii.eu/ Parametric jobs – faciliation of Instrument Elements usage in Grid applications INGRID 2009 Katarzyna Bylec , Szymon Mueller, Mateusz Pabiś, Mariusz Wojtysiak, Paweł Wolniewicz Poznań Supercomputing and Netwoking Center 03.04.2009, Alghero, Italy
22
Embed
Http:// Parametric jobs – faciliation of Instrument Elements usage in Grid applications INGRID 2009 Katarzyna Bylec, Szymon Mueller, Mateusz.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
http://www.dorii.eu/
Parametric jobs – faciliation of Instrument Elements usage in Grid
applications
INGRID 2009
Katarzyna Bylec, Szymon Mueller, Mateusz Pabiś, Mariusz Wojtysiak, Paweł Wolniewicz
Poznań Supercomputing and Netwoking Center
03.04.2009, Alghero, Italy
Outline
Background
DORII and HORUS application
JSDL and its extension
Parameter Sweep capabilities
Technical implementation of HORUS workflow
JAVA library for Parameter Sweep
New oportunities = new problemsSimplify access to the Grid → wider community → new users →
new use cases → new needs → extending classical e-Infrastructure → new problems
Problem: Instrument elements as a virtualisation of data sources - data processing
Input: do the same with many input files
Output: output the same file with changed content
or output many different files as a result of slightly changed processing Goal – to streamline the process – 2 levels:
Automatization
Describing the logic of job
Real life example - DORII
Situation:
well established communities (earthquake, environmental science, experimental science)
applications not or only partially integrated in the European e-Infrastructures
Applications' needs:
To make the daily work more efficient
To automatize the jobs' flow
Enhance usage of scientific devices
Solutions:
Integrate applications to e-Infrastructure
Convert applications into Grid workflows
Case study: HORUS_bench Description:
Insitituto de Hidráulica Ambiental, Universidad de Cantabria
Used to process data gathered via HORUS system
images of Puntal beach, Santander, Spain to measure beach user density, calculate the wather line, etc.
Requirements:
User chooses set of processing algorithms which constitue binary model to run over input images
The same model is run over GB of images data
HORUS_bench workflow
SE
SE
SE
Predefined binary models
Archived images
HORUS_bench output
IE – cameras on the beach
0. make photos
0.1 send photos to application
0.2 store photos on SE
1. model input data
2. processing model
3. computing 4. store results
HORUS workflow adaptation
DORII
E-Infrastructure (access to IEs)
VCR and Workflow System
Workflows: advantages
Automation of the process
Single Sing-On (MyProxy)
All application's task managed from one point
Monitoring of workflow execution
Hide the Grid complexity from user
Problems
Execution of thousands of the same jobs for different input data
JSDL specification
JSDL = Job Submission Description Language
JSDL Working Group in Open Grid Forum
Project created: 09/25/2003, version 1.0 available
Goal:
to specify an abstract standard of job description language that is independent of underlying middleware