Web: Email: [email protected] Running applications on interoperable Grid infrastructures, focusing on OMII-UK supported software - HPC-BP.

Web: www.omii.ac.uk Email: [email protected]

Running applications on interoperable Grid infrastructures,

focusing on OMII-UK supported software -

HPC-BP Interoperability TutorialOGF28, Munich

[email protected]

Steve Crouch, David Wallom, Matteo Turilli, Morris Riedel, Shahbaz Memon, Balazs Konya, Gabor Roczei, Peter

Stefan, Andrew Grimshaw, Mark Morgan , Katzushige Saga, Justin Bradley, Richard Boardman


Objectives

• To give participants practical experience of:o Using individual middleware clients to submit jobs

to HPC-BP compliant serviceso Using the HPC-BP interop demo framework, used

for previous HPC-BP demos, to submit jobs to HPC-BP compliant services

• To give participants opportunity (and starting point) to learn about:

o Basic techniques and approaches for interoperability – what do I need, and how can I do this?

o Some of the limitations of standards support across middlewares – what can’t I do?


Tutorial Approach

• ‘Presentation-lite’

• Learn at your own pace via online web tutorial…• …or follow my lead

• Pragmatic

• Generous in terms of time

• Tutorial remains available after OGF28

• Ask for help!


Schedule

• Session 1: Using individual clients to invoke HPC-BP services

o Overview of the demo + demo, Introduction to GridSAMo Download, Install and Configure GridSAM o Submit a Trivial Compute-only JSDL Job to HPC-BP

Compliant Services o Download, Build and Configure the BES++ Client o Running the BES++ Client Against HPC-BP Compliant

Services • Session 2: Using HPC-BP demo framework to

invoke multiple HPC-BP services simultaneouslyo Download, Install and Configure the Demo Framework o Running the Demo Against Multiple HPC-BP Compliant

Services o The Demo in Detail: Adding Another Endpoint to the

Demo


The Interoperability Demonstrator


Background

• Motivation:o Researchers are often reaching the limits of locally

available resources to conduct researcho They are beginning to realise the potential of using

much larger-scale resourceso Compute resources are becoming more numerous and

available across Europe

• However, using different Grid middleware deployments is traditionally difficult

o Middleware clients for different deployments not compatible

o Require different security policies/configuration for each


Background

• Possible solutions:o Maintain infrastructure that enables use of different clients for

each middleware – interoperation• Not scalable - user learning curve, operation and maintenance

o Each middleware supports a common service interface, enabled through adoption of accepted open standards – interoperability

• Need only learn, use and maintain single client infrastructure

• Still leaves security!

• What can be practically achieved, in terms of interoperability, with middlewares that adopt OGF compute-related standards?

o What is possible?o Limitations?

• Demonstrate through proof-of-concept, client-side, application-focused demo


History

• Initiated by UK National Grid Service, OMII-UK and FZJ

• Initially shown at OGF27, Banff, Canada, Oct 09

• SuperComputing, Nov 09

• ETSI Plugtests, FZJ, UK AHM, Dec 09

• GIN-CG, OGF28, Mar 10• Demonstrators: David

Wallom, Peter Stefan, Morris/Shahbaz Memon, Steve Crouch

Video available at http://www.omii.ac.uk/wiki/Videos


Compute Related Standards - OGF

Job ManagementOGSA-BES(GFD 108)

Job ManagementOGSA-BES(GFD 108)

HPC Domain Specific ProfileHPC Basic Profile

(GFD 114)

HPC Domain Specific ProfileHPC Basic Profile

(GFD 114)

ArchitectureOGSA EMS Scenarios

(GFD 106)

Use CasesGrid Scheduling Use Cases

(GFD 64)

EducationISV Primer (GFD 141)

AgreementWS-Agreement

(GFD 107)

Programming InterfaceDRMAA

(GFD 22/133)

Programming InterfaceSAGA

(GFD 90)

AccountingUsage Record

(GFD 98)

InformationGLUE Schema 2.0

(GFD. 147)

Job Definition

File TransferHPC File Staging

(GFD 135)

File TransferHPC File Staging

(GFD 135)

Job DescriptionJSDL

(GFD 56/136)

Job DescriptionJSDL

(GFD 56/136)

Application DescriptionHPC Application

(GFD 111)

Application DescriptionHPC Application

(GFD 111)

Application DescriptionSPMD Application

(GFD 115)

Job ParameterizationParameter Sweep

(GFD. 149)

Extend

Uses

Produces

Describes

Supports

Profiles


Standards/Data Protocols/Security Supported

• Standards:o HPC Basic Profile v1.0

• OGSA BES (Basic Execution Service) v1.0• JSDL (Job Submission Description Language) v1.0• HPC Profile Application Extension v1.0

o HPC File Staging Profile – UNICORE, GridSAM• Data protocols:

o UNICORE, ARC, BES++ – ftpo GridSAM – GridFTP

• Security:o Direct middleware -> certificate CA trust (just

import CAs)


Participation

• Currently:o DEISA/FZJ – UNICORE, SuSE, AMD 64-bit, 1 coreo NorduGrid/NIIF – ARC NOX Release, Debian Linux, i686, 16

coreo UK NGS/OMII-UK – GridSAM, Scientific Linux 4.7, AMD 64-bit,

256 coreo NAREGI-NII/Platform Computing – BES++, 2 nodes

• Coming soon:o University of Virginia Campus Grid – GENESIS2, Ubuntu

Linux, i686, 8 coreo POZNAN Supercomputing Centre – SMOA Computing

• Platform Computing BES++ Client used as interop client


Example Application: Plasma Charge Minimization

• Provided by David Wallom, NGS

• Undergraduate project• Total system energy

minimization of point charges around the surface of a sphere

• Three different applications o Pre processing – generate input

fileso Main processing – parallel

distributed processingo Post-processing – choose optimal

solution


System Requirements

• System requirements:o Linux - see the Linux client pre-requisites in

OMII-UK Development Kit supported platforms

o Sun Java JDK 1.6 or aboveo C compiler - gcc and related development

libraries o Lexical analyser - flex o Parser generator - bison

• Soon to appear on OGF Forge – hopefully by end of week


JSDL Template<?xml version="1.0" ?><JobDefinition xmlns="http://schemas.ggf.org/jsdl/2005/11/jsdl"> <JobDescription> <Application> <HPCProfileApplication xmlns="http://schemas.ggf.org/jsdl/2006/07/jsdl-hpcpa"> <Executable>@MINEM_INSTALL_LOCATION@/update_file</Executable> <Argument>input.txt</Argument> <Argument>output.txt</Argument> <Output>stdout.txt</Output> <Error>stderr.txt</Error> @OPTIONAL_WORKING_DIR_ELEMENT@ </HPCProfileApplication> @OPTIONAL_JOBRESOURCE_CREDENTIAL@ </Application>

<DataStaging> <FileName>input.txt</FileName> <CreationFlag>overwrite</CreationFlag> <Source> <URI>@INPUT_FILE_URI@</URI> </Source> @OPTIONAL_HPCFSP_CREDENTIAL@ </DataStaging> <DataStaging> <FileName>output.txt</FileName> <CreationFlag>overwrite</CreationFlag> <Target> <URI>@OUTPUT_FILE_URI@</URI> </Target> @OPTIONAL_HPCFSP_CREDENTIAL@ </DataStaging>

<DataStaging> <FileName>stdout.txt</FileName> <CreationFlag>overwrite</CreationFlag> <Target> <URI>@STDOUT_FILE_URI@</URI> </Target> @OPTIONAL_HPCFSP_CREDENTIAL@ </DataStaging> <DataStaging> <FileName>stderr.txt</FileName> <CreationFlag>overwrite</CreationFlag> <Target> <URI>@STDERR_FILE_URI@</URI> </Target> @OPTIONAL_HPCFSP_CREDENTIAL@ </DataStaging> </JobDescription>

</JobDefinition>


Endpoint Configuration

# UNICORE interop config fileendpoint_file=unicore.xmlapplication_type=HPCProfileApp

licationapplication_type_namespace=ht

tp://schemas.ggf.org/jsdl/2006/07/jsdl-hpcpa

working_dir=data_mode=ftpdata_input_base=ftp://

zam1161v01.zam.kfa-juelich.de:8004/ogf27/unicore

data_output_base=ftp://zam1161v01.zam.kfa-juelich.de:8004/ogf27/unicore

minem_install=/tmp/minemmyproxy=no

hpcfsp=yeshpcfsp_username=interopdatahpcfsp_password=89zukunft()

auth_utoken=yesauth_x509=yesauth_x509_credential=auth/

client.pemauth_x509_keypass=not_usedauth_x509_cert_dir=auth/

certificatesauth_utoken_username=ogfauth_utoken_password=ogf


How it Fits Together…

BES++Client

UNICORE

GridSAM

ARC

FTP

GridFTP

FTPClient Job Service

Data Service

Minem

Application

minem-interop.pl

MyProxy

Minem

Minem

Minem

Security Service

1. Create Minem input files

1. Create Minem input files

12. Generate JSDLs from

template2. Generate JSDLs from

template 2

7. Select best result7. Select best result

7

8. Generate/upload image to web server8. Generate/upload image to web server

8

5. Monitor jobs until completion

5. Monitor jobs until completion

5 BES++

FTP

4. Submit JSDLs across middlewares

4. Submit JSDLs across middlewares

4

6. Download output files6. Download output files

6

3. Upload input files3. Upload input files

3

Minem


The Demo…


Future Work• Standards integration:

o Integrate GENESIS II and SMOA Computingo Replacement of BES++ Client with SAGA

• SAGA BES adapter currently in development!• Schedule across BES/non-BES endpoints (e.g. Globus)

o GLUE2 (e.g. using OMII-UK Grimoires software)• Service discovery (static)• Dynamic allocation (dynamic)

o Integrate CREAM-BES?• Security: ‘Static’ trust set up of security, proper VO set

up?• Middleware client ‘audit’ of interoperability?

o Leads to ability to configure and use different middleware HPC-BP clients…

• Use of HARC for advance reservation• Clean up the code, upload to OGF Forge within GIN-CG• Participation very much an open process – if you wish to

donate an HPC-BP compliant endpoint, please let me know!


Verifi

ed/In

crea

sing

Inte

roper

abili

ty

Future Direction

• Interface:o Workflow engine

integration• To replace/provide

alternative to the Perl script

• Taverna2 good candidate

o Application abstraction

• Use of endpoints:o Utilise production-

level deploymentso Utilise production-

level security

Abstraction level

Uti

lise

pro

du

cti

on

-le

vel

de

plo

ym

en

ts

Now

Future


Dissemination

• Thanks to the OMII-UK publicity machine:o HPCWire: http://www.hpcwire.com/offthewire/European-

Grid-Interoperability-Goes-Global-79343767.html o SuperComputing Online:

http://www.supercomputingonline.com/latest/european-interoperability-goes-global

o EGEE: http://www.eu-egee.com/index.php?id=193&tx_ttnews[tt_news]=125&tx_ttnews[backPid]=65&cHash=90bb3f97cc

o http://www.d4science.eu/aggregator/sources/2?page=1 o http://www.it-tude.com/grid_interoperability_eu.html o http://www.beliefproject.org/zero-in/zero-in-fourth-issue-

emagazine/news o + numerous OMII-UK website articles & UK NGS articles

• Just type ‘European Interoperability Goes Global’ into Google…


GridSAM

OMII-UKLondon e-Science Centre, Imperial College, LondonInstitute of Computing

Technology, Chinese Academy of Sciences (Beijing)


GridSAM Overview

• What is GridSAM to the resource owners?o A web service to uniformly expose a computational

resource• Condor (via local or SSH submission)• Portable Batch Scheduler (PBS) (via local or SSH submission)• Globus• Sun GridEngine• Platform Load Sharing Facility (LSF)• Single machine through Fork or SSH

o Acts as a client to these resources• What is GridSAM to end-users?

o A means to access computational resources in an open standards-based uniform way

o A set of end-user command-line tools and client-side APIs to interact with GridSAM Web Services

• Submit and monitor compute jobs• Cross-protocol file transfer (gsiftp, ftp, sftp, WebDav, http, https,

soon SRB, iRODS) via Commons-VFS (http://sourceforge.net/projects/commonsvfsgrid)


Supported OGF Standards

• OGSA Basic Execution Service (BES) v1.0

• JSDL v1.0• HPC Basic Profile v1.0• HPC Profile Application Extension v1.0• HPC File Staging Profile v1.0• HPC Common Case Profile: Activity

Credential v0.1• JSDL SPMD Application Extension v1.0


GridSAM – Publications & Enabled Activities

+ in 2009/2010 – ICHEC Bioinformatics Portal, eSysBio, NAREGI/RENKEI


For Resource Owners…

ComputationalResourceManager

ComputationalResourceManager

DRMDRM

…

One of:PBS (Torque/OpenPBS/PBSPro)LSF, Condor, Sun GridEngine, Globus, Fork

GridSAMService

GridSAMService

X509certificate

Linux Many flavours:RHEL 3,4,5, Fedora 7,8, Scientific Linux 4Java: JDK 1.5.0+

Linux + JavaLinux + Java

Tomcat/AxisTomcat/Axis

Tomcat: 5.0.23, 5.0.28, 5.5.23Axis: v1.2.1

Persistence provided by one of:Hypersonic, PostgreSQL, or existing MySQL


For End-Users…

GridSAMService

GridSAMService

JSDL

HTTPS/HTTP

WS-Security:X509User/Password

Service InterfaceService Interface

Any/all of:GridSAM native interface, OGSA-BES v1.0, HPC Basic Profile v1.0

…

MyProxy (for Globus/GridFTP)

MyProxy (for Globus/GridFTP)

Globus-

styl

e

Proxy

Certif

icat

e

+ MyProxycredentials

Windows/Linux + Java

Windows/Linux + Java

GridSAM ClientGridSAM Client

Many flavours:RHEL 3,4,5, Fedora 7,8, Debian, Ubuntu, Scientific Linux 4, Windows XP, Windows VistaJava: JDK 1.5.0+

AxisAxisX509certificate

Generic BES/HPC Basic ProfileClient

Generic BES/HPC Basic ProfileClient

OSGA-BESOSGA-BESHPC Basic Profile

HPC Basic Profile


Open Community Development

• GridSAM is Open Source, Open Community Development

• GridSAM SourceForge project:

o 99.03% activity, 1 release/month

o SVN source code repositoryo Developer & discuss

mailing lists

http://sourceforge.net/projects/gridsam/


GridSAM e.g. with Condor

• A staged event-driven architectureo Submission pipeline is constructed as a network of

stages connected by event queueso Each stage performs a specific action upon incoming

events

Example Pipeline: Condor


Planned Future Developments

• For end-users:o Full support for JSDL Resource selection

across PBS, Globus, Condor & Fork DRMso JSDL Parameter Sweep Extensiono Support for SRB and iRODS

• For resource owners:o LCAS/LCMAPS supporto Packaging option as a standalone, manually

configurable web archive (WAR) file

• Direct PBS deployment throughout NGS sites


The tutorial begins… all you need is to go to:

http://www.omii.ac.uk/wiki/HPCBPTutorial

Web: Email: [email protected] Running applications on interoperable Grid infrastructures, focusing on OMII-UK supported software - HPC-BP.

Documents

client o

o unicore

o limitations

compatible o

maintenance o

o researchers

research o

o unive