NW-GRID Campus Grids Workshop Liverpool31 Oct 2007 NW-GRID Campus Grids Workshop Liverpool31 Oct 2007 Moving Beyond Campus Grids Steven Young Oxford NGS.

NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007NW-GRID Campus Grids WorkshopLiverpool 31 Oct 2007

Moving Beyond Campus Grids

Steven YoungOxford NGS Manager

Oxford e-Research Centre

University of Oxford

Partner Joining Consultant

National Grid Service


Beyond Campus Grids: My brief

• Connecting Campus Grids to broader grids

• How Campus Grids can be used to bridge to broader area grids like the NGS or NW-GRID

• What are the impediments?

• What can Campus Grids do to make things easier?


Straw Poll

• Who’s running a Campus Grid?– Condor-based, Other batch system (SGE,

PBS, LSF), Globus-based

• Level of service?– Grown organically, Well developed service

• User engagement?– Handful of heavy users, embedded into

institution’s teaching and research


Talk Summary

• Joining the NGS– The NGS Site Joining Process

– Benefits & difficulties

• Providing your users with a uniform interface to resources– Options for local deployment

– Useful NGS services and resources

• Workshop discussion


NGS expansion

Two levels of membership1. Affiliates run compatible stack, integrated

support2. Partners also contribute “significant

resources”.• Run compatible middleware

– as defined by NGS Minimum Software Stack– and verified by compliance test suite

• Partners define their level of service commitment through a Service Level Definition


Joining Quickstart

• http://www.ngs.ac.uk/guide/

• Partner’s Joining Guide

• Minimum Software Stack

• Conformance Testing

• Quickstart guide


Joining Procedure

• Initial contact

• Registration of Partner

• Contacts: Security, Operations, Helpdesk

• Testing of Site

• Validation of Site

• Continuous Monitoring


NGS Minimum Software Stack

• Access, Job submission, File transfer• Globus GRAM, GridFTP, IGTF CAs, GSIssh, SRB

• Information Publishing• GLUE schema, MDS

• People Management• edg-mkgridmap, VOMS

• Compiling• C/C++/Fortran/Java

• Resource Control• Accounting, Job Scheduling


Levels of Conformance

• Level 0: Globus, CAs, GLUE, MDS, VO Management, Accounting, Resource Scheduler

• Level 1: Level 0 + gsissh, compilers, SRB clients

• Level 2: Level 1 + Resource Broker, SRB servers/vaults/MCAT


Conformance Testing

• Inca framework: http://inca2.ngs.ac.uk• “When the partner and the buddy are satisfied

that the partner site can pass all the tests the partner site is progressed to the validation process.”

• Site needs to pass tests for a 7 day period.• Final report, SLD are written• Approval by NGS Technical Board and GOSC

Board• Continuous monitoring: “failures must be

corrected within 2 working days.”


Benefits

• Access to wider grid expertise

• Work put in to formalise processes for being part of NGS is not wasted

• NGS brand has value


Difficulties

• Justifying the effort of joining– Full Economic Costing

• Split focus– NGS software stack and interfaces may not

be what your users care about


Uniform Interface for Users

Things that could be deployed locally

• Condor-G

• Nimrod

• Gridway


Condor-G

• http://www.cs.wisc.edu/condor/condorg/

• Condor-G

• OxGrid is based on Condor-G technology– Some additional “magic” to generate Grid

resource ClassAds.

• Glide-in mechanisms


Nimrod-G

• http://www.csse.monash.edu.au/~davida/nimrod

• Tool for Distributed Parametric Modelling

• Actuators which can pass jobs to Globus resources as well as other local batch systems (Condor, PBS, SGE, LSF).

• Nimrod Portal


Gridway Metascheduler

• http://www.gridway.org/

• Works on top of Globus services– Which you will have if you become an NGS

affiliate.


Useful NGS services (1)

• NGS Resource Broker– A gLite Workload Management System and

User Interface node is available for use.

• NGS VOMS service– We can host VOMS instances for institutions,

groups, virtual organisations.

• NGS GridSAM EPR– Belfast provide GridSAM end point references

for NGS sites.


Useful NGS services (2)

• NGS Application Repository portal– JSDL repository for user applications. Could

allow generalised job submission to more than NGS resources.\

• NGS RUS and UAS– NGS UAS User Accounting System aims to

conform to OGF UR (Usage Record) and OGF RUS (Resource Usage Service) specifications.


Workshop session

• Discussion around how practical issues surrounding:– Joining of campus grids to NGS

• Eg. Globus interface node in front of a Condor pool

– Implementation of middleware solutions


Questions?


Shibboleth

• JISC funded projects to work on Shibboleth Grid integration

• Shibboleth SP used as a portal using “some X509 certificate mechanism” in the background to provide access to Grid resources.

NW-GRID Campus Grids Workshop Liverpool31 Oct 2007 NW-GRID Campus Grids Workshop Liverpool31 Oct 2007 Moving Beyond Campus Grids Steven Young Oxford NGS.

Documents