Resource Brokering: the EuroGrid/GRIP approach

Resource Brokering: the EuroGrid/GRIP approach

Donal Fellows, John Brooke, Jon MacLaren

E-Science NorthWest @ University of Manchester UK

http://www.esnw.ac.uk

2

Grid Interoperability

In European and Japanese Grid projects there are two major middleware systems deployed, Globus (US) and Unicore (Europe/Japan).

Globus is mainly deployed in cluster-based Grids and Unicore in projects with complex heterogeneous architectures (e.g. specialist HPC architectures).

The FP 5 project GRIP began looking at the question of how resource requests could be handled from Unicore to Globus and the FP6 project takes this work forward into the world of service-based architectures (e.g. OGSA)

Starting point - GRIP

EU Funded FP5 Project as part of Information Society Technologies Programme IST 2001-32257http://www.grid-interoperability.org/

4

A Dual Job-Space

Thus we have a space of “requests” defined as a vector space of the computational needs of users over a Grid. For many jobs most of the entries in the vector will be null.

We have another space of “services” who can produce “cost vectors” for costing for the user jobs (providing they can accommodate them).

This is an example of a dual vector space.

A strictly defined dual space is probably too rigid but can provide a basis for simulations.

The abstract job requirements will need to be agreed. It may be a task for a broker to translate a job specification to a “user job” for a given Grid node.

5

4 - Dual Space

Cost vector

Job vectorCost

User Job

Scalar cost

in tokens

1

2

6

Computational resource

Computational jobs ask questions about the internal structure of the provider of computational power in a manner that an electrically powered device does not.For example, do we require specific compilers, libraries, disk resource, visualization servers?What if it goes wrong, do we get compensation? If we transfer data and methods of analysis over the Internet is it secure?

A resource broker for high performance computation is a different order of complexity to a broker for an electricity supplier.

7

EuroGrid: Meteo-Grid

8

Resource Requestor and Provider Spaces

Resource requestor space (RR), in terms of what the user wants: e.g. Relocatable Weather Model, 10^6 points, 24 hours, full topography.Resource Provider space (RP), 128 processors, Origin 3000 architecture, 40 Gigabytes Memory, 1000 Gigabytes disk space, 100 Mb/s connection.We may even forward on requests from one resource provider to another, recasting of O3000 job in terms of IA64 cluster, gives different resource set.Linkage and staging of different stages of workflow require environmental support, a hosting environment.We can have multiple offers in RP space for the same RR values

9

Abstract Functions for a resource broker

1. Resource discovery, for workflows as well as single jobs.

2. Resource capability checking, do the offering sites have ALL necessary capability and environmental support for instantiating the workflow.

3. Inclusion of Quality of Service policies in the offers.

4. Information necessary for the negotiation between client and provider and mechanisms for ensuring contract compliance.

Document submitted to GPA-RG group of GGF.

10

OrganizationFirewalls

Users

VirtualOrganisationBrokers

SystemBrokers

ComputeResources

Brokers as Virtual Organizations

11

ClientClient

Broker ServiceBroker Service

Broker ServiceBroker Service

Broker ServiceBroker Service

Broker ServiceBroker Service

Broker ServiceBroker Service

Broker ServiceBroker Service

ClientClient

ClientClient

ClientClient

ClientClient

Broker ServiceBroker Service

ResourceResource

ResourceResource

ResourceResource

ResourceResource

ResourceResource

ResourceResourceBroker ServiceBroker Service

VO Layer Specialist Layer Site Layer

Replication

Federated Brokering

12

Persistent Virtual EnvironmentsPersistent Virtual Environments

MetaschedulingService

MetaschedulingService

Workflow ManagerWorkflow Manager

ClientsClients

BrokerBroker

Chargeable Schedulable GridServicesChargeable Schedulable GridServices

Other BrokersOther Brokers

Site Feedback Policy Manager

Site Feedback Policy Manager

Resource Usage MonitorResource Usage Monitor

Banking ServicesBanking Services

Brokering and OGSA Services

13

Possible OGSA Broker

Interoperating OGSA services

NetworkJob

Supervisor

UnicoreGateway

Unicore Client

TSI

OGSA Server A

GT3/4

ResourceDatabase

ResourceBroker

UserDatabase

14

BrokerBrokerNJSNJS

IDBIDB

NJSNJS

IDBIDB

GatewayGateway

TSITSI TSITSI

LRC

LRC

TSI SuppliesDynamic Datato IDB

Users Contact NJSes or Broker (for site-wide brokering)

Delegate (site-wide brokering only) Delegate (site-wide brokering only)

Potential to Share (Partial?) IDBs between NJSes (CSAR Config?)

Site Configuration

15

To outside world

UNICORE Components

EUROGRID Broker

Globus Components

GRIP Broker

Key:

Inheritance relation

Whole-Site Broker

Compute Resource Compute Resource

SingleVsiteBrokerSingleVsiteBroker

NJSNJSIDBIDB UUDBUUDB

ExpertBrokerExpertBroker

DWDLMExpertDWDLMExpert OtherOther

LocalResourceCheckerLocalResourceChecker

UnicoreRCUnicoreRC Globus2RCGlobus2RC

OntologicalTranslatorOntologicalTranslator

SimpleTranslatorSimpleTranslatorMDSGRAMTSI

ICMExpertICMExpert

Look up staticresources

Look upconfiguration

Verify delegatedidentities

Delegate to application-domain expert codeDelegate to Grid architecture-specificengine for local resource check

Pass untranslatable resources to Unicore resource checker

Look up resourcesLook updynamicresources

Delegate resource domain translation

Converts UNICORE resource requests to XPath search terms for GT3 Index Service & set of untranslatable resources to use UNCORE standard techniques upon.

Look up translations appropriateto target Globus resource schema

Broker hosted in NJS

SimpleTranslator converts delegated UNICORE resource requests into LDAP search terms for GT2 MDS & set of untranslatable resources to use UNICORE standard techniques upon.

TicketManagerTicketManagerGet & check signed tickets (contracts)

Look up signing identity

OntologyOntology

GT3

Globus3RCGlobus3RC

AbstractBrokerAbstractBroker

WholeUsiteBrokerWholeUsiteBroker

R-GMAR-GMA

HostVsite MapHostVsite Map

Use R-GMA to provideinformation for all Usitecomponent hosts

Experts may use LRC

Look up resources

UoM Broker architecture

16

Broker functions

• A simple Resource Check request: “Can this job run here”, checks static qualities like software resources (e.g. Gaussian98) as well as capacity resources like quotas (disk quotas, CPU, etc.)

• A Quality of Service request: returns a range of turnaround time, and cost, as part of a Ticket. If the Ticket is presented (within its lifetime) with the job, the turnaround and cost estimates should be met.

17

Grid Resource Description Problem

Two Independent Grid SystemsUnicore (http://www.unicore.org/)Globus (http://www.globus.org/)

Both Need to Describe Systems that run Compute JobsVery Different Description Languages

Unicore’s Resource model, part of the AJO FrameworkGlobus’s GLUE Schema (DataTAG, iVGDL) for GT2 and GT3

For interoperability, we want to take a Unicore job and run it on Globus resources

Therefore, we need to translate the Job’s Resource Requirements between the two Systems

18

Methodology fortranslation servce

Address Data Transformation Issues for Translating Attributes

Find a technology that has these characteristics:can model the two ontologieshas support for linking abstract concepts to code fragmentseasily allows someone to update mappingsis appropriate for a video conferencing settingwrites modelling information to a file format that can be used by other applications

Use the data files created by the application to run the translator service.

19

Unicore: Modelling Resources

20

GLUE:Modelling resources

21

GLUE: Marking up transcripts

22

GLUE: Provenance Information

23

Compatible Concepts

Unicore Ontology GLUE Ontology

Network Performance Glue SI00 Benchmark

Network Performance Glue SF00 Benchmark

Floating Point Performance Glue SI00 Benchmark

Floating Point Performance Glue SF00 Benchmark

Data Processing Performance Glue S100 Benchmark

Data Processing Performance Glue SF00 Benchmark

Maximum Memory Capacity Request Host Virtual Main Memory Available

Maximum Memory Capacity Request Subcluster Virtual Main Memory Available

Minimum Memory Capacity Request Host RAM Main Memory Available

Minimum Memory Capacity Request Subcluster RAM Main Memory Available

Priority Value Priority

24

Translation Service Prototype

25

Conclusions

• Interoperability of grid resource requests is at the heart of the abstract idea of computational resource that can cross Grid domain boundaries• We wish to provide application users with seamless access to resources, they should not need to know details of the machines on which they run.• High level abstractions do not yet exist as standards, so we have to create ontologies that can translate differing modelling abstractions for Grid resources.• Our current translations lose much information in crossing between current middleware systems (e.g. Globus and Unicore).

26

Continuation of interoperability research

Research Centre Jülich(Project manager)

Consorzio Interuniversitarioper il Calcolo Automatico dell’Italia Nord Orientale

Fujitsu Laboratories of Europe

University of Warsaw

Intel GmbH

University of Manchester

T-Systems SfR

http://www.unigrids.org

27

GLUE: Container Classes

GLUE has container classes that include “Computing Element”, “Cluster”, “Subcluster” and “Host”. From the heading “Representing Information”, the GLUE document indicates:“…hosts are composed into sub-clusters, sub-clusters are grouped

into clusters, and then computing elements refer to one or more clusters.”

These container objects may hold any number optional auxiliary classes that actually describe the GRID features.

28

GLUE: Auxiliary Classes

The documentation provides few details about the nature of a Host other than that it is a “physical computing element”. Much of the meaning for Host has to be derived from what it might contain. Consider the following two valid definitions:

A Host is a physical computing element characterized by an Architecture, a Processor and an Operating System.

A Host is a physical computing element characterized by Main Memory, a Benchmark, a Network Adapter and an Operating System

29

Map conceptsbetween ontologies

Unicore and GLUE have different philosophies for describing resources :-(In Unicore, the resources are described in terms of resource requests In GLUE, resources are described in terms of the availability of resources.

30

Broker

BrokerNJSNJS

IDBIDB

TSI/HostTSI/Host GT3GT3

GatewayGateway

ClientClient

Broker

Broker

NJSNJS

HostHost Host

Host

GatewayGateway

ClientClient

R-GMAR-

GMANJSNJS NJSNJS

HostHost

Site-Wide BrokeringNormal EUROGRID/GRIP Brokering

Local Brokering Configurations

31

RR space

RP space

RP space

RP spaceRR space

request

request

Request referral

sync

Figure 1: Request from RR space at A mapped into resource providers at B and C, with C forwarding a request formulated in RR space to RP space at D. B and D synchronize at end of workflow before results returned to the initiator A.

AB

CD

RR and RP Spaces