Grid Computing Middle Ware

7/27/2019 Grid Computing Middle Ware

1/49

1

Grid Computing Middleware

Presenter:Xon Xay

Nguyen Tuan VietTram The Phien


2/49

2

Outline Introduction

Definition of Grid Middleware

Purpose of Middleware

Grid layered architecture

Technology Globus

GLite


3/49

3





Technology Globus

GLite


4/49

4

Grid environment


5/49

5

Introduction Three main issues to confont in Grid

environment:

Heterogeneity multiplicity of resources

Scalability

Adaptability - Resource managers or

applications dynamic to extract themaximum performance from the availableresources and services.


6/49


7/49

7

Grid Middleware System software between applications

and operating system Provide services to application

Discovery, storage, execution, information,service integration, resource monitoring, failuredetection and recovery,

Hide heterogeneous of the Gridenvironment

Provide standardised interfaces to services.


8/49

8





Technology Globus

GLite


9/49

9

Purposes of Middleware Open, general-purpose and standard

Standard protocols

Defines the contents and sequence of messageexchanges used to request remote operation

Important and essential to achieve theinteroperability that Grid depends on

Standard APIs Interfaces to code libraries

Facilitate construction of Grid components byallowing code components to be reused


10/49


11/49

11

Grid layered architecture- The neck: a small setof core abstractions andprotocols- Top of hourglass:many different high-level behaviors can bemapped

- Base of hourglass:can be mapped ontomany differentunderlying technologies


12/49

12

Component of Middleware Connectivity layer

Resource layer

Collective layer


13/49

13

Connectivity layer Communicating easily and securely

Communication protocol: exchange data

Authentication protocol: cryptographicallysecure mechanism for verifying the identity ofusers and resources. Single sign-on

Delegation Integration with various local security solutions

User-based trust relationships


14/49

14

Resource layer Sharing single resources

Define protocols (and APIs and SDKs) for the

secure negotiation, initiation, monitoring,control, accounting, and payment of sharingoperations on individual resources

Information protocols - obtain information

about the structure and state of a resource Management protocols - negotiate access to

a shared resource


15/49

15

Collective layer Coordinating multiple resources

Implement a wide variety of sharing

behaviors without placing new requirementson the resources being shared

ExampleDirectory services, co-allocation,scheduling, brokering services, Monitoring and

diagnostics services, Data replication services,

Grid-enabled programming systems,


16/49

16

Collective layer- Example

Application use co-reservation serviceprotocol to request end-to-end networkreservation We define a co-

reservation serviceprotocol

Implement co-reservation service

Call co-allocation API toimplement co-allocationoperation


17/49

17

Outline

Introduction




Technology Globus

GLite


18/49

18

Globus Toolkit

Evolution

Globus Toolkit version 2




19/49

19

Globus evolution

1997

GT2.0

2003

GT3.0OGSA

2005

GT4.xWSRF

Proof of concept Standardization


20/49

20


Community-based, open-architecture, opensource set of services and software librariesthat support Grids and Grid applications.

De facto standard for Grid computing

Focus on usability and interoperability

Middleware GT2 Connectivity layer GT2 - Resource layer

GT2 - Collective layer


21/49

21

GT2 ConnectivityAuthentication protocol

Public-key based GSI protocols

GSI - Grid Security Infrastructure

authentication, communication protection,authorization

GSI builds on and extends the TLS protocols

TLS - Transport Layer Security

TLS - defines a protocol to provide privacy anddata integrity between two communicatingapplications.


22/49

22

GT2 Connectivity layerCommunication protocol

Internet layered protocol architecture

Internet layer - IP and ICMP

Transport - TCP, UDP

Application layer - DNS, OSPF, RSVP,


23/49

23

GT2 - Resource layer

A Grid Resource Information Protocol baseon Lightweight Directory Access Protocol(LDAP)

HTTP-based GRAM protocol GRAM - Grid Resource Access and Management allocation of computational resources monitoring and control of computation on

resources GridFTP - a management protocol for data

access An extended version of the File Transfer Protocol


24/49

24

GT2 - Collective layer

build on Globus Connectivity and Resourceprotocols

Meta Directory Service introduces Grid Information Index Servers (GIISs)

LDAP information protocol

Access resource-specific GRISs to obtain resource state

GRRP used for resource registration.

DUROC resource co-allocation library providesan SDK and API for resource co-allocation


25/49

25


Build on and significantly extend GT2concepts and technologies

Base on Open Grid Services Architecture Factoring of component behaviors identify

essential Grid functions and express

Service orientation enable a uniform treatment

of all network entities Align with Web services WSDL (Web Services

Definition Language)


26/49

26


developed by The Globus Alliance

OGSA requires stateful services

WSRF Web Service ResourcesFramework

developed by OASIS (http://www.oasis-open.org)

WSRF provides the stateful services thatOGSA needs

l i hi b OGS G S


27/49

27

Relationship between OGSA, GT4, WSRF,and Web Services


28/49

28

gLite Toolkit

The Enabling Grids for E-sciencE project(EGEE)

Middleware stack that combinescomponents developed in various relatedprojects.


29/49

29

gLite middleware

A layer betweenservices and resources

gLite follows a ServiceOriented Architecture


30/49

30

gLite - guiding principles

Service oriented approach Allow for multiple interoperable implementations

Lightweight (existing) services Easily and quickly deployable Use existing services where possible

Condor, EDG, Globus, LCG,

Portable Being built on Scientific Linux and Windows

Security Sites and Applications


31/49

31

gLite - guiding principles

Performance/Scalability & Resilience/FaultTolerance Comparable to deployed infrastructure

Co-existence with deployed infrastructure Co-existence with LCG-2 and OSG (US) are

essential for the EGEE Grid services

Site autonomy Reduce dependence on global, central services

Open source license


32/49

32

gLite Architecture


33/49

33

gLite - Access

All of the gLite services are accessiblevia APIs and CLIs.

API - Application Programming Interface

CLI - Command Line Interface


34/49

34

gLite - Security Services


35/49

35

gLite - Security Services

Authentication identify entities PKI (X.509) infrastructure with CA third party for signature

Authorization allows or denies access to specific services three basic authorization models, classified as agent, push

and pull.

Dynamic Connectivity Service tackle the problems that arise when connectivity is restricted

by the resource owner.

Auditing monitoring and providing for post-mortem analysis of

security related events.


36/49

36

gLite - Infomation and Monitoring Services


37/49

37


Infomation and Monitoring Services a vitallow-level component of any grid


38/49

38


Job Monitoring use the APIs offered by the apache logging

services (log4j)

Service Discovery a facility offered to both end users and to other

services to locate suitable services.

Network Performance Monitoring define a standard interface to monitoring

frameworks.

NPM Publisher for use by gLite components.


39/49

39

gLite - Helper Services


40/49

40

gLite - Helper Services

Configuration and Instrumentation Service common, standard-based configuration and

instrumentation functionality to the gLite services.

Bandwidth Allocation and Reservation Service control and balance the usage of the network and

categorise and prioritise traffic flows.

Agreement Service implements the communication protocol used to

exchange information about Service LevelAgreements (SLAs) and defines the SLA structure.


41/49

41

gLite - Data Services


42/49

42

gLite - Data Services

Catalog

store information about the data and metadata

Storage Element is the Grid service responsible for saving/retrieving

files to/from some data store which can provide awide range of quality of service to the user.

Data movement provide scalable and robust managed data

transfer between Grid sites


43/49

43

gLite - Job Management Services


44/49

44


Accounting accumulates information about the usage of Grid

resources Computing element

representing a computing resource job management and other capabilities

Workload Management comprises a set of Grid middleware components

responsible for the distribution and managementof tasks across Grid resources


45/49

45


Job Provenance

keep track of the definition of submitted jobs,execution conditions and environment.

Package Manager

is a helper service that automates the process ofinstalling, upgrading configuring, and removing

software packages from a shared area


46/49

46

Comparison

Globus Toolkit - faster andmore efficient

gLite - sophisticated anddeveloper friendly

Document

Globus documents - sketchyand missing

gLite documents - sufficient,available


47/49

47

Summary




Technology

Globus

GLite


48/49

48

Reference

I. Foster and C. Kesselman, The Grid: Blueprint for a NewComputing Infrastructure. Morgab Kaufmann Publishers, 1999.

Fran Berman, Anthony J. G. Hey and Geoffrey C. Fox, Gridcomputing: Making the Global Infrastructure a Reality. John

Wiley & Sons Ltd, 2003. Globus project: http://www.globus.org/alliance/ Grid Computing Slides Nguyen Tuan Anh Grid Computing Slides Pham Tran Vu vonLaszewski--grid-middleware 03-grid-concepts-architecture-middleware-adler-vortrag Websites of gLite (http://www.eu-egee.org)


49/49

Q&A

Grid Computing Middle Ware

Documents