The Anatomy of the Grid: An Integrated View of Grid Architecture Carl Kesselman USC/Information Sciences Institute Ian Foster, Steve Tuecke Argonne National Laboratory
The Anatomy of the Grid: An Integrated View of Grid
ArchitectureCarl Kesselman
USC/Information Sciences Institute
Ian Foster, Steve Tuecke
Argonne National Laboratory
Outline and Goals
Define what is (and isn’t) a grid in terms of a layered protocol architecture
Understand the role of protocols, services, APIs and tools in the deployment and use of Grids
Understand what role Globus plays Understand the role of other technologies
What is a Grid?
Grid language has been driven by genesis from metacomputing, but…
In practice, the Grid is about resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations
Focus on how to enable, maintain and control the sharing of resources to achieve a common goal
Some Useful Definitions
Network Protocol A formal description of message formats and
a set of rules for exchange of messages Rules define sequences of message
exchange, and potentially resulting behavior Protocol may define state-change in endpoint
Network Enabled Services Defines a set of capabilities Protocol defines interaction with service
All services require protocols, although not all protocols are to services
More definitions
Resource Entity that is to be shared Provides some capabilities, that can be
accessed via interface (API) or protocol Application Programmer Interface (API) Software Development Kit (SDK)
Package that enables application development, consisting of one or more APIs, and programming tools
Protocols Make the Grid
Protocols and APIs Protocols enable interoperability APIs enable portability
Sharing is about interoperability, so … Grid architecture should be about protocols
Grid Services Architecture: Previous Perspective
Grid-enabled archives, networks, computers, display devices, etc.; associated local services
Protocols, authentication, policy, resource management, instrumentation, discovery, etc., etc.
GridFabric
GridServices
ApplnToolkits
Applns
...
… a rich variety of applications ...
Remoteviz
toolkit
Remotecomp.toolkit
Remotedata
toolkit
Remotesensorstoolkit
Async.collab.toolkit
Characteristics of Grid Services Architecture
Identifies separation of concerns Isolates Grids from languages and specific
programming environments Makes provisions for generic and
application specific functionality Protocols not explicit in architecture
fails to make clear distinction between language, service and networking issues
Layered Grid Protocol Architecture
Connectivity
Fabric
Resource
Grid
Application
User
Important Points
Being Grid-enabled requires speaking appropriate protocols Protocol only requirement, not reachability Protocols can be used to bridge local resources
or “local Grids” Intergrid as analog to Internet
Built on Internet protocols Independent of language and implementation
Focus on interaction over network Services exist at each level
Protocols, services and interfaces
Languages/Frameworks
Fabric Layer
Applications
Local Access APIs and protocols
Grid Service APIs and SDKs
Grid ServicesGrid Service Protocols
Resource APIs and SDKs
Resource ServicesResource Service Protocols
User Service ProtocolsUser Service APIs and SDKs
User Services
Connectivity APIs
Connectivity Protocols
How does Globus fit in?
Defines connectivity and resource protocols
Enables definition of grid and user protocols Globus provides some of these, others
defined by other groups Defines range of APIs and SDKs that
leverage Resource, Grid and User protocols
Fabric
Local access to logical resource May be real component, e.g. CPU, software
module, filesystem May be logical component, e.g. Condor pool
Protocol or API mediated Fabric elements include:
SSP, ASP, peer-to-peer, Entropia-like, and enterprise level solutions
Connectivity Protocols
Two classes of connectivity protocols underlie all other components
Internet communication Application, transport and internet layer
protocols I.e., transport, routing, DNS, etc.
Security Authentication and delegation Discussed below
Security
Protocols TLS with delegation
Services K5ssl, Globus Authorization Service
APIs GSS-API, GAA, SASL, gss_assist
SDKs GlobusIO
Resource Protocols
Resource management, Storage system access Network quality of service Data movement Resource information
Resource Management
Protocols GRAM+GARA (on HTTP)
Resource services Gatekeeper, JobManager, SlotManager
APIs and SDKs GRAM API, JavaCog Client, DUROC
Data Transport
Protocols Grid FTP, LDAP for replica catalog
Services FTP, LDAP replica catalog
APIs and SDKs GridFTP client library, copy URL API, replica
catalog access, replica selection
Resource Information
Protocol LDAP V3, Registration/Discovery protocol
Service GRIS
APIs & SDKs C API; JNDI, PerlLDAP, ….
Grid Protocols Grid Information Index Services
LDAP and Service registration protocol, … GIIS service LDAP APIs and specialized information API
Co-allocation and brokering GRAM (HTTP+RSL) DUROC service DUROC client API, end-to-end reservation API
Grid Protocols (cont)
Online authentication, authorization services HTTP MyProxy, Group policy servers Myproxy API, GAA API,
Many others (e.g.): Resource discovery (Matchmaker) Fault recovery
User Protocols
In general, there are many of these, they tend to be on off, and not well defined
Examples: Portal toolkits (e.g. Hotpage) Netsolve Cactus framework
Summary
Grids are about sharing, not making big MPI jobs
Well defined protocol architecture is essential to understanding and progress Provides a framework for figuring out where
the pieces fit Globus provides many basic protocols and
services Interoperability between different Grid
components