Globus Toolkit Futures:An Open Grid Services
Architecture
Ian Foster
Carl Kesselman
Jeffrey Nick
Steven Tuecke
Globus Tutorial, Argonne National Laboratory, January 29, 2002
April 18, 2023 Open Grid Services Architecture 2
The Grid World: Current Status Dozens of major Grid projects in scientific &
technical computing/research & education Considerable consensus on key concepts and
technologies– Open source Globus Toolkit™ a de facto standard for
major protocols & services
– Far from complete or perfect, but out there, evolving rapidly, and large tool/user base
Industrial interest emerging rapidly Opportunity: convergence of eScience and
eBusiness requirements & technologies
April 18, 2023 Open Grid Services Architecture 3
Overview
eBusiness requirements and the need for a services-based model– Jeff Nick, IBM
Open Grid Services Architecture overview– Ian Foster
Grid service specification & demonstration of OGSA prototype– Steven Tuecke
Open Grid Services ArchitectureOverview & Globus Toolkit Evolution
Ian Foster
Mathematics and Computer Science Division
Argonne National Laboratory
and
Department of Computer Science
The University of Chicago
April 18, 2023 Open Grid Services Architecture 5
The Grid Problem
Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations
April 18, 2023 Open Grid Services Architecture 6
Why Grids? (1) eScience A biochemist exploits 10,000 computers to
screen 100,000 compounds in an hour 1,000 physicists worldwide pool resources for
peta-op analyses of petabytes of data Civil engineers collaborate to design, execute,
& analyze shake table experiments Climate scientists visualize, annotate, &
analyze terabyte simulation datasets An emergency response team couples real time
data, weather model, population data
April 18, 2023 Open Grid Services Architecture 7
Why Grids? (2) eBusiness Engineers at a multinational company
collaborate on the design of a new product A multidisciplinary analysis in aerospace
couples code and data in four companies An insurance company mines data from partner
hospitals for fraud detection An application service provider offloads excess
load to a compute cycle provider An enterprise configures internal & external
resources to support eBusiness workload
April 18, 2023 Open Grid Services Architecture 8
Intelligent Infrastructure:Distributed Servers and Services
April 18, 2023 Open Grid Services Architecture 9
The Grid World: Current Status Dozens of major Grid projects in scientific &
technical computing/research & education Considerable consensus on key concepts and
technologies– Open source Globus Toolkit™ a de facto standard for
major protocols & services
– Far from complete or perfect, but out there, evolving rapidly, and large tool/user base
Industrial interest emerging rapidly Opportunity: convergence of eScience and
eBusiness requirements & technologies
April 18, 2023 Open Grid Services Architecture 10
Globus Toolkit: Evaluation (+)
Good technical solutions for key problems, e.g.– Authentication and authorization
– Resource discovery and monitoring
– Reliable remote service invocation
– High-performance remote data access This & good engineering is enabling progress
– Good quality reference implementation, multi-language support, interfaces to many systems, large user base, industrial support
– Growing community code base built on tools
April 18, 2023 Open Grid Services Architecture 11
Globus Toolkit: Evaluation (-) Protocol deficiencies, e.g.
– Heterogeneous basis: HTTP, LDAP, FTP
– No standard means of invocation, notification, error propagation, authorization, termination, …
Significant missing functionality, e.g.– Databases, sensors, instruments, workflow, …
– Virtualization of end systems (hosting envs.) Little work on total system properties, e.g.
– Dependability, end-to-end QoS, …
– Reasoning about system properties
April 18, 2023 Open Grid Services Architecture 12
“Web Services” Increasingly popular standards-based framework for
accessing network applications– W3C standardization; Microsoft, IBM, Sun, others
WSDL: Web Services Description Language– Interface Definition Language for Web services
SOAP: Simple Object Access Protocol– XML-based RPC protocol; common WSDL target
WS-Inspection– Conventions for locating service descriptions
UDDI: Universal Desc., Discovery, & Integration – Directory for Web services
April 18, 2023 Open Grid Services Architecture 13
The Need to SupportTransient Service Instances
“Web services” address discovery & invocation of persistent services– Interface to persistent state of entire enterprise
In Grids, must also support transient service instances, created/destroyed dynamically– Interfaces to the states of distributed activities
– E.g. workflow, video conf., dist. data analysis Significant implications for how services are managed,
named, discovered, and used– In fact, much of our work is concerned with the
management of service instances
April 18, 2023 Open Grid Services Architecture 14
Open Grid Services Architecture Service orientation to virtualize resources From Web services:
– Standard interface definition mechanisms: multiple protocol bindings, multiple implementations, local/remote transparency
Building on Globus Toolkit:– Grid service: semantics for service interactions
– Management of transient instances (& state)
– Factory, Registry, Discovery, other services
– Reliable and secure transport Multiple hosting targets: J2EE, .NET, “C”, …
April 18, 2023 Open Grid Services Architecture 15
OGSA Service Model System comprises (a typically few) persistent
services & (potentially many) transient services All services adhere to specified Grid service
interfaces and behaviors– Reliable invocation, lifetime management,
discovery, authorization, notification, upgradeability, concurrency, manageability
Interfaces for managing Grid service instances– Factory, registry, discovery, lifetime, etc.
=> Reliable, secure mgmt of distributed state
April 18, 2023 Open Grid Services Architecture 16
The Grid Service A (potentially transient) Web service with specified
interfaces & behaviors, including– Creation (Factory)
– Global naming (GSH) & references (GSR)
– Lifetime management
– Registration & Discovery
– Authorization
– Notification
– Concurrency
– Manageability
April 18, 2023 Open Grid Services Architecture 17
Factory
A Grid service with Factory interface can be requested to create a new Grid service instance– Reliable creation (once-and-only-once)
Create operation can be extended to accept Grid-service-specific creation parameters
Returns a Grid Service Handle (GSH)– A globally unique URL
– Uniquely identifies the instance for all time
– Based on name of a home mapper service
April 18, 2023 Open Grid Services Architecture 18
Mapper A GSH is a stable name for a Grid service, but does
not allow client to actually communicate with the Grid service
A Grid Service Reference (GSR) is a WSDL document that describes how to communicate with the Grid service– Contains protocol binding, network address, …
– May expire (I.e. GSR information may change) The Mapper interface allows a client to map from a
GSH to a GSR– http get on GSH also returns a GSR
April 18, 2023 Open Grid Services Architecture 19
Lifetime Management GS instances created by factory or manually;
destroyed explicitly or via soft state– Negotiation of initial lifetime with Factory
SoftStateDestruction interface supports– GetTerminationTime message for inquiry
>Notification interface also allows for lifetime notification
– SetTerminationTime message for keepalive Soft state lifetime management avoids
– Explicit client teardown of complex state
– Resource “leaks” in hosting environments ExplicitDestruction interface also available
April 18, 2023 Open Grid Services Architecture 20
Discovery A Grid service instance may maintain a set of
service information– XML fragments encapsulated in standard <name,
type, TTL-info> containers Discovery interface allows clients to query the
Grid service instance for this information– Query operation, plus supporting operations
> Extensible query language support
See also Notification interfaces– Allows notification of service existence and about
service information
April 18, 2023 Open Grid Services Architecture 21
Registry
The Registry interface may be used to discover a set of Grid service instances– Returns a WS-Inspection document containing the
GSHs of a set of Grid services
– Also returns policy associated with the set
– Also available through Discovery interface The RegistryManagement interface allows for
soft-state registration of a Grid service– A set of Grid services can periodically register their
GSHs into a registry service, to allow for discovery of services in that set
April 18, 2023 Open Grid Services Architecture 22
Authorization
Protocol binding handles authentication during invocation of Grid service operation– Gives service URI for authenticated subject
Grid service instance should apply authorization policy on all operations– May be site-, service-, instance-, etc., specific
OGSA defines standard interfaces for remote management of access control policy– OperationAuthorizationManagement
– SubjectEquivalency
April 18, 2023 Open Grid Services Architecture 23
Notification Interfaces NotificationSource for client subscription
– One or more notification generators> Generates notification message of a specific type
> Typed interest statements: E.g., Filters, topics, …
> Supports messaging services, 3rd party filter services, …
– Soft state subscription to a generator NotificationSink for asynchronous delivery of
notification messages A wide variety of uses are possible
– E.g. Dynamic discovery/registry services, monitoring, application error notification, …
April 18, 2023 Open Grid Services Architecture 24
Using OGSAto Construct Grid Environments
Factory RegistryService
FactoryH2R
Mapper
ServiceService Service ...
...
(a) Simple HostingEnvironment
Factory RegistryService
FactoryH2R
Mapper
ServiceService Service ...
...
F R
F M
SS S
F R
F M
SS S
(b) Virtual HostingEnvironment
E2EFactory
E2E Reg
E2E H2RMapper
...
F1
R
M
SS S
F2
R
M
SS S
E2E S E2E S E2E S
(c) Compound Services
In each case, Registry handle is effectively the uniquename for the virtual organization.
April 18, 2023 Open Grid Services Architecture 25
OGSA and the Globus Toolkit Technically, OGSA enables
– Refactoring of protocols (GRAM, MDS-2, etc.)—while preserving all GT concepts/features!
– Integration with hosting environments: simplifying components, distribution, etc.
– Greatly expanded standard service set Pragmatically, we are proceeding as follows
– Develop open source OGSA implementation> Globus Toolkit 3.0; supports Globus Toolkit 2.0 APIs
– Partnerships for service development
– Also expect commercial value-adds
April 18, 2023 Open Grid Services Architecture 26
Globus Toolkit Refactoring
Grid Security Infrastructure (GSI)– Used in Grid service network protocol bindings
Meta Directory Service 2 (MDS-2)– Native part of each Grid service:
> Discovery, Registry, RegistryManagement, Notification
Grid Resource Allocation & Mngt (GRAM)– Gatekeeper -> Factory for job mgr instances
GridFTP– Refactor control channel protocol
Other services refactored to used Grid services
April 18, 2023 Open Grid Services Architecture 27
Summary:Evolution of Grid Technologies
Initial exploration (1996-1999; Globus 1.0)– Extensive appln experiments; core protocols
Data Grids (1999-??; Globus 2.0+)– Large-scale data management and analysis
Open Grid Services Architecture (2001-??, Globus 3.0)– Integration w/ Web services, hosting environments,
resource virtualization
– Databases, higher-level services Radically scalable systems (2003-??)
– Sensors, wireless, ubiquitous computing
April 18, 2023 Open Grid Services Architecture 28
For More Information The Globus Project™
– www.globus.org Grid architecture
– www.globus.org/research/papers/anatomy.pdf
Open Grid Services Architecture (soon)– www.globus.org/research/
papers/ogsa.pdf
– www.globus.org/research/papers/gsspec.pdf
Grid Service Specificationand Demonstration
Steven Tuecke
Mathematics and Computer Science Division
Argonne National Laboratory
April 18, 2023 Open Grid Services Architecture 30
Use of Web Services (1)
A Grid service interface is a WSDL portType A Grid service definition is a WSDL extension
(serviceType) containing:– A set of one or more portTypes supported by
the service
– portType & serviceType compatibility statements, to support upgradability
> For discovery of compatible services when interfaces are upgraded
– Implementation version information
April 18, 2023 Open Grid Services Architecture 31
Use of Web Services (2)
A GSR is a WSDL document with extensions:– Extension to service element to reference
serviceType
– Service element extensions to carry the GSH, and the expiration time of the GSR
A GSH is an URL, with the following properties:– Globally unique for all time
– http get on GSH + “.wsdl” returns GSR
– Can derive GSH to Mapper from it Registry returns WS-Inspection documents