PDC Enabling Science Globus – status & perspective Olle Mulmo.
Post on 18-Dec-2015
216 Views
Preview:
Transcript
PDCEnabling Science
Globus – status & perspective
Olle Mulmo <?????@pdc.kth.se>
PDCEnabling Science
What this talk will cover
• Background: Why we are Where we are today• Where we are today• Where we will go tomorrow
[Insert proper acknowledgement to everyoneI have stolen slides from here]
PDCEnabling Science
Metacomputing...
• A concept conceived of in the (late?) ’80s• 1990’s: Dramatic increase in network bandwitdh
• Opens up for entire new problem areas• Solving ”larger-than-a-supercomputer” problems• Remote visualization• Remote instrumentation• Collaboration (VR)
• 1995:• I-WAY, proof-of-concept demo/project
• 1996:• The Globus Project sees the day
PDCEnabling Science
The Globus approach
• Middleware approach• A set of orthogonal APIs
that applications can use• Command-line tools that
use the APIs (for scripting)• IP hourglass model
• Portability requirements• No perl on the Cray at the time• Poor C++ support• -> Implementation using bourne shell
(sh) and ANSI C only
Diverse global services
Coreservices
Local OS
A p p l i c a t i o n s
PDCEnabling Science
Hot technologies half a decade ago
• Network communication• Raw ATM vs TCP/IP?• Use Nexus to avoid technology lock-in• Already ported to several UNIX flavors• Automatic data conversion• Optimization for local communication (AAL5,TCP,MPI,SHMEM)
• Information Discovery• Queryable, Extensible (”give me a computer with 128M RAM”)• Use LDAP (Lightweight Directory Access Protocol)• Add TTL on all published data• Add heart beats to know the test bed status
PDCEnabling Science
Hot technologies half a decade ago (cont.)
• Security• Hot potato: Local control a must• -> mapfiles (what users running on what accounts)• PKI evolving, big push from SSL• -> let’s use it
• Resource Allocation• No standard: myriads of different local schedulers• -> RSL (Resource Specification Language) + wrappers
PDCEnabling Science
First round: the GUSTO testbed
• 1997: The Globus Metacomputing Toolkit• MDS (Metacomputing Directory Service)• GRAM (Globus Resource Allocation Manager)• DUROC (multi-site coordinated startup)
• No security (plain-text passwords)• No file transfers
PDCEnabling Science
The GUSTO testbed (cont.)
• 1998: Globus release 1.0• Security added: GSI (Globus Security Infrastructure)
• PKI (or Kerberos) below GSS-API• SSL handshakes• Proxy certificate delegation• No encryption due to export laws
• File transfer added:• GASS (fetch files over https:// or file://)
PDCEnabling Science
The GUSTO testbed (cont.)
• ... not really that heavily used, though...
PDCEnabling Science
Lessons learned from Globus v 1.x
• Severe performance problems with MDS• A centralized LDAP server didn’t scale very well• -> MDS-2 framework (GRIS and GIIS)
• Complaints on configuration and build times• You got everything and all the ”flavors”• debug,nodebug, nothreads,pthreads,native threads, with MPI,
without MPI, ...• -> Packaging technology
• One of the multi-site jobs often got stuck in a queue / poor network performance• -> GARA (Advanced reservation)
PDCEnabling Science
Lessons learned from Globus v 1.x
• Users don’t want to hack their favorite applications• Easy access
• They have NT on their desktops• Mass storage accessibility is important • -> Data and replica management work begins
PDCEnabling Science
1999: The year of the Grid
• The Concept of a Grid evolves• Resource sharing & coordinated problem solving in dynamic,
multi-institutional virtual organizations
PDCEnabling Science
1999-now: The Grid boom...
• The Grid book covers a lot of groundin the community
• Lots of grid projects spring to life• Increase in funding• Commercial interests (Microsoft,Sun,IBM,...)
• Grid Forum is founded• ”The place to standardize Grid-related stuff”• Many of the protocols and approaches tried
out by Globus are adopted as a first shot
PDCEnabling Science
Some renaming is required...
• GSI -> Grid Security Infrastructure• GRAM -> Grid Resource Allocation Management• MDS -> Monitoring and Discovery Service
• Never say ”Metacomputing” again...
PDCEnabling Science
Back to the drawing board: Globus v 2.0
• New packaging technology (not RPM)• Precompiled binaries• GridFTP• MDS-2: decentralized LDAP framework• Security on all communications channels
• Repackaged with add-ons by several DataGrid projects• GriPhyN, EDG, NMI, ...
PDCEnabling Science
Where we are today
• The Globus Toolkit™ centers around four key protocols• Connectivity layer:
• Security: Grid Security Infrastructure (GSI)
• Resource layer:• Resource Management: Grid Resource Allocation Management
(GRAM)
• Information Services: Grid Resource Information Protocol (GRIP) and Index Information Protocol (GIIP)
• Data Transfer: Grid File Transfer Protocol (GridFTP)
PDCEnabling Science
A second look
• Many different protocols and methods are used• LDAP, HTTP, FTP, ...
• Missing or lack of support for• Databases• Notification systems• Error propagation
• The security part needs more work• Dynamic, Multi-VO Authorization schemes• Audit• Proxy delegation (IETF draft not likely to become an RFC)
PDCEnabling Science
Let’s go shopping: Hot technologies of today
• XML & Web services• Generic and extensible• Many tools available• Backed up by industry
• Attribute Certificates• Give a certificate holder certain privilegies (access to files,
resources,...)• NMI is pursuing this path• -> Globus ships CAS (similar concept)
PDCEnabling Science
Web services means generic encapsulation
• ”Corba rewritten in XML” (and then some)• WSDL: Web Services Definition Language
• Defines a service and what methods that can be invoked• Stub compilers exist for several programming languages• -> Easy to wrap your old application
• SOAP: Simple Object Access Protocol• Invocation protocol, operating on an instance of a WSDL target
• WS-Inspection and UDDI• Conventions (naming etc.) for querying and locating WSDL
documents with certain properties
PDCEnabling Science
Open Grid Services Architecture (OGSA)
• Web Services for the Grid• WSDLs for some key components
• Factory, Registry, Grid Service• Dynamic properties (lifetime, version compatibility) are added• Open-source implementation in Java, .NET & C
• Guidelines, conventions and best practices• Global naming• Lifetime managment• Notification framework• ...
PDCEnabling Science
OGSA
• Still a draft (continously revised)• Alpha release of Core implementation (Java)
• The Factory and generic Grid Service part
• Base implementation next• Current Globus functionality refactored
as Grid Services• Alpha at the end of this summer
• Globus 3.0 (Q1 ’03?)• Old APIs will still work (or so they say)• ...but the wire protocols are new GT3 Core
GT3 Base Services
Other GridServicesGT3
DataServices
PDCEnabling Science
Next steps: your guess is as good as mine...
• Experiences learned with v2 and v3• Access to tertiary storage and databases• How much can be gotten ”for free” from other WS
initiatives?• Other forms of security infrastructures?
• Passport / Liberty Alliance technologies• ”MyProxy” effort
• Continued work on authorization & delegation
top related