The future of Globus (Grid meets Cloud) Ian Foster Computation Institute University of Chicago & Argonne National Laboratory
Mar 27, 2015
The future of Globus(Grid meets Cloud)
Ian Foster
Computation Institute
University of Chicago & Argonne National Laboratory
2
UC/ANL/ISI perspective on Globus
Globus is a community of contributors This presentation is focused on UC/ANL/ISI
planned contributions Globus funding @ UC/ANL
NSF CDIGS: development & support DOE CEDPS: Globus.org data grid R&D caBIG/caGrid: for biomedical applications Many others…
3
UC/ANL leadership and structure
Steve Tuecke resuming day-to-day technical and project leadership
Product teams Adopted Agile scrum methods Jira.globus.org with Greenhopper
User services Consulting, operations and support Led by Paul Dave’
Grid = federation
Cloud = hosting
5
Anatomy of the Grid
Application
Fabric“Controlling things locally”: Access to, and control of resources
Connectivity“Talking to things”: communication (Internet protocols) and security
Resource“Sharing single resources”: negotiating access, controlling use
Collective“Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services
InternetTransport
Application
Link
Inte
rnet P
roto
col A
rchite
cture
“The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Foster, Kesselman, Tuecke, Intl Journal of High Performance Computing Applications, 15(3), 2001.
6
Cloud Taxonomy
Software-as-a-Service (SaaS)
Platform-as-a-Service (PaaS)
Infrastructure-as-a-Service (IaaS)
7
Some strengths of Globus
Grid Security Infrastructure ecosystem GridFTP ecosystem GT4 Java Core use by caGrid Strong community
8
Some issues Facing Globus
Aging technology base Apache Axis 1.x, PureTLS
Sustainability Limited adoption of some components
MDS, RFT Confusing duplication
GRAM2 vs GRAM4 Limited collective layer, end-to-end
functionality
9
Globus ToolkitAssortment of Components for Grid Builders
Focus on Connectivity and Resource layers GRAM, GSI-OpenSSH: Run programs GridFTP: Access file systems OGSA-DAI, caGrid: Access databases GSI, Myproxy, GAARDS: Security XIO, Java Core, C Core: Communication
A few simple Collective layer components RLS: Replica tracking RFT: Reliable file transfer
10
GRAM5
Now you can have reliability and scalability Backward compatible with GRAM2
2 minor exceptions Job rendezvous for MPICH-G, stdout/err streaming
Tested w/ GRAM4 Java & C clients, Condor-G Improved functionality
Exit codes Better logging Enhanced metrics Etc.
11
Reliable File Transfer (RFT)
Replaced by Globus.org
12
MDS
Mostly used for service registries/catalogs Starting new IIS effort with this focus
Better tools (e.g. Nagios) for monitoring
13
Other Components Continuing
GridFTP RLS Myproxy GSI-OpenSSH
GAARDS Introduce OpenNebula Nimbus …
14
Metrics
New metrics collector More reliable and scalable
New metrics generators GRAM5 Myproxy Others coming
15
Globus Toolkit Roadmap
GT 5.0: 4Q2009 GridFTP, GRAM5, RSL, Myproxy, GSI-OpenSSH C libraries: GSSAPI, XIO, C Core, etc jGlobus: Security, GRAM client, GridFTP client
GT 5.2: 2Q2010 Focus: Native Packaging
GT 5.4: 4Q2010 Focus: Usability
GT 4.x maintenance and support will continue at least through end of 2010
16
CruxJava SOA for Grid Applications
Core: Apache CXF + Globus security Also: Introduce + OSGi + ServiceMix ESB WSRF subset compatibility supported Taverna workflows
Replacement for GT4 Java Core Programming model not the same
17
Many custom Collective layers
Most Grid deployments have custom, domain-specific Collective layer Built on common Resource layer components E.g. Data transfer and mirroring, workflows, …
Challenges: Expensive to develop Expensive to operating and supporting Useful for narrow community
How do we make these capabilities available to more users?
18
Globus.org ServiceCollective Layer Grid Services via Cloud SaaS
End-to-end collective layer functionality targeted toward end users Generalize lessons from custom Grids Focus on ease of use, federation
Hosted and supported by Globus team Initial focus on file transfer
Near term: Add sync, mirroring, caching Long term: Add job execution, workflows, VO
management
19
Applying SaaS Techniques
Service: Built as scale-out web application Hosted on Amazon Web Services
Client: Minimize software deployment Web 2.0
AJAX + REST Notification via email, IM, SMS, Twitter, etc. Enable mash-ups
CLI 2.0 ssh cli.globus.org …
Dynamic deployment of resource integrators Mobile clients
20
Web 2.0 GUI
21
iPhone Client
22
Upcoming Events
GlobusWorld: March 2-4 @ Argonne
Globus governance
24
“Governance” issues
Who decides what is called Globus? How do we encourage contributions? How do we encourage the development of
a healthy ecosystem of producers and consumers?
How do we achieve sustainability?
25
Governance history
First, Globus Project Argonne and USC/ISI
Then, Globus Alliance NCSA, Edinburgh, U.Chicago Commitment to cooperate
And now, dev.globus Apache-like governance structure Globus Management Committee Dev.globus incubator process