www.ci.anl.gov www.ci.uchicago.edu Hosted services for managing shared cyberinfrastructure Ian Foster Argonne National Laboratory & The University of Chicago Joint work with Rachana Ananthakrishnan, Josh Bryan, Kyle Chard, Mattias Lidman, Steven Tuecke, and others
I was invited to talk at the 18th GENI Engineering Conference (http://groups.geni.net/geni/wiki/GEC18Agenda) on experiences in the Grid community with creating and operating large shared infrastructures. I chose to focus on our experiences using Software as a Service (SaaS: aka Cloud) to reduce barriers to the use of the capabilities required to create and operate virtual organizations.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
www.ci.anl.govwww.ci.uchicago.edu
Hosted services for managing shared cyberinfrastructure
Ian FosterArgonne National Laboratory & The University of Chicago
Joint work with Rachana Ananthakrishnan, Josh Bryan, Kyle Chard, Mattias Lidman, Steven Tuecke, and others
GENI Engineering Conference, NYC, October 28, 2013
www.ci.anl.govwww.ci.uchicago.edu
Using cloud services to accelerate discoveryIan FosterArgonne National Laboratory & The University of Chicago
Joint work with Rachana Ananthakrishnan, Josh Bryan, Kyle Chard, Mattias Lidman, Steven Tuecke, and others
GENI Engineering Conference, NYC, October 28, 2013
www.ci.anl.govwww.ci.uchicago.edu
3
Cyberinfrastructure
• “a technological and sociological solution to the problem of efficiently connecting laboratories, data, computers, and people with the goal of enabling derivation of novel scientific theories and knowledge” [Wikipedia]
“The Anatomy of the Grid,” 2001 The … problem that underlies the Grid concept is coordinated
resource sharing and problem solving in dynamic, multi-institutional virtual organizations. The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem-solving and resource-brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization (VO).
www.ci.anl.govwww.ci.uchicago.edu
5
Large Hadron Collider
Grid technology accelerates discoveryHiggs discovery “only possible because of the extraordinary achievements of … grid computing”—Rolf Heuer, CERN DG
• User-managed group creation, management• Flexible control over admission policies and visibility• Groups can be used in authorization decisions
26
Example: kBase• Every kBase user
added to kbase_users• Subgroups also
created• Groups used for
access control
I
UV
G
www.ci.anl.govwww.ci.uchicago.edu
27
Group membership interface
27
www.ci.anl.govwww.ci.uchicago.edu
28
Branded sites
Open Science Grid University of ChicagoXSEDE
DOE kBase Indiana University University of Exeter
Globus Online NERSC NIH BIRN
www.ci.anl.govwww.ci.uchicago.edu
29
Implementation and deployment
Elastic Load Balancer
Monitoring
Logging
OSSEC
Nexus
REST APIWeb
Nexus
REST APIWeb
Nexus
REST APIWeb
www.ci.anl.govwww.ci.uchicago.edu
30
Globus Nexus usage as of 9/13
• >12,000 users and 4977 linked identities
• 557 groups totaling:– 1638 active members– 229 pending or
invited members– 162 rejected or
suspended members• Largest group (kbase)
has 402 members
Nov-10
Feb-11
May-11
Aug-11
Nov-11
Feb-12
May-12
Aug-12
Nov-12
Feb-13
May-13
Aug-130
2,000
4,000
6,000
8,000
10,000
12,000
14,000
Tota
l use
rs
1 11 21 31 41 51 61 71 81 91 1011111211311
10
100
1000
Use
rs in
gro
up
www.ci.anl.govwww.ci.uchicago.edu
31
Identities and groups in XSEDE• Proposal: Replace current ad-hoc systems with
Globus Nexus identity and group service– Reduce complexity, reduce cost, increase capability
• Careful process of documentation and review– “Architecture and development requirements: User
and identity management”– “User management proposal: Affected use cases”– “User management proposal: Motivating stories”– “Proposal: Refactoring XSEDE identity and group
capabilities”• Hope to reach closure by end of 2013
www.ci.anl.govwww.ci.uchicago.edu
32
Cloud services to accelerate discovery
Accelerate discovery and innovation worldwide by providing research IT as a service
Leverage software-as-a-service to• provide millions of researchers with
unprecedented access to powerful tools; • enable a massive shortening of cycle times in
time-consuming research processes; and• reduce research IT costs dramatically via