© Ton van Alebeek Quantum Leap in Open Source Collaboration Bridging the gap between campus infrastructures Harold Teunissen et al. April 2012 - #I2SMM12
Nov 29, 2014
© Ton van Alebeek
Quantum Leap in Open Source Collaboration
Bridging the gap between campus infrastructures
Harold Teunissen et al.April 2012 - #I2SMM12
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Cyberinfra in the Netherlands
2
• All ICT activities for Higher Education and Research in the Netherlands are brought under one umbrella
Shared Professional and Educational Services
Scientific Computing & Storage
Commercial ICT Products & Services
National Research & Education Network
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Big Data — when size does matter
3
• More is not less...- Large scale and global research - Collaboration in Virtual Organizations- Shared resources- High dependence on ICT
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Collaborations and Partnerships
4
• Virtual Research Team- Focussed on doing research- Small scale - Temporary and elastic- Little ICT awareness
• “Virtual” Infrastructure Provider- Focussed on providing infrastructure for specific discipline- Well-organized with grands and budgets- Longer term collaborations and better ICT awareness- Not virtual (e.g. CLARIN, Lifewatch, LHC, Project Bamboo)
• Virtual Collaboration* = People + Groups + Resources + Multi-Disciplines + Services + Policies + Funding + ...
Virtual Laboratory
Virtual Project
Virtual Colla-boration
Virtual Breeding Environment
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Researchers knocking on our door
5
Radio Astronomy — Pulsar Research
Climate Modeling for Scientist and Decision Makers
CineGRID — 4K+ Video Distribution Testbed
Next Generation DNA SequencingJungle Computing and Multi-Model Multi-Kernel Simulations
Centralized Imaging for Large Scale Population Imaging Studies
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
e-VBLI relies on intl. networks
6
• DNA reads of 750 individuals, 300 TB data• Creating a “reference genome” and looking at variations
in individuals to find cause of diseases• Hospitals generate data, and carry out the 1st analysis
• Next Generation Networking for Next Generation Sequencing!
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Genomics in the Netherlands
7
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA 8
Virtual Collaboration B
Virtual Collaboration A
Multi-Disciplinary Collaboration
ƒ
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA 8
Enabled by
Multi-Disciplinary Collaboration
ƒ
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
e-Research without limits
9
Network C
Network BNetwork A
Open Lightpath Exchange
ƒ
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Enabled by
e-Research without limits
9
ƒ
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Circle of Pain...
10
Relationship
AuthorizationAccess Control
UsersVirtual Collaboration
Resources & Services
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Multi-Domain Service Delivery
11
Virtual Collaboration A
Domain II Domain III Domain IV Domain V Domain VI
Domain VIIDomain I
Key words — trust, mutual understanding and standardization!
S3
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Multi-Domain Service Delivery
11
Virtual Collaboration A
Domain II Domain III Domain IV Domain V Domain VI
Domain VIIDomain I
Key words — trust, mutual understanding and standardization!
S3
What happens if domain is removed
or added?
What happens if domain is removed
or added?
What happens if a domain is removed, added or modified?
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
OpenConext
• Provides the glue and lubrication middleware to make campus, cloud based services and resources usable for collaboration for both inter campus and virtual collaboration scenarios
• OpenConext o!ers the Identity and Group Infrastructure
• O!er the platform software for large, virtual collaboration-type collaboration to run-you-own
• OpenConext allows you to create your own collaboration environment that organizes your applications, services and resources
12
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA 13
Core Components
• Federated Identity Management — SAML• Groups Management — Grouper• Social Network 'Portal' technology — OpenSocial• Collaboration Services and Resources
- Document Sharing- Video Collaboration- Learning Systems- Data Storage- Data Mining- Workspaces- Workflows- Et cetera
9000CALORIES
100% SATISFACTION
Virtual Organization
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Eco-system and supply chain
14
VirtualCollaboration
DNA Sequencer
Research Data StorageCollaboration Portal
ƒ
Bandwidth on demand
09 12 14
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Dynamic Network Service 1.0
15
Application
Network Management System
BoD GUI
NOC Engineer
Admin GUI Legacy API
Users
• User Management• Resource Management• Topology & Discovery• Path finding• Scheduling & Reservation• Network Element
Provisioning & Association
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Resource Reservation 101
16
Network C
Network BNetwork A
Open Lightpath Exchange
ƒ
Bandwidth on Demand
258am-10pm
APRIL
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Resource Reservation 101
16
Network C
Network BNetwork A
Open Lightpath Exchange
ƒ
Bandwidth on Demand
258am-10pm
APRIL
2610am-12pm
APRIL
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Resource Reservation 101
16
Network C
Network BNetwork A
Open Lightpath Exchange
ƒ
Bandwidth on Demand
258am-10pm
APRIL
2610am-12pm
APRIL
2612.10pm-3pm
APRIL
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Dynamic Network Service 2.0
17
NOC Engineer Users Applications
Network Management System
Bandwidth on Demand ServiceNSI
NSIBoD GUIAdmin GUI
CIENA NMS
NSI
NOC Engineer
Admin GUI
• Topology & Discovery• Path finding• Scheduling & Reservation• Network Element
Provisioning & Association
• Group Management• Resource Management
Inter Domain
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
New SURFnet7 and BoD
18
• All Institutes are upgraded to a MultiService Port• Allows for On Demand Lightpath Services• Provide connectivity towards Cloud providers and
Netherlight• O!ers fixed lightpath Quality of Service (bandwidth,
latency, availability, et cetera)• Separation of lightpaths through VLAN tagging
10 Gbps MultiService Port
2 Gbps 2 Gbps 4 Gbps 2 Gbps
Lightpath Lightpath On Demand Lightpath Available
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA 19
Fusion between users and services
Institute X
BoD Admin Group
Physical Resource Group X
Physical Resource Group Y
BoD Physical Ports
Bandwidth on Demand Service
Institute Y
Physical Resource Group Z
Institute Z
VirtualUser Group
Virtual Collaboration
Virtual Resource Group
Admin
Users
+ + =
Institute’s Admin
Ports are allocated to Institutes...
Institute’s Admin Institute’s Admin
...and can be combined into a Virtual Resource
Group and then used by the collaboration
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA 20
Architecture
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Simplified view
21
SAML2 Federation
SAML2 Identity ProviderADFS TBD
ConextEngine
JANUS
OpenSocialContainer
Web Portal
Service Registry
Policy Management
Attribute Management
Group Management
OpenSocialRest
XACML
TBDGadget
Grouper
OpenSocial RestOAuth
SAML2 SP SAML2 Metadata
Bandwidth on Demand Service
* Developed by SURFnet
Network Management System
CIENA NMS
NSI
NGE / Network
NGE
XACML
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
BoD Service — Work in Progress
22
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Virtual Ports Assignment
23
Bandwidth on Demand
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Virtual Ports Overview
24
Bandwidth on Demand
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Reservations
25
Bandwidth on Demand
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
Physical Ports
26
Bandwidth on Demand
Quantum Leap in Open Source Collaboration - I2SMM12 - Arlington, VA
BoD uptake in the Netherlands
27
Institute # of sites
Genome of the Netherlands 12 (2 Intl)
Population Imaging Studies 3
Visual Analysis of Flooding Scenarios 3
Connecting climate model data 3
Distributed Multi-Model / Multi-Kernel Simulations 3
Proteomics 4
CineGrid NL 4
Pulsar Research 3
FOM Rijnhuizen 2
Visualization of climate data 2
Visionair 2
DAS-4 5
NEXPReS 5+ (intl)