IRNC Special Projects: IRIS and DyGIR

IRNC Special Projects:IRNC Special Projects:IRIS and DyGIRIRIS and DyGIR

Eric Boyd, Internet2October 5, 2011

• International Research Network Connections (IRNC) is an NSF Office of Cyberinfrastructure program to enable collaboration, in the research and education community, on a global scale. – These grants facilitate hardware and software solutions to foster

access to remote instruments, data, and computational resources located throughout the world.

• Internet2 was awarded 2 IRNC Special Projects (SP) awards in May of 2010:– IRIS - The International Research Instrumentation System

• Performance Monitoring using perfSONAR-ps software package– DyGIR - Dynamic Gateway for International Research

• Building dynamic circuits using OSCARS software package

• Internet2 was awarded 1 MRI award in August of 2010:– DYNES –DYnamic NEtwork System

• Deploying equipment to build a distributed virtual instrument2 – 04/22/23, © 2011 Internet2

IRNC Program

Campus

3 – 04/22/23, © 2009 Internet2

Region

4 – 04/22/23, © 2009 Internet2

Country

5 – 04/22/23, © 2009 Internet2

World

6 – 04/22/23, © 2009 Internet2

• Protocol– IDC (DICE)– NSI (OGF NSI WG)– “OpenFlow Peering” (GENI, NDDI)

• Software– OSCARS (Internet2 and ESnet)– OSCARS / DRAGON (Internet2)– AUTOBAHN (GÉANT)– DRAC (NORTEL / SURFNET)– Others …

• Hardware / Operations– Campus / Region (Campuses, Regionals, Internet2 through DYNES)– National Networks (Internet2, Esnet)– International Links (IRNC Program)– NRENs and CRENs (Nation-specific and continent-specific funding)

Components of E2E Dynamic Circuit Infrastructure

7 – 04/22/23, © 2009 Internet2

• Protocol– perfSONAR (OGF NMC WG)– perfSONAR with support for network research slicing (GENI I&M,

NDDI)• Software

– perfSONAR-ps (ESnet and Internet2)– perfSONAR-MDM (GÉANT)

• Hardware / Operations– Campus / Region (Campuses, Regionals, Internet2 through DYNES)– National Networks (Internet2, Esnet)– International Links (IRNC PRONET)– NRENs and CRENs (Nation-specific and continent-specific funding)

Components of E2E Performance Monitoring Infrastructure

8 – 04/22/23, © 2009 Internet2

• Software Development– perfSONAR-ps (IRIS, DOE Office of Science)– OSCARS (DyGIR, DOE Office of Science)

• Hardware– Campus / Region (DYNES)– National (BTOP, DOE)– International Links (IRNC PRONET)

• Network Engineering and Operations– Campus / Region (DYNES, Campus or Regional Contribution)– National (Internet2, ESnet)– International Links (IRNC PRONET with initial help from IRIS, DyGIR)

Funding of OSCARS / perfSONAR-ps

9 – 04/22/23, © 2009 Internet2

• We know how to build operational dynamic services using IDC protocol (which is widely deployed, but not a standard)

• We know how to build operational performance services using perfSONAR protocol (which is widely deployed and nearly a standard)

• OGF NSI WG is developing a standard (can think of it as an evolution of the IDC protocol)

• GLIF is pushing the envelope, promoting functionality and interoperability on a global scale and move to a standard

• OpenFlow is pushing the envelope under the hood, within and outside of GENI– Initially connect OpenFlow “islands” by Layer 2 VLANs using IDC

protocol– Eventually, we’ll figure out what “OpenFlow peering” means

How do the pieces fit?

10 – 04/22/23, © 2009 Internet2

• NSF Grant # 0962704• IRIS will provide a software framework to simplify the task

of end-to-end network performance monitoring and diagnostics– Based on the widely deployed perfSONAR-PS infrastructure

and protocols– Facilitates broader deployment of perfSONAR enabled

resources; increasing the likelihood of diagnostic resources being available along the end-to-end paths

• Integrates with existing deployments on R&E networks (ESnet, GÉANT, Internet2, Regional and NRENs) as well as those maintained by scientific VOs (USATLAS, LHCOPN, eVLBI, REDDnet)

• Will work with IRNC ProNet awardees to customize deployment for target network functionality

• Target end date is April 201311 – 04/22/23, © 2011 Internet2

IRNC SP:IRIS

• perfSONAR-PS v3.2.1 release imminent– New features supporting IRNC Pronet awardees

• Nagios Plug-ins• Improved IPv6 support• Improved bandwidth/latency analysis

• Continue to meet with all awardees to advise on hardware choices and best practices for tool deployments

IRNC SP:IRIS Development Status

12 – 04/22/23, © 2009 Internet2

• NSF Grant # 0962705• DyGIR will provide a component based solution for

scheduling dynamic circuits on IRNC ProNet infrastructure– Utilizes the OSCARS software suite, developed by ESnet– Integrates circuit statistics and networking monitoring via

the perfSONAR-PS framework• Capabilities will integrate with existing backbone networks

(ESnet SDN, GÉANT AutoBAHN, Internet2 ION), as well as emerging campus deployments (DYNES – an NSF MRI Funded effort).

• Will work with IRNC ProNet awardees to customize deployment for target network functionality

• Target end date is April 2013

13 – 04/22/23, © 2011 Internet2

IRNC SP:DyGIR

• OSCARS 0.6 is in testing phase– Release expected Fall, 2011

• Brocade switch was selected for ACE– OSCARS port will (?) be needed, work should be complete 1st

quarter 2012

• OSCARS ported to support OpenFlow (NOX controller)– Demonstrated at SC11 - SCinet Research Sandbox

IRNC SP:DyGIR Development Status

14 – 04/22/23, © 2009 Internet2

• ACE– perfSONAR-PS test points to be available, along with periodic

monitoring to select locations within GEANT– Planned IDC at MANLAN 1st Quarter of 2012

• AmLight– IDC available in the AMPATH exchange for use on International

Links– IDC Peering with RNP established– perfSONAR-PS monitoring available

• GLORIAD– Discussed current DYGIR and IRIS activities– Currently have ability to create VLANs on infrastructure. Looking

to integrate with dynamic circuit networks over next couple of years.

– Exploring current GLORIAD peerings and heavy users to determine what use cases could benefit from dynamic circuits.

– Exploring best way to capitalize on perfSONAR – specifically looking at ways to publish passive flow data currently collected by GLORIAD using perfsONAR protocols

15 – 04/22/23, © 2011 Internet2

IRNC Outreach to ProNET Awardees

• TransLight/Pacific Wave– Established working group to install perfSONAR-PS

monitoring software at endpoints and participants in the region

– Have put out a questionnaire to participants and end sites to determine perfSONAR availability. Some deployments already happening

– Evaluating dynamic capabilities• TransPAC3– Have traded topology information and are determining

what switches in the infrastructure to put under dynamic control. Will likely peer with ION and JGN2.

– Currently has perfSONAR-PS test points, and periodic tests with APAN

16 – 04/22/23, © 2011 Internet2

IRNC Outreach to ProNET Awardees

• Data movement to support science:– Increasing in size (100s of TBs in the LHC World,

approaching PB sizes)– Becoming more frequent (multiple times per day)– Reaching more consumers (VO sizes stand to

increase, more VOs)– Time sensitivity (data may grow “stale” if not

processed immediately)• Traditional networking:– R&E or Commodity “IP” connectivity is subject to

congestion by other users – Supporting large sporadic flows is challenging for

the engineers, and frustrating for the scientists

17 – 04/22/23, © 2011 Internet2

DYNES Motivation

• Solution– Dedicated bandwidth (over the entire end to end path)

to move scientific data– Invoke this “on demand” instead of relying on

permanent capacity (cost, complexity)– Exists in harmony with traditional IP networking– Connect to facilities that scientists need to access– Integration with data movement applications

• Invoke the connectivity when the need it, based on network conditions

• Proposed Deployment:– Software and hardware support spanning domain

boundaries• Campus• Regional• Backbone

– Integration with existing technologies and deployments

18 – 04/22/23, © 2011 Internet2

DYNES Motivation

• What is it?:– A nationwide cyber-instrument spanning ~40 US

universities and ~14 Internet2 connectors• Extends Internet2’s ION service into regional networks and

campuses, based on OSCARS implementation of IDC protocol (developed in partnership with ESnet)

• High-performance file store at sites

• Who is it?– A collaborative team including Internet2, Caltech,

University of Michigan, and Vanderbilt University – Community of regional networks and campuses– LHC, astrophysics community, OSG, WLCG, other

virtual organizations

DYNES Summary

19 – 04/22/23, © 2011 Internet2

20 – 04/22/23, © 2011 Internet2

DYNES Projected Topology (October 2011)

• September 2011• Demonstration of end-to-end Dynamic Circuit

capabilities– International collaborations spanning 3 continents

(South America, North America, and Europe)– Use of several software packages

• OSCARS for inter-domain control of Dynamic Circuits• perfSONAR-PS for end-to-end Monitoring• FDT to facilitate data transfer over IP or circuit networks

– Science components – collaboration in the LHC VO (ATLAS and CMS)

– DYNES, IRIS, and DyGIR NSF grants touted

21 – 04/22/23, © 2011 Internet2

GLIF 2011

22 – 04/22/23, © 2011 Internet2

GLIF 2011 - Topology

23 – 04/22/23, © 2011 Internet2

GLIF 2011 - Participants