Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing

1/40

Introduction to perfSONAR

perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring

Szymon Trocha, Poznań Supercomputing and Networking Center, Poland

26th APAN Meeting, 4-8 August 2008, Queenstown, New Zealand

2/40

• „Internet measurements is fun”

• „We need to share our data”

N.Brownlee, „Measuring Internet Evolution - If We Don't Measure, We Don't Know What's Happening!”, APAN26

3/39

Performance Measurement Goals

• Increase network awareness• Set user expectations accurately

• Reduce diagnostic costs• Performance problems noticed early • Performance problems addressed efficiently• Network engineers can see & act outside their

domains

• Transform application design• Incorporate network intuition into application

behavior

4/40

Why perfSONAR ?

5/39

Network Boundaries

• Users un-interrupted communication • Network’s administrative domain

• perfSONAR offers• Standard access to the information• Access to already deployed tool instances • GUIs to present and analyse the information

• Benefits• Better understanding, better ownership• Reduces users downtime, increase user

satisfaction

6/39

Quick answers and solutions

• Diagnosing takes time

• After some time, the user moves on• Lost opportunities• Problem disappear

• perfSONAR offers• The same view to the operators• Pre-authorised access

• Benefits• Retention/creation of opportunities

7/39

Resources Discovery

• Data is there

• Difficult to find measurement capability

• perfSONAR offers• Discovery mechanism

• Benefits• Save staff time

8/39

Bringing Evidence

• Difficulty to bring evidences about the existence or non-existence of a problem

• perfSONAR offers• Access to on-demand and historical information

• Delay• Packet loss

• Interface errors and discards

• Throughput transfers

• Benefits• Reduce the realm of uncertainty for the user• Improve staff and user satisfaction

9/39

Applications

• User interest in what they use the network for

• Network Operators for the network itself

• perfSONAR offers• Access to historical TCP throughput transfers and

delay information

• Benefits• Provide a better understanding about what the

user experiences

10/39

Vision: Performance Information is …

• Available• People can find it• “Community of trust” allows access across administrative

domain boundaries

• Ubiquitous• Widely deployed

• Paths of interest covered• Reliable

• Consistently configured correctly and maintained

• Valuable• Actionable

• Analysis suggests course of action• Automatable

• Applications act on data

11/40

Use Cases

12/39

Use Cases

• Backbone• Collected from Internet2, GÈANT2

• State and regional networks• Collected from MCNC (US)

• Application community• Collected from LHC

13/39

Backbone Use Case (Internet2 network)

• Traditional requirements• Collect multiple performance metrics over a wide geographic area

(passive/active)

• Alarms (real-time analysis for ‘exceptional’ conditions)

• Ad-hoc measurements by ‘peers’• Data requests by ‘peers’

• Contemporary additions• Topological correlations

• Spatial – node and link connections, including inter-domain ones• Layer – Map L3 paths to lower-level infrastructure

• Federated administration• Need ability to share information and even control (with adequate authorization)• This ability must extend beyond direct peers

• Automation• User applications require more direct access to topology/performance information

14/39

Backbone Use Case (GÈANT2 network)

• NOC requirements• Bring evidence about where the problem is

located• Give indication if the problem is rather on the

network or on the end-site• Monitor e2e links for specific projects

• PERT requirements• Multi-domain access to already deployed and well

known tool instances that can be started on demand

• Have a trustable baseline against which data results can be compared• TCP throughput

15/39

State and regional Use Case (MCNC)

• Simple deployable measurement systems in edge networks• Some local administration should be possible• Remote (centralized) administration and data

collection is required• Performance and capacity planning

• Challenges• K12 network admin skill levels/time availability• Middleboxes (firewalls/NATs)• Outdated hardware

See: http://www.internet2.edu/presentations/spring08/20080423-netperf-hunt.pdf

16/39

Application CommunityUse Case (LHC)

• Autonomous system per-site is required• Sites must be able to monitor from their site to

other sites of interest (Tier-2 want to actively probe upstream Tier-1s and downstream Tier-3s)

• Must allow for multiple levels of participation from sites

• Must have ability to determine what diagnostics or tools are available from remote sites

• A formal plan is being developed within the US-LHC community and the Internet2 HENP SIG

See: https://wiki.internet2.edu/confluence/display/PSPS/Tier-2-BCP *draft*

17/40

perfSONAR Overview

18/39

What is perfSONAR

• A collaboration• Production network operators focused on designing and

building tools that they will deploy and use on their networks to provide monitoring and diagnostic capabilities to themselves and their user communities

• An architecture & a set of protocols• Web Services Architecture• Protocols based on the Open Grid Forum (OGF) Network

Measurement Working Group (NM-WG) Schemas

• Several interoperable software implementations• Java, Perl, Python…

• A deployed measurement infrastructure

19/39

Proud of Our Collaborators

20/39

perfSONAR Protocols

• Get perfSONAR protocols standardized within the Open Grid Forum -NMC Working Group

21/39

perfSONAR Architecture

• Interoperable network measurement middleware (SOA)• Modular• Web services-based• Decentralized• Locally controlled

• Integrates• Network measurement tools and archives• Data manipulation• Information Services

• Discovery• Topology• Authentication and authorization

• Based on• Open Grid Forum Network Measurement Working Group

schema

22/39

perfSONAR Client Developments

• Most tuned to specific services currently

• Different user focus (micro vs macro view)• These represent what is possible

• Data is available• Location is known (due to LS)• It is expected that individual application

communities and NOCs will want something more tuned to what they care about

23/39

Available in production

• MDM 3.0 release• Bundle of perfSONAR MDM installers comprising

Java and Perl web services as well as an admin guide

• Originated in GÈANT2

• PS 0.9 release• Perl implementation including CPAN distribution• Originated in Internet2 and UoD

• Visualization toolsSee: http://www.perfsonar.net/download.html

24/39

perfSONAR is active

• Developers meetings• Regular calls• Service desk (EU)

• User support

• Monitoring

• Publications• http://wiki.perfsonar.net/jra1-

wiki/index.php/PerfSONAR_and_JRA1_Publications

25/40

Example Tools

26/39

perfSONAR Tools Examples

• Client applications• perfSONAR UI (ISTF)• CNM (DFN)

• Web Based• GMAPS (SLAC)• DCN Administrative Tool (Internet2)• perfAdmin (Internet2)• pingER Analysis (FNAL)• perfSONAR-BUOY (Internet2)• E2E Link Monitoring (DFN)

• Different ways of using infrastructure and presenting data

27/39

perfSONAR UIhttp://perfsonar.acad.bg/perfsonar.jnlp

• Utilization / errors / discards

• Throughput• Looking Glass• Flows• Circuits

• Developed by ISTF

28/39

Gmaps (1/3)http://packrat.internet2.edu:8008/

• Utilization

• Developedby SLAC

29/39

Gmaps (2/3)http://packrat.internet2.edu:8008/

• ping

30/39

Gmaps (3/3)http://packrat.internet2.edu:8008/

• BWCTL

31/39

DCN Administrative Toolhttps://dc211.internet2.edu/cgi-bin/dcn2.cgi

• TopologyService

• DevelopedbyInternet2

32/39

perfAdmin (1/2)https://dc211.internet2.edu/cgi-bin/perfAdmin.cgi

• BWCTL

• DevelopedbyInternet2

33/39

perfAdmin (2/2)https://dc211.internet2.edu/cgi-bin/perfAdmin.cgi

• PingER

34/39

pingER (1/2)http://tukki.fnal.gov/pinger/pingerUI.pl

• Ping

• Developedby FNAL

35/39

pingER (2/2)http://tukki.fnal.gov/pinger/pingerUI.pl

• Delay• Loss

• IPD

36/39

perfSONAR-BUOY (1/2)http://owamp.net.internet2.edu/owamp_grid.cgi

• OWAMP• BWCTL

• Developed by Internet2

37/39

perfSONAR-BUOY (2/2)http://owamp.net.internet2.edu/owamp_path.cgi/LAT4/HOUS/CHIC/now

38/39

CNMhttp://wiki.perfsonar.net/jra1-wiki/index.php/CNM

Access restricted to project participants

• Utilization

• Developedby DFN

39/39

e2e Link Monitoring

Access restricted to project participants

• Path status

• Developed by DFN

• Multiple organizations providing data

40/39

http://www.perfsonar.net

[email protected]