Instrumentation and Measurement Background

Instrumentation and Measurement Background

GENI I&M Motivation

• To conduct useful and repeatable experiments, GENI experimenters require extensive and reliable instrumentation and measurement capabilities to gather, analyze, present and archive Measurement Data (MD).

• In addition, the GENI operations staff require extensive and reliable instrumentation and measurement capabilities to monitor and troubleshoot the GENI suite and its constituent entities. Some of this data will be made available to experimenters, to help them conduct useful and repeatable experiments.

FIBRE KOM - 29/10/11 2

Relationship to GENI Architecture

The Instrumentation and Measurement WG focuses on the instrumentation and measurement infrastructure that will be deployed and used in GENI.

FIBRE KOM - 29/10/11 3

GENI I&M Services Goals (1/3)

• Provide broad data gathering, analysis and archival capability that is sufficient for scientific mission, operations, and success of the infrastructure.

• Remove the burden on researcher to become a system and network measurement infrastructure expert so that researcher can better focus on the science in the experiments

• Measure details of GENI behavior with high precision and accuracy in a ubiquitous, extensible, highly available, secure, and integrated manner without adversely impacting experiments.

FIBRE KOM - 29/10/11 4

GENI I&M Services Goals (2/3)

• Provide drill-down performance transparency of system and network resources at hop, link, path and slice levels in terms of availability, health status, and diagnosis of perceived as well as impending problems.

• Allow and make-it-easy for various user groups to access and control functions involving interactions between I&M sub-services encompassing resources such as instrumentation taps in the network, time sensors, software-based and hardware-based measurement probes, router/switch MIBs, and (short-term/long-term) measurement data archives.

FIBRE KOM - 29/10/11 5

GENI I&M Services Goals (3/3)

• Provide performance transparency of the status of the individual I & M sub-service components and their interfaces with other sub-services to ensure correctness of measurements provisioned.

• Through tight integration with the CFs, provide mechanisms to handle security, privacy and access control of measurement data archives to allow access only to authorized users, and also provide different data views based on authorization privileges.

• Make it easy for data providers (e.g., instrumentation device designers, experiments who implement custom instrumentation devices, etc) to supply data (within legal and ethical limits) to other entities.

FIBRE KOM - 29/10/11 6

GENI I&M User Groups

• Experimenters• Opt-In Users: users who participate or “opt-in” to a GENI

experiment to utilize resources, applications or services.• Central (i.e., GMOC) Operators• Aggregate Providers and Operators• Archive Providers and Operators: Groups that catalog indexes

of GENI-related measurement datasets in a repository and provide tools for users to share, annotate, search and cite the measurement datasets.

• Researchers that use Archived Measurement Data: Users that utilize the measurement datasets provided by Archive Providers in order to test hypotheses, and promote reproducible research.

FIBRE KOM - 29/10/11 7

GENI Key I&M Services

• Measurement Orchestration (MO) service• Measurement Point (MP) service• Measurement Information (MI) service• Measurement Collection (MC) service• Measurement Analysis and Presentation (MAP) service• Measurement Data Archive (MDA) service

FIBRE KOM - 29/10/11 8

GENI I&M Services Overview

Wide range ofMP Services

ExperimenterOperator

Researcher

MP

MP

MPs

MAP

MI

MC

MC MAP

MDA

MO

Measurement Orchestration (MO) Service

FIBRE KOM - 29/10/11 10

Measurement Point (MP) Services

FIBRE KOM - 29/10/11 11

Measurement Point (MP) Services

FIBRE KOM - 29/10/11 12

Measurement Point (MP) Services

FIBRE KOM - 29/10/11 13

Measurement Information (MI) Service

FIBRE KOM - 29/10/11 14

Measurement Collection (MC) Services

FIBRE KOM - 29/10/11 15

Measurement Analysis and Presentation (MAP) Service

FIBRE KOM - 29/10/11 16

Measurement Data Archive (MDA) Service

FIBRE KOM - 29/10/11 17

Experimenter Gathering MD from his/her Slice

Experimenter

MP

MP MC

MC MAP

MO

MDA

Experimenter Gathering MD from his/her Slice

FIBRE KOM - 29/10/11 19

Operator Gathering MD from GENI Infrastructure

Operator

MP

MP

MAP

MI

Operator Gathering MD from GENI Infrastructure

FIBRE KOM - 29/10/11 21

Experimenters Gathering MD from their Slice and from GENI Infrastructure

FIBRE KOM - 29/10/11 22

Experimenters Gathering MD from their Slice and from GENI Infrastructure

FIBRE KOM - 29/10/11 23

GENI I&M Architecture Overview:Types of Services

• Type 1 Service: Service contained within a slice.

• Type 2 service: Common service platform plus multiple slivers dedicated to multiple experiments.

• Type 3 service: Common service with shared data provided to multiple experiments.

• Type 4 Service: MDA service with a portal for sharing data

FIBRE KOM - 29/10/11 24

perfSONAR

PERFormance Service Oriented Network monitoring ARchitecture

Analysis & Visualization

Measurement Infrastructure

Data Collection Performance

Tools

Analysis & Visualization

Measurement Infrastructure

API

API

perfSONAR as a Middleware

FIBRE KOM - 29/10/11 26

Measurement Points

Data Services

MeasurementArchives

Transformations

Service Configuration

Auth(n/z)Services

Infrastructure

Information Services

Topology

Service Lookup

Analysis/Visualization

User GUIs

Web Pages

NOC Alarms

perfSONAR Architecture Overview

FIBRE KOM - 29/10/11 27

perfSONAR

• Base network measurement schema– OGF Network Measurement Working Group (NM-WG)

• Topology Schema– OGF Network Markup Language (NML-) WG– Includes Topology Network ID

• perfSONAR Protocol Documents– OGF Network Measurement and Control (NMC-) WG

FIBRE KOM - 29/10/11 28

MonIPÊ Infrastructure

FIBRE KOM - 29/10/11 29

Visualização dos Resultados

30

Results visualization

Results visualization

• Backbone transition from 2.5 to 10Gbps

FIBRE KOM - 29/10/11 32

perfSONAR-MDM

• Developed within the scope of the GÉANT project• Written in Java, was designed as a monitoring solution for LHC

(Large Hadron Collider).• Available in Debian or RPM packages

• https://forge.geant.net/forge/display/perfsonar/Home

FIBRE KOM - 29/10/11 33

perfSONAR-MDM 3.3 Components

• RRD Measurement Archive• SQL Measurement Archive• SSH/Telnet Measurement Point• Lookup Service• Authentication Service and Authorization Service• OPPD (previously called BWCTL) Measurement Point) • E2EMon Measurement Point

FIBRE KOM - 29/10/11 34

perfSONAR-PS

• http://psps.perfsonar.net/• Collaboratively developed by:

– ESnet, Fermilab, Georgia Tech, Indiana University, Internet2, SLAC, The University of Delaware

• Written in Perl. Available for independent deployment or through the Performance Toolkit (pS-PT).

FIBRE KOM - 29/10/11 35

pS-PT

• perfSONAR Performance Toolkit (pS-PT)• Current version: 3.2.1

– LiveCD or Net-Install based on CentOS 5.5– Easy to deploy

• Boot with the instalation CD• http://psps.perfsonar.net/toolkit/

– User-friendly Web Interface• https://pspt.nuperc.unifacs.br

FIBRE KOM - 29/10/11 36

FIBRE KOM - 29/10/11 37

perfSONAR-OS Services

• SNMP Measurement Archive• perfSONAR-BUOY• perfAdmin GUI• Lookup Service• Topology Service• Status Measurement Archive• PingER Services• PingER GUI• LS Registration Daemon

FIBRE KOM - 29/10/11 38

pS-PT

• Metrics– Throughput– Delay– Ping– SNMP Utilization– Cacti graphs with host performance information – Scheduled tests:

• Throughput, one-way delay, and ping.

FIBRE KOM - 29/10/11 39

LAMP

• Leveraging and Abstracting Measurements with perfSONAR• perfSONAR is a multi-domain performance monitoring

framework, which defines a set of protocols standards for sharing data between measurement and monitoring systems.

• Objectives– Collaborate on defining a common but extensible format for

data storage and exchange.• Use perfSONAR NM-WG schema as starting point

– Develop a representation of GENI topolgy to be used to describe measurements and experiment configuration

– Collaborate on a common GENI I&M architecture

FIBRE KOM - 29/10/11 40

perfSONAR-based GENI I&M

• Operations & Management:– Federated GENI is perfect match to perfSONAR– perfSONAR SNMP MA setup to query ProtoGENI switches on

Internet2• Experiments:

– perfSONAR architecture is modular and extensible– Integration with GENI will need

• Measurement Portal• Measurement Orchestration

FIBRE KOM - 29/10/11 41

Onelab Monitoring

MySlice

FIBRE KOM - 29/10/11 43

TopHat (Topát)

• http://www.top-hat.info/• PlanetLab Europe’s active measurement component• Part of OneLab’s effort: global federated environment• TopHat provides PlanetLab applications with a topology

monitoring service for the entire lifecycle of an experiment.• Uses DIMES and ETOMIC for specialized Services

FIBRE KOM - 29/10/11 44

Experiment Lifecycle Support

• Experiment set up:– Assist users in choosing the nodes for their experiments

based on measured characteristics of the network• Run time:

– Provides info to support adaptative applications and experiment control.

• Post-morten data:– Data are archived for further analysis.

FIBRE KOM - 29/10/11 45

Topology Information

• Sources at IP level:– BGP feeds:

• RouteViews, Team Cymrum, and PWhoIs– Active measurements:

• ScriptRoute (PL native)• Other: eg, perfSONAR

FIBRE KOM - 29/10/11 46

TopHat and perfSONAR

• Measurement tools:– “perfSONAR offers a uniform set of tools from a set of peer entities”– TopHat federates heterogeneous systems

• Focus:– perfSONAR focus on troubleshooting across network boundaries– TopHat focus on experiment support

• TopHat could benefit from accessing router information via perfSONAR (queue lengths and packet drop rates).

FIBRE KOM - 29/10/11 47

TopHat Architecture

perfSONAR

FIBRE KOM - 29/10/11 48

Core API

• Get: – request information and measurements about nodes and

paths• Filter:

– filters a ser of nodes or paths• SetCallback:

– Configures conditions on which the system will react by triggering a callback function

FIBRE KOM - 29/10/11 49

DIMES

• www.netdimes.org • distributed scientific research project, aimed at studying the

structure and topology of the Internet, with the help of a volunteer community (similar in spirit to projects such as SETI@Home).

• Traceroute measurement module• Large number of vantage points (ca. 1700 measurement

agents)

FIBRE KOM - 29/10/11 50

ETOMIC

• Infrastructure with GPS synchronized servers.• With special cards for a precision of 10 ns• Performs one way delay and available bandwidth

measurements pre-scheduled tests• Deployment on PlanetLab and other points

• Similar to the results produced by OWAMP and BWCTL MPs in perfSONAR.

FIBRE KOM - 29/10/11 51

SONOMA

• On-demand tests• Precision on the order of ten’s of microseconds• Uses web services for test requests

FIBRE KOM - 29/10/11 52

OML

The Orbit Measurement Library

OML

• A client library (liboml2) for instrumenting your applications; plus

• A measurement server for collecting and storing measurements, remotely.

• Tutorial: http://groups.geni.net/geni/attachment/wiki/Gec9InstMeasWorkshopAgenda/OML_tutorial.pptx

54FIBRE KOM - 29/10/11

Monitoring in OFELIA

Monitoring in OFELIA (D5.1)

• The control framework should perform monitoring tasks, for both the components conforming the control framework, and the resources on the testbed.

• Main focus: monitoring resource usage in order to avoid congestion and guarantee slices isolation.

FIBRE KOM - 29/10/11 56

Monitoring in the FIBRE-BR testbed

Initial ideas/proposals

Initial Ideas

• Ideally we would monitor:– Experiments (slices)– Infrastructure (Testbed)

• User groups:– Experimenters– Central Operators– Aggregate (Island) Managers

• Initial proposals:– Leverage on existing monitoring– Federation through perfSONAR

58FIBRE KOM - 29/10/11

Existing Monitoring by Aggregate

• OF Aggregate:– Flow information:

• SNMP, Controller (and FlowVisor) info, directly from OF API• PlanetLab Aggregate:

– TopHat, other tools• OMF Aggregate:

– OML• ProtoGENI Aggregate:

– LAMP

FIBRE KOM - 29/10/11 59

Initial Ideas

• Adapting LAMP to the FIBRE CMF• Collect and Export OpenFlow Data• Integration of OML to perfSONAR

FIBRE KOM - 29/10/11 60

Adapting LAMP to the FIBRE CMF

• LAMP was developed for ProtoGENI• We plan to study and possibly adapt it for use with the FIBRE-

CMF

61FIBRE KOM - 29/10/11

Collect and Export OpenFlow Data

• Information about slices, queues, response time, flow aggregation and flow setup time, not limiting the monitoring framework to these parameters.

• OpenFlow switch table and port statistics can be collected at its controller, e.g., NOX.

• The collected data will be made available through a measurement archive (MA) and shared to other monitoring tools.

FIBRE KOM - 29/10/11 62

Integration of OML to perfSONAR

• OML main feature is the possibility to add Measurement Points (MP) to applications or services and it is a researcher’s decision to enable it at run-time [White, 2010].

• OML focuses mainly on mobile networks and FIBRE proposed monitoring framework could benefit from this expertise and could integrate this service to perfSONAR, at least through a MA interface to OML’s server.

• It is also interesting to analyze and monitor the communication between the client and the server in FIBRE’s federated environment.

FIBRE KOM - 29/10/11 63

Interconnection Monitoring

• connections among FIBRE-BR Islands, FIBRE-EU islands, and intercontinental link

• routers SNMP data (wherever available).• [passive measurements ?!]

64FIBRE KOM - 29/10/11