User-Perceived Performance Measurement on the Internet

Bill Tice

Thomas Hildebrandt

CS 6255

November 6, 2003

Introduction

The Internet is different from LANs– Diversely administered– Users are relatively distant from the network

administrators– IP is the only service provided by the network– SNMP or something like it? No…

Overview

Why user-perceived performance measurement is important and also difficult

Existing solutions to the global network measurement problem– NIMI– E2E piPEs– Client-side proxies and other solutions

Performance Measurement on the Internet

IP was designed around the end-to-end argument: certain functions are not required at low levels of a system [1]

As a result, we have a “dumb network” with no built-in performance measurement architecture

The tools we can count on are limited

Performance Measurement on the Internet: Example Tools

Packet Internet Groper (ping) – Useful to test connectivity and RTT between hosts

Traceroute – Provides an approximation of the network topology in the forward direction

Iperf – Measures TCP and UDP performance, including bandwidth, delay jitter and packet loss

Performance Measurement on the Internet

Ping and traceroute rely on ICMP, which is increasingly being blocked by network administrators

Active measurement based on the use of common network services is more robust and perhaps more realistic– DNS queries– HTTP requests

Performance Measurement Architectures

There have been projects to create measurement architectures either by deployment at nodes within domains or at the users– NIMI– E2E pIPEs– AMP, Medusa Proxy, Liston Proxy, Network

Weather Service …

NIMI(National Internet Measurement Infrastructure)

Software system for building network measurement infrastructures

Diversely administered Facilitate many kinds of measurements Extensible and Modular

NIMI Architecture

Measurement servers (probes)– 2 daemons: nimid and scheduled– Measurement Modules

NIMI Architecture

Configuration Point of Contact (CPOC)– Configuration and Control Servers– Several per domain– Delegation of configuration access

NIMI Architecture

Measurement Client (MC)– Only ‘user’ point of contact with NIMI system– Measurement requests

NIMI Architecture

Data Analysis Client (DAC)– Data Collection– Post-Processing– Run at central location or at MC

NIMI Measurement Modules

NIMI has no knowledge of measurements– Plug-ins

Wrapped for a standard API Current Modules

– traceroute, treno, zing, mflect, traffic, ftp

NIMI Measurements

Request Received from MC– Access Control List

scheduled creates pending measurement Results sent to DAC by nimid All communications encrypted

NIMI Deployment

35 hosts (as of 2001)– Research laboratories and Universities

Georgia Tech

Dead?

NIMI Difficulties

Laboratory Conditions– High bandwidth– Dedicated Resources– Not representative of Internet as a whole

NIMI Difficulties

Meaningful data for User Perceived measurements– Not widely distributed

NIMI Difficulties

Hosts administration difficulties– Tools requiring privileged access– Updates

NIMI Difficulties

Difficult to distribute– What’s in it for me?– Not attractive to average user

E2E piPEs

End-to-end Performance Improvement Performance Environment System– A framework to indicate performance capabilities

and locate performance problems along the path between two computers connected by the Abilene network

E2E piPEs Architecture

E2E piPEs Architecture

OWAMP

One-Way Active Measurement Protocol: An Internet2 project

A UDP-based protocol to precisely measure network characteristics:

– Loss– Delay– Jitter

http://owamp.internet2.edu/ - Under construction

E2E piPEs Status

“The initial deployment, which includes the Abilene backbone network and two campuses only, is scheduled for Fall 2003.”

“piPEfitters” are still developing the system – one suggestion is to place PMPs at the end hosts.

Other Tools: Proxy-Based

Liston Proxy– Between browser and the Web– Handles DNS resolution and content requests– Logs information of interest

DNS responsiveness Response time

Proxy-Based Tools

Medusa Proxy– Cool name– Monitors performance

DNS Akamai edge servers vs. origin servers

Proxy-Based Disadvantages

Privacy issues Overhead Limited Scope

Distributed Tools

AMP Network– Distributed physical nodes– High Performance Computing (HPC)

Network measurement, not user-perceived

Distributed Tools Disadvantages

Wide Distribution– Need large amounts of diverse data– How to do it

Updates Security

Applications

Network Weather Service– Predicting network performance for

applications– Can be run by user to predict their

performance

Applications

Lots– Zing– Ping– IPerf– Traceroute

NM Applications Disadvantages

Limited Scope Generally Stateless

User-Perceived Performance Measurement: Why?

Because the Internet was designed following the end-to-end principle, end-to-end performance is ultimately the most meaningful to measure

It is difficult to deploy a measurement architecture in the Internet backbone

Network users see end-to-end performance directly and could be effectively used as monitor points

User-Perceived Measurement: How?

The infrastructure– NIMI-like plug-in measurement modules– Standardized communication between

components

User-Perceived Measurement: How?

Distribution– Have to make it something users want to run

User-Perceived Measurement: How?

Processing– Centralized data collection and post-processing

References

[1] J.H. Saltzer, D.P. Reed, and D.D. Clark. “End to End Arguments in System Design.” http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.txt

[2] Vern Paxson, Andrew Adams, and Matt Mathis. “Experiences with NIMI.” In Proceedings of Passive and Active Measurement, 2000. http://citeseer.nj.nec.com/paxson00experiences.html

[3] Internet2. E2E piPEs. http://e2epi.internet2.edu/E2EpiPEs/e2epipe_index.html

[4] Richard Liston and Ellen Zegura. “Using a Proxy to Measure Client-Side Web Performance.” Proceedings of the 6th International Web Caching and Content Distribution Workshop, Boston, MA, June 1999. http://www.cc.gatech.edu/~liston/pubs/proxy_wcw01.ps.gz

Network Weather Service http://www.npaci.edu/envision/v15.2/nws.html