University Computing Services EDUCAUSE Mid-Atlantic Regional Conference 16 January 2003 An Infrastructure and Accounting Response to Peer to Peer Traffic.

University Computing ServicesUniversity Computing Services

EDUCAUSE Mid-Atlantic Regional Conference16 January 2003

An Infrastructure and An Infrastructure and Accounting Response to Accounting Response to

Peer to Peer Traffic VolumePeer to Peer Traffic Volume

Dr. Michael R MundraneDirector of Telecommunications

Rutgers University Computing Services

CopyrightCopyright

Copyright Michael R Mundrane 2003. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

AgendaAgenda

• Statement of Problem

• Objectives

• Approach

• Results

• Conclusions

Statement of ProblemStatement of Problem

Is he kidding? P2P is the problem!

Network EvolutionNetwork Evolution

• Sporadic

• Unequally funded

• Unstructured

• Immediacy

• Complex

• Point services

• Faculty centric

Application ModelsApplication Models

• Limited customer interface • Minimal administration• Centralized management• Centralized storage• hub and spoke infrastructure• Minimal bandwidth

Terminal Host

• Rich customer interface

• Medium administration

• Centralized management

• Hybrid storage (server and client)

• Tiered network infrastructure

• Bandwidth server/s dependant

Client Server

• Rich user interface

• High touch administration

• Distributed management (costly)

• Distributed storage (difficult to manage)

• Complex mesh infrastructure

• High bandwidth

Peer Peer

Cooperative?!?Cooperative?!?

A. Badges? We don’t see no stinking badges!

Q. Excuse me, would you please forward the business activity associated with your traffic so that we can adjust our records?

ObjectivesObjectives

More than near term survival!

Essential CharacteristicsEssential Characteristics

• Preserve behavior

• Ensure access

• Moderate impact

• Protect privacy

• Avoid value judgments

• Apply to new applications

AssumptionsAssumptions

• Large number of hosts

• Small number of problems

• Service consumers

• Many random light providers

• Few heavy providers

• Responsive community

Just Use Traffic ShapingJust Use Traffic Shaping

• Cisco routers

• CAR – traffic class

• MicroCAR – identified flow

Gigabytes

8024,1024,1

sec400,86

Kilobits128

Just Use QoSJust Use QoS

• Classification

• Differentiation

• Admission control

• Provisioning

• Bandwidth

• Latency

• Jitter

QoS DifferentiationQoS Differentiation

P2P Other

10Mbit 90Mbit

QoS DifferentiationQoS Differentiation

10Mbit

Differentiation w/o admission control only

defers the problem!

Rutgers NetworkRutgers Network

• 40,000+ hosts

• 1200+ networks

• 200+ routers

• 17 zones

• 7 campuses

• 3 regions

• 1 autonomous system

ApproachApproach

No single solution!

Best Network PracticesBest Network Practices

• Modular

• Layered

• Aggregated

• Scalable

• Uniform

• Deterministic

• Comprehensible

DeviceDevice DeviceDevice

Intra-building Backbone

Building

Intra-building BackboneIntra-building Backbone

RUNet ~ 1200

BuildingBuilding BuildingBuilding

Inter-building Backbone

Inter-building BackboneInter-building Backbone

RUNet 17

ZoneZone ZoneZone

Intra-campus Backbone

Campus

Intra-campus BackboneIntra-campus Backbone

RUNet 7

CampusCampus CampusCampus

Inter-campus backbone

Region

Inter-campus BackboneInter-campus Backbone

RUNet 3

MANMAN MANMAN

Inter-region Backbone

Autonomous System

Inter-region BackboneInter-region Backbone

RUNet 1

CharacteristicsCharacteristics

• Geographic independence

• Shallow topology

• Similar (not optimal) paths

• Low latency

• Uniform characteristics

• 1 autonomous system

Collect DataCollect Data

• Netflow

• Source/Destination address

• Source/Destination ports

• Protocol

• Packets/Octets/Flows

• Start/End time

Raw DataRaw Data

• 10 minute granularity

• Each source

• Each destination

• 1,000,000 addresses

• 10,000,000 records

• 1 Gigabytes, 1 day

Rollup DataRollup Data

• Rutgers sources/sinks

• Data >= 1024, 10 minutes

• Data >= 6*1024, 1 hour

• Data >= 24*6*1024, 1 day

• 20,000 unique hosts

• 20,000 records

• 1 Megabyte

Filtered DataFiltered Data

• Rutgers sources/sinks

• Data >= 512 Megabytes, 1 Day

• 125 unique hosts

• 125 records

• 50 Kilobytes

ReductionReduction

10,000,000 99.799%20,000 0.200%125 0.001%

10,020,125

Addresses

1,000,000 98.027%20,000 1.961%125 0.012%

1,020,125

Records1,073,741,824 99.898%

1,048,576 0.098%51,200 0.005%

1,074,841,600

1,000 90.090%100 9.009%10 0.901%

DistributionDistribution

• Reread entire data set

• Limit to filtered only

• Rollup based on external address

• Preserve individual distributions

• Useful to reduce contact

Questionable DistributionQuestionable Distribution

Good DistributionGood Distribution

Storage

Process ModelProcess Model

Rollup

Internet

NetflowFilterDistribution

Analyze

Residence AssumptionsResidence Assumptions

• RFC1918 address space

• Large number of hosts

• Small number of problems

• Service consumers

• No service providers

• Unresponsive community

Set LimitsSet Limits

• 2048 MB download

• 512 MB upload

• 7 day granularity

• Sliding window

• Enforcement

ReferenceReference

• 4 movies

• 400 songs

• 45,000 web pages

• 2048 Megabytes

Oracle

Process ModelProcess Model

Rollup

Enforce

GatherInternet

Netflow

Custom ACL

Traffic ShapingTraffic Shaping

• 1 Day on

• 7 Days off

• Multiplexed

• 1:8 ratio

• Automatic

• Aggregated

• Not legalistic

Impact

Differentiated ServiceDifferentiated Service

• Residence facilities

• Other locations

• Two traffic classes

• 1:2 host distribution

• 1:1 bandwidth allocation

• CAR enforced

ResultsResults

Some pains, some gains!

Extra EffortsExtra Efforts

• Registration

• Port Address Translation

• Split horizon DNS

• Help desk/Appeals

• Address hopping

• Proxy services

• Oracle

90% Data Sinks90% Data Sinks

99.99% Data Sinks99.99% Data Sinks

90% Data Sources90% Data Sources

99.99% Data Sources99.99% Data Sources

Internet TrafficInternet Traffic

ConclusionsConclusions

• Modest applications with broad demographics have profound impact.

• Students have free time.

• Network best practices never more important.

• Cooperative generic methods can be effective (w/ encouragement).

• No magic bullet.

Questions?

mundrane@td.rutgers.edu

University Computing Services EDUCAUSE Mid-Atlantic Regional Conference 16 January 2003 An Infrastructure and Accounting Response to Peer to Peer Traffic.

qos slide

mbit slide

services faculty centric

flow slide

autonomous system slide

peer traffic volume

copyright statement

written permission

Documents

Atlantic Sturgeon Stock Assessment Peer Review Report ·...

Diyu educause

Impact from Washington: How will national policy changes...

1 Seminar 4A - Effective Security Practices Eoghan Casey,...

1/17/07 1SCC-SSM Supporting Academic Needs: A Strategic...

30533681 ERP System Acquisitions: Proven Strategies in...

MOOCs EDUCAUSE

An Introduction to EDUCAUSE and the EDUCAUSE/Internet2...

1 Good Digital Citizenship: Examining Student Peer-to-Peer.....

How Collaboration Created an Online Help Desk and Knowledge....

EDUCAUSE 08

Educause 2008

EDUCAUSE Data, Research & Analytics Leah Lang, EDUCAUSE CSG....

Going mobile with a web based strategy (educause quarterly)....

Educause 2010

Presentation to EDUCAUSE Mid-Atlantic Regional Conference...