Griffin Update: Toward an Agile, Predictive Infrastructure Anthony D. Joseph UC Berkeley http:// www.cs.berkeley.edu/~adj/ Sahara Retreat, January 2004 DETER
Griffin Update: Toward an Agile, Predictive Infrastructure
Anthony D. Joseph
UC Berkeleyhttp://www.cs.berkeley.edu/~adj/
Sahara Retreat, January 2004
DETER
2
Outline
Griffin– Motivation– Goals– Components
Tapas Update Tapestry Update REAP/MINO Update Beyond Griffin: DETER
3
Near-Continuous, Highly-Variable Internet Connectivity
Connectivity everywhere: campus, in-building, satellite…– Projects: Sahara (01-04), Iceberg (98-01), Rover (95-97)
Most applications support limited variability (1% to 2x)– Design environment for legacy apps is static desktop LAN– Strong abstraction boundaries (APIs) hide the # of RPCs
But, today’s apps see a wider range of variability– 35 orders of magnitude of bandwidth from 10's Kb/s 1 Gb/s– 46 orders of magnitude of latency from 1 sec 1,000's ms– 59 orders of magnitude of loss rates from 10-3 10-12 BER– Neither best-effort or unbounded retransmission may be ideal– Also, overloaded servers / limited resources on mobile devices
Result: Poor/variable performance from legacy apps
4
Griffin Goals
Users always see excellent ( local, lightly loaded) application behavior and performance
– Independent of the current infrastructure conditions– Move away from “reactive to change” model– Agility: key metric is time to react and adapt
Help legacy applications handle changing conditions– Analyze, classify, and predict behavior– Pre-stage dynamic/static code/data (activate on demand)
Architecture for developing new applications– Input/control mechanisms for new applications– Application developer tools
5
Griffin: An Adaptive, Predictive Approach
Continuous, cross-layer, multi-timescale introspection– Collect & cluster link, network, and application protocol events– Broader-scale: Correlate AND communicate short-/long-term events and
effects at multiple levels (breaks abstractions)– SOLVED: Building accurate models of correlated events
Convey app reqs/network info to/from lower-levels– Break abstraction boundaries in a controlled way– OPEN: Extensible interfaces to avoid existing least common denominator
problems Overlay more powerful network model on top of IP
– Avoid standardization delays/inertia– Enables dynamic service placement– PARTIAL: Efficient interoperation with IP routing policies
6
Some Enabling Infrastructure Components
Tapas network characteristics toolkit– Measuring/modeling/emulating/predicting delay, loss, …– Provides micro-scale network weather information– Mechanism for monitoring/predicting available QoS
REAP protocol modifying / application building toolkit– Introspective mobile code/data support for legacy / new apps– Provides dynamic placement of data and service components– MINO E-mail application, COMPASS service instance locator
Tapestry, Brocade, and Mobile Tapestry– Overlay routing layer providing efficient application-level object
location and routing– Mobility support, fault-tolerance, varying delivery semantics
7
Outline
Griffin– Motivation– Goals– Components
Tapas Update Tapestry Update REAP/MINO Update Beyond Griffin: DETER
8
Tapas Update
Accurate modeling and emulation for protocol design– Models/artificial traces that are statistically indistinguishable
from real network traces: delay, error, congestion– Study interactions between protocols at different levels
Project completed (1998-2003)– Multitracer trace analysis tool– Two highly-accurate network models (MTA, M3)– Domain analysis tool– Highly-accurate Tapas-based link simulator
PhD dissertation– Almudena Konrad, “TAPAS: A Research Paradigm for the
Modeling, Prediction, and Analysis of Non-stationary Network Behavior,” (Ph.D., December 2003)
9
Tapestry Update
Distributed Object Location and Routing (DOLR) overlay network
Improved static resilience (talk tomorrow)– Pre-computed backup paths enable near- instantaneous fail-over
(3 paths/router entry)– Better dynamic resilience through improved repair algorithms to
handle long-term faults– IEEE JSAC article pending
Support for rapid, hierarchical mobility– Scaleable mobility for large crowds traveling together– IPTPS paper in submission
10
Tapestry Static Resilience (Sim)
00.10.20.30.40.50.60.70.80.9
1
0 0.05 0.1 0.15 0.2
Proportion of IP Links Broken
% o
f A
ll P
air
s R
ea
ch
ab
le
Instantaneous IP Tapestry / FRLS
11
REAP/MINO/COMPASS Update
Introspective code / data migration in 3-tier hierarchies– Distributes server load, empowers limited devices– Provides illusion of high connectivity
Combines static trace analysis w/ dynamic monitoring of clients to predict appl’n / communication behavior
– Identify and optimize code/data placement– Analyzing EECS IMAP server traces for user session length
and inter-session mobility (see poster) Testbed technologies:
– REAP code migration toolkit– MINO E-mail OceanStore application– COMPASS: service instance location service (talk tomorrow)
12
User IMAP Session Lengths (processed to remove auto checks)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 2000 4000 6000 8000 10000
session length (seconds)
frac
tio
n o
f se
ssio
ns
50% of sessions <1000 seconds (17 minutes)
1800 seconds (30 minutes) = IMAP server's timeout setting
20% of sessions > 6000 secs
13
Outline
Griffin– Motivation– Goals– Components
Tapas Update Tapestry Update REAP/MINO Update Beyond Griffin: DETER
14
Cyber DEfense Technology Experimental Research (DETER)
NSF and DHS sponsored cyber-defense research project– Approx $10M total ($2.4 for UCB)
DETER Goals:– Design and construction of a testbed for network security
experiments,– Research on experimental methodology for network security, and– Research on network security.
DETER: focus on 1), but it needs to do some of 2) and 3) Goal: Duplicate observed attack effects in the testbed
– E.g., self-congestion for worms
DETER
15
Related Goals
Vendor-heterogeneous environment– Reflects real-world, implementation interactions– Open source versus commercial code (e.g., timers)– Behavior under load/attack
Create a researcher’s electronic notebook– Network topologies, attack traces and generators– Background traffic traces and generators
Many requirements (some conflicting!)– Versatility, Controllability, Accessibility, Usability– Functionality, Transparency, Fairness, Containment– Security, Fidelity, Integrity
DETER
16
Background
People: – Anthony Joseph, Ruzena Bajcsy, Shankar Sastry, David
Culler, Doug Tygar, David Wagner, Eric Fraser (staff), Yih-Chun Hu (postdoc)
– Small initial user community (usability versus containment) Hardware
– First cluster of ~64 PCs at USC/ISI West (Jan/Feb 04)– Second cluster at UCB (Mar/Apr 04)
Similar to ISI cluster, but with more hw routers Three experiment areas (EMIST)
– Worms, routing attacks, DDoS attacks Major demo of experimental results in DC in June 04
– Future: DHS, HSARPA, and White House “exercises”– E.g., LiveWire, DarkScreen, JWIG2004
DETER
17
Preliminary UCB Architecture Proposal
latigid
Bay Networks
Com3
1 2 3 4 5 6
7 8 9101112
AB
12x
6x
8x
2x
9x
3x
10x
4x
11x
5x
7x
1x
Eth
erne
t
A
12x
6x
8x
2x
9x
3x
10x
4x
11x
5x
7x
1x
C
L2 switches
L3 routers
Mgmt L2 switch
Mgmt 1
FW1
Hub
FW2
Internet
Mgmt 2
Serial links
File Server
1 2 3 4 5 6
7 8 9101112
AB
12x
6x
8x
2x
9x
3x
10x
4x
11x
5x
7x
1x
Eth
erne
t
A
12x
6x
8x
2x
9x
3x
10x
4x
11x
5x
7x
1x
C
L2 switch
DNS
DMZ
Sniffer Servermonitoring/analys is
Sniffer
Pwr Ctlr
Pwr Ctlr Pwr Ctlr
Pwr Ctlr
DETER
18
Some Collaboration Opportunities
Research opportunities– Measuring application behavior under attack
Web servers, file servers, etc.– Strategies for mitigating attacks
Worm defenses, DDoS traceback and block, hardening routing protocols
– Operations and management Substantial knowledgebase from commercial customers (Tiger
teams) Donations
– VIFs: Cluster or security experience/research– Remote administration tools, remote SW installation setup tools– Nodes, Firewall machines, L2/L3 routers, HW sniffers, etc
DETER