CHEP2000 - Padova, February 2000 Network Engineering @ SLAC S. Luitz, D. Millsom, D. Salomoni, J.Y. Kim, A. Zele.

CHEP2000 - Padova, February 2000

Network Engineering @ SLAC

S. Luitz, D. Millsom, D. Salomoni, J.Y. Kim, A. Zele

CHEP2000 - Padova, February 2000

Summary

IntroductionMajor IssuesNetwork ArchitecturePerformanceRetrospective

CHEP2000 - Padova, February 2000

Introduction

SLAC - Stanford Linear Accelerator Center High Energy Physics Stanford Synchrotron Radiation Laboratory

1300 full-time employees, 700 on-site collaborators

Collaborate with 200 institutions internationally Mixture of real-time data acquisition, numerical

analysis, business services/administration Very large quantities of data, e.g. 1-2

petabytes/year for the BaBar experiment

CHEP2000 - Padova, February 2000

Major Issues

Bandwidth demand doubles every 1.5 years High reliability and availability requirement, both

for experimental work and business services Network performance requirements dictate use

of leading-edge technology Leading-edge technology challenges reliability International collaboration dictates need for

open network Security - DOE requirements, open network is

more prone to hacking

CHEP2000 - Padova, February 2000

Major Issues...

Network services have become so essential that failures can bring organizations to a standstill They are often more critical than the phone

One reason: critical information (data and applications) has moved into the (often-times central) computer/storage facilities

SLAC: particular issue - very high bandwidth/availability for data acquisition

CHEP2000 - Padova, February 2000

Bandwidth Growth

CHEP2000 - Padova, February 2000

Network Architecture Layer one

Star topology Fiber/Copper Redundant Ethernet Full/Half Duplex, 10/100/1000 Mbps Backbone almost entirely Gigabit Ethernet (with Gigabit EtherChannel)

Islands BaBar detector/data acquisition Main Control Center Business Services

Layer two Switched VLAN (~45 switches, ~40 VLANs)

Layer three Centralized routing

CHEP2000 - Padova, February 2000

DMZ

InternetModems,

ISDNxDSL

SLAC Switched LAN Spring 2000

ESA

Legacy

SSRL

OldServers

MCC3

20 Buildingedge switches

BaBar

MCC2

SSRL

MCC1

10BaseT

FDDI/CDDI

100BaseT

100BaseFL

Gigaswitch

Router

Switch

Hub

1Gbit FL

4Gbit FL

Concentrator

IR26 Farm edge switches

4 Server switches

BSD

DMZ Switch

Netscout

Monitoring

CoreRouters

Switches

CHEP2000 - Padova, February 2000

CHEP2000 - Padova, February 2000

CHEP2000 - Padova, February 2000

External Connections

622 Mb/s

155 Mb/s( 622 Mb/s)

45 Mb/s

155 Mb/s

2 Mb/s

CHEP2000 - Padova, February 2000

Performance

A number of tools developed in-house to monitor the status of the network E.g., port, backplane, CPU utilization, device

reachability, L2/L3 traceroute, DMZ traffic statistics

No components are currently approaching saturation

CHEP2000 - Padova, February 2000

CHEP2000 - Padova, February 2000

CHEP2000 - Padova, February 2000

Retrospective

The network design has proved to be scalable, highly available and provides high bandwidth in a secure environment

Some lessons learned: be careful with management and configuration of layer-2 switching

and spanning trees (e.g., watch for VTP configuration issues) and with CGMP/IGMP multicast support

do not attempt to mix standards (e.g., Cisco ISL and IEEE 803.1Q) even when theoretically possible

adequate security and openness are often conflicting requirements Open issues:

effective monitoring (e.g., SPAN) and data gathering in a high-speed switched environment

secure SNMP monitoring (SNMPv3 support) and secure access to the network devices (SSH)