Network Planning Task Force - University of Pennsylvania · NPTF 2014 Schedule University of Pennsylvania - Network Planning Task Force 3 July 21st – •Information Security Update

University of Pennsylvania

Network Planning Task Force

September 8, 2014

Deke Kassabian [email protected]

mailto:[email protected]

NPTF Membership

Daniel Alig / Joe Cruz, Wharton

Charles Buchholtz, SEAS

Lena Buford / Tejash Patel, Annenberg

Cathy DiBonaventura, School of Design

Brian Doherty, SAS

David Domico, SRFS

John Eckman, Housing Services

Jeff Fahnoe, Dental

Mike Herzog / Didi Sariyska, GSE

Andre Jenkins / Michael Morris / Robert Colligon, PSOM

Marilyn Jost / Victoria Iannotta, FRES

Sue Kennedy, Business Services

David Kern, Public Safety

Kay McDonnell / Christine Droesser, Law

Grover McKenzie, Library

Donna Milici / Sam Smith, Nursing

Dominic Pasqualino, OACP

Smith Ragsdale / Brian Sherman, VET

Joseph Shannon, Finance

Mary Spada, VPUL

Marilyn Spicer, College Houses

Michael Weaver, Budget Mgmt. Analysis

Ira Winston, SEAS, SAS, Design

ISC Representation

Deke Kassabian, ISC (Chair)

Mark Wehrle, ISC

Jim Choate, ISC

Josh Beeman, ISC

Michel van der List, ISC

Mark Aseltine / Amy Phillips, ISC

Gary Delson / Geoff Filinuk, ISC

2 University of Pennsylvania - Network Planning Task Force

NPTF 2014 Schedule


July 21st –

• Information Security Update

September 8th –

• Network and Server Infrastructure (other than wireless)

October 13th –

• Wireless, Identity and Access Management, Penn+Box, Next Gen WWW

November 10th –

• Any remaining content

• Working through FY16 Rates

CSF and Port Rental / Maintenance

IP charges are 20% of the CSF

Headcount charges are 80% of the CSF

FY'14 FY'15

IP Charges: $1.61 / month $1.57 / month

FY'14 FY'15 10Base-T $4.75/month $4.75/month

100Base-T $4.75/month $4.75/month

1000Base-T $7.00/month $4.75/month

10 GbE $80.00/month $80.00/month

PoE ports $2.00/month $0.00/month

ISC Branding, Marketing & Website Project 4

Initial CSF Rate Change Information

Assuming an overall 2.75% staff salary

increase, and holding all else constant, costs

funded through CSF would increase by

roughly 1.45% for FY16.

This is NOT a rate announcement, just initial

information. Other factors will affect the

eventual rate.

Rate discussion will continue throughout the

fall meetings, and we will revisit the

recommended rate at the November 10th

meeting.


FY15

CSF Rate

$1.57/IP/Mnth

FY16

CSF Rate

Not yet set

FY’15 CSF Bundle of Services

Campus Backbone Infrastructure

Internet and Internet2 access

Rate limits on ResNet

IPv6, Multicast, and Advanced Networking

Public Wireless Subsidy

Cap on billable wireless IPs

NAP Operations, Fiber &Cable Mgmt

NOC Services and Network Management

Penn's Main Web and Central Pages

Online Directory and LDAP access

Classlists and SMTP Mail Relay

University Calendar Service

Infrastructure Services (DNS, DHCP, NTP

Penn+Box Storage & Collaboration

Network Address Translation (FY15)

Eduroam and IoT Support (FY15)

Enterprise Social Networking (FY15)

Security/ID Management

Kerberos, KITE, RADIUS

Penn WebLogin (CoSign and Shibboleth)

The InCommon Federation

Enterprise InCommon Cert Service

Authorization (Penn Groups)

PennNames and Penn Community Services

Wireless Authentication & Support

NetReg

DNSSEC

Vulnerability Scanning

Security Tools, Education, and Response

PennKey School Support

PGP Whole Disk Encryption LSP Support

XpressConnect

Enhanced AirPennNet Guest Services (FY’13)

Intrusion Detection System

SafeDNS (FY15)

6

6

University of Pennsylvania - Network Planning Task Force

Today’s Agenda

MAGPI Update

Next Generation PennNet

• Core and border routers, building routers

Science DMZ status

Update from the Network Architecture & Security initiative

IP address utilization and IP Address Management (IPAM)

Core server infrastructure

• DNS, DHCP, NTP, Kerberos, RADIUS and SafeDNS

Proposed wired port policy

Networking for the Penn Wharton China Center

Open Discussion


MAGPI Update

MAGPI is the Penn-sponsored and operated Regional Optical Network for PA, NJ, DE.

MAGPI was founded in 1997 with 2 members; reached a high point of 500 supported R&E institutions through 37 connections in 2011.

Consolidated in early 2014, MAGPI now supports 8 large pipe connections with a strong focus on “big data” research.


What is MAGPI?

MAGPI Update

The Pennsylvania Research and Education Network (PennREN) is now independently offering Internet2 access.

MAGPI’s focus is now primarily to serve the largest Higher-Ed members within our region, transitioning smaller members to PennREN.

Negotiations for 3 year commitments with 6 major regional universities are in late stages.

MAGPI and Penn have established a new 100Gb connection to Internet2.


Changes for MAGPI

Next Generation PennNet (NGP)

NGP is an ongoing project to significantly improve performance and reliability of PennNet and to meet the network goals of major campus applications. The primary components are:

• Wiring closet switches

• Building backbones and building routers

• Central wireless infrastructure

• Campus core routers

• Campus border routers

• Core server infrastructure

• Fiber links and Wave Division Multiplexing to key locations in the metro region

• High capacity links to the Internet and Internet2

• Science DMZ

University of Pennsylvania - Network Planning Task Force 10

PennNet Core, Border, and Science

DMZ Topology


NGP Accomplishments for FY 2014

Completed deployment of new core and border routers.

• Provided 10x increase on core bandwidth to 100Gb, and 100Gb connectivity to Internet2

Provided 10Gb connectivity to most building entrance routers and closet switches.

Completed a major upgrade of wireless controller infrastructure.

Continued progress on closet switch upgrades.

Replaced remaining older building routers to allow for higher density 10Gb connections.


Building Entrance (BE) Routers

Replaced remaining BE routers (especially for larger buildings) this summer.

New BE routers have greater 10Gb port capacity and allow for closet switch upgrades in larger buildings.

Replacing data center routers with newer larger capacity routers for better performance, higher density, additional features.


NGP Progress

Switches:

• 1,158 of 1,916 closet switches replaced with 10Gb capable closet switches.

• 151 of 239 buildings are completed. We are currently working in another 28 buildings.

• Most remaining locations are residential and remote buildings off the main campus.

10Gb Connections to Campus Buildings:

• 40 uplink connections are yet to be upgraded.

• Some low volume locations will remain at 1Gb.


NGP Goals for FY 2015

Complete 10Gb upgrades to remaining BE locations.

Complete data center router upgrades.

Upgrade standard PennNet connections to 1000BaseT.

Continue closet switch deployment.

Continue migration of multimode fiber circuits to single mode fiber circuits.

Continue deployment of the Science DMZ.

Deploy central NAT infrastructure at network border.


Science DMZ

In April we reported that Penn was successful in getting a Campus Cyberinfrastructure award from the National Science Foundation.

This award was primarily for bringing high bandwidth to campus researchers and establishing a Science DMZ.

A Science DMZ is “a portion of the network designed to optimize for high-performance scientific applications rather than for general purpose business systems or enterprise computing.” *

* Definition from ESNet: https://fasterdata.es.net/science-dmz/


Science DMZ Progress at Penn

Campus 100 Gbps connection now operational.

Dedicated 100 Gbps Science DMZ switches and circuits deployed.

Initial connections provisioned or underway.

• Connection to South Bank for Dr. Srolovitz (the PI on the Penn grant proposal) is underway.

• Other interfaces soon to be allocated to researchers who wrote support letters.

Deployment of measurement infrastructure and OpenFlow test lab equipment is in progress.

Please see the diagram in subsequent slide.


PennNet Core, Border, and Science

DMZ Topology


Network Architecture And Security

Working Group (NASWG)

Last year at NPTF, we introduced the idea of a new group to take a fresh look at PennNet design, particularly in the areas of segmenting and protecting networks.

Since then, a large collaborative team has formed.

• Participation from many schools and centers, sharing local network designs, challenges, solutions

• Participants joined forces to look at new approaches to solving common problems in:

– network segmentation and extension,

– network filtering,

– network access management,

– secure remote access, and

– network visibility.




Current methods of achieving network segmentation:

• Routed Virtual LAN (VLAN) segments within a building

• Dedicated fiber circuits between buildings

• Routed private VLANs protected by Access Control Lists (ACLs) on the routers

• Site-to-Site infrastructure Virtual Private Networks (VPNs)

Current methods do not scale in a cost-effective way, nor do they meet all segmentation and reach use cases.

Virtualization and overlay technologies would enable new approaches to network segmentation and extension.





In combination with the goals of the IRC, and leveraging what is being built for the Science DMZ, NASWG can bring new opportunities for PennNet architecture and security design.



In combination with the goals of the IRC, and leveraging what is being built for the Science DMZ, NASWG can bring new opportunities for PennNet architecture and security design.


IP Address Management (IPAM) Strategy

PennNet address use growing by 30% annually, largely due to increases in the numbers of wireless devices on campus. Without additional action, we would run out of IPv4 addresses in the next few years.

Strategies include:

Use of Network Address Translation (NAT) in appropriate situations. Possibilities include AirPennNet, AirPennNet-Guest, AirPennNet-Device, Residential Networks.

Use of RFC1918 for internally routed applications.

Continue deployment of IPv6 infrastructure across campus.

Consider outsourcing AirPennNet-Guest (17,868 IPs currently allocated), relying upon ISP address space.


IPv4 Conservation Efforts

Recent progress has delayed IPv4 exhaustion by 1 year (to Fall 2017).

• Completed conversion of IP addresses on all PennNet Phone networks (recovered 19,000 IP Addresses).

• Bundled wireless AP groups to efficiently share IP addresses.

Preparing to implement NAT services on PennNet.

• NAT purchase and design is complete

• Initial deployment by end of Fall 2014

• Begin conversion of targeted locations, start of CY2015

• Potential gains from NAT: 32,000+ addresses

24

NAT Traffic Flow

ISC Branding, Marketing & Website Project 25

IPv4 Utilization Year Over Year

26

0 10000 20000 30000 40000 50000 60000 70000 80000 90000

AirPennNet

AirPennNet-Device

AirPennNet-Guest

Central Services

Dark

GreekNet

Management/Infrastructure

PennNet (Standard)

PennNetPhone

Private Networks

Resnet

44390

2097

17167

1440

14199

1836

10093

80118

18232

12599

11572

47801

567

17301

1440

30057

1836

9241

80858

0

13055

11572

Sep-14

Oct-13

Core Server Infrastructure

Recent refresh of campus DNS, DHCP, NTP, Kerberos and RADIUS servers.

Common features

• Higher capacity, smaller/cheaper hardware

• Modern software versions

• Substantially increased throughput

• Simplified administration

DNS [Deployed August 2013]

• Anycast IP addresses

• Smaller fault zones, increased availability

DHCP [Deployed July 2013]

• Simplified redundancy model

• Peak sustained request rate of 114/second or 50% capacity

• Exploring static DHCP possibilities




Common features





NTP [Deployed January 2014]

• On-board reference clocks (GPS, AM)

Kerberos KDCs [Deployed June 2014]

• Newer Kerberos code, plus removal of local patches

• Enables high-availability administration




Common features





RADIUS [Deployed August 2014]

• Peak authentication rates up 31% this year to 113/second (sustained for five minutes)

• New service running at roughly 30% of capacity (from 93%)

• New design eliminates risk of cascading failures

• Changes also enables next stage projects (EduRoam, IoT)


Anycast

virtual

servers

Physical

servers

Primary Campus DNS Resolvers


Anycast DNS

Anycast

virtual

servers

Physical

servers

SafeDNS Resolvers


Anycast DNS

r dns1a

r dns1

128. 91. 18. 1

r dns1asdns2a

sdns2

128. 91. 49. 2

sdns2asdns1a

sdns1

128. 91. 18. 2

sdns1a

Primary Secondary Tertiary

PennNet SafeDNS client

The Road to SafeDNS


Initial pilot128. 91. 19. 240

128. 91. 19. 241

Enhanced pilot128. 91. 18. 2

128. 91. 49. 2

Productionservice

128. 91. 18. 2

128. 91. 49. 2

New users

Front end

Back end

CY14 CY15

Launch

SeptemberNPTF

Evaluate cloud-based service

Adapt proven UPenn solutions

Finalize deployment

OctoberSUG

Campusannouncement

Decisionpoint

Next Steps for DNS/NTP

As SafeDNS moves to production ISC will continue to work with community to migrate to:

• Either RDNS or SDNS service

• High-performance NTP time sources

Proposed Goal: Retire old NOC1, NOC2, NOC3 servers and recover their IP addresses.

• Temporarily maintain legacy DNS addresses as cache-only servers

• Work with community on a schedule to retire old server addresses


Proposed Wired Port Policy

Purpose is to identify and remove inactive wired ports leading to reduced infrastructure and billing costs.

Initially proposed at July NPTF.

Small working team met in August to discuss policy development.

Proposal details:

• Inactive wired ports disabled after 45 days.

• Additional waiting period of 15 to 30 days, with reactivation if “dead port” ticket is reported.

• Disconnect process follows if no tickets reported.

• Reports 2x/year (Fall/Spring to avoid winter and summer breaks).


Proposed Wired Port Policy

Next steps

• Seek approval to proceed from NPTF.

• Reconvene the working group.

• Develop policy and fully document process and risks vs. rewards.

• Take the resulting policy through the Network Policy Committee process.

• Present to IT Roundtable for final recommendation and adoption.


Penn Wharton China Center

Global Engagement Initiative to Extend Penn Presence in Beijing, China

Penn will have classroom presence in World Financial Center Building 16th Floor.

Major technology collaborators are FRES, ISC, and Wharton IT.

ISC assisting Wharton IT staff and the Penn project team with technical design and security consulting for wireless, Telepresence, and network connectivity from China to Penn’s main campus.

PWCC site will have connection to the local Internet and a dedicated Ethernet private line connection to Philadelphia.

• Ethernet for telepresence and Penn traffic for faculty, staff and students visiting PWCC

Work is in progress on selection of IT Integrators and telecommunication service providers.

Targeted turn up and move in date is January 2015.


Open Discussion

Questions, clarifications, concerns on any topic covered, or additional

topics to be researched.


Next Meeting

October 13th

Currently planned topics • Identity and Access Management update

• AirPennNet and AirPennNet-Guest updates

• EduRoam

• Wireless for Internet of Things devices

• Cellular and DAS

• Penn+Box

• Project ButtonUP

• NextGen WWW


Network Planning Task Force - University of Pennsylvania · NPTF 2014 Schedule University of Pennsylvania - Network Planning Task Force 3 July 21st – •Information Security Update

Documents