Leading edge technologies in the HEAnet network Brian Nisbet & John Lyons.

Leading edge technologies in the HEAnet network

Brian Nisbet

&

John Lyons

<[email protected]>

Overview of talk

HEAnet FAQ - what, why, who, where. National and international links

- Gigabit ethernet in the WAN Some services

- Mirroring

- Cisco Distributed Director Some projects IP routing fundamentals

- IP routing in HEAnet using OSPF, IS-IS & BGP

What?

HEAnet is the national network for education and research

Not-for-profit, not-for-loss too. Operating since 1983 Incorporated in 1997 Located in Dublin PoPs in Cork, Dublin, Citywest, Galway, Limerick &

Kilcarbery

Why?

Intercommunication Resource sharing Economies of scale Research collaboration Network technology National strategy

Who?

40 institutions (universities, ITs, …) Owned by the members Small central staff Collaboration with members International associations/partnerships

Where? HEAnet Ltd., Ground Floor, Brooklawn House,

Crampton Ave, Shelbourne Rd, Dublin 4

Tel: +353-1-6609040 Fax: +353-1-6603666

E-mail: [email protected] or [email protected]

GPS Coordinates: 53.3381ºN 6.2403ºW

Web: http://www.heanet.ie

Network Map

DCU Bandwidth Usage

DCU – MRTG (12th –20th March)

Maximum bandwidth available to DCU : 63 Mbps ATM over STM1

Mantova Pic

Gigabit Backbone

Citywest – Galway (March 12th)

Citywest – NOC March 17th –25th

Gigabit Ethernet WAN

Bigger, Better, Faster? Pros

Mass Produced Less Demanding of Hardware Less Complex to Configure Higher Bandwidth Cheaper

Gigabit Ethernet WAN

Cons Slower convergence times More difficult to troubleshoot Next steps may lose some of the

advantages 10GE (add another zero)

Complicates things by adding more framing for long haul.

May cost the same as STM-64.

Gigabit Ethernet WAN

National Backbone Links Router to router, very straightforward.

Client Links HEAnet to TCD, Ratelimited 120Mbps

over GE. Cisco 7200 onsite in TCD. Point-to-point

from there into a Routing-Switch. Allows better visibility of circuit and

development of Routing-Switches makes things cheaper.

International Links

155Mbps: Teleglobe, Abilene & StarTap 622Mbps: GlobalCrossing 1Gbps: JANET, Ca*net4, INEX 2.5Gpbs: Géant

Some Operational Services

National Information Server IP address and Domain registration Listserv Statistics Seminars, training Databases Webhosting Mirroring IPv6 NTP Network security evaluation Hot standby (Cisco DD)

Mirroring - ftp.heanet.ie

2.2 Terabyte Server with dual GigE connectivity IPv6 Capable in HTTP/FTP and RSYNC One of the busiest mirrors in Europe and an official mirror of

RedHat, Debian, FreeBSD Over 60 projects mirrored Developed custom Apache patches to tweak service. [email protected]

Hotstandby Web services

HEAnet offer failover protection for webservers using Cisco “Distributed Director” technology

Router (Cisco 3620/7200) acts as an authoritative DNS server with a TTL of 0

Clients cannot cache the DNS records Router monitors the primary webserver with full TCP

connections on port 80 at specified intervals (eg. Every 10 seconds)

Cisco Distributed Director

Router has IP address of both main webserver and backup “standby server”

If the router cannot connect to the main webserver it will redirect queries to standby

Continues to initiate connections to main webserver and reverts back to it when the webserver recovers

In HEAnet CDD is simply set-up to telnet to webserver on port 80. If it receives a response it considers the webserver to be up but more granularity can be added.

Has limitations

- IPv6 not supported

- https not supported

Some HEAnet service developments

IPv6 (Also a service)

Test traffic project

Multicast

MAN

Honeypots

Videoservices

Netflow

PKI

National Backbone

Next Generation Internet

IP routing Fundamentals

Routing is the process of selecting the interfaces through which a packet should be sent

Forwarding is the process of actually moving this packet once the route has been determined

Classful routing only passes network addresses, classless routing uses both network and subnet addresses

Classless interdomain routing (CIDR) was brought in in a bid to reserve IPv4 addresses and allows blocks of class C addresses to be combined/aggregated eg. 192.168.1.0 /21

More fundamentals …

Variable length subnet masks (VLSM) - greater granularity eg.192.168.0.0/28

Distance Vector protocols use hop count as a metric, link state (eg. OSPF) use cost. Hybrid protocols like EIGRP can use complex rules involving delay, reliability

Convergence is the length of time after a network change (Interface/line down, router added etc.)that routing tables take to update

An Autonomous system (AS) is a group of IP networks operated by one or more network operator/s which has a single and clearly defined external routing policy

Rip v1/v2

RIPv1 – RFC1058 - Classful, Periodic updates, transfers entire routing tables, distance vector, uses holddown timers.

RIPv2- Added classless, authentication, multicast support.

Uses “routing by rumour” as it only sees directly connected routers

Both of these routing protocols are inefficient and are now only used on very small networks. They have been

phased out in favour of more “intelligent”Interior gateway protocols (IGP’s) such as OSPF and IS-IS.

OSPF Overview

Features

- Supports CIDR/VLSM

- Fast convergence

- Uses multicast addresses for hello packets

- Load balancing

- Uses areas to cut down CPU usage Uses hello packets to establish adjacencies initially via

a 6 step process Once adjacency established it uses regular Link State

advertisements (LSA) and Link state updates (LSU) to describe links & adjacencies changes

OSPF

9 types of LSA, 5 of which are used extensively :

1-Router link

2-Network link

3-Network summary

4-ASBR summary

5-AS external.

(other 4 for TOS) Link state databases (LSDB) built from info in LSA’s Each router in an Area has the same LSDB and using this &

the SPF algorithm (Dijkstra) to build its routing table It’s an efficient & quiet routing protocol as it can have separate

LSDB for each area & only sends LSA updates

OSPF

Works under different network topologies differently

- Broadcast multiaccess (LAN)

- Point to Point (eg. Serial connection)

- Point to Multipoint (eg ATM using multiple VC’s)

- Point to Multipoint (Non broadcast multi access (NBMA)) Reacts quickly to network changes (Line down, router death)

- Hello interval every 10 secs

- Dead interval (Link down) is 4 times hello interval Routing descisions based on cost of 100000000/1

- Cost ranges from 0 to 65535 with 0 being the best

- 100Mb = OSPF cost of 1

- OSPF can do load balancing between same cost routes

OSPF in HEAnet

In HEAnet we have only one area (backbone: area 0)although we are investigating the use of multiple areas in the future (or using IS-IS)

In conjunction with static routes and BGP we use OSPF to route traffic throughout our internal network

Routers in DCU have full visibility of all routers running OSPF

in our AS ie. They have the same LSDB as the other routers in the AS

Does have its limitations:

- Currently running OSPF v2 which doesn’t handle IPv6

- OSPF v3 does but isnt considered stable enough yet

- For IPv6 we need to run another routing protocol IS-IS

IS-IS

IS-IS (Intermediate System to Intermediate System). Part of the OSI standard.

ISO/IEC 10589, RFC 1195 and RFC 2763 Not originally designed for IP. Uses Link-State PDUs, similar to OSPF

LSAs. Extensible Protocol, updated to deal with

IPv6 faster than OSPF. Add more Type-Length-Values to the PDUs

to handle more protocols.

IS-IS in HEAnet

Used purely for IPv6 routing within the HEAnet network.

Communication between routers. IPv6 connections to clients.

BGP

BGP (Border Gateway Protocol) is an EGP (External Gateway Protocol) and an IGP (Internal Gateway Protocol).

RFC 1771 Deals with AS’ (Autonomous Systems) Exchanges routing information between

peers. Makes routing decisions based on a number

of different criteria applied to each route.

BGP

These criteria feed into the BGP Best Path Algorithm. Each step is gone through until a definite routing decision can be made.

Invalid paths are ignored. Common values/criteria:

Local Preference AS Path Length Multi Exit Discriminator (MED)

Will eventually choose the path to the router with the lowest IP address.

BGP in HEAnet

We have BGP peerings with all of the networks we directly connect to eg Géant, JANET, Ca*net4, Global Crossing, Teleglobe etc.

A large number of the routers in HEAnet form part of our internal BGP mesh.

Allows for proper route selection to any accessible point in the internet from anywhere in HEAnet.

Contacts

http://www.heanet.ie

[email protected]

Questions ?