Cisco Net Workers - Deploying Interior Gateway Protocols (2007)

© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential

TECRST-2021

13881_06_2007_c1 1


TECRST-2021

13881_06_2007_c1 2

Deploying Interior Gateway Protocols

TECRST-2021


TECRST-2021

13881_06_2007_c1 3

Deploying Interior Gateway Protocols

Design Theory

Working with Addressing and Summarization

Working with Hierarchy

Working with Topologies

Working with Redistribution

Transitioning Routing Protocols

BGP


TECRST-2021

13881_06_2007_c1 4

Design Theory


TECRST-2021

13881_06_2007_c1 5

Design Theory

Design Goals

Resiliency

Simplicity

Functional Separation

Hiding Reachability

Hiding Topology

Virtualization


TECRST-2021

13881_06_2007_c1 6

―… a reliable network delivers virtually every packet accepted by the network, to the right destination, within a reasonable amount of time…‖

Optimal Routing Design

Cisco Press®


TECRST-2021

13881_06_2007_c1 6


TECRST-2021

13881_06_2007_c1 7

Design Goals

Networks deliver packets!

A network is judged on its ability to support applications

All the other elements of network design support this single goal

The three primary goals:

Resiliency (Reliability)

Simplicity


DeliverPackets

Adjust to Real World Changes

Device Failure Business Changes

High AvailabilityRedundancy Scaling

Reduced Downtime

Fast TroubleshootingFast Recovery

Simplicity



TECRST-2021

13881_06_2007_c1 8

Design Goals

Another view of network design is to determine why networks fail

Device failure, resolved through:

Resiliency

High availability techniques


Negative feedback loops, resolved through:

Simplicity


The Same Goals!

Resiliency

Simplicity


NetworkFailure

High AvailabilityRedundancy

Reduced Downtime

Fast TroubleshootingFast Recovery

Simplicity


Device Failure Feedback Loops


TECRST-2021

13881_06_2007_c1 9

Notes on OPEX

Operational Expenses are directly tied to:

Day to day costs of running the network

The costs of downtime

Do these network design principles impact operational expenses?


TECRST-2021

13881_06_2007_c1 10

Notes on OPEX

Resiliency

Manages the costs of downtime

Simplicity

Manages the costs of monitoring and changing the network



Manages the costs of monitoring and changing the network



TECRST-2021

13881_06_2007_c1 11

Design Goals

Provides alternate paths to route around failures

Resiliency

Easier to grasp and troubleshoot

Simplify configurations, reducing human error

Downtime includes troubleshooting time

Simplicity

Enables simplified configurations

Allows complexity in one part of the network to be hidden from other parts of the network

Divide and conquer

FunctionalSeparation


TECRST-2021

13881_06_2007_c1 12

Resiliency

Resiliency is the ability of the network to adjust to changing conditions

Two dimensions

How many packets inserted at the edge of the network do not make it to their destination?

How long is it between unplanned network failures, and how long does it take to fix the network when it‘s broken?

In general: Avoid Brittleness!


TECRST-2021

13881_06_2007_c1 13

Resiliency

What Are You Planning For?

Yes NoSevere Weather with Local Power Failure?

No YesFootball Playoffs?

Yes YesBeginning of School?

Yes NoSpring Break?

The Worst Case or the Common Case?


TECRST-2021

13881_06_2007_c1 14

Resiliency

It‘s Important to Understand:

Mean Time Between Failures (MTBF)

How long the device or system runs before failing

Mean Time To Repair (MTTR)

How long it takes to repair the device or system after a failure

Uptime, or Reliability

How many ―9‘s‖

Total Time/(MTBF+MTTR)

Statistical Analysis


TECRST-2021

13881_06_2007_c1 15

Resiliency

Break failure domains apart

A single failure impacts less of the network

Improves Troubleshooting

Troubleshooting is split and test

Splitting the failure domain presplits the troubleshooting domains

Decreases MTTR



TECRST-2021

13881_06_2007_c1 16

Resiliency

The simplest path to increased resiliency is adding redundancy...

Not so fast!

Resiliency must be balanced against simplicity and functional separation

Redundancy doesn‘t always add resiliency

A

10.1.1.0/24

B

Redundancy


TECRST-2021

13881_06_2007_c1 17

Resiliency

There are other resilient techniques besides redundancy

High availability

Fast convergence


TECRST-2021

13881_06_2007_c1 18

―Could I explain this at 2AM to a TAC Engineer who lives halfway across the world?‖

The 2AM Rule of Thumb


TECRST-2021

13881_06_2007_c1 18


TECRST-2021

13881_06_2007_c1 19

Simplicity

Simplicity Encompasses:

Network Design

Covered throughout the remainder of this presentation

Management Simplicity

Configuration Simplicity


TECRST-2021

13881_06_2007_c1 20

Simplicity

Choose the simplest configuration that will do the job

Choose the easier configuration to change in the future

Choose the configuration that contains the intent

Examples

Use prefix lists for route filtering, rather than access lists

Use tags for filtering redistributed routes, rather than building a long list of networks



TECRST-2021

13881_06_2007_c1 21

Simplicity

OSPF Network

Install new router...

Examine configuration of hub router

Examine configuration of existing spoke router

Configure new router

Connect to network

Network breaks!

Why?

hub_router#show run

....

interface s0/0

ip address 10.1.1.100 255.255.255.0

....

spoke_router#show run

....

interface s0/0

ip address 10.1.1.200 255.255.255.0

....

new_router#show run

....

interface s0/0

ip address 10.1.1.80 255.255.255.0

....



TECRST-2021

13881_06_2007_c1 22

Simplicity

Why were the interface IP addresses set up this way?

The interface isn‘t a point-to-point, so it must be a multipoint

The DR must be the hub router...

What ensures this? The interface IP addresses!

This is not obvious!

A specific control is buried under a normal looking configuration

hub_router#show run

....

interface s0/0

ip address 10.1.1.100 255.255.255.0

....


....

interface s0/0

ip address 10.1.1.200 255.255.255.0

....

new_router#show run

....

interface s0/0

ip address 10.1.1.80 255.255.255.0

....



TECRST-2021

13881_06_2007_c1 23

Simplicity

What if we use the OSPF interface priority, instead?

The reason for the configuration directly relates to what the configuration does

This makes network maintenance simpler

Rules of thumb:

Apply the most obvious configuration possible

Apply the configuration as close to the point of control as possible

hub_router#show run

....

interface s0/0

ip address 10.1.1.100 255.255.255.0

ip ospf priority 240

....


....

interface s0/0

ip address 10.1.1.200 255.255.255.0

ip ospf priority 0

....

new_router#show run

....

interface s0/0

ip address 10.1.1.80 255.255.255.0

ip ospf priority 0

....



TECRST-2021

13881_06_2007_c1 24


Allows us to hide information

Allows us to break the network into multiple failure domains

The amount of separation between the failure domains depends on the the strength of the separation

Watch out for fate sharing (should cover this later in the presentation)

Two Types:

Hierarchy

Virtualization

Can be mixed/blended

Many grey areas between these


TECRST-2021

13881_06_2007_c1 25


Going back to our design goals

Redundancy (Resiliency)

Breaking the network up into smaller pieces allows us to design, understand, and troubleshoot smaller pieces

This adds to the resiliency of the network

Simplicity

Breaking the network up into smaller pieces allows us to break a single large problem into a number of smaller, simpler problems


TECRST-2021

13881_06_2007_c1 26


What Do We Gain by Hiding Information?

Improved Stability

Improved Convergence

A tradeoff

Some types of information hiding cost more, in processing time, than the cost of computing across the information in the first place

Essentially, try to hide the right amounts of information in the right places...

Apparent Simplicity

A tradeoff

Sometimes, the cost of overall complexity is higher than the offsets in increased simplicity in one specific area or topology


TECRST-2021

13881_06_2007_c1 27


Topological

Divide the network along topological ―choke points‖

Aggregate reachability information

Aggregate topology information

Aggregate traffic flows

Distribution

Access

Core

Aggregation

Two Directions


TECRST-2021

13881_06_2007_c1 28


Logical

Divide the network into multiple topologies

Divide topology information between topologies

Leak minimal information between topologies

The most common implementation

Split ―outside routes‖ from ―next hop routes‖

Advertise in two different routing protocols, an EGP and an IGP

Two Directions


TECRST-2021

13881_06_2007_c1 29

Hiding Reachability

IP addressing is built around the concept of summarizing reachability information

A doesn‘t advertise each of the host addresses attached to its interface, but rather a range of addresses, or a network address

19

2.1

68

.1.0

/29

.1.2

.3

.5

.4

.6

A

192.168.1.1192.168.1.2192.168.1.3192.168.1.4192.168.1.5192.168.1.6

192.168.1.0/29


TECRST-2021

13881_06_2007_c1 30

Hiding Reachability

In the same way, summarizing multiple networks into one advertisement just increases the scope of reachable hosts

192.168.1.0/29 and 192.168.1.8/29 can be aggregated (summarized) to one advertisement, 192.168.1.0/28

To routers and devices beyond the summarization point, all the hosts from 192.168.1.0 through 192.168.1.15 are reachable through A 1

92

.16

8.1

.8/2

9

.1.2

.3

.5

.4

.6

19

2.1

68

.1.0

/29

.1.2

.3

.5

.4

.6

19

2.1

68

.1.0

/28

A


TECRST-2021

13881_06_2007_c1 31

(192.168).00000001.00000000

(192.168).00000010.00000000

(192.168).00000011.00000000

(192.168).00000000.00000000

24

bits

22

bits

28 destinations

210 destinations

Hiding Reachability

Seen from the binary perspective, as you make the prefix length shorter, you move the network/host separation line to the left

As you move the red line to the left, you encompass more reachable destinations in the same advertisement, but you have fewer advertisements


TECRST-2021

13881_06_2007_c1 32

Hiding Reachability

192.168.1.0/24, 192.168.2.0/24, and 192.168.3.0/24 can be advertised as 192.168.0.0/22

Rather than three networks, each with 255 addresses (253 hosts), A advertises a single network, with 1024 addresses

192.168.1.0/24

192.168.2.0/24

192.168.3.0/24

253 hosts

192.168.0.0/22

1 network

1024 addresses

3 networks

255 addresses each


TECRST-2021

13881_06_2007_c1 33

192.168.1.0/24

192.168.2.0/24

192.168.3.0/24

A

B

C

D

192.168.0.0/22

Hiding Reachability

Address summarization also hides changes in the network

Even if the link between A and C fails, A can still advertise the 192.168.0.0/22 address space (as long as 192.168.2.0/24 isn‘t reachable via some other path)

Routers beyond A don‘t need to know about the reachability or topology change

Summary doesn‘t

change!


TECRST-2021

13881_06_2007_c1 34

Hiding Reachability

One way of looking at hierarchical design is to determine the difference summarization makes statistically

If we know the rate at which prefixes change state within a network, we can predict how many state changes any given router will need to adjust to in a given time period

For instance suppose we know the average prefix will change once every month. What impact will this have on a large network?

Assessing the Impact


TECRST-2021

13881_06_2007_c1 35

1000 routes

1000 routes

1000 routes

1000 routes

4000+100 routes

400+100 routes

Hiding Reachability

1000 routes each failing once/month means 4100/30 = 136.7 state changes per day in the core of this network

Summarizing each 1000 route area into 100 routes reduces the core to 500, rather than 4100, routes

Summarization hides individual route changes, so we are down to 100/30 = 3.3 state changes per day



TECRST-2021

13881_06_2007_c1 36

Hiding Topology

Topology information describes how devices are interconnected in the network

While topology information is useful, we‘d like to hide this information at some point in the network

Hiding topology information reduces the amount of data routers need to process when converging

C

A

B

10.1.1.0/24

D

A is connected to BA is connected to CB is connected to DC is connected to DD is connected to10.1.1.0/24

C is connected to 10.1.2.0/24

B is connected to10.1.2.0/24


TECRST-2021

13881_06_2007_c1 37

Hiding Topology

Hiding topology information also hides information about changes in the topology

C advertises reachability to 10.1.1.0/24

If the F to G link fails, C can still reach 10.1.1.0/24 (although the metric might change)

If B can still use C to reach 10.1.1.0/24, does B need to know about the F to G link failure?

No!

C

A B

10.1.1.0/24

G

D

E F

C can reach

10.1.1.0/24, and

I‘m connected to

C!

Hide

topology

here


TECRST-2021

13881_06_2007_c1 38

Virtualization

Virtualization is placing two apparently separate resources on top of a single resource

If every application stream over every IP pair over every logical subnet had its own physical path, there would be no virtualization

Virtualization is an extremely powerful tool

It allows multiple logical topologies to reside on a single underlying topology or network

Red

DWDM over fiber

Blue

100

802.1q VLANs

101

Silver

Virtual Topologies

Gold

xxx

TCP/IP Sessions

yyy


TECRST-2021

13881_06_2007_c1 39

Virtualization

Virtualization always introduces fate sharing

If an underlying topology, or network, fails, all overlaying topologies fail as well

This is fate sharing

Fate sharing makes virtualization complex to design and troubleshoot

The more ―global‖ the virtualization, the more added complexity

Red

DWDM over fiber

Blue

100

802.1q VLANs

101

Silver

Virtual Topologies

Gold

xxx

TCP/IP Sessions

yyy


TECRST-2021

13881_06_2007_c1 40

Virtualization

Control Plane Only

EGP (BGP) over IGP (EIGRP, OSPF, or IS-IS)

Separates control plane information into internal and external

Fairly simple to implement and deploy

Data Plane Only

L3 Tunneling (most implementations), including L3VPNs

Multiple forwarding tables with a single routing protocol database (or instance)

Moderately simple to implement and deploy

L2VPNs Multiple virtual Layer 2 networks on top of a single IP network

Multiple routing and forwarding tables

Moderately simple to implement and deploy

Virtual Networks Such as MTR

Multiple virtual topologies on a single IP infrastructure

Multiple routing and forwarding tables

Difficult to implement and deploy


TECRST-2021

13881_06_2007_c1 41

Working with Addressing and Summarization


TECRST-2021

13881_06_2007_c1 42

Addressing and Summarization

Address Allocation

Summary Metrics

Aggregation Issues

Aggregation Techniques


TECRST-2021

13881_06_2007_c1 43

Address Allocation

A hierarchical topology isn‘t enough to hide reachability information—the way the addressing is laid out in the network is also critical

There are several possible methods you can use to assign addresses within a network

Allocating addresses as they are requested is a common method

This only creates summarization points if you happen to get address allocation requests that coincide with the topology of the network

10.1.2.0/24

10.1.1.0/24

10.1.3.0/24

Can‘t

summarize hereI asked

second!

I asked

first!

I asked

third!


TECRST-2021

13881_06_2007_c1 44

Address Allocation

Assigning addresses based on the political structure of the organization is another method

10.1.x.x is marketing

10.2.x.x is sales

This only creates summarization points if the political structure of the corporation follows the logical topology of the network 10.1.1.0/24

10.1.2.0/24

10.1.3.0/24

10.2.3.0/24

10.2.2.0/24

10.2.1.0/24

Can‘t

summarize

here


TECRST-2021

13881_06_2007_c1 45

10.2.1.0/24

10.2.2.0/24

10.1.2.0/24

Address Allocation

Assigning address by the geographic location of the device or network is also common

10.1.0.0/16 is Nevada

10.2.0.0/16 is California

This only creates summarization points if the topological and geographical layouts of the network coincide, which isn‘t always the case

10.1.1.0/24

Can‘t

summarize here

California

Nevada


TECRST-2021

13881_06_2007_c1 46

Address Allocation

Addressing needs to follow the network topology to create summarization points

Any scheme will create summarization points as long address allocation happens to follow the network topology

But, it‘s best just to use topological addressing from the start

Creates summarization points

Allows flexibility in moving sections of a network from one place to another (moving connections to network regions)

10.1.2.0/24

10.1.1.0/2410.1.3.0/24

10.1.1.0/24

10.1.2.0/24

10.1.3.0/24

10.2.3.0/2410.2.2.0/24

10.2.1.0/24

10.2.1.0/24

10.2.2.0/24

10.1.2.0/2410.1.1.0/24

California

Nevada


TECRST-2021

13881_06_2007_c1 47

Address Allocation

Start with a very large address space

Summarization always wastes address; this is a natural consequence of hiding reachablity

You could use private address space

It might be possible to gain huge summarizable address spaces by deploying IPv6 in the future

Try to balance between

Conserving address space

Providing room to grow without breaking summarization

10.1.2.0/24

10.2.2.0/24

10.3.2.0/24

10.3.1.0/24

10.2.1.0/24

10.1.1.0/24

10.1.0.0/16

10.2.0.0/16

10.3.0.0/16


TECRST-2021

13881_06_2007_c1 48

Address Allocation

Several techniques can be used to conserve address space, where needed

Use /31‘s on point-to-point links to conserve address space

Avoid IP unnumbered, for management reasons—you can‘t reach the remote device if the remote link fails

Don‘t be frightened of odd length masks where it makes sense

10.1.2.0/24

10.2.2.0/24

10.3.2.0/24

10.3.1.0/24

10.2.1.0/24

10.1.1.0/24

10.1.0.0/16

10.2.0.0/16

10.3.0.0/16

/31 on point-to-point


TECRST-2021

13881_06_2007_c1 49

Summary Metrics

In all interior gateway protocols, the summary metric is dependant on the metrics of the components

The metric of the highest or lowest cost component route is chosen as the summary metric

A

BC

10

.1.0

.0/2

4

Co

st 1

0

10

.1.1

.0/2

4

Co

st 2

0

10

.2.0

.0/2

4

Co

st 1

0

10

.2.1

.0/2

4

Co

st 2

0

10.1.0.0/23

Cost 20

10.2.0.0/23

Cost 20


TECRST-2021

13881_06_2007_c1 50

10.1.0.0/23

Cost 20

10.2.0.0/23

Cost 20

Summary Metrics

If the component the metric was taken from flaps, the summary flaps as well!

You‘re using the summary to hide reachability information, but it‘s passing metric information through, and the routers beyond the summary are still working to keep up with the changes

A

BC

10

.1.0

.0/2

4

Co

st 1

0

10

.1.1

.0/2

4

Co

st 2

0

10

.2.0

.0/2

4

Co

st 1

0

10

.2.1

.0/2

4

Co

st 2

0

10.1.0.0/23

Cost 10

10.2.0.0/23

Cost 20


TECRST-2021

13881_06_2007_c1 51

Summary Metrics

EIGRP takes its summary metric from the component route with the smallest metric

OSPF takes its summary cost from the component route with the smallest metric

If no compatible rfc1583 is configured, in which case the cost from the component with the largest cost is used

IS-IS takes its summary cost from the component route with the largest cost


TECRST-2021

13881_06_2007_c1 52

Summary Metrics

Use a loopback interface to force the metric to remain constant

Create a loopback interface within the summary address range with a higher or lower metric than any other component

The summary will use the metric of the loopback, which doesn‘t ever go down

A static route to null0 on the summarizing router can also be used

You can sometimes use a route map to force the summary‘s metric to always be the same

A

B

10

.1.0

.0/2

4

Co

st 1

0

10

.1.1

.0/2

4

Co

st 2

0

10.1.0.0/23

Cost 10

loopback 0

ip address 10.1.1.1 255.255.255.255

ip ospf cost 10

10.1.0.0/23


TECRST-2021

13881_06_2007_c1 53

Aggregation Issues

B and C are advertising 10.1.0.0/23 to A with a metric of 30

A has two routes to 10.1.0.0/23

B with a cost of 30

C with a cost of 40

A forwards traffic to 10.1.1.1 to B (40), although this is not the optimal route to reach 10.1.1.0/24 (30)

Summarization hides information, so the best path may not always be chosen

C

10 20

10.1.0.0/24 10.1.1.0/24

A

B

D E20

1010

10

.1.0

.0/2

3 (3

0)

10

.1.0

.0/2

3 (3

0)

Summary Suboptimal Routing


TECRST-2021

13881_06_2007_c1 54

Aggregation Issues

When summarizing down the hierarchy in OSPF, we can use manual summaries instead of stub areas

Always prefer to summarize more information rather than less C

10

20

10.1.0.0/24 10.1.1.0/24

A B

D

E

20

1010

Are

a b

ord

er

10

.1.0

.0/2

3

area 1 range 10.1.0.0 255.255.254.0

10

.1.0

.0/2

31

0.1

.1.0

/24

area 1 range 10.1.0.0 255.255.254.0

area 1 range 10.1.1.0 255.255.255.0

no discard-route



TECRST-2021

13881_06_2007_c1 55

Aggregation Issues

It‘s also possible to use LSA type 3 filtering to solve this problem

Permit only a default plus some number of longer prefix routes to allow optimal routing to those destinations C

10

20

10.1.0.0/24 10.1.1.0/24

A B

D

E

20

1010

Are

a b

ord

er

0.0

.0.0

/0

ip prefix-list AREA_1_OUT seq 10 permit 0.0.0.0

!

router ospf 1000

area 1 filter-list prefix AREA_1_OUT out

0.0.0.0/0

0.0

.0.0

/01

0.1

.1.0

/24

ip prefix-list AREA_1_OUT seq 10 permit 0.0.0.0/0

ip prefix-list AREA_1_OUT seq 20 permit 10.1.1.0/24

!

router ospf 1000

area 1 filter-list prefix AREA_1_OUT out



TECRST-2021

13881_06_2007_c1 56

Aggregation Issues

IS-IS automatically summarizes down the hierarchy

You can use route leaking to leak more specific routes when optimal routing towards the core is important

C

10

20

10.1.0.0/24 10.1.1.0/24

A B

D

E

20

1010

L1

/L2

bo

rde

r

0.0

.0.0

/0

0.0

.0.0

/01

0.1

.1.0

/24

access-list 100 permit ip 10.1.1.0 0.0.0.255

!

router isis

redistribute isis ip level-2 into level-1 distribute-list 100

metric-style wide



TECRST-2021

13881_06_2007_c1 58

Aggregation Issues

EIGRP always requires either summarization or filtering to reduce routing information from the core towards the edge

There are several techniques we can use to summarize routing information towards the edge and allow more specific information to leak to prevent suboptimal routing

As with all the other protocols, you need to carefully weigh the gains in network stability and scaling against the gains from optimal routing!

C

10

20

10.1.0.0/24 10.1.1.0/24

A B

D

E

20

1010



TECRST-2021

13881_06_2007_c1 59

Aggregation Issues

Rather than summarizing, redistributed static routes paired with distribute lists can be used

C

10

20

10.1.0.0/24 10.1.1.0/24

A B

D

E

20

1010

10

.1.0

.0/2

31

0.1

.1.0

/24

ip route 10.1.0.0 255.255.254.0 null0

!

access-list 10 permit 10.1.0.0 0.0.1.255


!

router eigrp 100

redistribute static

default-metric 1000 1 255 1 1500

distribute-list 10 out serial 0/0

10

.1.0

.0/2

3

ip route 10.1.0.0 255.255.254.0 null0

!


!

router eigrp 100

redistribute static


distribute-list 10 out serial 0/0



TECRST-2021

13881_06_2007_c1 60

Aggregation Issues

Another option is to create a pair of summaries containing the more and less specific routes

EIGRP also allows leaking more specifics past a summary C

10

20

10.1.0.0/24 10.1.1.0/24

A B

D

E

20

1010

0.0

.0.0

/01

0.1

.1.0

/24

interface serial 0/0

ip summary-address 10.1.1.0 255.255.255.0 250

ip summary-address 0.0.0.0 0.0.0.00

.0.0

.0/0


ip summary-address 10.1.1.0 255.255.255.0 250

ip summary-address 0.0.0.0 0.0.0.0



TECRST-2021

13881_06_2007_c1 61

Aggregation Issues

Routers B and C are summarizing 10.1.0.0/24 and 10.1.1.0/24 into a single advertisement, 10.1.0.0/23, towards A

Routers B and C are also advertising a default route only towards each other through 10.1.0.0/24 and 10.1.1.0/24

10.1.0.0/23 10.1.0.0/23

10.1.0.0/24

10.1.1.0/24

A

B C0.0.0.0/0

Distance Vector Summary Black Holes


TECRST-2021

13881_06_2007_c1 62

Aggregation Issues

If Router B loses its link to 10.1.0.0/24, what happens?

Router B isn‘t learning about 10.1.0.0/24 through C, since C is only advertising a default route—so B no longer knows how to get there

The routes advertised by B and C to A look the same before and after the failure

10.1.0.0/24

10.1.1.0/24

A

B C

10.1.0.0/23 10.1.0.0/23

0.0.0.0/0

10.1.1.0/24 isn‘t learned from A



TECRST-2021

13881_06_2007_c1 63

Aggregation Issues

A could still forward traffic destined to 10.1.0.1 to B

We have a summarization black hole

If A is load sharing per packet, every other packet will be dropped

If A is load sharing per session, then some hosts will be able to reach destinations on 10.1.0.0/24, and others won‘t

10.1.0.0/24

10.1.1.0/24

A

B C

10.1.0.1

10.1.0.0/23 10.1.0.0/23



TECRST-2021

13881_06_2007_c1 64N

o

Su

mm

ari

za

tion

Aggregation Issues

One way to solve this problem is to always have at least one unsummarized link between the summarizing routers

The summarizing routers always have someplace to send the traffic if they lose connectivity to the link

Another option is not to summarize both up the hierarchy and down the hierarchy

This reduces network scaling!

10.1.0.0/23 10.1.0.0/23

10.1.0.0/24

10.1.1.0/24

A

B C0.0.0.0/0

Don‘t summarize up and down



TECRST-2021

13881_06_2007_c1 65

Aggregation Issues

Routers E and F are not intended to transit traffic between C and D

Routers C and D issue summaries containing 10.1.1.0/24

Router A chooses D as its best path to the summary

The link from Router D to Router E fails

How can we prevent Router D from using the link through F to reach 10.1.1.0/24?

A

C D

B

10.1.1.0/24

E

F

10.1.2.0/24

10.1.1.0/24

10.1.0.0/16

Link State Summary Suboptimal Routing


TECRST-2021

13881_06_2007_c1 66

Aggregation Issues

Place a link between C and D within the same area as E and F

The link cost between C and D should be lower than the link cost through F, causing D to route through this new link

New link

A

C D

B

10.1.1.0/24

E

F

10.1.2.0/24

10.1.1.0/24

10.1.0.0/16

Link State Summary Suboptimal Routing


TECRST-2021

13881_06_2007_c1 67


In this network, it appears almost impossible to summarize at any point because of the addressing

Summarize anyway!

Router B can advertise 10.1.0.0/22

Routes which don‘t fall within this summary range will be leaked through to Router A

10.1.1.0/24

10.2.1.0/24

10.1.2.0/24

10.2.4.0/24

10.1.3.0/24

10.2.2.0/24

10.1.4.0/24

10.2.3.0/24

10.1.5.0/24

10.2.4.0/24

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC

Leaking More Specifics


TECRST-2021

13881_06_2007_c1 68


Summarizing to 10.1.0.0/22 on Router B will reduce the number of routes at Router A by two

10.1.0.0/22

10.2.1.0/24

10.2.4.0/24

10.2.2.0/24

10.1.4.0/24

10.2.3.0/24

10.1.5.0/24

10.2.5.0/24

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC



TECRST-2021

13881_06_2007_c1 69


We can do the same thing with the 10.2.0.0 networks on Router C, with 10.2.0.0/21, dropping the number of routes on Router A by two more

The more specific information is still leaked through the summary, so routing still works

10.1.0.0/22

10.2.1.0/24

10.2.4.0/24

10.2.0.0/21

10.1.4.0/24

10.1.5.0/24

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC



TECRST-2021

13881_06_2007_c1 70


If one of the networks behind Router B fails, traffic for that network will be forwarded to Router C

At C, it will be discarded because of the NULL0 route automatically created with the summary

The only danger here is that the link from A to C may be overwhelmed with the extra traffic

10.1.0.0/22

10.2.1.0/24

10.2.4.0/24

10.2.0.0/21

10.1.4.0/24

10.1.5.0/24

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC

Packets dropped to null 0



TECRST-2021

13881_06_2007_c1 71


It‘s also useful to leak more specifics along with (or through) an aggregate

C should receive as few routes as possible

But still optimally route to 10.1.1.0/24 and 10.1.2.0/24 dynamically

There are several ways to accomplish this

Redistributed static routes and route filters

Overlapping Aggregates

Route Leaking (EIGRP)

10.1.0.0/16

10

.1.0

.0/1

6

10

.1.0

.0/1

6

A B

C

10.1.1.0/24 10.1.2.0/24



TECRST-2021

13881_06_2007_c1 72


router eigrp 100

redistribute static route-map aggroutes


distribute-list 20 out serial0/0

!

ip route 10.1.0.0 255.255.0.0 null0

!

route-map agg-routes permit 10

match ip address 10

match interface serial 0/0

!



router eigrp 100

redistribute static route-map aggroutes


distribute-list 20 out serial0/0

!

ip route 10.1.0.0 255.255.0.0 null0

!

route-map agg-routes permit 10

match ip address 10

match interface serial 0/0

!



10.1.0.0/16

10

.1.0

.0/1

6

10

.1.0

.0/1

6

A B

C

10

.1.1

.0/2

4

10

.1.2

.0/2

4

10.1.1.0/24 10.1.2.0/24



TECRST-2021

13881_06_2007_c1 73


EIGRP allows overlapping summaries

Set the administrative distance on the longer prefix so it‘s not installed...


....

ip summary-address eigrp 1 10.1.0.0 255.255.0.0

ip summary-address eigrp 1 10.1.1.0 255.255.255.0 255

Interface serial 0/0

....

ip summary-address eigrp 1 10.1.0.0 255.255.0.0

ip summary-address eigrp 1 10.1.2.0 255.255.255.0 255

10.1.1.0/24 10.1.2.0/24

10.1.0.0/16

10

.1.0

.0/1

6

10

.1.0

.0/1

6

A B

C

10

.1.1

.0/2

4

10

.1.2

.0/2

4



TECRST-2021

13881_06_2007_c1 74


EIGRP can leak more specific routes through a summary, as well

CSCed01736, 12.3(11.01)T

route-map LeakList permit 10

match ip address 1

!

access-list 1 permit 10.1.2.0

!

interface Serial0/0

ip summary-address eigrp 1

10.1.0.0 255.255.0.0 leak-map LeakList

10.1.1.0/24 10.1.2.0/24

10.1.0.0/16

10

.1.0

.0/1

6

10

.1.0

.0/1

6

A B

C

10

.1.1

.0/2

4

10

.1.2

.0/2

4

route-map LeakList permit 10

match ip address 1

!

access-list 1 permit 10.1.1.0

!

interface Serial0/0


10.1.0.0 255.255.0.0 leak-map LeakList



TECRST-2021

13881_06_2007_c1 75


We can also get some gains by trying to do less, and using smaller summary blocks

Router B can advertise 10.1.2.0/23, saving one route

Router C can advertise 10.1.4.0/23

Router C can advertise 10.2.2.0/23

The gains might seem small, but with enough work, they can build up into significant savings

10.1.1.0/24

10.2.1.0/24

10.1.2.0/23

10.2.4.0/24

10.2.2.0/23

10.1.4.0/23

10.2.4.0/24

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC

Smaller Aggregates


TECRST-2021

13881_06_2007_c1 76

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC


We can combine the larger summaries with the smaller summaries to have the most impact

These are two very effective tools if used together, with a little planning

10.1.0.0/22

10.2.1.0/24

10.2.4.0/24

10.2.0.0/21

10.1.4.0/23

Smaller Aggregates


TECRST-2021

13881_06_2007_c1 77


Balance this sort of optimization with the maintenance work it produces in the network

Leaking routes through summaries means checking what adding a new route will do to the summaries and the routing

Summarizing on small blocks means considering the summaries when moving a set of addresses

10

.1.1

.0/2

4

10

.2.1

.0/2

4

10

.1.2

.0/2

4

10

.2.4

.0/2

4

10

.2.2

.0/2

4

10

.1.4

.0/2

4

10

.2.3

.0/2

4

10

.1.5

.0/2

4

10

.1.3

.0/2

4

10

.2.5

.0/2

4

A

BC

10.1.0.0/22

10.2.1.0/24

10.2.4.0/24

10.2.0.0/21

10.1.4.0/23

Smaller Aggregates


TECRST-2021

13881_06_2007_c1 78

Hiding Topology


TECRST-2021

13881_06_2007_c1 79

Hiding Topology

Topology information is naturally hidden in distance vector protocols, beyond the next hop

C and D only advertise that they can reach 10.1.1.0/24, not that they are connected to D, which is then connected to 10.1.1.0/24

C

A

B

10.1.1.0/24

DI can reach 10.1.1.0/24

I can reach 10.1.1.0/24

I can reach 10.1.1.0/24

I can reach 10.1.1.0/24

Distance Vector


TECRST-2021

13881_06_2007_c1 80

Hiding Topology

Distance vector protocols can still have too much topology information

Multiple parallel links can slow down convergence because of overwhelming topology information

General EIGRP rule of thumb: There should be no more paths in the topology table than are allowed to be installed in the routing table

(show ip eigrp topology all vs.maximum paths)

A

10.1.1.0/24

B

Distance Vector


TECRST-2021

13881_06_2007_c1 81

Hiding Topology

In link state protocols, routers flood information about the state of their links to all other routers, carrying topology information to all the routers in the network

All the routers receiving the flooded link state information are said to be in the same flooding domain

We summarize topology information into reachability information at a flooding domain border

C

A B

10.1.1.0/24

G

D

E F

Border

Connected to E, F, and 10.1.1.0/24

Connected to D and G

Connected to D and G

Connected to C, E, and F

Link State Flooding Domains


TECRST-2021

13881_06_2007_c1 82

Hiding Topology

OSPF

Flooding Domain == Area

Flooding Domain Border == Area Border Router

Link State Summary == Type 3

Contains only reachability and cost information, no topology

External == Type 5

Contains only reachability and cost information, no topology

Autonomous System Border == Type 4

How to reach a router injecting reachability information from outside OSPF (type 5‘s)

C

A

B

D

10.1.2.0/24

10.1.1.0/24

10.1.2.0/24 external AA BB AB 10.1.1.0/24B CC B

10.1.2.0/24 external AA C10.1.1.0/24 C

Redistributed

Are

a

bo

rde

r

Area 0

10

.1.3

.0/2

4

10.1.3.0/24 C



TECRST-2021

13881_06_2007_c1 83

10

.1.3

.0/2

4

Hiding Topology

Decoding OSPF Stub Areas

―Stub‖ == no link state summaries (type 3)

―Totally‖ == no external information (type 4 or 5)

―Not so‖ == Externals injected as type 7‘s and translated at the border

Stub area receives external routing information from outside the area only (no redistribution within the area)

C

A

B

D

Are

a

bo

rde

r

10.1.2.0/24

10.1.1.0/24

10.1.2.0/24 external AA Cdefault C

Redistributed

Stub Area

No information about 10.1.1.0/24


Area 0

10.1.3.0/24 C



TECRST-2021

13881_06_2007_c1 84

10

.1.3

.0/2

4

Hiding Topology

Totally stubby areas receive no information about reachability to external or internal destinations

In a ―Not So Stubby Area (NSSA),‖ or a ―Totally Not So Stubby Area (Totally NSSA),‖ D could originate information about destinations external to OSPF

You should use stub areas by default

Supply minimal information where possible

Consider suboptimal routing when necessary

C

A

B

D

Are

a

bo

rde

r

10.1.2.0/24

10.1.1.0/24

Default C

Redistributed

Totally Stub

No information about 10.1.1.0/24 or 10.1.2.0/24


Area 0

10.1.3.0/24 C



TECRST-2021

13881_06_2007_c1 85

Hiding Topology

1000 routers: 90 to 100 ms




350

300

200

150

100

50

250

Milliseco

nd

s

5000

10000

15000

20000

25000


Considering SPF run time for a link state protocol, convergence times vary around the number of routers and the number of routes:


TECRST-2021

13881_06_2007_c1 86

Hiding Topology

Changing the number of routes can make up to a 10 millisecond difference in SPF run time

Changing the number of routers can make up to a 200 millisecond difference in SPF run time

The number of routers is the primary determinant in SPF run time

350

300

200

150

100

50

250

Milliseco

nd

s

5000

10000

15000

20000

25000

10 ms

200 ms



TECRST-2021

13881_06_2007_c1 87

Hiding Topology

This isn‘t always the case

The primary cost in convergence is route installation time

Varies platform to platform, and Cisco IOS® to Cisco IOS

350

300

200

150

100

50

250

Milliseco

nd

s

5000

10000

15000

20000

25000

10 ms

200 ms



TECRST-2021

13881_06_2007_c1 88



TECRST-2021

13881_06_2007_c1 89


Hierarchical Design

Two Layer Hierarchy

Three Layer Hierarchy


TECRST-2021

13881_06_2007_c1 90

Hierarchical Design

Zones (or Nodes)

A topologically defined part of the network

Attached to other parts of the network through choke points

Choke Points

Places where zones or nodes are connected together

Zones

Choke Points

Basic Concepts


TECRST-2021

13881_06_2007_c1 91

Hierarchical Design

Each zone represents a failure domain

Choke points provide:

A place to aggregate reachability information

A place to aggregate topology information

A place to aggregate traffic flows

A place to apply traffic policy

Basic Concepts


TECRST-2021

13881_06_2007_c1 92

Hierarchical Design

There are two basic designs:

Two layer

Three layer

Which one is right for a specific network?

Rule of Thumb:

Balance simplicity, optimal routing, and functional separation

How Many Layers?


TECRST-2021

13881_06_2007_c1 93

Hierarchical Design

Geography

Networks contained in smaller spaces lend themselves to two layers

Networks with more ―reach‖ lend themselves to three layers

Topology Depth

The maximum number of hops from one edge to another

The greater the depth, the more layering will help the design

How Many Layers?


TECRST-2021

13881_06_2007_c1 94

Hierarchical Design

Topology Design

The more complex the design, the more splitting the network up into zones will help the design

Policy Implementation

Traffic engineering tends to prefer two layer designs

Resource restriction policies tend to prefer three layer designs

How Many Layers?


TECRST-2021

13881_06_2007_c1 95

Hierarchical Design

Moving the boundary between two pieces of the network may create a choke point which didn‘t exist before

With the logical boundary point behind the lower routers, based on the divisional structure, there‘s no place to summarize

10

.1.0

.0/2

4

10

.1.2

.0/2

4

10

.2.0

.0/2

4

10

.2.2

.0/2

4

10

.1.1

.0/2

4

10

.1.3

.0/2

4

10

.2.1

.0/2

4

10

.2.3

.0/2

4

Sales

Marketing

Logistics

Engineering

No

summarization Logical

boundary

points

Creating Choke Points


TECRST-2021

13881_06_2007_c1 96

Hierarchical Design

The logical network structure no longer follows the corporate departments

We now have a point at which we can summarize routes!

Logical

boundary

point10.1.0.0/22

10.2.0.0/22

10

.1.0

.0/2

4

10

.1.2

.0/2

4

10

.2.0

.0/2

4

10

.2.2

.0/2

4

10

.1.1

.0/2

4

10

.1.3

.0/2

4

10

.2.1

.0/2

4

10

.2.3

.0/2

4

Sales

Marketing

Logistics

Engineering


What happens if we move the logical boundary point up one layer?


TECRST-2021

13881_06_2007_c1 97

Hierarchical Design

In this case, moving the logical boundary point down one layer can be used to improve summarization

With EIGRP, it‘s just a matter of configuring summaries in the best possible place

With OSPF and IS-IS, some restructuring of the area or routing domain borders may be needed to change where summarization takes place

Logical

boundary

point

10

.1.0

.0/2

4

10

.1.2

.0/2

4

10

.1.1

.0/2

4

10

.1.3

.0/2

4

10

.2.1

.0/2

4

10

.2.3

.0/2

4

10

.2.0

.0/2

4

10

.2.2

.0/2

4



TECRST-2021

13881_06_2007_c1 98

Hierarchical Design

Sometimes, you need to change the topology to build a choke point

A full mesh is just a hierarchical network in disguise!



TECRST-2021

13881_06_2007_c1 99

Hierarchical Design

Separating complexity from complexity through choke points amplifies the benefits of hierarchy

Sometimes, logical or physical topology changes are needed to separate complexity from complexity



TECRST-2021

13881_06_2007_c1 100

Two Layer Hierarchy

The core gets traffic from one topological area of the network to another

High Speed Switching is the focus

Within the core, avoid

Policy (the more complex the more to avoid it) within the core

Reachability and topology aggregation

Core

Aggregation

Basic Concepts


TECRST-2021

13881_06_2007_c1 101

Two Layer Hierarchy

Core routers should summarize routing information towards the aggregation layer

Typically, the fewer number of routes advertised towards the edge, the better

Routing policy may also be implemented at the core edge

How many and what routes will be accepted from each aggregation area, etc.

Core

Aggregation Summary

Policy

Basic Concepts


TECRST-2021

13881_06_2007_c1 102

Two Layer Hierarchy

The aggregation layer provides user attachment points

Information about the edge should be hidden from the core using summarization and topology hiding techniques

Core

Aggregation Summarize

Basic Concepts


TECRST-2021

13881_06_2007_c1 103

Two Layer Hierarchy

Policy should be placed at the edge of the network

Traffic acceptance (based on load and traffic type)

Filtering unwanted traffic

Security policy

Layer 2 and Layer 3 filters apply at the edge

Core

Aggregation

Policy

Basic Concepts


TECRST-2021

13881_06_2007_c1 104

Two Layer Hierarchy

A moderate number of routers are attached to the network

The network doesn‘t have a large wide area component

Distances are small, and all links are similar in speed

Core

Basic Concepts

Small and medium scale campus networks are often modeled as two layer networks


TECRST-2021

13881_06_2007_c1 105

Customers

Two Layer Hierarchy

ISP networks are often modeled on a two layer hierarchy as well

The core is often mesh or a set of rings, with each POP modeled as a ring or a two layer hierarchy

Topology information is summarized between the POPs and the network core

Address summarization is generally from the core towards the POPs

Core

POP

POP

POP

POP

POP

Basic Concepts


TECRST-2021

13881_06_2007_c1 106

Two Layer Hierarchy

In an EIGRP network, the hierarchy is created through summarization, rather than through some protocol defined boundary

There are no ―areas‖ or other ways of dividing a network built into EIGRP itself, since topology information is hidden at each hop in the network anyway

EIGRP


TECRST-2021

13881_06_2007_c1 107

Two Layer Hierarchy

Summarization from the edge towards the core hides details about the user access points from the core

Summarization towards the core can cause routing black holes, however

Summarization

EIGRP


TECRST-2021

13881_06_2007_c1 108

Two Layer Hierarchy

Summarization from the core towards the edge can hide details about the core from the edge routers, as well

This type of summarization can cause suboptimal routing, however

Summarization

EIGRP


TECRST-2021

13881_06_2007_c1 109

Two Layer Hierarchy

OSPF creates edges through areas, using Area Border Routers (ABRs)

Typically, with a two level hierarchy, the ABRs are at the edge of the core

The core is area 0

Area Border

OSPF


TECRST-2021

13881_06_2007_c1 110

Two Layer Hierarchy

Summarization is configured at the ABR, on the edge of the edge/aggregation areas and the core

Summarization can also be configured to reduce the amount of reachability information carried into the areas

Area 0

Su

mm

ari

za

tion

OSPF


TECRST-2021

13881_06_2007_c1 111

Two Layer Hierarchy

To remove virtually all reachability information into the areas, declare them totally stub or not so totally stub areas

Use totally stub areas when there is a single area border, or when suboptimal routing of traffic exiting the area isn‘t an issue

Use stub areas when there is more than one area border, and optimal routing of traffic leaving the area is important

Area 0

network .... area 1 stub

network .... area 2 stub nosummary

OSPF


TECRST-2021

13881_06_2007_c1 112


The core gets traffic from one topological area of the network to another: High Speed Switching

Within the core, avoid

Policy (the more complex the more to avoid it) within the core

Aggregation

Core

Distribution

Access

Basic Concepts


TECRST-2021

13881_06_2007_c1 113


Address summarization and aggregation occur at the distribution layer

Address Summarization

Within the distribution layer

At the edge of the distribution layer and the core

At the edge of the distribution layer and the access layer

At both edges of the distribution layer

Traffic Aggregation

High to low speed link transitions

Core

Distribution

Access

Summary

Tra

ffic

ag

gre

ga

tio

n

Basic Concepts


TECRST-2021

13881_06_2007_c1 114


The distribution layer is where most of the policy in a three layer network should reside

Routing Policy

Routes accepted from the access layer

Routes will be passed from the core into the access layer

Traffic Engineering

Directing traffic into the best core entry point

Access layer failover

Traffic filters

This should take all the policy load off the network core

Core

Distribution

Access

Policy

Basic Concepts


TECRST-2021

13881_06_2007_c1 115


Summarization should be avoided between distribution layer routers!

This can cause a lot of odd and hard to troubleshoot problems within the network

Focus summarization and policy up and down the layers, rather than along the layers

Core

Distribution

Access

No

su

mm

ari

za

tio

n!

Basic Concepts


TECRST-2021

13881_06_2007_c1 116


The access layer provides ports for the users to plug in to

Traffic filtering and packet policies are implemented here

Traffic acceptance (based on load and traffic type)

Filtering unwanted traffic at Layer 2 and Layer 3

Security policy

Core

Distribution

Access

Policy

Basic Concepts


TECRST-2021

13881_06_2007_c1 117


Deeper hierarchy doesn‘t change EIGRP‘s fundamental design concepts

The distribution layer should be the blocking point for EIGRP queries

Provide minimal information toward the core

Provide minimal information toward the access

Access layer routers should be considered for configuration as EIGRP stubs

We discuss EIGRP stubs more in hub and spoke topology considerations

Distribution

Access

Core

Summarize

Consider stubs

EIGRP


TECRST-2021

13881_06_2007_c1 118


For OSPF, the question is whether to place the area borders in the distribution layer, or in the core

The answer to this question is, as always, ―it depends‖

There are two rules of thumb we can work with, though:

Separate complexity from complexity

Place area borders to reduce suboptimal routing and to increase summarization

Distribution

Access

Core

OSPF


TECRST-2021

13881_06_2007_c1 119


Complex areas include

Full mesh topologies

Large scale hub and spoke

Highly redundant topologies

Try to separate complex topologies from one another with an area border

You can vary the location of the area borders placing them in the distribution or access layers, depending on the network design

Highly parallel data center

Full mesh core


Highly redundant campus

OSPF


TECRST-2021

13881_06_2007_c1 120

OSPF Two Layer Hierarchy

To remove virtually all reachability information into the areas, declare them totally stub or not so totally stub areas

Use totally stub areas when there is a single area border, or when suboptimal routing of traffic exiting the area isn‘t an issue

Use stub areas when there is more than one area border, and optimal routing of traffic leaving the area is important

network .... area 1 stub

network .... area 2 stub nosummary

Highly parallel data center

Full mesh core


Highly redundant campus


TECRST-2021

13881_06_2007_c1 121



TECRST-2021

13881_06_2007_c1 122


Link State Point-to-Point Broadcast

Controlling Physical Parallelism

Hub and Spoke

Full Mesh

Link State Border Connections


TECRST-2021

13881_06_2007_c1 123


Normally, if a set of routers are connected over a broadcast link, each router would form a neighbor relationship with every other router on the link

This can cause a large amount of flooding over the single broadcast network

To reduce flooding and apparent network complexity, link state protocols elect one router to control flooding

OSPF: Designated Router

IS-IS: Designated Intermediate System

A

B C

D E


TECRST-2021

13881_06_2007_c1 124


To reduce flooding:

In OSPF, a router that receives new information floods it to the DR, which then refloods it to the other connected routers

In IS-IS, the first router to receive new information floods it, and the DIS coordinates database synchronization between the routers

To reduce apparent complexity:

Each connected router advertises a link to the DR/DIS

The DR/DIS advertises a 0 cost link to each connected router

This converts the full mesh to a set of point-to-point links

A

B

D

C

E


TECRST-2021

13881_06_2007_c1 125


If there are only two routers on the broadcast link the DR/DIS adds complexity, rather than removing it

Point-to-point high speed Ethernet segments used in campus environments, data centers, etc.

What could be advertised as a point-to-point is actually advertised as two point-to-points to the DR/DIS

We could reduce the apparent complexity, again, by treating the link as a point-to-point link, rather than as a broadcast link

A

B

D


TECRST-2021

13881_06_2007_c1 126


draft-ietf-isis-igp-p2p-over-lan describes a method for OSPF and IS-IS to treat a broadcast link with only two devices attached as a point-to-point link

Implemented in IS-IS with CSCdu51410, using the isis network interface command

Implemented in OSPF as well, using the ip ospf network interface command

A

B

D

A

B

D

interface FastEthernet 0

isis network point-to-point

ip ospf network point-to-point

....


TECRST-2021

13881_06_2007_c1 127


More redundancy is better, right?

Not always...

There are 64 paths between these two hosts, 26


TECRST-2021

13881_06_2007_c1 128


There Are Several Reasons for Redundancy in a Network:

To provide multiple attachment points for servers and hosts in case of a link or device failure

To provide alternate links through the network in case of link or device failure

To provide optimal routing to services

To provide load sharing in heavily utilized areas


TECRST-2021

13881_06_2007_c1 129


It‘s common to build networks with back-to-back routers for redundancy

The routing protocol sees each of these links as a possible transit path, so each link adds another set of paths the routing protocol must consider when calculating the best path

You want to route to these links, not through them

RP Transit

Paths

HSRP Peers

Server Farm Example


TECRST-2021

13881_06_2007_c1 130

router ospf 100

passive-interface fastethernet 0/0




....

router ospf 100

passive-interface default

no passive-interface fastethernet 1/0

....

-or-


The solution to this is passive-interface

Configuring an interface as passive in EIGRP, OSPF, or IS-IS will cause it not to form neighbor relationships across the link

These networks will still be advertised as reachable destinations, but they will never be advertised as transit links

Server Farm Example


TECRST-2021

13881_06_2007_c1 131


It‘s common to build out alternate links in a network

Adds network resiliency

Can provide optimal routing to resources

Adds additional bandwidth in congested areas of the network

The second link also adds moderate complexity, and more information, into the network

Backup path

Optimal routing

Additional bandwidth


TECRST-2021

13881_06_2007_c1 132


Adding a third link almost always approaches the point of diminishing returns, and adds much more network complexity

When considering adding more redundancy, always balance the increased resiliency against the added complexity

Increased network convergence times

Increased management effort

Increased troubleshooting times


TECRST-2021

13881_06_2007_c1 133

2.5

0 10000

Seconds

Routes

Feasible successor


The impact of greater levels of redundancy on convergence times can be seen in routing protocol scalability testing

Using EIGRP, with a single backup path, it takes about 1.3 seconds for a router with 10000 routes to converge when the best path fails

Best path

fails


TECRST-2021

13881_06_2007_c1 134


Adding the third path increases convergence time to 2 seconds

Adding the fourth path increases convergence time to 2.25 seconds

2.5

0 10000

Seconds

Routes

Best path

fails


TECRST-2021

13881_06_2007_c1 135


High availability studies also show the impact of adding the third link is not all that great

Adding a second link will increase reliability significantly

Adding a third link approaches the point of diminishing returns

Combined with the impact of slower convergence times, higher management costs, and slower troubleshooting, the total downtime in a network may actually increase with the addition of large amounts of redundancy

99.50

99.60

99.70

99.80

99.90

100.00

1 link 2 links 3 links 4 links

Relia

bili

ty


TECRST-2021

13881_06_2007_c1 136


Try to hide this complexity from other parts of the network, if possible

Summarize just the parallel links into a single advertisement at both sides if you‘re using a distance vector protocol

Summary

Summary

If you‘re adding more links to increase the available bandwidth in a specific place in the network


TECRST-2021

13881_06_2007_c1 137


Layer 2 bundling (such MLPPP or EtherChannel®) may be useful to reduce the Layer 3 complexity when using multiple links to build required bandwidth

But be careful of issues with processor utilization due to bundling overhead, troubleshooting complexity, etc.

Link bundle


TECRST-2021

13881_06_2007_c1 138


Consider using High Availability (HA) techniques to reduce overlapping redundancy

Stateful Switchover/ NonStop Forwarding with redundant hardware in the same box may be able to replace redundant connections to network connected devices

Single high

availability device


TECRST-2021

13881_06_2007_c1 139


Balance between complexity and resiliency

Hide the additional complexity created by redundant links where possible

Summarization

Link bundling (but balance against overhead)

Consider High Availability techniques to reduce heavy redundancy for resiliency


TECRST-2021

13881_06_2007_c1 140

Hub and Spoke

Hub and spoke networks are often built over point-to-multipoint networks

If the hub is configured to treat the entire point-to-multipoint network as a single interface, it can transmit multicast and broadcast packets which are received by all spoke routers

Layer 3 on the hub router will not notice a single circuit failure

Packets transmitted

here are received by

all spokes

Packets transmitted

here are received

only by the hub router

interface s0/0

ip address 10.1.1.1 255.255.255.0

Basic Design


TECRST-2021

13881_06_2007_c1 141

Hub and Spoke

The hub router can also be configured to treat each spoke‘s circuit as an individual point-to-point circuit on a subinterface

If end-to-end signaling is in use, a failed circuit will cause the subinterface to fail

Packets

transmitted

here are received

by one spoke

Packets transmitted

here are received

only by the hub router

interface s0/0.1 point-to-point

ip address 10.1.1.0 255.255.255.254

....


ip address 10.1.1.2 255.255.255.254

....


ip address 10.1.1.4 255.255.255.254

interface s0.1 point-to-point

ip address 10.1.1.x 255.255.255.254

....

Basic Design


TECRST-2021

13881_06_2007_c1 142

Hub and Spoke

You can mitigate the single point of failure in the routers using high availability techniques

Highly

available

Basic Design

In single homed hub and spoke networks, the hub router, spoke routers, and the links themselves are all single points of failure


TECRST-2021

13881_06_2007_c1 143

Hub and Spoke

Summarize towards the core

Number the remote links out of the same address space as the remote networks, if possible

Use /31‘s to conserve address space for point-to-points

Send the remotes a default only

If you can‘t address the links out of the summary address space, then use a distribute list to filter them from being advertised back into the core of the network

0.0.0.0/0

Summary

only

192.168.1.0/24

192.168.2.0/24

192.168.2.0/24

access-list 10 deny 192.168.0.0 0.0.0.255

access-list 10 permit any

....

router eigrp 100

distribute-list 10 out

Basic Design


TECRST-2021

13881_06_2007_c1 144

Hub and Spoke

All the same principles apply to dual homed hub and spoke networks

Summarize or filter the links to the remotes

Use /31‘s on point-to-points to conserve address space

Provide as little information as possible to the remotes

Something more than a default route may be required to provide optimal routing

Avoid Summary Black Holes!

0.0.0.0/0

Summary

only

192.168.1.0/24

192.168.2.0/24

192.168.2.0/24

Basic Design


TECRST-2021

13881_06_2007_c1 145

Hub and Spoke

How do we limit the amount of information passed down to the remote sites?

You can summarize at A and B towards the remote routers

The summary will generate a local route with an administrative distance of five

The external default route learned from D will have an administrative distance of 170

What happens?

Internet

EIGRP

A B

C

DExternal

default

route

D* 0.0.0.0/0 is a summary, 00:08:41, Null0


0.0.0.0 0.0.0.0

Basic Design: Administrative Distance


TECRST-2021

13881_06_2007_c1 146

Hub and Spoke

If two routing protocols provide a route to the same destination, how do we choose between them?

Their metrics are not comparable

An administrative distance is added to each route learned based on the protocol installing the route

Static routes can be configured with a distance

This can create a floating static

The route will not be used unless the dynamic protocols have no route to that destination

router#show ip eigrp topology

P 10.0.1.0/24, 1 successors, FD is 2681856

via 10.1.1.1 (2681856/2169856)

router(config)#ip route 10.0.1.0

255.255.255.0 null0

router(config)#ip route 10.0.1.0

255.255.255.0 null0 200

distance 90

distance 1

distance 200

The static

route wins

The EIGRP

route wins

Basic Design: Administrative Distance


TECRST-2021

13881_06_2007_c1 147

Basic Hub and Spoke Design

The route generated by the summary is called a discard route

What would happen if this route isn‘t created?

Configure two routers back to back with overlapping summaries

Generate a packet towards 10.1.2.1 from either router

At A, the best path is through 10.1.0.0/16 to B

At B, the best path is through 10.0.0.0/8 to A

Routing Loop

10

.0.0

.0/8

10

.1.0

.0/1

6

ip summary-address eigrp 1 10.0.0.0

255.0.0.0


255.255.0.0

10.1.1.0/24

10.2.1.0/24

A

B

10.1.2.1

Basic Design: The Discard Route


TECRST-2021

13881_06_2007_c1 148

Hub and Spoke

In this case, the locally generated discard route wins

The route learned from D will not be installed in the local table

Hosts behind C will not be able to reach destinations on the Internet

There are ways to prevent this discard route from being installed, but we need to be careful with the design

Routing Loops

Routing Black Holes

There is enough rope here to hang yourself!

D* 0.0.0.0/0 is a summary, 00:08:41, Null0


0.0.0.0 0.0.0.0

Internet

EIGRP

A B

C

DExternal

default

route



TECRST-2021

13881_06_2007_c1 149

Internet

EIGRP

A B

C

DExternal

default

route

Hub and Spoke

To remove the discard route

In EIGRP, add an administrative distance after the ip summary address command

In OSPF, use the command no discard-route under the routing process

What happens if A loses its path to D?

C will now prefer the internal learned through A over the external learned trough B

We have a black hole


0.0.0.0 200D* 0.0.0.0/0 [170/409600] via <A>

[170/409600] via <A>

D* 0.0.0.0/0 [90/409600] via <A>

[90/409600] via <A>



TECRST-2021

13881_06_2007_c1 150

Hub and Spoke

You can also use floating static routes at the two hub routers and redistribute them into the routing protocol

Distribute list 10 only allows the default route to be advertised to the remotes

Distribute list 20 prevents a default route from being leaked back into the core

This has the same problem if a single link back towards the core and the injected external route both fail

There are other situations under which this also fails

A

C

B

access-list 10 permit host 0.0.0.0

access-list 20 deny host 0.0.0.0


....

ip route 0.0.0.0 0.0.0.0 null0 250

....

router eigrp 100

redistribute static

distribute-list 10 out <remote 1>



distribute-list 20 out <core>

Basic Design: Summary Black Hole


TECRST-2021

13881_06_2007_c1 151

Full routing information

Basic Hub and Spoke Design

One solution is to have a link between the summarizing routers across which they share full routing information

Conditional advertisement of routing information is another possible solution

OSPF can conditionally generate a default route

EIGRP has conditional advertisement as a planned feature

Internet

EIGRP

A B

C

DExternal

default

route

Basic Design: Summary Black Hole


TECRST-2021

13881_06_2007_c1 152

Hub and Spoke

EIGRP can run over either a multipoint interface at the hub router or point-to-point subinterfaces

A single multipoint interface is easier to configure but it can be harder to troubleshoot

Use summarization at the hub routers to reduce information into the network core

Provide as little information to the remotes as possible

Declare the remote routers as stubs

0.0.0.0/0

Summary

only

192.168.1.0/24

192.168.2.0/24

192.168.2.0/24

Single multipoint

or several

point-to-points

router eigrp 100

eigrp stub connected

....

EIGRP


TECRST-2021

13881_06_2007_c1 153

Hub and Spoke

Multiple Interfaces

Processor/Process Scalability is the primary limiting factor

Same Interface

Queue Congestion/Drops bottleneck is the primary limiting factor

Theoretical Limitations

EIGRP has a limitation of 2000 peers per interface, currently

EIGRP Scaling


TECRST-2021

13881_06_2007_c1 154

Hub and Spoke

The blue line shows the rate at which the convergence time increases as EIGRP neighbors are added to hub routers and does not pass 500

The red line shows the convergence time if the neighbors added are all configured as EIGRP stub routers and scales to over 1000 peers

Measure initial bring up convergence until all neighbors are established and queues empty

Dual Homed Remotes, NPE-G1 with 1G RAM, 3000 prefixes advertised to each spoke

2

5

9

0 500 1000 1500

Number of Neighbors

Tim

e (

min

ute

s)

Test performed with 12.3(14)T1

Non-Stub

EIGRP Stub

EIGRP Scaling


TECRST-2021

13881_06_2007_c1 155

Hub and Spoke

The blue line with the steep slope shows the rate at which the failover convergence time increases as EIGRP neighbors are added to a single hub router

The red line shows the failover convergence time if the neighbors added are all configured as EIGRP stub routers and is extremely linear in behavior

Primary Hub failed, time measured for EIGRP to complete failover convergence


0

1

60

0 200 400 600 800 1000 1200 1400 1600

Number of Neighbors

Tim

e (

min

ute

s)


EIGRP Stub

Non-Stub

EIGRP Scaling


TECRST-2021

13881_06_2007_c1 156

Hub and Spoke

Most EIGRP Neighbors Seen

800 deployed in live, working networks

1400 is the largest number ever tested in a lab environment

Key Strategy for achieving scalability is design!

Stub for EIGRP hub and spoke environments is a must

Minimize advertisements to spokes

EIGRP Scaling


TECRST-2021

13881_06_2007_c1 157

Hub and Spoke

B and D don‘t receive C‘s packets, so they think A has the highest IP address, and elect A as DR

C elects itself as DR

Flooding will fail miserably in this situation

―A is DR‖ ―C is DR‖ ―A is DR‖

―C is DR‖

A

B C D

interface s0/0

ip address 10.1.1. 255.255.255.0


....interface s0

ip ospf priority 0

....

OSPF

OSPF can treat a multipoint link as a broadcast network, but we need to be careful about designated router (DR) issues


TECRST-2021

13881_06_2007_c1 158

A

B C D

Hub and Spoke

Set the OSPF DR priorities so the hub router is always elected DR

Set the spokes to 0 so they don‘t participate in DR election

The remote sites won‘t be able to reach each other without some special considerations, either

Maps pointing each remote‘s address to A‘s circuit can solve this

―A is DR‖ ―C is DR‖ ―A is DR‖

―C is DR‖

interface s0/0

ip address 10.1.1. 255.255.255.0


....interface s0

ip ospf priority 0

....

OSPF


TECRST-2021

13881_06_2007_c1 159

Hub and Spoke

OSPF can treat a multipoint link as a non-broadcast network

Each spoke router must be manually configured as a neighbor

In a large hub and spoke environment, this would be very difficult to maintain

The remote sites can‘t reach each other using this method

Circuit maps pointing each remote to each other remote can be used to resolve this

interface s0/0

ip ospf network non-broadcast

....

router ospf 100

neighbor 10.1.1.2

neighbor 10.1.1.3

neighbor 10.1.1.4

interface s0

ip ospf network non-broadcast

....

router ospf 100

neighbor 10.1.1.1

A

B C D

OSPF


TECRST-2021

13881_06_2007_c1 160

Hub and Spoke

You can also configure the serial interface at the hub router as a point-to-multipoint type

All the remotes are in a single IP subnet

OSPF treats each remote as a separate point-to-point link for flooding

OSPF will advertise a host route to the IP address of each spoke router to provide connectivity

10.1.1.2/32

10.1.1.3/32

10.1.1.4/32

...

interface s0/0

ip address 10.1.1.1 255.255.255.0

ip ospf network point-to-multipoint

interface s0

ip address 10.1.1.x 255.255.255.0

ip ospf network point-to-point

A

B C D

OSPF


TECRST-2021

13881_06_2007_c1 161

Hub and Spoke

OSPF can also use point-to-point subinterfaces, treating each one as a separate point-to-point link

These uses more address space, and requires more administration on the router

Use /31 addresses for these point to point links


ip address 10.1.1.0 255.255.255.254

....


ip address 10.1.1.2 255.255.255.254

....


ip address 10.1.1.4 255.255.255.254

interface s0.1 point-to-point

ip address 10.1.1.x 255.255.255.254

....

OSPF


TECRST-2021

13881_06_2007_c1 162

Hub and Spoke

Network Type Advantages Disadvantages

Single interface at the hub treated as an OSPF broadcast network

ip ospf network-type

broadcast

Single IP subnet

Fewer nodes in the SPF tree

Manual configuration of each spoke with the correct OSPF priority

Remote-to-remote connectivity difficult

Single interface at the hub treated as an OSPF nonbroadcast network

ip ospf network-type

nonbroadcast

Single IP subnet

Fewer nodes in the SPF tree

Manual configuration of the hub and spokes with correct unicast neighbors

Remote-to-remote connectivity difficult

Single interface at the hub treated as an OSPF point-to-multipoint network

Single IP subnet

No configuration per spoke

Additional host routes inserted in the OSPF database and routing table

Individual point-to-point interface at the hub for each spoke

ip ospf network-type point-

to-point

Can take advantage of end-to-end signaling for down state

Lost IP address space

More routes in the OSPF database and routing table

OSPF


TECRST-2021

13881_06_2007_c1 163

Hub and Spoke

If possible, make them totally stubby

If there is redistribution at the spokes, make the area totally not-so-stubby

Area 1

router ospf 100

area 1 stub no-summary

....

router ospf 100

redistribute rip metric 10

....

router ospf 100

area 1 nssa no-summary

....

OSPF

The areas the spokes are placed in should always be the ―most stubby‖ you can get away with


TECRST-2021

13881_06_2007_c1 164

Hub and Spoke

If you need to leak some routing information from area 0 into the spoke areas, use type 3 LSA filtering at the border to remove as much information as possible

OSPF Hub and Spoke Areas, currently in development, would allow an area where the spoke routers only receive the default route

Area 1

ip prefix-list 10 permit 10.1.1.0/24 ge 25

ip prefix-list 10 deny all

....

router ospf 100

area 1 filter-list prefix-list 10 in

OSPF


TECRST-2021

13881_06_2007_c1 165

Hub and Spoke

Once you‘ve determined how to configure the hub‘s interface, you need to decide how to divide the remote sites among flooding domains

If the hub and spoke section of the network is small, and fits well within some other area structure, then the entire hub and spoke can be placed in this single flooding domain

OSPF


TECRST-2021

13881_06_2007_c1 166

Hub and Spoke

Remember each spoke router receives all the topology information from all the other spoke routers

OSPF

If the hub and spoke is large enough, you‘ll want to split it off as its own flooding domain


TECRST-2021

13881_06_2007_c1 167

Hub and Spoke

Low speed links and large numbers of spoke may require multiple flooding domains

Balance the number of flooding domains on the hub against the number of spokes in each flooding domain

The link speeds and the amount of information being passed through the network determine the right balance

OSPF


TECRST-2021

13881_06_2007_c1 168

Hub and Spoke

Dual homed remotes make the division of flooding domains significantly more difficult

If all the spoke routers will fit, put both the hubs and all the spokes in a single flooding domain

OSPF


TECRST-2021

13881_06_2007_c1 169

Two links, one in each flooding domain

Hub and Spoke

You should build links between the hub routers within each flooding domain in some way to prevent routing black holes

Put two links between the area borders, one in each area or flooding domain

OSPF

If all the spokes will not fit into a single flooding domain, split the hub and spoke up into multiple areas or flooding domains


TECRST-2021

13881_06_2007_c1 170

Hub and Spoke

The blue line shows the rate at which the startup convergence time increases as OSPF neighbors are added to the hub routers and peaks at the 700 router mark

The red line starts and ends below the green line showing the startup convergence time if the neighbors added are all configured as OSPF neighbors are added to a Totally Stubby area

Measure initial bring up convergence until all neighbors are established and queues empty, SPF completes


0

50

100

150

200

250

300

350

400

0 100 200 300 400 500 600 700 800

Number of Spokes

Co

nv

erg

en

ce T

ime (

seco

nd

s)


Single Area

Totally Stubby Area

OSPF Scaling


TECRST-2021

13881_06_2007_c1 171

Hub and Spoke

The blue line, ending above the red line, shows the rate at which the failover convergence time increases as OSPF neighbors are added to a single hub router

The red line shows the failover convergence time if the neighbors added are all configured as OSPF neighbors are added to a Totally Stubby area

Primary Hub failed, time measured for OSPF to complete failover convergence


0

5

10

15

20

25

30

35

40

45

50

0 100 200 300 400 500 600 700 800

Number of Spokes

Co

nv

erg

en

ce T

ime (

seco

nd

s)


Single Area

Totally Stubby Area

OSPF Scaling


TECRST-2021

13881_06_2007_c1 172

Hub and Spoke

Most OSPF Neighbors Seen

200 Deployed in live, working networks

600 is the largest number ever tested in a lab environment

Key Strategy for achieving scalability is design!

Minimize advertisements to spokes

Area placement is the key to summarization, filtering, etc.

Use the most stubby area possible

OSPF Scaling


TECRST-2021

13881_06_2007_c1 173

Full Mesh

Full mesh topologies are complex:

2 routers == 1 link

3 routers == 3 links




...

Adjacencies == nodes(nodes-1)/2


TECRST-2021

13881_06_2007_c1 174

Full Mesh

60 node TEST network

1770 links

NPE-G1, NPE-400s

All devices on same physical Ethernet (via a switch), full mesh created with GRE Tunnels

Three tests performed

Initial convergence, measured from interface bring up

Flap a transit link, such that a routing adjacency will reset

Flap a stub network, to measure prefix propagation

This test does not consider stability, only convergence!

Scaling Tests


TECRST-2021

13881_06_2007_c1 175

Full Mesh

EIGRP OSPF Default Timers OSPF Tuned Timers

Initial Convergence 1:13 1:13 1:18

Link Flap 0:51 0:43 0:41

Prefix Flap 0:15 0:09 0:03

Scaling Tests


TECRST-2021

13881_06_2007_c1 176

Full Mesh

Flooding routing information through a full mesh topology is also complicated

Each router will, with optimal timing, receive at least one copy of every new piece of information from each neighbor on the full mesh

There are several techniques you can use to reduce the amount of flooding in a full mesh

Mesh groups reduce the flooding in a full mesh network

Mesh groups are manually configured ―designated routers‖

New information

OSPF


TECRST-2021

13881_06_2007_c1 177

interface serial x

ip ospf database-filter all out

....

Full Mesh

Pick one or two routers to flood into the mesh, and block flooding on the remainder

This will reduce the number of times information is flooded over a full mesh topology

New information

OSPF


TECRST-2021

13881_06_2007_c1 178

Full Mesh

Routes must be advertised between every pair of peers in the mesh so each router has the correct next hop and routing information

Number the links so they can be summarized to a single advertisement at the edge

Number the links so the link information can be filtered out at the edge

Summarize

EIGRP


TECRST-2021

13881_06_2007_c1 179

Full Mesh

Consider High Availability ring topologies, such as SRP, SONET rings, and others as an alternative to full mesh high speed networks in POPs and other enclosed networks

This can provide resiliency against a single failure in the network, and simplify the topology from the perspective of routing dramatically


TECRST-2021

13881_06_2007_c1 180


Be careful with links between border routers in OSPF and IS-IS

Traffic prefers to stay within the flooding domain no matter what the actual link costs are

To reach A, we will take the higher cost link if the border link is in the backbone

To reach B, we will take the higher cost link if the border link is in the area or L1 domain

This is because we are removing topology information at the border, and always trust routes with more explicit topology information

100

10 10

A

B100

10.1.1.0/24, cost 10

100

10 10

A

B100

10.1.1.0/24, cost 10


TECRST-2021

13881_06_2007_c1 181


In OSPF, we have to decide which traffic we want to route optimally

The ability to place a single link in two areas is under consideration within the OSPF working group

In IS-IS, we can place the link in both the L1 and L2 routing domains, and optimally route both ways

100

10 10

A

B100

10.1.1.0/24, cost 10

100

10 10

A

B100

10.1.1.0/24, cost 10


TECRST-2021

13881_06_2007_c1 182



TECRST-2021

13881_06_2007_c1 183


Alternatives to Redistribution

Single Point of Redistribution

Multiple Points of Redistribution


TECRST-2021

13881_06_2007_c1 184


When connecting to an outside network, creating static routes at the edge, and redistribute those, instead of redistributing live routing information

This prevents misconfigurations and rapid topology changes in the other network from impacting you

It also prevents someone from injecting false information to attack your routing system

BigShoes, Inc

10.1.0.0/16

MediumSocks, LTD

10.2.0.0/16

Redistribute

EIGRP to OSPF

Redistribute

OSPF to EIGRP

ip route 10.1.0.0 255.255.255.0 s0/0

!

router ospf 100

redistribute state metric 10

ip route 10.2.0.0 255.255.255.0 s0/0

!

router eigrp 100

redistribute state metric 1000 1 255 1 1500


TECRST-2021

13881_06_2007_c1 185


Even if you must have live routing data, don‘t redistribute between IGPs to connect to an outside network; this opens serious security holes in routing

Instead, use eBGP, so you can do policy based filtering on the routes you‘re receiving

BigShoes, Inc

MediumSocks, LTD

Redistribute

EIGRP to OSPF

Redistribute

OSPF to EIGRP

eBGP

AS65000

AS65001


TECRST-2021

13881_06_2007_c1 186


Use redistribution when permanently merging two networks into a single administrative domain

Use redistribution as a transition strategy when switching routing protocols

Use redistribution to split off a section of the network for security, experimental, or administrative reasons

BigShoes, Inc

MediumSocks, LTD

BigShoes, Inc

MediumSocks, LTD

Redistribute

EIGRP to OSPF

Redistribute

OSPF to EIGRP

Socks&Shoes, Corp


TECRST-2021

13881_06_2007_c1 187

Single Point of Redistribution

Single points of redistribution are simple to manage and control

There is little or no chance of routing loops or other problems with single points of redistribution

They are also single points of failure; consider using high availability methods to reduce the risk

EIG

RP

OS

PF

router ospf 100

redistribute eigrp 100 metric 10

....

!

router eigrp 100

redistribute ospf 100 metric 1000 1 255 1 1500

....

Single point of failure


TECRST-2021

13881_06_2007_c1 188


Multiple points of redistribution resolve the single point of failure

The cost is dramatically increased network complexity and the possibility of permanent routing loops E

IGR

P

OS

PF

router ospf 100


....

!

router eigrp 100


....

router ospf 100


....

!

router eigrp 100


....


TECRST-2021

13881_06_2007_c1 189


A route is injected into EIGRP as an external; this route is redistributed through B into OSPF

The route is transmitted to A through OSPF, and redistributed into EIGRP

The metric is set manually in redistribution at A to something lower than the original external injected into EIGRP

B prefers this route, building a routing loop

A

EIG

RP

OS

PF

BMetric 10 Metric 2816000

10.1.1.0/24

Metric 2560256

Metric

2688000

Metric 25 Metric 2560256


TECRST-2021

13881_06_2007_c1 190


There Are Three Ways to Prevent This Routing Loop:

Only redistributing live routing information in one direction

Filtering routes based on the network advertised to prevent feedback

Filtering routes using routing tags to prevent feedback


TECRST-2021

13881_06_2007_c1 191


If live routing data is only needed in one direction (normally, this is true), redistribute a static in one direction, and between protocols in the other direction

ip route 10.2.1.0 255.255.255.0 serial 0/0

....

router ospf 100


....

router eigrp 100

redistribute static 100 metric 1000 1 255 1 1500

....

A

EIG

RP

OS

PF

B

10.1.1.0/2410.1.2.0/24

Single Redistribution Direction


TECRST-2021

13881_06_2007_c1 192


To filter based on prefixes, configure access lists which match the address ranges used by each section of the network

Use these access lists to filter routes redistributed between protocols



....

router ospf 100

redistribute eigrp 100 metric 10 distribute-list 10

....

router eigrp 100

redistribute ospf 100 metric 1000 1 255 1 1500 distribute-list 20

....

10.1.1.0/2410.1.2.0/24

A

EIG

RP

OS

PF

B

Filtering Based on Prefixes


TECRST-2021

13881_06_2007_c1 193


EIGRP and OSPF can set tags on their external routes

Set the tag when redistributing between the protocols; deny tagged routes at the redistribution point

route-map usetags deny 10

match tag 1000

route-map usetags permit 20

set tag 1000

....

router ospf 100

redistribute eigrp 100 metric 10 route-map usetags

....

router eigrp 100

redistribute ospf 100 metric 1000 1 255 1 1500 route-map usetags

....

10.1.1.0/2410.1.2.0/24

A

EIG

RP

OS

PF

B


TECRST-2021

13881_06_2007_c1 194

Transitioning Protocols


TECRST-2021

13881_06_2007_c1 195


Basics

Cutover At Once

Splitting the Problem

Using Redistribution

Using Administrative Distance


TECRST-2021

13881_06_2007_c1 196


There is a quick and easy way to transition from one protocol another without any network downtime

Perhaps—If you discover it, let me know

It‘s impossible to transition from one routing protocol to another in a really large network

It‘s almost always difficult, but never impossible

Basics


TECRST-2021

13881_06_2007_c1 197


It‘s never worth the trouble of switching routing protocols

That depends...

Would the cost benefits outweigh the transition costs?

Differentials in overall equipment costs in the future

Convergence speeds on specific network topologies

Other factors

You sometimes don‘t have a choice, such as when merging two networks

Basics


TECRST-2021

13881_06_2007_c1 198


―We want faster convergence...‖

Generally convergence is a matter of design, rather than protocol

―Our network design is hub and spoke, so it fits better for EIGRP...‖

Can‘t argue with this one…

―We want a standards based protocol…‖

What, so you can install some ―other‖ vendor‘s equipment? Are you insane????

―We‘re all studying for our CCIEs, and need exposure to other protocols…‖

Basics

What reasons have we heard in the field for switching routing protocols?


TECRST-2021

13881_06_2007_c1 199

Cutover at Once

Start from one end of the network

Telnet to the other end hop by hop, removing routing at each step

Apply new routing protocol at the router farthest away

Back out, applying new routing protocol hop by hop

Telnet

Telnet

Telnet

Removerouting

Removerouting

Configure routing

Configure routing

Removerouting

Configure routing

Removerouting

Configure routing

Don‘t count on routed reachability while you are switching the routing protocol


TECRST-2021

13881_06_2007_c1 200

Cutover at Once

If the network destabilizes, take a break

At each router, wait until the network has converged before moving to the next router

When you configure routing on a given router, wait until the routing protocol is quiescent

For instance, for EIGRP, look at show ip eigrp neighbors, and wait until

the Q Count is 0 on all interfaces

This technique should be used whether converting manually or when using a script

Telnet

Telnet

Telnet

Removerouting

Removerouting

Removerouting

Removerouting

Configure routing

Configure routing

Configure routing

Configure routing

Wait for convergence





TECRST-2021

13881_06_2007_c1 201

Cutover at Once

If the process stalls or fails, each device should be left completely in a known state

There should be no chance of partial configurations

Only one of three states should be possible

The old routing protocol is completely configured

No routing is configured

The new routing protocol is completely configured


TECRST-2021

13881_06_2007_c1 202

Cutover at Once

Create each new routing configuration in a locally accessible file

Remember not to count on reaching a server

At each router

Open file with new routing protocol configuration commands

no router xxxx to remove

the old routing protocol

config t

(Copy/paste)

copy run start

reload

router-b(config)#no router xxxx

A

B

router-a#telnet <b>

host#telnet <a>

host#<edit config b>

router xxxx

network xx.xx.xx.xx

....

router-b(config)#router xxxx

router-b(config-rtr)#....

<Copy>

<Paste>

router-b(config-rtr)#exit

router-b#copy run start

router-b#reload

(Text editor)

router xxxx

network xx.xx.xx.xx

....

Make the Process Atomic


TECRST-2021

13881_06_2007_c1 203

Cutover at Once

Create each new routing configuration

Copy the configuration into a file on the local flash of each device

To convert

Telnet to each router

Remove routing

no router xxxx

Copy the local file from flash to the startup configuration

copy <file> run

Reload the router

You can also copy the new configuration directly to the startup configuration and reload, rather than to the running configuration

router-b(config)#no router xxxx

A

B

router-a#telnet <b>

host#telnet <a>

router-b(config)#exit

router-b#copy slot0:newconfig run

router-b#copy run start

router-b#reload

....

router-b>

<ctrl>+<shift>+6

x

router-a#copy slot0:newconfig run

router-a#copy run start

router-a#reload



TECRST-2021

13881_06_2007_c1 204

Cutover at Once

Failure Point Result

Before the old routing protocol is removedRouter can be reached through old routing protocol or direct connections (interface addresses are not removed)

After the old routing protocol is removedRouter can be reached through direct connections

After the new routing protocol is configured

Router can be reached through new routing protocol or direct connections

After the new routing protocol is configured and saved, and router is reloaded

Router can be reached through new routing protocol or direct connections



TECRST-2021

13881_06_2007_c1 205


In really large networks, you might have to split the problem into pieces

Consider the network as a set of smaller networks, and convert each part separately

Where can you split a network?

Hierarchical division points

Aggregation points


TECRST-2021

13881_06_2007_c1 206


In a two layer hierarchy, the only real choice is to split the network along the core/aggregation divide

Each ―lobe‖ within the aggregation layer can be converted separately

The network core can be converted separately

core

aggregation

Hierarchical Division Points


TECRST-2021

13881_06_2007_c1 207


In a three layer hierarchy, the split points are going to depend on the size of each ―lobe‖ and ―layer‖ in the network

Each access layer ―lobe‖ can be converted separately

The core can be converted as one unit

The distribution layer can either be converted with the core, with the access layer, or separately, in ―lobes,‖ etc.

core

distribution

access



TECRST-2021

13881_06_2007_c1 208


Which part should you convert first?

Start at the edge and work in?

Start at the core and work out?

This question applies to both

Converting individual pieces of the network

The order in which to convert network pieces

Ed

ge

in?

Co

re O

ut?



TECRST-2021

13881_06_2007_c1 209


Typically, it‘s easier to work from the edge in…

This tends to work with aggregation and network design, rather than against it

Provides a set of ―lower risk‖ areas to work in, and perfect techniques

But… in some cases, core out might be easier

I‘ve just never seen a network where it is…

Ed

ge

in?

Co

re O

ut?



TECRST-2021

13881_06_2007_c1 210


Another good place to divide the network is at aggregation points

This will often be along hierarchical boundaries, anyway…

If you choose different aggregates in the new protocol, both protocols can run at the same time, along the edges

This allows you to convert one section of the network at a time

Aggregation Points


TECRST-2021

13881_06_2007_c1 211


Once you‘ve split the network up into pieces to convert, how do you actually convert each piece, and still have a working network?

One ―easy‖ answer is redistribution…

A

B

C

router ospf 100

network 0.0.0.0 0.0.0.0 area 0

router ospf 100

network 0.0.0.0 0.0.0.0 area 0

area 0 range 10.1.0.0 0.0.255.255

....

router eigrp 100

network 0.0.0.0

router eigrp 100

network 0.0.0.0

Redistribute here?


TECRST-2021

13881_06_2007_c1 212


Redistribution is probably one of the most ―counted on‖ tools to convert from one routing protocol to another

But, it‘s a lot like playing with fire…

You can cook a really nice omelet, or you can get really burnt!


TECRST-2021

13881_06_2007_c1 213


In a simple network design, it‘s easy to move redistribution around as you convert

How many networks have simple linear topologies like this one, though?

New protocol

Old protocol

Old protocol

New protocol

Old protocol

Redistribution

New protocol

Old protocol

Redistribution

Old protocol

New Protocol

New protocolNew protocol

Old protocol

Old protocol

New protocol

Old protocol

Redistribution


TECRST-2021

13881_06_2007_c1 214


The more complex a network‘s topology is, the more places redistribution is required to convert from one protocol to another

More points of redistribution means:

More complexity in moving the protocol conversion over at each step

More chances for human error in configurations

More complex problems if the network fails during conversion

etc.

New Protocol

Redistribution

Old Protocol


TECRST-2021

13881_06_2007_c1 215

10.1.1.0/24


Create the new protocol on all the routers

Set the administrative distance so the new protocol never wins

Take the old protocol off

router ospf 100

network 0.0.0.0 0.0.0.0 area 0

distance 200

no router rip

B

C

D

A

Distance Vector to Link State

When converting from a Distance Vector to Link State protocol…


TECRST-2021

13881_06_2007_c1 216


Add OSPF to all routers

At B, remove RIP

Does this work?

A has a route to 10.1.1.0/24 through OSPF

B has a route to 10.1.1.0/24 through OSPF

C has a route to 10.1.1.0/24 through RIP

D has a connected route to 10.1.1.0/24

This works…

10.1.1.0/24

router ospf 100

network 0.0.0.0 0.0.0.0 area 0

distance 200

no router rip

B

C

D

A



TECRST-2021

13881_06_2007_c1 217


Things can get harder around aggregation points and area borders

Suboptimal routing is the rule rather than the exception

In some cases, suboptimal routing can become extreme

Area 0

Area 1Totally Stub

RIP

ip summary-address 10.1.0.0 255.255.0.0 10.1.1.0/24

Only path to 10.1.2.1 is through C

Best path to 10.1.2.1 is through B

Throw traffic to 10.1.2.1 away through the discard

route



TECRST-2021

13881_06_2007_c1 218


Add EIGRP to all routers

Remove RIP from B

D advertises 10.1.1.0/24 through RIP and EIGRP

C receives 10.1.1.0/24 in both RIP and EIGRP, but doesn‘t advertise it through EIGRP because the RIP route is installed in the routing table

B has no route to 10.1.1.0/24

This doesn‘t work!

10.1.1.0/24

router eigrp 100

network 0.0.0.0 0.0.0.0

distance eigrp 190 200

no router rip

B

C

D

A

Distance Vector to Distance Vector


TECRST-2021

13881_06_2007_c1 219


Note B is the first router that doesn‘t have a route to 10.1.1.0/24

This technique won‘t work in the general case, then, but it is useful in some cases, even with distance vector protocols

10.1.1.0/24

router eigrp 100

network 0.0.0.0 0.0.0.0

distance eigrp 190 200

no router rip

B

C

D

A

B is the first router without a route to 10.1.1.0/24

Distance Vector to Distance Vector


TECRST-2021

13881_06_2007_c1 220


You can use the administrative distance to your advantage when using a ―cutover at once‖ technique

Rather than removing the old routing protocol at each step, then installing the new one…

Configure the new routing protocol at each router, making certain the new protocol doesn‘t take routing over

To convert the network, walk through the each router, changing one of the two protocol‘s administrative distance to make the new protocol win, and the old protocol lose

Telnet

Telnet

Telnet

Removerouting

Removerouting

Removerouting

Removerouting

Configure routing

Configure routing

Configure routing

Configure routing





Combined with a Cutover


TECRST-2021

13881_06_2007_c1 221


If you use this technique…

Watch for unpredictable routing as you‘re converting, especially if you‘re converting from a distance vector protocol to a link state protocol

Be careful not to rely on routing to modify routing

Never count on a routed path to reach a router that you‘re working on

Always telnet hop by hop when converting

Don‘t be too hasty to back out, if things start looking wrong

Troubleshoot the problem

Make certain it doesn‘t relate to both protocols running at the same time

Combined with a Cutover


TECRST-2021

13881_06_2007_c1 222


For protocols that rely on the administrative distance to sort routes…

EIGRP

BGP

Do not reverse the administrative distance of their routes

Don‘t make external EIGRP routes preferred over internal EIGRP routes

This is a certain path to routing loops and major network failures

Warning


TECRST-2021

13881_06_2007_c1 223

BGP


TECRST-2021

13881_06_2007_c1 224

BGP

BGP Basics

Route Reflectors

BGP Cores

Outside Connections

BGP/IGP Interaction


TECRST-2021

13881_06_2007_c1 225

BGP Basics

Interior Gateway Protocols:

Automatic discovery

Generally trust your IGP neighbors

Routes go to all IGP neighbors

Exterior Gateway Protocols

Specifically configured peers

Connecting with outside networks

Set administrative boundaries


TECRST-2021

13881_06_2007_c1 226

BGP Basics

Autonomous System: A network(s) sharing the same routing policy

Possibly multiple IGPs

Usually under single administrative control

Contiguous internal connectivity

Numbering range from 1 to 65,535—globally unique—―AS Number‖

Private range: 64512–65535


TECRST-2021

13881_06_2007_c1 227

BGP Basics

Learns multiple paths via internal and external BGP speakers

Picks THE bestpath, installs it in the IP forwarding table, forwards to EBGP neighbors (not IBGP)

Policies applied by influencing the bestpath selection


TECRST-2021

13881_06_2007_c1 228

BGP Basics

Summary of Peering Operation:

TCP connection established (port 179)

Both peers attempt to connect—there is an algorithm to resolve ―connection collisions‖

Exchange messages to open and confirm the connection parameters

Initial exchange of entire table

Incremental updates after initial exchange

Keepalive messages exchanged when there no updates


TECRST-2021

13881_06_2007_c1 229

BGP Basics

External (eBGP) connections are to BGP peers in other autonomous systems

Internal (iBGP) peers are to BGP peers in the same autonomous system

A

B

BGP core

IGP Area

C

eBGP

sessionrouter bgp 65000

neighbor 10.1.1.1 remote-as 65000

router bgp 65000



router bgp 65001


iBGP

session

Peering


TECRST-2021

13881_06_2007_c1 230

BGP Basics

When B learns a route from C through eBGP, it sets the next hop towards the destination to C

When it advertises this route to A, through iBGP, it does not reset the next hop

A need to learn how to reach C through some other method than BGP

An IGP needs to underlie BGP

eBGP

session

A

B

BGP AS

BGP AS

C

Next hop is

set to C

Next hop

remains C

A needs to learn

how to reach C

Peering


TECRST-2021

13881_06_2007_c1 231

BGP Basics

Routes learned from eBGP peers are readvertised to iBGP peers

Routes learned from iBGP peers are not readvertised to other iBGP peers

iBGP peers have to be fully meshed, or some other technique needs to be used to distribute iBGP routes through an autonomous system

A

B

BGP AS

C

C

eBGP

session

iBGP

session

iBGP

session

Learn eBGP routes

Readvertise

eBGP routes to

iBGP peers

Don‘t readvertise

iBGP routes to

iBGP peers

iBGP session

Peering


TECRST-2021

13881_06_2007_c1 232

Route Reflectors

B receives 10.1.1.0/24 with an AS Path of {65000,65001}

C receives 10.1.1.0/24 with an AS Path of {65001,65000}

D receives 10.1.1.0/24 with an AS Path of {65001,65000}

B receives the same route with the same attributes, setting up a loop!

eB

GP

AS65001

10.1.1.0/24

10.1.1.0/24

10.1.1.0/24

AS65000A

B

C

D10.1.1.0/24

Basics

We know that iBGP doesn‘t guarantee loop free routing through an AS


TECRST-2021

13881_06_2007_c1 233

Route Reflectors

What we need is an AS Path to prevent loops within the AS!

RFC2796, BGP Route Reflection, defines two BGP attributes to provide loop detection within an AS

Originator ID

Set to the ID of the router injecting the route into the AS

Cluster List

Each route reflector the route passes through adds their ID to this list

Basics


TECRST-2021

13881_06_2007_c1 234

Route Reflectors

B receives 10.1.1.0/24 with an AS Path of {65000,65001}

C receives 10.1.1.0/24 with an AS Path of {65001,65000}, but adds A‘s Router ID as the Originator ID

C also starts a Cluster List, and adds its own local Router ID into the list

eB

GP 10.1.1.0/24

10.1.1.0/24

AS65000A

B

CD

neighbor <B> route-reflector-client

neighbor <D> route-reflector-client


neighbor <C> route-reflector-client

AS65001

Basics


TECRST-2021

13881_06_2007_c1 235

Route Reflectors

D receives 10.1.1.0/24 with an AS Path of {65001,65000} and an Originator ID of A

D adds its own router ID to the Cluster list

Before sending the route to A, D compares the Originator ID and the Cluster ID list to see if A‘s router ID matches any ID on either one

D finds A‘s ID as the Originator ID, so it doesn‘t send the route to A

eB

GP 10.1.1.0/24

10.1.1.0/24

10.1.1.0/24

10.1.1.0/24

AS65000A

B

C

AS65001

D





Basics


TECRST-2021

13881_06_2007_c1 236

Route Reflectors

eB

GP

AS65000A

B

C

AS65001

D





10.1.1.0/24AS Path: {65001, 65000}

10.1.1.0/24AS Path: {65001, 65000}Originator ID: ACluster List: {C}

10.1.1.0/24AS Path: {65001, 65000}

Originator ID: ACluster List: {C,D}

Basics


TECRST-2021

13881_06_2007_c1 237

Route Reflectors

A route reflector is an iBGP speaker that reflects routes learned from iBGP peers to other iBGP peers

Route reflectors add the Originator ID and the Cluster List to routes they reflect

Route reflectors are designated by configuring some of their iBGP peers as route reflector clients

Route reflectors

neighbor <X> route-reflector-client


Basics


TECRST-2021

13881_06_2007_c1 238

Route Reflectors

A route reflector clientis just an iBGP speaker

There is no special configuration for a route reflector client

Route reflectors



Route reflector client

Basics


TECRST-2021

13881_06_2007_c1 239

Route Reflectors

A cluster is a route reflector and its clients

Route reflector clusters may overlap

Route reflectors




Cluster

Basics


TECRST-2021

13881_06_2007_c1 240

Route Reflectors

A non-client is any route reflector iBGP peer that is not a route reflector client

Each route reflector is also a non-client of each other route reflector in this network

Route reflectors must be fully iBGP meshed with non-clients

Route reflectors




Cluster

Non-client

Basics


TECRST-2021

13881_06_2007_c1 241

Route Reflectors

When reflecting a route, a route reflector always:

Creates a Cluster List if one doesn‘t exist

Adds its router ID (or the configured cluster ID) to the Cluster List

If no Cluster List exists, adds the router ID of the peer it received the route from as the Originator ID

When sending a route, a route reflector always follows normal BGP processing rules

Basics


TECRST-2021

13881_06_2007_c1 242

Route Reflectors

Send the route to all clients

Send the route to all non-clients

eBGP peer Non-client

Client

Client

SendSend

Send

Basics

If a route reflector receives a route from an eBGP peer:


TECRST-2021

13881_06_2007_c1 243

Route Reflectors

Reflect the route to all clients

Reflect the route to all non-clients

Send the route to all eBGP peers


Client

Client

Reflect

Reflect

Send

Basics

If a route reflector receives a route from a client:


TECRST-2021

13881_06_2007_c1 244

Route Reflectors

Reflect the route to all clients

Send the route to all eBGP peers


Client

Client

Reflect

Send

Reflect

Basics

If a route reflector receives a route from a non-client:


TECRST-2021

13881_06_2007_c1 245

Route Reflectors

A advertises 10.1.1.0/24 to B

B sends 10.1.1.0/24 to D

D sends 10.1.1.0/24 to E

E reflects 10.1.1.0/24 to C

D chooses the path through B (via C)

C chooses the path through E (via D)

We have a permanent routing loop!

A

B

D

E

C

eB

GP

B is a client of D

C is a client of E

1

2

3

4

Basics


TECRST-2021

13881_06_2007_c1 246

Route Reflectors

Always configure the reflector topology to follow the physical topology

No route reflector client should ever peer through a route reflector the client isn‘t peered to

C (a client) should not be peered to E (a reflector) through D (a reflector) without being peering to D as well as E

In this case, making C a client of D would resolve the loop

A

B

D

E

C

eB

GP

B is a client of D

C is a client of E

1

2

3

4

Basics


TECRST-2021

13881_06_2007_c1 247

Route Reflectors

All of the route reflectors will need to be fully meshed

Reflectors still follow the normal rules of iBGP route propagation between themselves

This full iBGP mesh between reflectors can still contain so many routers that it presents a scaling problem

ClusterCluster

Full iBGP mesh between reflectors

Hierarchical Route Reflectors


TECRST-2021

13881_06_2007_c1 248

Route Reflectors

To resolve this, route reflectors can be deployed in a hierarchy

A single router can be a reflector client and a reflector

ClusterCluster

Cluster

Client and reflector



TECRST-2021

13881_06_2007_c1 249

Hierarchal Route Reflectors

An unlimited number of tiers that can be used

The edges of route reflector tiers are a natural place to reduce the amount of routing information being carried in the lower tiers

The same topology rule applies: The reflector topology must follow the physical topology to prevent loops and black holes

Suboptimal routing can actually be worse, and harder to figure out



TECRST-2021

13881_06_2007_c1 250

Route Reflectors

Use the divide con conquer approach to convert from a full iBGP mesh to route reflectors

Divide network into multiple clusters, using the physical topology as a guide to the logical divisions

Pick out one router to act as the reflector in each cluster, making certain reflection follows the physical topology

Remove redundant iBGP sessions as you configure reflectors in each cluster

Deployment


TECRST-2021

13881_06_2007_c1 251

Route Reflectors

If you‘re going to use hierarchal route reflectors, do the outer edge first, leaving the core full mesh iBGP until the outer edge is done

Continue using a single IGP—the next-hop is unmodified by reflectors unless set via an explicit route-map

Deployment


TECRST-2021

13881_06_2007_c1 252

Route Reflectors

A client may peer with more than one reflector, in different clusters

A client that peers to only one reflector has a single point of failure

Clients should peer to at least two reflectors to provide redundancy

How many reflectors should a single route reflector be peered to?

Should redundant reflectors be in the same cluster or should they be in separate clusters?

Deployment


TECRST-2021

13881_06_2007_c1 253

Route Reflectors

How many route reflectors should a single client be peered to?

Two considerations are important:

Network configuration and management

Router memory and processing requirements

If A is the client of only one reflector, it only receives one copy of the route to 10.1.1.0/24

B

A

E

10.1.1.0/24neighbor <a> route-reflector-client

Deployment


TECRST-2021

13881_06_2007_c1 254

Route Reflectors

E new route reflector A becomes a client of adds more configuration and management

Each new route reflector A becomes a client of adds another path to 10.1.1.0/24

This increases the amount of memory A requires to operate, and also increases A‘s processing requirements

B C D

A

E


Deployment


TECRST-2021

13881_06_2007_c1 255

Route Reflectors

Each new client B, C, and D are peered to also increased their processing requirements

At some point, the additional reflectors will stop adding to the resilience of the network, and make management and memory requirements similar to a full iBGP mesh

B C D

A

E


Deployment


TECRST-2021

13881_06_2007_c1 256

Route Reflectors

Some redundancy is needed

Too much burns memory on RRCs because the client learns the same information from each RR

Also burns memory on the RRs because they learn multiple paths for each route introduced by a RRC

Two or three reflectors peer cluster should be plenty

Deployment


TECRST-2021

13881_06_2007_c1 257

Route Reflectors

Assume A and B have the same route reflector clients configured

These two reflectors are redundant

Should they be configured with the same cluster ID or different cluster IDs?

neighbor <c> route-reflector-client

neighbor <d> route-reflector-client

neighbor <c> route-reflector-client

neighbor <d> route-reflector-client

A B

C D

E

10.1.1.0/24

Deployment


TECRST-2021

13881_06_2007_c1 258

Route Reflectors

Assume A and B are using the same cluster ID, 10.10.10.10

E advertises 192.168.1.0/24 to D

D sends this route to its reflector, B

B adds a Cluster List and the Originator ID, and reflects the route to A and C

When A receives this route, it notes its local cluster ID is already in the Cluster List (since A and B have the same cluster ID), and rejects the route

A B

C D

E

192.168.1.0/24

192.168.1.0/24

192.1

68.1

.0/2

4

192.168.1.0/24Cluster: 10.10.10.10

Deployment


TECRST-2021

13881_06_2007_c1 259

Route Reflectors

If the A to D link fails, A won‘t have any path to 192.168.1.0/24, since it is rejecting the route from B

If the B to C link fails, C won‘t have any path to 192.168.1.0/24, since A is rejecting the route from B, and won‘t reflect it to C

This configuration only protects against some link failures, not all of them

A B

C D

E

192.168.1.0/24

Deployment


TECRST-2021

13881_06_2007_c1 260

Route Reflectors

One way to resolve this problem is to configure the iBGP sessions between the router‘s loopbacks, rather than their physical interfaces

If the A to B link fails, the A to B iBGP session stays up (through C), so A maintains connectivity to 192.168.1.0/24

If the B to C link fails, the B to C iBGP session stays up (through A), so C maintains connectivity to 192.168.1.0/24

A B

C D

E

192.168.1.0/24

Deployment


TECRST-2021

13881_06_2007_c1 261

Route Reflectors

Another option is to configure A and B with different cluster IDs

Now, when A receives B‘s reflected route, it will keep the route, since the cluster ID in the Cluster List doesn‘t match its own cluster ID

A will run the BGP bestpath algorithm, and advertise either its path through B or its path through D to C

A B

C D

E

192.168.1.0/24

192.168.1.0/24

192.1

68.1

.0/2

4

192.168.1.0/24Cluster: 10.10.10.10

Deployment


TECRST-2021

13881_06_2007_c1 262

Route Reflectors

If the A to D link fails, A will still have the path through B to reach 192.168.1.0/24

If the B to C link fails, C will still have the path through A to reach 192.168.1.0/24

This provides full redundancy

A B

C D

E

192.168.1.0/24

Deployment


TECRST-2021

13881_06_2007_c1 263

Route Reflectors

A now also has two routes to 192.168.1.0/24, one through D, and one through B

Each additional path A must hold and process adds additional memory and processor overhead

This solution is less scalable than A and B being configured with the same cluster ID

A B

C D

E

192.168.1.0/24

Deployment


TECRST-2021

13881_06_2007_c1 264

Route Reflectors

RedundancyAdministrative

FactorsAttribute

Combinations

Reflector Memory

Consumption

Same Cluster ID

100% with sessions between

loopbacks

Easy to identify network

regions based on cluster ID

MediumOne path from

each client

Different Cluster ID 100%

Easy to identify reflection chain

based on Cluster List

High

One path from each client and one path from each reflector

Deployment


TECRST-2021

13881_06_2007_c1 265

BGP Cores

When the network becomes ―too large‖ for an interior gateway protocol to manage

When the core of the network becomes an ―internal service provider,‖ connecting several large, independent networks with separate support staffs, policies, and (possibly) interior gateway protocols

Why Use a BGP Core?


TECRST-2021

13881_06_2007_c1 266

BGP Cores

How do you know your network is too big for a single interior gateway protocol domain or instance to handle?

When the network fails on a regular basis

When the network never converges (constant churn)

The upper limit on most interior gateway protocols is about 5,000 to 10,000 routes

The more complex the network is in terms of available alternate paths, the fewer routes the IGP will be able to manage

Why Use a BGP Core?


TECRST-2021

13881_06_2007_c1 267

BGP Cores

But…

If you have deployed the scaling techniques we‘ve talked about, you shouldn‘t hit these limits until the network is truly gigantic!

BGP cores deployed for scaling are generally a sign the network design needs to be rethought

In some cases, however, the network design is just what it is, and we have to do what we can to make it work

Why Use a BGP Core?


TECRST-2021

13881_06_2007_c1 268

BGP Cores

Some networks are not networks, but rather internetworks

An internetwork is made up of multiple smaller networks, each one under separate administrative control

An interior gateway protocol may work as a ―core protocol,‖ as long as the network isn‘t too large, and the administrators all work together well

OSPF Core

Finance

(EIGRP)

HQ (RIP)

Redistribute here

Why Use a BGP Core?


TECRST-2021

13881_06_2007_c1 269

BGP Cores

Each administration team can better control routing information flow

A major failure in one part of the network is less likely to impact the core or other sections of the network

Less finger pointing means a smoother running, more stable network

OSPF Core

Finance

(EIGRP)

HQ (RIP)

Redistribute here

BGP Core

Why Use a BGP Core?

It‘s better to use a policy based protocol in the core, however:


TECRST-2021

13881_06_2007_c1 270

BGP Cores

Consider administrative division points

Divide up complex areas of the network as much as possible

Consider physical and topological choke points

Consider places where you could summarize, if at all possible

Complex

topological areas

BGP core

Deployment

Determine where the boundaries of the core should be


TECRST-2021

13881_06_2007_c1 271

BGP Cores

Don‘t ever redistribute all the routing data from BGP into the IGP at the edge; routes should be injected in a very controlled manner

If possible, inject just the default into the IGP

To provide optimal routing, you can inject summaries into the IGP as well, but this should be limited to one or two routes

BGP core

0.0.0.0/0

10.1.0.0/16

10.2.0.0/16

Deployment


TECRST-2021

13881_06_2007_c1 272

BGP Cores

There are several possible ways to manage getting routes into the IGP from the BGP core

The primary factor is in whether the filtering should be done by the administrators of the IGP areas, or the BGP core

Don‘t pass full routes to the IGP area routers unless you want the filtering done by the IGP area administrators

eBGP

session

A

B

BGP core

IGP Area

Redistribute eBGP

learned routes into the

IGP

Generate or permit a default

and other routes towards

the IGP area edge

Pass the entire BGP table to

the IGP area edge

Redistribute filtered

eBGP learned routes

into the IGP

Generate a default

and other routes

into the IGP

Pass no routes to the

IGP area edge

Deployment


TECRST-2021

13881_06_2007_c1 273

BGP Cores

If the core doesn‘t have a default, you can generate a default on the edge

router

router bgp <AS number>

neighbor 10.1.1.1 default-information originate

If the core has a default you can pass on through the edge, but you want to make

certain there is always a default route

supplied to the IGP areas

ip route 0.0.0.0 0.0.0.0 null0 200

!


!

route-map 0-only permit 10

match ip address 10

!


neighbor 10.1.1.1 distribute-list 10 out

redistribute static route-map 0-only


If the core has a default and you want it to be dynamically provided to the IGP

areas


!





A

B

BGP core

IGP Area

Redistribute all eBGP learned routes into the IGP here

Deployment


TECRST-2021

13881_06_2007_c1 274

BGP Cores

Pass the more specific into the IGP area into

the core using a

distribute-list to filter out all the other routes



!



Generate it using a summary (but

remember to watch out

for summary black holes)

!


aggregate-address 10.1.0.0 255.255.0.0 summary-only

A

B

BGP core

IGP Area

Redistribute all eBGP learned routes into the IGP here

Deployment


TECRST-2021

13881_06_2007_c1 275

BGP Cores

If the IGP area edge router is receiving full routing information, filtering redistribution into the IGP is required

A

B

BGP core

IGP Area

Full BGP routing

information



!

route-map localin permit 10

match ip address 10

!

router eigrp 100

redistribute bgp <AS number> route-map localin

Deployment


TECRST-2021

13881_06_2007_c1 276

BGP Cores

If the core edge router isn‘t providing any routing information to the IGP area edge, a locally generated default can be created

A

B

BGP core

IGP Area

No routing

information

OSPF router ospf 100

default-information originate always

EIGRP ip route 0.0.0.0 0.0.0.0 null0 200

!

router eigrp 100

redistribute static metric ....

IS-IS router isis

default-information originate

Deployment


TECRST-2021

13881_06_2007_c1 277

BGP Cores

Filter or summarize from the IGP areas into the core; be careful of routing black holes

Be very careful with complex filtering techniques at the edge; consider maintenance requirements carefully

BGP core

Summarize

and filter

Deployment


TECRST-2021

13881_06_2007_c1 278

BGP Cores

Filter the default route and any routes learned from BGP when redistributing into BGP at the IGP area edge

Filtering routing information using a list of specific

prefixes




!

route-map nolocalout permit 10

match ip address 10

!


redistribute ospf 100 route-map nolocalout

Tagging routes into the IGP, and filtering on the tags

redistributing from the IGP


!

route-map tagfilter deny 10

match tag 100

match ip address 10

route-map tagfilter permit 20

set tag 100

!


redistribute OSPF route-map tagfilter metric 10

!

router ospf 100

redistribute bgp <AS number> route-map tagfilter

A

B

BGP core

IGP Area

Deployment


TECRST-2021

13881_06_2007_c1 279

BGP Cores

What autonomous system numbers should you use when deploying a BGP core?

It depends on whether or not the BGP core is going to be tied into the network‘s connectivity to the outside networks, including the Internet

Deployment


TECRST-2021

13881_06_2007_c1 280

BGP Cores

If the BGP running in the core is not going to touch, in any way, connections to outside networks, use private AS numbers throughout, even for the network core

BGP core

Internet

Partner

DMZ

Routes generated at the

edge, rather than passed

through from the core

Deployment


TECRST-2021

13881_06_2007_c1 281

BGP Cores

If routes are passed through the BGP core, a public AS number can be used for the core

The IGP areas can be assigned private AS numbers

Advertisements from the IGP areas can be filtered at the edge towards the outside networks

The routing information can be aggregated at the edge

BGP core

Internet

Partner

Routing

information

passes through

the core

Deployment


TECRST-2021

13881_06_2007_c1 282

BGP Cores

If each IGP area is considered a network under separate administrative control, the BGP core can become a ―mini service provider,‖ offering various services to the ―client networks,‖ even though they are all within the same large organization

For instance, one such service would be the provisioning of MPLS VPN tunnels through the core between IGP areas and outside partners, or between IGP areas

BGP core

Internet

Partner

MPLS VPN to a

partner

MPLS VPN

between IGP

areas

Deployment


TECRST-2021

13881_06_2007_c1 283

BGP Cores

The BGP core could also provide quality of service forwarding, using QPPB to transport quality of service information to the edges of the core

Communities carried in BGP, along with access lists and AS path lists, can be used to classify packets on the edges of the BGP core

This classification is then used to modify the way packets are forwarded through the network

BGP core

Internet

Partner

Routes from partner

marked for QOS

service level

Packets marked based on BGP

transported QOS information

Deployment


TECRST-2021

13881_06_2007_c1 284

BGP Cores

The BGP core could be used as a basis for providing high quality connectivity to the Internet (and partners)

Optimized Exit Routing (OER) can determine the best path to given destinations, and steer traffic along that path

For more information, attend the Optimized Edge Routing (OER) presentation

BGP core

Internet

Internet

OER steers

traffic along the

best exit point


TECRST-2021

13881_06_2007_c1 285

BGP Cores

MPLS VPNs through a BGP Core

http://www.cisco.com/en/US/tech/tk436/tk428/technologies_configuration_example09186a00800a6c11.shtml

RST-1601, Introduction to MPLS VPNs

RST-2602, Deploying MPLS VPNs

RST-3605, Troubleshooting MPLS VPNs

Quality of Service BGP Propagation

http://www.cisco.com/en/US/partner/products/hw/routers/ps133/products_configuration_guide_chapter09186a008007df4f.html#1015477

Optimized Exit Routing

RST-4311, Advances in Routing Protocols

Deployment

http://www.cisco.com/en/US/tech/tk436/tk428/technologies_�configuration_example09186a00800a6c11.shtml



http://www.cisco.com/en/US/partner/products/hw/routers/ps133/products_configuration_guide_chapter09186a008007df4f.html




TECRST-2021

13881_06_2007_c1 286

BGP Cores

A has two paths to 10.0.0.0/8 with the same metric down to the router ID

It will mark one of them as the best path, and send all traffic along the link to that exit point

iBGP multipath allows A to load share between these two paths

AS65000

AS65001

10.0.0.0/8

A

B C

D E

router-a#sh ip bgp 10.0.0.0

65001

192.168.1.1 from 192.168.1.1

(192.168.1.1)

Origin IGP, metric 0, localpref 100,

valid, internal,

65001

192.168.2.2 from 192.168.2.2

(192.168.2.2)

Origin IGP, metric 0, localpref 100,

valid, internal, best

All traffic

sent through

C to

10.0.0.0/8

Deployment


TECRST-2021

13881_06_2007_c1 287

BGP Cores

Flag multiple iBGP paths as ‗multipath‘

Each path must have a unique NEXT_HOP

Number of multipaths configured are inserted in the routing table

maximum-paths ibgp <1–6>

Only the bestpath is advertised to A‘s BGP peers

http://www.cisco.com/en/US/partner/products/sw/iosswrel/ps1839/products_feature_guide09186a0080087b00.html

http://www.cisco.com/cgi-bin/Support/Bugtool/ onebug.pl?bugid=CSCdp72929

Deployment

http://www.cisco.com/en/US/partner/products/sw/�iosswrel/ps1839/products_feature_guide09186a0080087b00.html




http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCdp72929





TECRST-2021

13881_06_2007_c1 288

BGP Cores

AS65000

AS65001

10.0.0.0/8

A

B C

D E

router bgp 65000

maximum paths ibgp 2

....


65001

192.168.1.1 from 192.168.1.1 (192.168.1.1)

Origin IGP, metric 0, localpref 100, valid,

internal, multipath

65001

192.168.2.2 from 192.168.2.2 (192.168.2.2)

Origin IGP, metric 0, localpref 100, valid,

internal, multipath, best

....

router-s#sh ip route 10.0.0.0

Routing entry for 10.0.0.0/8

* 192.168.1.1 , from 192.168.1.1 , 00:00:09 ago

Route metric is 0, traffic share count is 1

AS Hops 1

192.168.2.2 , from 192.168.2.2 , 00:00:09 ago


AS Hops 1

Traffic is

load shared

across

both links

Deployment


TECRST-2021

13881_06_2007_c1 289

AS65000

AS65001

A

D

C

AS65002

10.0.0/8

AS65003

B

BGP Cores

If two paths are learned from different autonomous systems, it‘s impossible to load share between them


65001 65002

192.168.1.1 from 192.168.1.1 (192.168.1.1)

Origin IGP, metric 0, localpref 100, valid, internal,

65003 65002

192.168.2.2 from 192.168.2.2 (192.168.2.2)

Origin IGP, metric 0, localpref 100, valid, internal, best

Cannot load share

Deployment


TECRST-2021

13881_06_2007_c1 290

BGP Cores

Even when two paths are learned from the same AS through eBGP, BGP won‘t load share between them by default

But we could get load sharing by building a single multihop session between B and C

AS65000

AS65001

A

D

C

AS65002

10.0.0/8

AS65003

B

Only one route is

installed in the

routing table

Deployment


65001 65002

192.168.1.1 from 192.168.1.1 (192.168.1.1)

Origin IGP, metric 0, localpref 100, valid, internal,

65001 65002

192.168.2.2 from 192.168.2.2 (192.168.2.2)

Origin IGP, metric 0, localpref 100, valid, internal, best



* 192.168.1.1 , from 192.168.1.1 , 00:00:09 ago


AS Hops 2


TECRST-2021

13881_06_2007_c1 291

BGP Cores

router bgp 65000


neighbor 192.168.2.1 ebgp-multihop 2

neighbor 192.168.2.1 update-source

192.168.1.1

!

ip route 192.168.2.1 255.255.255.255 10.1.1.1

ip route 192.168.2.1 255.255.255.255 10.1.2.1


65001

192.168.2.1 from 192.168.2.1 (192.168.1.1)

Origin IGP, valid, internal, best



192.168.2.1, from 192.168.2.1, 00:00:09 ago


AS Hops 1

router-a#show ip route 192.168.2.1


* 10.1.1.1 from 0.0.0.0, 00:00:00 ago

Route metric is 0, traffic share count is

1

10.1.2.1 from 0.0.0.0, 00:00:00 ago

Route metric is 0, traffic share count is

1

A

B

AS65000

AS65001

192.168.1.1

192.168.2.1

eBGP

10

.1.1

.1

10

.1.2

.1

The eBGP session is set

up as a multihop session

between the loopbacks

There are multiple paths

between the loopbacks

There‘s only one path to

10.0.0.0/8, but there are

multiple paths to the next

hop; A load shares

between the two possible

paths to the next hop

Deployment


TECRST-2021

13881_06_2007_c1 292

Outside Connections

If the BGP running in the core is not going to touch, in any way, connections to outside networks, use private AS numbers throughout, even for the network core

BGP core

Internet

Partner

DMZ

Routes generated at the

edge, rather than passed

through from the core

Advertising Routes Outside


TECRST-2021

13881_06_2007_c1 293

Outside Connections

If routes are passed through the BGP core, a public AS number can be used for the core

The IGP areas can be assigned private AS numbers

Advertisements from the IGP areas can be filtered at the edge towards the outside networks

The routing information can be aggregated at the edge

BGP core

Internet

Partner

Routing

information

passes

through the

core



TECRST-2021

13881_06_2007_c1 294

Outside Connections

! permit anything in 10.1.4.0/20 to partner 1

ip prefix-list pl-ptner1 permit 10.1.40.0/20 ge 21

!

! permit anything from private as 65005 to partner 1

ip as-path access-list 100 permit ^.*_65005$

!

! route map putting partner 1’s filters together

route-map rm-ptner1 permit 10

match ip address prefix-list pl-ptner1

route-map rm-ptner1 permit 20

match as-path 100

route-map rm-ptner1 deny 30

!

! other filters as needed for other partners

!

router bgp <public as number>

! aggregate public address space to the internet

aggregate-address 192.168.40.0 255.255.248.0 summary-only

neighbor <internet> remote-as <isp as>

! build peering with partner 1 and put filters on

neighbor <partner1> remote-as <partner as>

neighbor <partner1> route-map rm-ptner1 out

Internet

Partner 1

BGP

Core



TECRST-2021

13881_06_2007_c1 295

Outside Connections

You can also use communities to express filtering from the IGP areas into outside networks

Communities are opaque ―route tags‖ which can carry policy on a per prefix basis in BGP

This could be combined with aggregation, as well, for public address space advertised into the Internet

Internet

Partner

BGP

Core

10.1.1.1

10.2.2.2

Apply communities

marking routes to be

filtered

Filter outbound to

partners based on

communities; aggregate

towards the Internet



TECRST-2021

13881_06_2007_c1 296

route-map to-ptner1 permit 10

match community 10

set community NO_EXPORT

route-map to-ptner1 deny 20

!

route-map to-ptner2 permit 10

match community 20

set community NO_EXPORT

route-map to-ptner2 deny 20

!

router bgp 65000

neighbor <partner 1> route-map to-ptner1 out

neighbor <partner 2> route-map to-ptner2 out

Internet

Partner 1

BGP

Core

! routes to advertise to partner 1

access-list 10 permit 10.2.8.0/24



!

route-map tocore permit 10

match ip address 10

set community 1000


match ip address 20

set community 2000

!

router bgp 65004

neighbor <bgp core> route-map tocore out



!


match ip address 10

set community 1000

!

router bgp 65005

neighbor <bgp core> route-map tocore out

Outside ConnectionsAdvertising Routes Outside


TECRST-2021

13881_06_2007_c1 297

Outside Connections

Make use of the NO_EXPORT community to prevent routes from leaking out of the BGP core

Make use of the NO_EXPORT community to prevent routes from leaking out from partner networks to their peers

In the future, more interesting filtering capabilities will be built on BGP communities

NOPEER community for BGP route scope controlhttp://www.ietf.org/rfc/rfc3765.txt

Controlling the redistribution of BGP routeshttp://www.ietf.org/proceedings/03mar/I-D/draft-ietf-ptomaine-bgp-redistribution-02.txt


http://www.ietf.org/rfc/rfc3765.txt

http://www.ietf.org/proceedings/03mar/I-D/draft-ietf-�ptomaine-bgp-redistribution-02.txt















TECRST-2021

13881_06_2007_c1 298

Outside Connections

Should you run BGP at all to connect to the Internet?

If you are connecting in a single place, no

Distribute a default into your network, and allow the ISP to originate the routes to your networks at their edge

If you are dual homed to the same ISP in the same physical location, there‘s no reason to run BGP

ISP

Enterprise

192.168.1.0/24

A

B

C

192.168.1.0/24

0.0.0.0/0

Internet Connection Considerations


TECRST-2021

13881_06_2007_c1 299

Outside Connections

If you are dual homed to the same SP in two different locations, you may want to accept at least partial routes at both locations, and use the MED to route optimally

If you always want to take the closest exit point out of your network, however, you don‘t need to run BGP

Enterprise

192.168.1.0/24

London Raleigh

ISP A

AS65000

London New York

Optimal

path to

London

Closest

exit path

to

London



TECRST-2021

13881_06_2007_c1 300

Outside Connections

If you are dual homed to two ISPs, you should run BGP to advertise routing information to both of them

This doesn‘t mean you should accept the full routing table from both service providers, however

You can still originate a local default route into your network, and accept no routes from either SP

0.0.0.0/0

ISP A

AS65000

Enterprise

192.168.1.0/24

AISP B

AS65001

192.168.1.0/24



TECRST-2021

13881_06_2007_c1 301

Outside Connections

Why would you accept partial routes?

So you can optimally route to destinations connected to one of the ISP‘s you‘re peering to, while allowing traffic to more distant destinations to flow along default routes

Typically, you will accept all of the routes originated by each ISP, and possibly the routes of each of their directly connected customers



TECRST-2021

13881_06_2007_c1 302

ISP A

AS65000

Enterprise

AS65002

ISP B

AS65001

ip as-path access-list 100 permit ^65000(_[1-9]*)\1*$

ip as-path access-list 110 permit ^65001(_[1-9]*)\1*$

!

router bgp 65002

neighbor <ISP A> remote-as 65000

neighbor <ISP A> filter-list 100 in

neighbor <ISP B> remote-as 65001

neighbor <ISP B> filter-list 110 in

Match any AS path starting

with AS65000

Match any single AS

number repeated any

number of times

Outside ConnectionsInternet Connection Considerations


TECRST-2021

13881_06_2007_c1 303

Outside Connections

You can also ask the ISP to filter the routes they are sending at the edge of their network, which reduces the load on your edge router

ISP A

AS65000

Enterprise

AS65002

ISP B

AS65001

Filter for connected

customer and

originated routes

Accept all

advertised

routes



TECRST-2021

13881_06_2007_c1 304

Outside Connections

You could ask the ISP to configure Outbound Route Filtering, which allows you to configure the filters, but the ISP router actually filters the routes

http://www.cisco.com/en/US/partner/products/sw/iosswrel/ps1839/products_feature_guide09186a0080087c26.html

http://www.ietf.org/internet-drafts/draft-ietf-idr-route-filter-10.txt

This only works for prefix based filters, not for AS Path filters right now

http://www.ietf.org/internet-drafts/draft-ietf-idr-aspath-orf-06.txt

AS Path ORF support is planned in Cisco IOS































TECRST-2021

13881_06_2007_c1 305

Outside Connections

ISP advertises some route to AS65002, which then readvertises the route to ISP B

ISP B chooses the path through AS65002 as the best path, directing all traffic for that destination through the customer‘s network

The customer network has become a transit

ISP A

AS65000

Enterprise

AS65002

ISP B

AS65001

172.18.1.0/24

Best path for

172.18.1.0/24



TECRST-2021

13881_06_2007_c1 306

Outside Connections

How can you prevent this from happening?

One common way is to count on lack of synchronization to prevent routes from being readvertised

Don‘t count on synchronization; at some point it will be off by default!

Filtering these routes is simple; a single line AS path access list will do the right thing



TECRST-2021

13881_06_2007_c1 307

ISP A

AS65000

Enterprise

AS65002

ISP B

AS65001ip as-path access-list 100 permit ^$

!

router bgp 65002

neighbor <ISP A> remote-as 65000

neighbor <ISP A> filter-list 100 out

neighbor <ISP B> remote-as 65001

neighbor <ISP B> filter-list 100 out

Outside ConnectionsInternet Connection Considerations


TECRST-2021

13881_06_2007_c1 308

ISP A

AS65000

Enterprise

AS65002

ISP B

AS65001

Outside Connections

You dual home to gain diversity in your routing path:

If a links fails due to backhoe fade, you still have a connection to the outside

If an ISPs fails, you still have a connection to the outside

What if the two physical links run through the same conduit?

What if both ISPs use the same upstream?

ISP C



TECRST-2021

13881_06_2007_c1 309

Outside Connections

The Problem:

Logical Diversity isn‘t the same as physical diversity

Diversity of any type at one point doesn‘t guarantee diversity throughout; things may recombine at some point

The Solution:

When dual homing, try to dual home from and to physically diverse points

If dual homing from the same physical location, consider using a single provider, and putting physical diversity in the contract

Try to ensure that your providers aren‘t dependant on each other, or on a common point behind them



TECRST-2021

13881_06_2007_c1 310

BGP/IGP Interaction

G advertises 10.1.1.0/24 to F through eBGP; F readvertises it to B through iBGP

B checks its local routing table, and finds that G is reachable, so it installs the route, and advertises 10.1.1.0/24 to A through eBGP

A

B

E

F

D

G

10.1.1.0/24

BGP AS

eBGP

eBGP

iBGP

10.1.1.0/24 via B

10.1.1.0/24 via G

G is reachable via D

BGP Synchronization


TECRST-2021

13881_06_2007_c1 311

BGP/IGP Interaction

A receives a packet for 10.1.1.1, and forwards it to B

B examines its routing table, and finds the next hop is G, a recursive route, and find the next hop of the recursive route is D, so it forwards the packet to D

D, since it‘s not running BGP at all, has no route to 10.1.1.0/24, so it drops the packet!

A

B

E

F

D

G

10.1.1.0/24

BGP AS

eBGP

eBGP

iBGP

10.1.1.0/24 via B

10.1.1.0/24 via G

G is reachable via D

No route to

10.1.1.0/24!

BGP Synchronization


TECRST-2021

13881_06_2007_c1 312

BGP/IGP Interaction

Synchronization solves this by forcing the IGP and BGP routing tables to match before a route can be advertised to a peer

B would not advertise 10.1.1.0/24 to A if the route isn‘t reachable via some path other than BGP

Unless you want 150,000 routes in your IGP, this isn‘t very useful

A

B

E

F

D

G

10.1.1.0/24

BGP AS

eBGP

eBGP

iBGP

No IGP route to

10.1.1.0/24; don‘t

advertise to

eBGP peers

BGP Synchronization


TECRST-2021

13881_06_2007_c1 313

BGP/IGP Interaction

The more general solution is to run BGP on D an E, and disable synchronization

This requires running full mesh iBGP on B, D, E, and F, or running route reflectors in the core

A

B

E

F

D

G

10.1.1.0/24

BGP AS

eBGP

eBGP

Full mesh iBGP

BGP Synchronization


TECRST-2021

13881_06_2007_c1 314

BGP/IGP Interaction

Conveys relative preference of entry points

Lowest MED is best—Default is no MED==0

Comparable only if paths are from same AS

Non-transitive—Do not pass MED from one AS to another

route-map: set metricset metric-type internal

AS 1

AS 6

AS 5

AS 2

AS 3

AS 4

CITY B

CITY A CITY A

BGP/IGP Interaction


TECRST-2021

13881_06_2007_c1 315

B

AS 1

AS 6

AS 2

AS

A

Configuration:

router bgp 1

neighbor x.x.x.x remote-as 2

neighbor x.x.x.x route-map set_MED out

!

route-map set_MED permit 10

match as-path 2

set metric-type internal

BGP/IGP InteractionSet MED to IGP Metric


TECRST-2021

13881_06_2007_c1 316

C

10

10

BGP/IGP Interaction

E is learning 10.1.1.0/24 through iBGP from D with a next hop of A

E examines the path to A, and finds an IGP route through D to A; it installs this route in the routing table

C is now inserted into the circuit; after a few seconds, the IGP has converged, and E now chooses C as the best path to A

A

B

D

E

10.1.1.0/24

eBGP

Full iBGP

mesh

20

20

Original

best path

to A

C starts and

provides a

better path to

A

Wait for BGP


TECRST-2021

13881_06_2007_c1 317

C

10

10

BGP/IGP Interaction

However, BGP takes much longer to converge if C is accepting full routes (about 150,000 routes) from A; at least five minutes

When E forwards packets to C for 10.1.1.1, C hasn‘t finished building its BGP tables, so it doesn‘t know how to reach this destination

C drops the packets

A

B

D

E

10.1.1.0/24

eBGP

full iBGP

mesh

20

20C has no path

to 10.1.1.0/24

Wait for BGP


TECRST-2021

13881_06_2007_c1 318

C

10

10

BGP/IGP Interaction

Instead, once the IGP has converged, C signals its IGP neighbors that they should not route this direction

The IGP remains in this state until BGP notifies the IGP it has converged

E will continue using D as its best path to A, even though a better one is available, until BGP converges on C

A

B

D

E

10.1.1.0/24

eBGP

Full iBGP

mesh

20

20

Don‘t use me yet!

Wait for BGP


TECRST-2021

13881_06_2007_c1 319

BGP/IGP Interaction

OSPF uses max-metric router-lsa on-startup wait-for-bgp to configure this feature

Available in 12.2T


IS-IS uses set-overload-bit on-startup wait-for-bgp to configure this feature

Available in 11.3

http://www.cisco.com/en/US/tech/tk365/tk381/technologies_tech_note09186a00800a4bb1.shtml

Wait for BGP

http://www.cisco.com/en/US/partner/products/sw/iosswrel/�ps1839/products_feature_guide09186a0080087c09.html



http://www.cisco.com/en/US/tech/tk365/tk381/technologies_�tech_note09186a00800a4bb1.shtml




TECRST-2021

13881_06_2007_c1 320

Summary


TECRST-2021

13881_06_2007_c1 321

Other References

ASIN: 1578701651 ISBN: 0201657732 ISBN: 1587051877


TECRST-2021

13881_06_2007_c1 322

Other References

ISBN: 1587050323 ISBN: 1578702208 ISBN: 0201657724


TECRST-2021

13881_06_2007_c1 323

Other References

ISBN: 0321127005 ISBN: 1587051095 ISBN: 0201379511


TECRST-2021

13881_06_2007_c1 324

Recommended Reading

Continue your Networkers at Cisco Live learning experience with further reading from Cisco Press

Check the Recommended Reading flyer for suggested books

Available Onsite at the Cisco Company Store


TECRST-2021

13881_06_2007_c1 325

Q and A


TECRST-2021

13881_06_2007_c1 326

Complete Your Online Session Evaluation

Win fabulous prizes; give us your feedback

Receive ten Passport Points for each session evaluation you complete

Go to the Internet stations located throughout the Convention Center to complete your session evaluation

Winners will be announced daily at the Internet stations


TECRST-2021

13881_06_2007_c1 327

Cisco Net Workers - Deploying Interior Gateway Protocols (2007)

Documents