Mpls vpn toi

1© 2001, Cisco Systems, Inc. Course NumberPresentation_ID

MPLS VPN TOIMPLS VPN TOI

[email protected]@cisco.com

2TOI-VPNeosborne © 2001, Cisco Systems, Inc.

AgendaAgenda

• How MPLS VPN works• What Code Is MPLS VPN In?• Platform Issues in Implementation• Lab Demo - config


How MPLS-VPN WorksHow MPLS-VPN Works• Concepts and goals• Terminology• Connection model• Forwarding• Mechanisms• Topologies• Scaling• Configuration


MPLS-VPN MPLS-VPN What is a VPN ?What is a VPN ?

• An IP network infrastructure delivering private network services over a public infrastructureUse a layer 3 backboneScalability, easy provisioningGlobal as well as non-unique private address

spaceQoSControlled accessEasy configuration for customers


VPN Models - The Overlay modelVPN Models - The Overlay model

• Private trunks over a TELCO/SP shared infrastructureLeased/Dialup linesFR/ATM circuitsIP (GRE) tunnelling

• Transparency between provider and customer networks

• Optimal routing requires full mesh over over backbone


VPN Models - The Peer modelVPN Models - The Peer model

• Both provider and customer network use same network protocol

• CE and PE routers have a routing adjacency at each site

• All provider routers hold the full routing information about all customer networks

• Private addresses are not allowed• May use the virtual router capability

Multiple routing and forwarding tables based on Customer Networks


VPN Models - MPLS-VPN: VPN Models - MPLS-VPN: The True Peer modelThe True Peer model

• Same as Peer model BUT !!!• Provider Edge routers receive and hold

routing information only about VPNs directly connected

• Reduces the amount of routing information a PE router will store

• Routing information is proportional to the number of VPNs a router is attached to

• MPLS is used within the backbone to switch packets (no need of full routing)


AgendaAgenda• Concepts and goals• Terminology• Connection model• Forwarding• Mechanisms• Topologies• Scaling• Configuration


MPLS-VPN TerminologyMPLS-VPN Terminology

• Provider Network (P-Network)The backbone under control of a Service

Provider

• Customer Network (C-Network)Network under customer control

• CE routerCustomer Edge router. Part of the C-

network and interfaces to a PE router



• SiteSet of (sub)networks part of the C-network and co-

locatedA site is connected to the VPN backbone through

one or more PE/CE links

• PE routerProvider Edge router. Part of the P-Network and

interfaces to CE routers

• P routerProvider (core) router, without knowledge of VPN



• Border routerProvider Edge router interfacing to other

provider networks

• Extended CommunityBGP attribute used to identify a Route-origin,

Route-target

• Site of Origin Identifier (SOO)64 bits identifying routers where the route has

been originated



• Route-Target64 bits identifying routers that should receive

the route

• Route DistinguisherAttributes of each route used to uniquely

identify prefixes among VPNs (64 bits)VRF based (not VPN based)

• VPN-IPv4 addressesAddress including the 64 bits Route

Distinguisher and the 32 bits IP address



• VRFVPN Routing and Forwarding Instance

Routing table and FIB table

Populated by routing protocol contexts

• VPN-Aware networkA provider backbone where MPLS-VPN is

deployed




MPLS VPN Connection ModelMPLS VPN Connection Model

• A VPN is a collection of sites sharing a common routing information (routing table)

• A site can be part of different VPNs• A VPN has to be seen as a community

of interest (or Closed User Group)• Multiple Routing/Forwarding

instances (VRF) on PE routers



• A site belonging to different VPNs may or MAY NOT be used as a transit point between VPNs

• If two or more VPNs have a common site, address space must be unique among these VPNs

Site-1

Site-3

Site-4

Site-2

VPN-AVPN-C

VPN-B



• The VPN backbone is composed by MPLS LSRsPE routers (edge LSRs)P routers (core LSRs)

• PE routers are faced to CE routers and distribute VPN information through MP-BGP to other PE routers

VPN-IPv4 addresses, Extended Community, Label

• P routers do not run BGP and do not have any VPN knowledge



VPN_A

VPN_A

VPN_B10.3.0.0

10.1.0.0

11.5.0.0

P P

PP PE

PE CE

CE

CE

VPN_A

VPN_B

VPN_B

10.1.0.0

10.2.0.0

11.6.0.0

CEPE

PECE

CE

VPN_A10.2.0.0

CE

iBGP sessions

• P routers (LSRs) are in the core of the MPLS cloud

• PE routers use MPLS with the core and plain IP with CE routers

• P and PE routers share a common IGP• PE router are MP-iBGP fully meshed



• PE and CE routers exchange routing information through:EBGP, OSPF, RIPv2, Static routing

• CE router run standard routing software

PE

CE

CE

Site-2

Site-1

EBGP,OSPF, RIPv2,Static



• PE routers maintain separate routing tablesThe global routing table

With all PE and P routesPopulated by the VPN backbone IGP (ISIS or OSPF)

VRF (VPN Routing and Forwarding)Routing and Forwarding table associated with one or more

directly connected sites (CEs)VRF are associated to (sub/virtual/tunnel)interfacesInterfaces may share the same VRF if the connected sites may

share the same routing information

PE

CE

CE

Site-2

Site-1

VPN Backbone IGP (OSPF, ISIS) EBGP,OSPF, RIPv2,Static



• Different site sharing the same routing information, may share the same VRF

• Interfaces connecting these sites will use the same VRF

• Sites belonging to the same VPN may share same VRF

PE

CE

CE

Site-2

Site-1



• The routes the PE receives from CE routers are installed in the appropriate VRF

• The routes the PE receives through the backbone IGP are installed in the global routing table

• By using separate VRFs, addresses need NOT to be unique among VPNs

PE

CE

CE

Site-2

Site-1

VPN Backbone IGP EBGP,OSPF, RIPv2,Static



• The Global Routing Table is populated by IGP protocols.

• In PE routers it may contain the BGP Internet routes (standard BGP-4 routes)

• BGP-4 (IPv4) routes go into global routing table

• MP-BGP (VPN-IPv4) routes go into VRFs



PE

VPN Backbone IGP

iBGP session

PE

P P

P P

• PE and P routers share a common IGP (ISIS or OSPF)

• PEs establish MP-iBGP sessions between them• PEs use MP-BGP to exchange routing

information related to the connected sites and VPNs

VPN-IPv4 addresses, Extended Community, Label


MPLS VPN Connection ModelMPLS VPN Connection ModelMP-BGP UpdateMP-BGP Update

• VPN-IPV4 addressRoute Distinguisher

64 bitsMakes the IPv4 route globally unique RD is configured in the PE for each VRFRD may or may not be related to a site or a VPN

IPv4 address (32bits)• Extended Community attribute (64 bits)

Site of Origin (SOO): identifies the originating site Route-target (RT): identifies the set of sites the

route has to be advertised to



Any other standard BGP attributeLocal PreferenceMEDNext-hopAS_PATHStandard Community...

A Label identifying:The outgoing interfaceThe VRF where a lookup has to be done (aggregate

label)The BGP label will be the second label in the label

stack of packets travelling in the core


MPLS VPN Connection ModelMPLS VPN Connection ModelMP-BGP Update - Extended communityMP-BGP Update - Extended community

• BGP extended community attribute Structured, to support multiple

applications 64 bits for increased range

• General form <16bits type>:<ASN>:<32 bit number>

Registered AS number <16bits type>:<IP address>:<16 bit number>

Registered IP address


MPLS VPN Connection ModelMPLS VPN Connection ModelMP-BGP Update - Extended communityMP-BGP Update - Extended community

• The Extended Community is used to:

Identify one or more routers where the route has been originated (site)

Site of Origin (SOO) Selects sites which should receive the route

Route-Target



• The Label can be assigned only by the router which address is the Next-Hop attributePE routers re-write the Next-Hop with their own

address (loopback interface address)“Next-Hop-Self” BGP command towards iBGP neighborsLoopback addresses are advertised into the backbone IGP

• PE addresses used as BGP Next-Hop must be uniquely known in the backbone IGP No summarisation of loopback addresses in the

core



PE-1 VPN Backbone IGP

PE-2

P P

P P

PE routers receive IPv4 updates (EBGP, RIPv2, Static)PE routers translate into VPN-IPv4

Assign a SOO and RT based on configurationRe-write Next-Hop attributeAssign a label based on VRF and/or interfaceSend MP-iBGP update to all PE neighbors

BGP,RIPv2 update for Net1,Next-Hop=CE-1

VPN-IPv4 update:RD:Net1, Next-hop=PE-1SOO=Site1, RT=Green, Label=(intCE1)

CE-1

Site-2

VPN-IPv4 update is translated into IPv4 address (Net1) put into VRF green since RT=Green and advertised to CE-2

Site-1

CE-2



Receiving PEs translate to IPv4 Insert the route into the VRF identified by the

RT attribute (based on PE configuration) The label associated to the VPN-IPv4 address will

be set on packet forwarded towards the destination


PE-2

P P

P PBGP,OSPF, RIPv2 update for Net1Next-Hop=CE-1


CE-1

Site-2


Site-1

CE-2



• Route distribution to sites is driven by the Site of Origin (SOO) and Route-target attributes

BGP Extended Community attribute• A route is installed in the site VRF

corresponding to the Route-target attributeDriven by PE configuration

• A PE which connects sites belonging to multiple VPNs will install the route into the site VRF if the Route-target attribute contains one or more VPNs to which the site is associated




MPLS ForwardingMPLS ForwardingPacket forwardingPacket forwarding

• PE and P routers have BGP next-hop reachability through the backbone IGP

• Labels are distributed through LDP (hop-by-hop) corresponding to BGP Next-Hops

• Label Stack is used for packet forwardingTop label indicates BGP Next-Hop (interior

label)Second level label indicates outgoing interface

or VRF (exterior label)


MPLS ForwardingMPLS ForwardingPacket forwardingPacket forwarding

• MPLS nodes forward packets based on the top label

• P routers do not have BGP (nor VPN) knowledge

No VPN routing informationNo Internet routing information


MPLS ForwardingMPLS ForwardingPenultimate Hop PoppingPenultimate Hop Popping

• The upstream LDP peer of the BGP next-hop (PE router) will pop the first level label

The penultimate hop will pop the label• Requested through LDP• The egress PE router will forward the packet

based on the second level label which gives the outgoing interface (and VPN)


MPLS ForwardingMPLS ForwardingMPLS Forwarding - Penultimate Hop PoppingMPLS Forwarding - Penultimate Hop Popping

PE2

PE1

CE1

CE2

P1 P2

IGP Label(PE2)VPN LabelIPpacket

PE1 receives IP packetLookup is done on site VRF BGP route with Next-Hop and Label is foundBGP next-hop (PE2) is reachable through IGP route with associated label

IGP Label(PE2)VPN LabelIPpacket

P routers switch the packets based on the IGP label (label on top of the stack)

VPN LabelIPpacket

Penultimate Hop PoppingP2 is the penultimate hop for the BGP next-hopP2 remove the top labelThis has been requested through LDP by PE2

IPpacket

PE2 receives the packets with the label corresponding to the outgoing interface (VRF)One single lookupLabel is popped and packet sent to IP neighborIP

packet

CE3


T1 T7T2 T8T3 T9T4 T7T5 TBT6 TBT7 T8

MPLS VPN MPLS VPN ForwardingForwarding

VPN_A

VPN_A

VPN_B10.3.0.0

10.1.0.0

11.5.0.0

P P

PP PE

CE

CE

CE

Data

<RD_B,10.1> , iBGP next hop PE1<RD_B,10.2> , iBGP next hop PE2<RD_B,10.3> , iBGP next hop PE3<RD_A,11.6> , iBGP next hop PE1<RD_A,10.1> , iBGP next hop PE4<RD_A,10.4> , iBGP next hop PE4<RD_A,10.2> , iBGP next hop PE2

<RD_B,10.2> , iBGP NH= PE2 , T2 T8• Ingress PE receives normal IP Packets from CE router

• PE router does “IP Longest Match” from VPN_B FIBVPN_B FIB , find iBGP next hop PE2PE2 and impose a stack of labels: exterior Label T2T2 + Interior Label T8T8

DataT8T2VPN_A

VPN_B

VPN_B

10.1.0.0

10.2.0.0

11.6.0.0

CEPE1

PE2CE

CE

VPN_A10.2.0.0

CE


MPLS VPN MPLS VPN ForwardingForwarding

VPN_A

VPN_A

VPN_B10.3.0.0

10.1.0.0

11.5.0.0

P P

PP PE

CE

CE

CE

T7T8T9TaTb

TuTwTxTyTz

T8, TA

T2 DataT8Data

T2 DataTB

outin /

• All Subsequent P routers do switch the packet Solely on Interior Label

• Egress PE router, removes Interior Label • Egress PE uses Exterior Label to select which VPN/CE

to forward the packet to. • Exterior Label is removed and packet routed to CE router

VPN_A

VPN_B

VPN_B

10.1.0.0

10.2.0.0

11.6.0.0

CEPE1

PE2CE

CE

VPN_A10.2.0.0

CE T2 DataData

TAT2




MPLS VPN mechanismsMPLS VPN mechanismsVRF and Multiple Routing InstancesVRF and Multiple Routing Instances

• VRF: VPN Routing and Forwarding Instance

VRF Routing Protocol Context

VRF Routing Tables

VRF CEF Forwarding Tables



• VPN aware Routing Protocols• Select/Install routes in appropriate routing

table• Per-instance router variables• Not necessarily per-instance routing processes• eBGP, OSPF, RIPv2, Static



• VRF Routing table contains routes which should be available to a particular set of sites

• Analogous to standard IOS routing table, supports the same set of mechanisms

• Interfaces (sites) are assigned to VRFsOne VRF per interface (sub-interface, tunnel or virtual-template)Possible many interfaces per VRF



StaticBGP RIPRouting processes

Routing contexts

VRF Routing tables

VRF Forwarding tables

• Routing processes run within specific routing contexts

• Populate specific VPN routing table and FIBs (VRF)

• Interfaces are assigned to VRFs



Site-1 Site-2 Site-3 Site-4

Logical view

Routing viewVRF

for site-1

Site-1 routesSite-2 routes

VRFfor site-4


VRFfor site-2

Site-1 routesSite-2 routesSite-3 routes

VRFfor site-3


Site-1

Site-3

Site-4

Site-2

VPN-AVPN-C

VPN-B

PE PE

PP

Multihop MP-iBGP


AgendaAgenda• Concepts and goals• Terminology• Connection model• Forwarding• Mechanisms• Topologies• Scaling• BGP-4 Enhancements

Cap. Negotiation, MPLS, Route Refresh, ORF

• Configuration


MPLS VPN TopologiesMPLS VPN Topologies

VPN_A

VPN_A

VPN_B10.3.0.0

10.1.0.0

11.5.0.0

P P

PP PE

PE CE

CE

CE

VPN_A

VPN_B

VPN_B

10.1.0.0

10.2.0.0

11.6.0.0

CEPE

PECE

CE

VPN_A10.2.0.0

CE

• VPN-IPv4 address are propagated together with the associated label in BGP Multiprotocol extension

• Extended Community attribute (route-target) is associated to each VPN-IPv4 address, to populate the site VRF

iBGP sessions


MPLS VPN TopologiesMPLS VPN TopologiesVPN sites with optimal intra-VPN routingVPN sites with optimal intra-VPN routing

• Each site has full routing knowledge of all other sites (of same VPN)

• Each CE announces his own address space• MP-BGP VPN-IPv4 updates are propagated

between PEs• Routing is optimal in the backbone

Each route has the BGP Next-Hop closest to the destination

• No site is used as central point for connectivity


MPLS VPN TopologiesMPLS VPN TopologiesVPN sites with optimal intra-VPN routingVPN sites with optimal intra-VPN routing

Site-1

VRFfor site-1

N1,NH=CE1N2,NH=PE2N3,NH=PE3

PE1

PE3

PE2

N1

Site-3

N3

N2

VPN-IPv4 updates exchanged between PEs RD:N1, NH=PE1,Label=IntCE1, RT=BlueRD:N2, NH=PE2,Label=IntCE2, RT=BlueRD:N3, NH=PE3,Label=IntCE3, RT=Blue

IntCE1

IntCE3

N1NH=CE1

Routing Table on CE1N1, LocalN2, PE1N3, PE1

EBGP/RIP/Static

VRFfor site-3

N1,NH=PE1N2,NH=PE2N3,NH=CE3

Routing Table on CE3N1, PE3N2, PE3N3, Local

N3NH=CE3

EBGP/RIP/Static

Site-2

IntCE2

Routing Table on CE2N1,NH=PE2N2,LocalN3,NH=PE2

N2,NH=CE2

EBGP/RIP/Static

VRFfor site-2

N1,NH=PE1

N2,NH=CE2

N3,NH=PE3


MPLS VPN TopologiesMPLS VPN TopologiesVPN sites with Hub & Spoke routingVPN sites with Hub & Spoke routing

• One central site has full routing knowledge of all other sites (of same VPN)Hub-Site

• Other sites will send traffic to Hub-Site for any destinationSpoke-Sites

• Hub-Site is the central transit point between Spoke-SitesUse of central services at Hub-Site



PE2

PE1

PE3

Site-1

N1

N3

VPN-IPv4 updates advertised by PE3RD:N1, NH=PE3,Label=IntCE3-Spoke, RT=SpokeRD:N2, NH=PE3,Label=IntCE3-Spoke, RT=SpokeRD:N3, NH=PE3,Label=IntCE3-Spoke, RT=Spoke

Site-3

Site-2

N2

IntCE3-Spoke VRF(Export RT=Spoke)N1,NH=CE3-SpokeN2,NH=CE3-SpokeN3,NH=CE3-Spoke

CE1

CE3-Spoke

CE2

CE3-HubIntCE3-Hub VRF(Import RT=Hub)N1,NH=PE1N2,NH=PE2

VPN-IPv4 update advertised by PE1RD:N1, NH=PE1,Label=IntCE1, RT=Hub


IntCE2 VRF(Import RT=Spoke)(Export RT=Hub)N1,NH=PE3 (imported)N2,NH=CE2 (exported)N3,NH=PE3 (imported)

IntCE1 VRF(Import RT=Spoke)(Export RT=Hub)N1,NH=CE1 (exported)N2,NH=PE3 (imported)N3,NH=PE3 (imported

BGP/RIPv2

BGP/RIPv2

• Routes are imported/exported into VRFs based on RT value of the VPN-IPv4 updates

• PE3 uses 2 (sub)interfaces with two different VRFs



PE2

PE1

PE3

Site-1

N1

N3

Site-3

Site-2

N2


CE1

CE3-Spoke

CE2

CE3-Hub

IntCE3-Hub VRF(Import RT=Hub)N1,NH=PE1N2,NH=PE2



BGP/RIPv2

BGP/RIPv2

• Traffic from one spoke to another will travel across the hub site

• Hub site may host central servicesSecurity, NAT, centralised Internet access



• If PE and Hub-site use BGP the PE should not check the received AS_PATHThe update the Hub-site advertise contains the

VPN backbone AS numberBy configuration the AS_PATH check is disabledRouting loops are detected through the SOO

attribute• PE and CE routers may use RIPv2 and/or static

routing


MPLS VPN Internet RoutingMPLS VPN Internet Routing

• In a VPN, sites may need to have Internet connectivity

• Connectivity to the Internet means:Being able to reach Internet destinationsBeing able to be reachable from any Internet

source

• Security mechanism MUST be used as in ANY other kind of Internet connectivity


MPLS VPN Internet RoutingMPLS VPN Internet Routing

• The Internet routing table is treated separately

• In the VPN backbone the Internet routes are in the Global routing table of PE routers

• Labels are not assigned to external (BGP) routes

• P routers need not (and will not) run BGP


MPLS VPN MPLS VPN Internet routing Internet routing VRF specific default routeVRF specific default route

• A default route is installed into the site VRF and pointing to a Internet Gateway

• The default route is NOT part of any VPNA single label is used for packets forwarded

according to the default routeThe label is the IGP label corresponding to the

IP address of the Internet gateway Known in the IGP



• PE router originates CE routes for the Internet Customer (site) routes are known in the site VRF

Not in the global tableThe PE/CE interface is NOT known in the global

table. However:A static route for customer routes and pointing to

the PE/CE interface is installed in the global table

This static route is redistributed into BGP-4 global table and advertised to the Internet Gateway

• The Internet gateway knows customer routes and with the PE address as next-hop



• The Internet Gateway specified in the default route (into the VRF) need NOT to be directly connected

• Different Internet gateways can be used for different VRFs

• Using default route for Internet routing does NOT allow any other default route for intra-VPN routing As in any other routing scheme



PE

PE

Internet

Site-1

PE-IG

Site-2

Network 171.68.0.0/16

Serial0

192.168.1.1

192.168.1.2

ip vrf VPN-Ard 100:1route-target both 100:1!Interface Serial0ip address 192.168.10.1 255.255.255.0ip vrf forwarding VPN-A!Router bgp 100no bgp default ipv4-unicastnetwork 171.68.0.0 mask 255.255.0.0neighbor 192.168.1.1 remote 100neighbor 192.168.1.1 activateneighbor 192.168.1.1 next-hop-selfneighbor 192.168.1.1 update-source loopback0!address-family ipv4 vrf VPN-A neighbor 192.168.10.2 remote-as 65502 neighbor 192.168.10.2 activate exit-address-family! address-family vpnv4 neighbor 192.168.1.2 activateexit-address-family!ip route 171.68.0.0 255.255.0.0 Serial0ip route vrf VPN-A 0.0.0.0 0.0.0.0 192.168.1.1 global

BGP-4

MP-BGP



PE

PE

Internet

Site-1

PE-IG

Site-2

Network 171.68.0.0/16

Serial0

192.168.1.1

192.168.1.2

Site-2 VRF0.0.0.0/0 192.168.1.1 (global)Site-1 routesSite-2 routes

Global Table and LFIB192.168.1.1/32 Label=3192.168.1.2/32 Label=5...

IP packetD=cisco.com

Label = 3 IP packetD=cisco.com




• PE routers need not to hold the Internet table

• PE routers will use BGP-4 sessions to originate customer routes

• Packet forwarding is done with a single label identifying the Internet Gateway IP addressMore labels if Traffic Engineering is used


MPLS VPN Internet RoutingMPLS VPN Internet RoutingSeparated (sub)interfacesSeparated (sub)interfaces

• If CE wishes to receive and announce routes from/to the InternetA dedicated BGP session is used over a separate

(sub) interface The PE imports CE routes into the global routing

table and advertise them to the InternetThe interface is not part of any VPN and does

not use any VRFDefault route or Internet routes are exported to the

CEPE needs to have Internet routing table



• The PE uses separate (sub)interfaces with the CEOne (sub)interface for VPN routing

associated to a VRFCan be a tunnel interface

One (sub)interface for Internet routing

Associated to the global routing table



PE

PE

Internet

Site-1

PE-IG

Site-2

Network 171.68.0.0/16

Serial0.1

192.168.1.1

192.168.1.2

ip vrf VPN-Ard 100:1route-target both 100:1!Interface Serial0no ip address!Interface Serial0.1ip address 192.168.10.1 255.255.255.0ip vrf forwarding VPN-A!Interface Serial0.2ip address 171.68.10.1 255.255.255.0!Router bgp 100no bgp default ipv4-unicastneighbor 192.168.1.1 remote 100neighbor 192.168.1.1 activateneighbor 192.168.1.1 next-hop-selfneighbor 192.168.1.1 update-source loopback0neighbor 171.68.10.2 remote 502!address-family ipv4 vrf VPN-A neighbor 192.168.10.2 remote-as 502 neighbor 192.168.10.2 activate exit-address-family! address-family vpnv4 neighbor 192.168.1.2 activateexit-address-family

BGP-4

MP-BGP

Serial0.2

BGP-4



PE

PE

Internet

Site-1

PE-IG

Site-2

Network 171.68.0.0/16

Serial0.1

192.168.1.1

192.168.1.2

Serial0.2

Serial0.1 Serial0.2 CE routing tableSite-2 routes ----> Serial0.1Internet routes ---> Serial0.2


PE Global TableInternet routes ---> 192.168.1.1192.168.1.1, Label=3

Label = 3 IP packetD=cisco.com





ScalingScaling

• Existing BGP techniques can be used to scale the route distribution: route reflectors

• Each edge router needs only the information for the VPNs it supports

Directly connected VPNs

• RRs are used to distribute VPN routing information


ScalingScaling

• Very highly scalable:Initial VPN release: 1000 VPNs x 1000 sites/VPN = 1,000,000 sitesArchitecture supports 100,000+ VPNs, 10,000,000+ sitesBGP “segmentation” through RRs is essential !!!!

• Easy to add new sites configure the site on the PE connected to it the network automagically does the rest

• See also platform issues, later on


MPLS-VPNMPLS-VPNScaling BGPScaling BGP

VPN_A

VPN_A

VPN_B10.3.0.0

10.1.0.0

11.5.0.0

P P

PP PE

PE CE

CE

CE

RR RRRoute Reflectors

VPN_A

VPN_B

VPN_B

10.1.0.0

10.2.0.0

11.6.0.0

CEPE1

PE2CE

CE

VPN_A10.2.0.0

CE

• Route Reflectors may be partitionedEach RR store routes for a set of VPNs

• Thus, no BGP router needs to store ALL VPNs information

• PEs will peer to RRs according to the VPNs they directly connect


MPLS-VPN ScalingMPLS-VPN ScalingBGP updates filteringBGP updates filtering

iBGP full mesh between PEs results in flooding all VPNs routes to all PEs

Scaling problems when large amount of routes. In addition PEs need only routes for attached VRFs

Therefore each PE will discard any VPN-IPv4 route that hasn’t a route-target configured to be imported in any of the attached VRFs

This reduces significantly the amount of information each PE has to store

Volume of BGP table is equivalent of volume of attached VRFs (nothing more)


MPLS-VPN ScalingMPLS-VPN ScalingBGP updates filteringBGP updates filtering

Each VRF has an import and export policy configuredPolicies use route-target attribute (extended

community)PE receives MP-iBGP updates for VPN-IPv4 routesIf route-target is equal to any of the import values

configured in the PE, the update is acceptedOtherwise it is silently discarded

PE

MP-iBGP sessions

VRFs for VPNsyellowgreen

VPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Green, Label=XYZ

VPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Red, Label=XYZ

Import RT=yellow

Import RT=green


MPLS-VPN ScalingMPLS-VPN ScalingRoute RefreshRoute Refresh

Policy may change in the PE if VRF modifications are done• New VRFs, removal of VRFs

However, the PE may not have stored routing information which become useful after a change

PE request a re-transmission of updates to neighbors• Route-Refresh

PE

VPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Green, Label=XYZVPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Red, Label=XYZ

Import RT=yellow

Import RT=green

Import RT=red1. PE doesn’t have red routes (previously filtered out)

2. PE issue a Route-Refresh to all neighbors in order to ask for re-transmission

3. Neighbors re-send updates and “red” route-target is now accepted


MPLS-VPN ScalingMPLS-VPN ScalingOutbound Route Filters - ORFOutbound Route Filters - ORF

PE router will discard update with unused route-target

Optimisation requires these updates NOT to be sentOutbound Route Filter (ORF) allows a router to tell

its neighbors which filter to use prior to propagate BGP updates

PE

VPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Green, Label=XYZVPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Red, Label=XYZ

Import RT=yellow

Import RT=green

1. PE doesn’t need red routes

2. PE issue a ORF message to all neighbors in order not to receive red routes

3. Neighbors dynamically configure the outbound filter and send updates accordingly


AgendaAgenda• Concepts and goals• Terminology• Connection model• Forwarding• Mechanisms• Topologies• Scaling• BGP-4 Enhancements

Cap. Negotiation, MPLS, Route Refresh, ORF

• Configuration


MPLS VPN - ConfigurationMPLS VPN - Configuration

• VPN knowledge is on PE routers• PE router have to be configured for

VRF and Route DistinguisherVRF import/export policies (based on Route-target)Routing protocol used with CEsMP-BGP between PE routersBGP for Internet routers

With other PE routersWith CE routers


MPLS VPN - ConfigurationMPLS VPN - ConfigurationVRF and Route DistinguisherVRF and Route Distinguisher

• RD is configured on PE routers (for each VRF)• VRFs are associated to RDs in each PE• Common (good) practice is to use the same

RD for the same VPN in all PEs But not mandatory

• VRF configuration commandip vrf <vrf-symbolic-name>rd <route-distinguisher-value>route-target import <community>route-target export <community>


CLI - VRF configurationCLI - VRF configuration


VRFfor site-1(100:1)








PE1 PE2

PP

Multihop MP-iBGP

ip vrf site1 rd 100:1 route-target export 100:1 route-target import 100:1ip vrf site2 rd 100:2 route-target export 100:2 route-target import 100:2 route-target import 100:1 route-target export 100:1

ip vrf site3 rd 100:3 route-target export 100:2 route-target import 100:2 route-target import 100:3 route-target export 100:3ip vrf site-4 rd 100:4 route-target export 100:3 route-target import 100:3

Site-1

Site-3

Site-4

Site-2VPN-A

VPN-C

VPN-B


MPLS VPN - ConfigurationMPLS VPN - ConfigurationPE/CE routing protocolsPE/CE routing protocols

• PE/CE may use BGP, RIPv2 or Static routes• A routing context is used for each VRF• Routing contexts are defined within the routing

protocol instanceAddress-family router sub-commandRouter rip

version 2address-family ipv4 vrf <vrf-symbolic-name>… any common router sub-command…



• BGP uses same “address-family” commandRouter BGP <asn>

...address-family ipv4 vrf <vrf-symbolic-name>… any common router BGP sub-command…

• Static routes are configured per VRFip route vrf <vrf-symbolic-name> …


MPLS VPN - ConfigurationMPLS VPN - ConfigurationPE router commandsPE router commands

• All show commands are VRF basedShow ip route vrf <vrf-symbolic-name> ...Show ip protocol vrf <vrf-symbolic-name>Show ip cef <vrf-symbolic-name> ……

• PING and Telnet commands are VRF basedtelnet /vrf <vrf-symbolic-name> ping vrf <vrf-symbolic-name>




PE1

PE2

PP

Multihop MP-iBGP

Site-1

Site-3

Site-4

Site-2

VPN-AVPN-C

VPN-B

VRFfor site-1(100:1)Site-1 routesSite-2 routes







ip vrf site3 rd 100:3 route-target export 100:23 route-target import 100:23 route-target import 100:34 route-target export 100:34ip vrf site-4 rd 100:4 route-target export 100:34 route-target import 100:34!interface Serial4/6 ip vrf forwarding site3 ip address 192.168.73.7 255.255.255.0 encapsulation ppp!interface Serial4/7 ip vrf forwarding site4 ip address 192.168.74.7 255.255.255.0 encapsulation ppp

ip vrf site1 rd 100:1 route-target export 100:12 route-target import 100:12ip vrf site2 rd 100:2 route-target export 100:12 route-target import 100:12 route-target import 100:23 route-target export 100:23!interface Serial3/6 ip vrf forwarding site1 ip address 192.168.61.6 255.255.255.0 encapsulation ppp!interface Serial3/7 ip vrf forwarding site2 ip address 192.168.62.6 255.255.255.0 encapsulation ppp




PE1

PE2

PP

Multihop MP-iBGP

Site-1

Site-3

Site-4

Site-2

VPN-AVPN-C

VPN-B

VRFfor site-1(100:1)Site-1 routesSite-2 routes







router bgp 100no bgp default ipv4-unicast neighbor 6.6.6.6 remote-as 100 neighbor 6.6.6.6 update-source Loop0! address-family ipv4 vrf site4

neighbor 192.168.74.4 remote-as 65504

neighbor 192.168.74.4 activate exit-address-family ! address-family ipv4 vrf site3


neighbor 192.168.73.3 activate exit-address-family ! address-family vpnv4 neighbor 6.6.6.6 activate neighbor 6.6.6.6 next-hop-selfexit-address-family

router bgp 100no bgp default ipv4-unicast neighbor 7.7.7.7 remote-as 100 neighbor 7.7.7.7 update-source Loop0! address-family ipv4 vrf site2


neighbor 192.168.62.2 activate exit-address-family ! address-family ipv4 vrf site1


neighbor 192.168.61.1 activate exit-address-family ! address-family vpnv4 neighbor 7.7.7.7 activate neighbor 7.7.7.7 next-hop-selfexit-address-family


SummarySummary

• Supports large scale VPN services• Increases value add by the VPN Service

Provider• Decreases Service Provider’s cost of

providing VPN services• Mechanisms are general enough to enable

VPN Service Provider to support a wide range of VPN customers

• See RFC2547


ip vrf odd rd 100:1 route-target export “Green” route-target import “Green”

Route TargetRoute Target


PE-2

P P

P PBGP,RIPv2 update for Net1,Next-Hop=CE-1


CE-1

Site-2


Site-1

CE-2

Receiving PE is inserting the route into the VRF identified by the RT attribute (based on PE configuration)

In this example RT = Green.


Inbound FilteringInbound Filtering

• Proprietary feature

VPN-IPv4 update is silently rejected when it reaches PE since there isn’t any VRF configured with import RT = Red.

Automatic (always on) rejection of all prefixes where at least one route target extended community attribute does not match any of route targets configured at the PE.

Any VRF configuration change triggers “Route Refresh”

PE creates a union of all configured RTs and automatically compares all incoming RTs for non null intersection

PE

MP-iBGP sessions

VRFs for VPNsyellowgreen

VPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Green, Label=XYZ


Import RT=yellow

Import RT=green


Route RefreshRoute Refresh

• Based on: draft-chen-bgp-route-refresh-01.txt

• When the inbound policy has been modified, the BGP speaker sends a Route-Refresh message to its neighbors

With AFI, Sub-AFI attributes• Neighbors will re-transmit all routes for that particular

AFI and Sub-AFI• Routers not refresh capable will reset BGP session• Used for vpnv4 sessions, for ipv4 sessions manual soft

refresh trigger: clear ip bgp neighbour x.x.x.x soft-in


• Policy may change in the PE if VRF modifications are done• New VRFs, removal of VRFs, RT addition or deletion• However, the PE may not have stored routing information

which become useful after a change• PE request a re-transmission of updates to neighbors via

Route-Refresh

PEVPN-IPv4 update:RD:Net1, Next-hop=PE-XSOO=Site1, RT=Green, Label=XYZ


Import RT=yellow

Import RT=green

Import RT=red1. PE doesn’t have red routes (previously filtered out)

2. PE issue a Route-Refresh to all neighbors in order to ask for re-transmission

3. Neighbors re-send updates and “red” route-target is now accepted

Route Refresh and filteringRoute Refresh and filtering


• One central site has full routing knowledge of all other sites (of same VPN)Hub-Site

• Other sites will send traffic to Hub-Site for any destinationSpoke-Sites

• Hub-Site is the central transit point between Spoke-SitesUse of central services at Hub-Site

Allow ASAllow AS


PE2

PE1

PE3

Site-1

N1

N3

VPN-IPv4 updates advertised by PE3RD:N1, NH=PE3,Label=IntCE3-Spoke, RT=SpokeRD:N2, NH=PE3,Label=IntCE3-Spoke, RT=SpokeRD:N3, NH=PE3,Label=IntCE3-Spoke, RT=Spoke

Site-3

Site-2

N2


CE1

CE3-Spoke

CE2

CE3-HubIntCE3-Hub VRF(Import RT=Hub)N1,NH=PE1N2,NH=PE2





BGP/RIPv2

BGP/RIPv2

• Routes are imported/exported into VRFs based on RT value of the VPN-IPv4 updates

• PE3 uses 2 (sub)interfaces with two different VRFs

Allow ASAllow AS


PE2

PE1

PE3

Site-1

N1

N3

Site-3

Site-2

N2 IntCE3-Spoke VRF(Export RT=Spoke)N1,NH=CE3-SpokeN2,NH=CE3-SpokeN3,NH=CE3-Spoke

CE1

CE3-Spoke

CE2

CE3-Hub

IntCE3-Hub VRF(Import RT=Hub)N1,NH=PE1N2,NH=PE2



BGP/RIPv2

BGP/RIPv2

• Traffic from one spoke to another will travel across the hub site

• Hub site may host central servicesSecurity, NAT, centralised Internet access

Allow ASAllow AS


• If PE and Hub-site use BGP the PE should not check the received AS_PATHThe update the Hub-site advertise contains the

VPN backbone AS numberBy configuration the AS_PATH check is disabled

Allow ASRouting loops are suppressed by the limit of

occurrence of provider ASN in the AS_PATHTherefore, PE will REJECT the update if its ASN

appears more than 3 times in the AS_PATH 3 is the default and can be overwritten with <opt>

Allow ASAllow AS


PE2

PE1

PE3

Site-1

192.168.0.5/32

N3

Site-3

Site-2

N2

CE1

CE3-Spoke

CE2

CE3-Hub

Allow ASAllow AS

ASN: 100

! address-family ipv4 vrf Hub neighbor 192.168.73.3 remote-as 250 neighbor 192.168.73.3 activate neighbor 192.168.74.4 remote-as 250 neighbor 192.168.74.4 activate neighbor 192.168.74.4 allowas-in <opt> no auto-summary no synchronization exit-address-family !

eBGP4 update: 192.168.0.5/32AS_PATH: 100 251

ASN: 251

ASN: 252

ASN: 250

eBGP4 update: 192.168.0.5/32AS_PATH: 250 100 251


PE2

PE1

PE3

Site-1

192.168.0.5/32

N3

Site-3

Site-2

N2

CE1

CE3-Spoke

CE2

CE3-Hub

Allow AS with ASN overrideAllow AS with ASN override

ASN: 100

! address-family ipv4 vrf Hub neighbor 192.168.73.3 remote-as 250 neighbor 192.168.73.3 activate neighbor 192.168.74.4 remote-as 250 neighbor 192.168.74.4 activate neighbor 192.168.74.4 allowas-in <opt> neighbor 192.168.74.4 as-override no auto-summary no synchronization exit-address-family

ASN: 250

ASN: 250

ASN: 250

eBGP4 update: 192.168.0.5/32AS_PATH: 100 100


eBGP4 update: 192.168.0.5/32AS_PATH: 250

VPN-IPv4RD:192.168.0.5/32, AS_PATH: 250

VPN-IPv4RD:192.168.0.5/32, AS_PATH: 250 100 100

eBGP4 update: 192.168.0.5/32AS_PATH: 100 100 100 100

Now the AS_PATH contains four occurrences of the provider ASN. This update will not be accepted anymore if the CE re-advertise it back to any PE


When BGP is used between PE and CE routers, the customer VPN may want to re-use ASN in different sites

Private ASN procedures already exist in order to strip the private ASN from the AS_PATH

However, these procedures have following constraints:Private ASN is stripped if only private ASN are present

in the AS_PATHPrivate ASN is stripped if NOT equal to the

neighbouring ASNPrivate ASN procedures do NOT allow the re-use of same

ASN in a MPLS-VPN environment

ASN OverrideASN Override


New procedures have been implemented in order to re-use the same ASN on all VPN sites

The procedures allows the use of private as well as public ASN

Same ASN may be used for all sites, whatever is their VPN



• With ASN override configured the PE does following

If the last ASN in the AS_PATH is equal to the neighbouring one, it is replaced by the provider ASN

If last ASN has multiple occurrences (due to AS_PATH prepend) all the occurrences are replaced with provider-ASN value

After this operation, normal eBGP operation occur:

Provider ASN is added to the AS_PATH



• ASN override feature is used in conjunction with SOO in order to prevent routing loopsIn case of multihomed sites

• SOO is not needed for stub sitesSites connected to a single PE

• Multi-homed sites need to use SOO




PE-1

CE-1

192.168.0.5/32

PE-2

CE-2

192.168.0.3/32

ASN: 250

ASN: 100ip vrf odd rd 100:1 route-target export 100:3 route-target import 100:3!interface Serial1 ip vrf forwarding odd ip address 192.168.73.7 255.255.255.0!router bgp 100 no synchronization no bgp default ipv4-unicast neighbor 192.168.0.6 remote-as 100 neighbor 192.168.0.6 update-source Loop0 neighbor 192.168.0.6 activate neighbor 192.168.0.6 next-hop-self no auto-summary ! address-family ipv4 vrf odd neighbor 192.168.73.3 remote-as 250 neighbor 192.168.73.3 activate neighbor 192.168.73.3 as-override no auto-summary no synchronization exit-address-family ! address-family vpnv4 neighbor 192.168.0.6 activate neighbor 192.168.0.6 send-community extended no auto-summary exit-address-family!ASN: 250



PE-1

CE-1

192.168.0.5/32

PE-2

CE-2

VPN-IPv4 update:RD:192.168.0.5/32AS_PATH: 250

eBGP4 update: 192.168.0.5/32AS_PATH:100 100

192.168.0.3/32

ASN: 250 ASN: 250

eBGP4 update: 192.168.0.5/32AS_PATH: 250

ASN: 100

PE-2 performs following actions:1- Replace last ASN with its own ASN2- Update AS_PATH with its own ASN3- Forward the update to CE-2

7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight PathRoute Distinguisher: 100:1 (default for vrf odd)*>i192.168.0.3/32 192.168.0.7 0 0 250 i*> 192.168.0.5/32 192.168.65.5 0 0 250 i

3640-5#sh ip b Network Next Hop Metric LocPrf Weight Path*> 192.168.0.5/32 192.168.73.7 0 100 100 i*> 192.168.0.3/32 0.0.0.0 0 i


ASN OverrideASN Overridewith AS_PATH prependwith AS_PATH prepend

PE-1

CE-1

192.168.0.5/32

PE-2

CE-2

VPN-IPv4 update:RD:192.168.0.5/32AS_PATH: 250 250 250

eBGP4 update: 192.168.0.5/32AS_PATH:100 100 100 100

192.168.0.3/32

ASN: 250 ASN: 250


ASN: 100

PE-2 performs following actions:1- Replace all occurrences of last ASN with its own ASN2- Update AS_PATH with its own ASN3- Forward the update to CE-2

7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight PathRoute Distinguisher: 100:1 (default for vrf odd)*>i192.168.0.3/32 192.168.0.7 0 0 250 i*> 192.168.0.5/32 192.168.65.5 0 0 250 250 250 i

3640-5#sh ip b Network Next Hop Metric LocPrf Weight Path*> 192.168.0.5/32 192.168.73.7 0 100 100 100 100 i*> 192.168.0.3/32 0.0.0.0 0 i


Site of OriginSite of Origin

• Used to identify the site • Extended Community type• Used to prevent loops when AS_PATH cannot be

used

When BGP is used between PE and multihomed sites

A BGP route is NOT advertised back to the same site

Even through different PE/CE connections



• SOO for eBGP learned routes

SOO is configured through a route-map command

• SOO can be applied to routes learned through a particular VRF interface (without the use of BGP between PE and CE)

SOO is then configured on the interface

SOO is propagated into BGP during redistribution


PE

CE

Site-1

Site of OriginSite of Originip vrf odd rd 100:1 route-target export 100:3 route-target import 100:3!interface Serial1 ip vrf forwarding odd ip address 192.168.65.6 255.255.255.0!router bgp 100 no synchronization no bgp default ipv4-unicast neighbor 192.168.0.7 remote-as 100 neighbor 192.168.0.7 update-source Loop0 neighbor 192.168.0.7 activate neighbor 192.168.0.7 next-hop-self no auto-summary ! address-family ipv4 vrf odd neighbor 192.168.65.5 remote-as 250 neighbor 192.168.65.5 activate neighbor 192.168.65.5 route-map setsoo in no auto-summary no synchronization exit-address-family ! address-family vpnv4 neighbor 192.168.0.7 activate neighbor 192.168.0.7 send-community extended no auto-summary exit-address-family!route-map setsoo permit 10 set extcommunity soo 100:65

7200-1#sh ip route vrf oddC 192.168.65.0/24 is directly connected, Serial2B 192.168.0.5 [20/0] via 192.168.65.5, 00:08:44, Serial27200-1#7200-1#sh ip bgp vpn all Network Next Hop Metric LocPrf Weight PathRoute Distinguisher: 100:1 (default for vrf odd)*> 192.168.0.5/32 192.168.65.5 0 0 250 i7200-1#sh ip bgp vpn all 192.168.0.5BGP routing table entry for 100:1:192.168.0.5/32, version 17Paths: (1 available, best #1) Advertised to non peer-group peers: 192.168.0.7 250 192.168.65.5 from 192.168.65.5 (192.168.0.5) Origin IGP, metric 0, localpref 100, valid, external, best Extended Community: SoO:100:65 RT:100:37200-1#

192.168.0.5/32



PE-1

CE-1 Site-1SOO=100:65

192.168.0.5/32

PE-2

CE-2

eBGP4 update: 192.168.0.5/32

intCE1

VPN-IPv4 update:RD:192.168.0.5/32, Next-hop=PE-1SOO=100:65, RT=100:3, Label=(intCE1)

eBGP4 update: 192.168.0.5/32

PE-2 will not propagate the route since the update SOO is equal to the one configured for the site


Selective ExportSelective Export

• PE may have to export VRF routes with different route-targetsExample: export management routes with

particular RT

• Export command accept route-mapRoute-map configured into VRFRoute-map match or deny statements with

extended community list


Selective ExportSelective Export

PE

CE

Site-1

ip vrf odd rd 100:1 export map RTMAP route-target import 100:3!…… !access-list 10 permit 192.168.0.5 0.0.0.0access-list 11 permit any!route-map RTMAP permit 10 match ip address 10 set extcommunity rt 100:3!route-map RTMAP permit 20 match ip address 11 set extcommunity rt 100:4!

192.168.0.5/32

192.168.50/24

VPN-IPv4 update:RD:192.168.0.5/32RT=100:3



Selective ImportSelective Import

• PE may have to import routes based on other criteria than only Route-Target

• Import command accept route-mapRoute-map configured into VRF

Route-map match or deny statements


Selective ImportSelective Import

PE

CE

Site-1

ip vrf odd rd 100:1 import map RTMAP route-target export 100:3!…… !access-list 10 permit 192.168.30.0 0.0.0.0!route-map RTMAP permit 10 match ip address 10!

192.168.0.5/32

192.168.50/24



B 192.168.30.0 [200/0] via 192.168.0.7, 02:17:48


Extended route-mapsExtended route-maps

• Added support for extended communities in route-maps

Route-Map match/set statements:

route-map <Name> permit 10

[no] match extcommunity <1-99>

[no] set extcommunity [rt|soo] <ASN:nn | IP-address:nn>

Defining Extended Community access list:

[no] ip extcommunity-list 1 [permit|deny] [rt|soo] <ASN:nn | IP-address:nn>


Internet routing - VRF specific default routeInternet routing - VRF specific default route

• The PE installs a default route into the site VRF• PE router originates CE routes for the Internet

• The default route points to the Internet router of the VPN backbone

Possibility to use different Internet gateways per VRF

• No VPN default route allowed


MPLS VPN TopologiesMPLS VPN TopologiesInternet routing - VRF specific default routeInternet routing - VRF specific default route

Site-2

PE

PE

Internet


Destination cisco.com is

covered by the default route to

PE-IG

Site-1

PE-IG

Site-2 VRFSite-1 routesSite-2 routes0.0.0.0/0 PE-IG

Global routing table with

Internet routes

Site-1 VRFSite-1 routesSite-2 routes


Internet routes


Internet routes


Label=PE-IG

Ip route vrf <Name> 0.0.0.0 0.0.0.0 PE-IG global


Direct Import (RT intersection)Direct Import (RT intersection)

• EBGP received prefixes are now added to the vrf table in the router thread itself.

• Requirement to have a non null intersection between RTs for every VRF has been removed.


CE to CE convergenceCE to CE convergence

• New BGP mechanism to be used in order to improve convergence time between sites

• BGP update origination, validation and advertisement• Other mechanisms in order to improve import and export

processes• BGP update next-hop validation (done at scanner on PE)

- scan-time adjustment.• BGP validates updates by verifying next-hop

reachability (first rule on PATH selection)• By default the next-hop validation is done once every

60 seconds• New command that allows to configure the timer bgp

scan-time <5-60>


CE to CE convergenceCE to CE convergence• BGP update advertisement interval (default):

• EBGP updates are propagated once every 30 seconds• iBGP updates are propagated once every 5 seconds

Default can be changed on a per neighbor basis

neighbor <ip_address> advertisement-interval <0-600>• BGP import/export process (IBGP learned into vrf on remote

PE)• By default import/export actions are performed once every

60 seconds

Command to modify the timer: bgp scan-time import <5-60>

Timer is configurable ONLY under address-family vpnv4


VRF Size Limit/WarningVRF Size Limit/Warning

• New VRF level configuration command:

• (config-vrf)# maximum routes <number> { <warn percent> | warn-only }

• When <warn-percent> of <number> is reached then a SYSLOG error message is issued

• If the number of routes in the VRF routing table reaches <number> then no more routes will be added, a SYSLOG error message will be issued when an attempt is made to add a route which is rejected, throttled to one message per-VRF in 10 minutes.


AgendaAgenda



What Code Is MPLS VPN In?What Code Is MPLS VPN In?

• Introduced in 12.0(5)T and 12.0(9)ST• Also in 12.1M and derivatves• 12.0(15)SL, 12.0(17)ST for ESR


AgendaAgenda



Things That Make Up MPLS-VPNThings That Make Up MPLS-VPN

• MPLS Forwrding – ENG-59293• TAG VPN Functional Spec – ENG-

17513• MPLS VPN on GSR E2 cards – ENG-

59451…as a reference to a HW implementation


Software-based platformsSoftware-based platforms

• If you are developing a new software-based platform (like 2600, 3600, 4500, etc), should be pretty simple

• Concentrate on testing different packet paths and interface types


Hardware-based platformsHardware-based platforms

• Label Imposition:could be 0, 1, or 2 labels

• Label Exposition: need to deal with aggregate label, very likely 2 lookups on the same packet


Label Imposition (Push)Label Imposition (Push)

• CE3<->CE4: PE3 imposes 0 labels, does regular FIB lookup in VRF table

• CE3->CE1: PE3 imposes 1 label (VPN label), IGP label is effectively PHP’d

• CE3->CE2: PE3 imposes 2 labels: (IGP label to PE2, VPN label)

• Explicit-null mitigates PHP

PE1CE1

PE3

CE2

CE3

PE2 P1

CE4


Label Exposition (Pop)Label Exposition (Pop)

• VPN advertises “aggregate label” for scalability

• Aggregate label leads to 2 lookups on egress PE (1 LIB, 1 FIB)

• Label lookup turns aggregate label into IP address within a VRF, IP lookup necessary to figure out correct L2 encap


Aggregate LabelAggregate Label

1. PE3 does MPLS lookup on VPN label, finds outgoing VRF

2. PE3 does IP lookup in VRF routing table, finds L2 encap, sends packet

PE1CE1

PE3 CE3

CE4

VPN label = 42IP packet Dst = 3.3.3.3

Label VRF42 Red

IP Address Port3.3.3.0/24 POS1/0


CPU Considerations

QOS Considerations

Platform Specific Considerations

PE Memory Considerations

Sizing Provider Edge (PE) Sizing Provider Edge (PE) RoutersRouters


PE to CE Connectivity Type

Sizing Provider Edge (PE) Sizing Provider Edge (PE) CPU ConsiderationsCPU Considerations

STATICOSPF

BGP-4# of provisioned

VRFs

# of VPN clients/routes

Amount of provisioned QOS

Several factors determine CPU Usage

# of backbone BGP peers

Packet forwarding CEF vs. process


Platform Processor Type Internal Clock Speed

NPE 225 RM5271 262 MHz

NPE 300 R7000 262 MHz

NPE 400 R7000 350 MHz

RSP 4 R5000 200 MHz

RSP 8 R7000 250 MHz

GRP R5000 200 MHz

Platform Processor TypesPlatform Processor Types


Baseline (No Traffic) CPU Comparison Baseline (No Traffic) CPU Comparison Small VPN: 500 VRFs (11 routes per-VRF)Small VPN: 500 VRFs (11 routes per-VRF)

NPE225 – 262 MHz

NPE300 – 262 MHz

NPE400 – 350 MHz

RSP8 – 250 MHz


# of neighbors and type of connectivity

Sizing Provider Edge (PE) Sizing Provider Edge (PE) Memory ConsiderationsMemory Considerations

STATICOSPF

BGP-4# of provisioned

VRFs# of local VPN

routes

Unique or non-unique RD allocation ?

Several factors determine Memory Usage

# of backbone BGP peers (paths)

# of remote VPN routes

Spread of IP addressing structure


Sizing Provider Edge (PE) Sizing Provider Edge (PE) Memory ConsiderationsMemory Considerations

BGP Memory

Several Areas of Memory Usage

Routing Table

MPLS CEF

IDB


Sizing Provider Edge (PE) Sizing Provider Edge (PE) BGP MemoryBGP Memory

BGP Memoryndc-brighton# show ip bgp v a s

BGP router identifier 10.3.1.9, local AS number 2

BGP table version is 21, main routing table version 21

1 network entries and 2 paths using 189 bytes of memory

2 BGP path attribute entries using 108 bytes of memory

2 BGP AS-PATH entries using 48 bytes of memory

1 BGP extended community entries using 24 bytes of memory

0 BGP route-map cache entries using 0 bytes of memory

0 BGP filter-list cache entries using 0 bytes of memory

BGP activity 8/58 prefixes, 8/6 paths, scan interval 15 secs

Mp = (N*128) + (P*60) + (Pa * 24) + (Ec * 24) Mp = Total memory used by PE in Bytes N = Number of BGP network entries P = Number of path entries Pa = Number of AS_PATH entries Ec = Number of Extended Community entries


Sizing Provider Edge (PE) Sizing Provider Edge (PE) Routing Table MemoryRouting Table Memory

Routing Table Memory

ndc-brighton# show memory summary | include IP: Control Block

0x60567BB0 33184 101 3351584 IP: Control Block

Each VRF consumes :

• 1 IP control block -> 33,184 bytes • 1 Network Descriptor Block (NDB) per route (64 bytes)• 1 Routing Descriptor Block (RDB) per path (144 bytes)

ndc-brighton# show ip route vrf testing summary

IP routing table name is testing(1)

Source Networks Subnets Overhead Memory (bytes)

connected 0 1 64 144

External: 0 Internal: 0 Local: 0

internal 1 1164Total 1 1 64 1308


Sizing Provider Edge (PE) Sizing Provider Edge (PE) MPLS Memory MPLS Memory

ndc-brighton# show memory allocating-process total | include TFIB tag_

0x60DC5D54 8101672 125 TFIB tag_rewrite chunk

0x60DC5DB4 4141564 64 TFIB tag_info chunk

0x60DC5DA4 65540 1 TFIB tag_info chunk

0x60DC5D44 65540 1 TFIB tag_rewrite chunk

ndc-brighton# show memory allocating-process total | include TIB

0x60FC7E10 24228 134 TIB entry

MPLS forwarding memory (TFIB) consumes one 'taginfo‘ (64 bytes) per route, plus one forwarding entry (104 bytes) for each path

MPLS Memory


Sizing Provider Edge (PE) Sizing Provider Edge (PE) IDB Memory IDB Memory

IDB Memory

ndc-brighton# show memory summary | include IDB

0x602F88E8 4692 9 42228 *Hardware IDB*

0x602F8904 2576 9 23184 *Software IDB*

Hardware IDB

Software IDB

Interface Description Block

Hardware IDB: 4692 bytes (One per physical interface) Software IDB: 2576 bytes (One per interface and per sub-interface)

Note: The amount of memory required will differ from platform to platform


PE VRF Memory Sizing PE VRF Memory Sizing NO VPN routesNO VPN routes

Used Memory 8,187,968 MB




VPN Memory ComparisonVPN Memory Comparison


PE Memory Sizing Design Rules PE Memory Sizing Design Rules

• ~ 60-70K per VRF33K for base VRF control block, other memory such as CEF, TFIB overhead, IDBs and so on

• ~800-900 bytes per route (includes CEF, TFIB and RIB Memory in BGP)

• Remember IOS uses memory!• Remember Internet Routes!• Remember to leave transient memory

Recommended to leave ~ 20MB free


PE Memory Sizing Design PE Memory Sizing Design ObservationsObservations

• 128 MB platforms are very limited(NPE 225, 3640 *NOT* suitable for full Internet table and VPNs!!!)

• 256 MB Minimum recommended on PE devices• Limit the number of RDs per VRF in the same

VPN unless you require iBGP load balancing with RRs


VRF and Route Limits SummaryVRF and Route Limits Summary

• VRF Limits Constrained mainly by CPUBetween 500 & 1000 VRFs for static routing (depending on platform – 10 routes per VRF)Between 250 & 500 VRFs if using EBGP or RIPv2 (depending on platform - 500 routes per VRF)

• VPN & Global Route Limits Constrained mainly by available memoryWith 256 MB, 200,000 routes total (IPv4 and VPNv4)

If Internet table is present, this reduces the memory available for VPNs (Current calculations are near 65 Meg for 100K Internet routes – with tightly packed attributes)


AgendaAgenda



Core TopologyCore Topology

SRP12N6

OC192N5

OC48N7

OC3PO SN2

OC3POSN3

OC48N4

OC48N8

OC12N10 OC12

N11OC12N12

OC12N13

ATM OC12

ATM OC12

POS 5/0 POS 0/0

POS 1/0 POS 1/0

POS 2/0

G SR 1

G SR 4 G S R 5

G S R 8

G S R 2

POS0/0

POS0/0

POS0/1

G SR 3

G SR 6

G S R 7

POS0/0

PO S3/0

POS2/0 POS1/0

POS 2/1

POS 1/1

POS1/0

POS1/1

POS1/0

to vpnto vpn


AS3402

G SR 1VXR 15VXR 14

VXR 13 VXR 16

N23

N20

N21

N 22

N 25

N24 BG P

RIP

AS 65001

G S R8VXR 12

VXR 11

VX R10

VXR 9

N26

N27

N29

N30

N31

N28

O SP F

BG P A S 65501

VPN toplogyVPN toplogy

NOTES:-VXR15,16,12,11 are PEs-VXR14,13,10,9 are CEs-all CEs have 192.168.1.x as their RID-GSR6 is VPNv4 RR