Virtualized PE for BGP/MPLS L3-VPN using Open …...Virtualized PE for BGP/MPLS L3-VPN using Open-Source Software NANOG 74 (October 2018) Bilal Anwer, Robert Bays, Vijay Gopalakrishnan,
Post on 22-Feb-2020
36 Views
Preview:
Transcript
VirtualizedPEforBGP/MPLSL3-VPNusingOpen-SourceSoftwareNANOG74(October2018)
BilalAnwer,RobertBays,VijayGopalakrishnan,BoHan,DewiMorgan,PatrickRuddy,AmanShaikh,SusheelaVaidya,ChengweiWangandGeorgeWilkie
2
IntroductionObjective• DemonstratefeasibilityofcreatingaBGP/MPLSL3-VPNvPEusingopen-sourcesoftware
Motivation• Use-caseforAT&T’sDANOS(DisaggregatedNetworkOS)• WhyL3-VPNvPEfromopen-sourcesoftware?– L3-VPN• Allowscreationofmultiplelayer-3virtualnetworksontopofasharedservice-providernetwork
• Widelyusedservicebyenterprises
– vPE• EnablerVNFwhichactsastheingressandegressforL3-VPNtrafficintheservice-providernetwork
– Opensourcesoftware• Allowsincreasedagilityinprovidingnewfeatureswhilereducingthecost
Challenges• Requiredfunctionalandintegration-relatedextensionstoopen-sourcecomponents
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
3
SoftwareComponentsofOpenSourcevPE Control-plane• FRR(5.1-dev,snapshote8f9540)forOSPF,LDPandZebra• GoBGP(version1.31.1=version1.31+ourenhancements)
Data-plane• AT&T-Vyatta’s(DPDK-based)data-plane
• Wealsoverifiedfeasibilitywith…– Linuxdata-plane(kernel4.14.4-mpls)
– VPPdata-plane(release1801+routerplug-inwithourenhancementswhichhavebeenup-streamed)
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
DANOSUse-Case
4
SoftwareArchitectureofDANOSOpenSourcevPE
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
Control-plane
FRROSPF
FRRLDPGoBGP
AT&T-VyattaDPDKbasedData-plane
FRRZebrazserv.api
vPE
LinuxForwardingTable
RIB
NIC
KernelFIB
NetLinkLinuxNetworkingStack
TAPInterface
LinuxUserSpace
LinuxKernelSpace
FIB
AT&T-VyattaRouteBrokerReliable&highscaleroutedownload
Existingclientsofnetlink
vCEred_east
Videoserver
172.16.0.2
RedEastOffice
5
VerifyingFeasibility
• Demonstratedfeasibilitybyconcurrentlyrunningtwovideostreams– KeepvideotrafficseparatedespitesameIPaddressesbeingusedbytwocustomers
– Eachclient/serverhasastaticroutepointingtoitsupstreamCE
– EachCEadvertisesappropriateprefixtothePENANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
Corerouter+
VPNv4RR
vPEwest vPEeast
OSPFLDP
VPNv4BGP
OSPFLDP
VPNv4BGP
VRFred VRFred
VRFblue VRFblue
ServiceProvider
SameIPaddress SameIPaddress
vCEblue_west
Videoclient
172.16.0.1
BlueWestOffice
vCEred_west
Videoclient
172.16.0.1
RedWestOffice
Videoserver
172.16.0.2
BlueEastOfficevCE
blue_east
Videostream
Videostream
6
PacketCaptureatCoreRouterduringVideoStreaming
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
redserver→redclient
redclient→redserver
blueclient→blueserver
blueserver→blueclient
7
ConfigurationDetailsEstablishLSP(LabelSwitchedPaths)betweenPEs• EnableIPandMPLSforwarding
• ConfigureOSPFandLDPonserviceproviderrouters
EnableL3-VPNservice• ConfigureVRFs• ConfigureeBGPsessionsbetweenPEsandCEs• ConfigureiBGPsessionsbetweenPEsandroutereflector
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
Note:WeusedAT&T/DANOSYangModulesforconfiguringvPEswherepossible,butshowequivalentLinux,FRRandGoBGPcommandsinsubsequentslides
8
TaleofTwoLoopbacksConfiguredtwoloopbackaddressesonvPEsandcorerouter• Loopback1– UsedforIPtrafficincludingcontrol-planetraffic• Loopback2– UsedforMPLStraffic• HencealltrafficfromVPNcustomers
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
9
vPEEast:ConfiguringMPLSForwarding
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
# Enable IPv4 forwarding $ sudo sysctl –w net.ipv4.ip_forward=1 # Load MPLS modules $ sudo modprobe mpls_router $ sudo modprobe mpls_iptunnel # Enable MPLS forwarding on the interface facing the core router $ sudo sysctl -w net.mpls.conf.ens4.input=1 # Allocate entries in MPLS label table $ sudo sysctl -w net.mpls.platform_labels=1048575
10
vPEEast:FRROSPFandLDPConfigurations
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
OSPFConfiguration LDPConfiguration
11
vPEEast:VRFConfiguration
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
# Create VRF blue and bring it up $ sudo ip link add blue type vrf table 1 $ sudo ip route add table 1 unreachable default metric 4278198272 $ sudo ip link set dev blue up # Add interface to vCE blue-east to VRF blue $ sudo ip link set dev ens6 master blue # Allow BGP to listen on port 179 over the VRF-bound interface $ sudo sysctl –w net.ipv4.tcp_l3mdev_accept=1 $ sudo sysctl –w net.ipv4.udp_l3mdev_accept=1
12
vCEBlueEast:FRRBGPConfiguration
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
AllowsconfigurationofeBGPsessiontovPEEastloopback
13
vPEEast:GoBGPConfiguration
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
Useofpolicyforsettingnext-hop-selfrequirespolicy
CommunicatetoLinuxkernelandVyattadata-planeviaFRRZebra
iBGPsessionwithRoute-Reflector
14
vPEEast:Defining“SetNext-HopSelf”policy
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
Matchontheroutereflectorasneighbor
Setloopback2ofthisPEasthenext-hop
15
vPEEast:AddingVRFandeBGPNeighborviaGoBGPCLI
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
CommandsforaddingblueVRFandeBGPsessiontovCEeast-blue
$ ip link show blue
11: blue: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP
mode DEFAULT group default qlen 1000
link/ether ae:07:ef:a3:f3:f7 brd ff:ff:ff:ff:ff:ff
$ gobgp add vrf blue id 11 rd 100:1 rt both 100:1
$ gobgp nei add 10.31.4.10 as 65101 vrf blue
UseofifIndexvalueassignedbyLinuxasVRFid
16
GoBGPConfigurationforRouterReflector
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
`CorerouterClients vCEs
vPEeast
vPEwest ServersvCEs
NoneedtocommunicatewithFRRZebrasinceVPNv4routesarenotinstalledinforwardingtable
17
Implementation:L3-VPNSupportinGoBGPKeybuildingblocks• InternetroutingwithBGP– Messagehandling,routecomputation,andpolicies
• PartitionofroutingtableintoglobalandVRF– AssignBGPsessionstoappropriatepartition• VPNv(4|6)BGPaddressfamily– IPprefix,RouteDistinguisher(RD)andMPLSlabel
• Routetargets(RTs)– ToassociaterouteswithVRF(s)
Interactionwith“outsideworld”• AllowconfigurationofVRF(s)– AssociateaneBGPsessionwithCEtoaVRF• HandlearoutereceivedfromaCE
• HandlearoutereceivedfromRR(orremotePEs)
• CommunicatewithZebraNANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
Existingsupportwasadequate
Neededsomeenhancements
18
GoBGP:HandlingaRouteReceivedfromanAdjacentCE• InstalltherouteinVRFforwardingtableviaZebra– AllowsvPEtosendtrafficcomingfromotherPEstotheCE
• AttachVRFlabeltotheroutebeforesendingtoRR
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
GoBGPpullrequest1587
vCEVRF VRF Global
Control-plane
FRRZebra
ZebraAPI
vPE
GoBGP
Data-plane
VRF VRF GlobalRIB
VRF VRF GlobalFIB
VRF VRF GlobalBGPtable
RR
IPv4BGProuteforprefixpp
Ourenhancement;pullrequest1572acceptedintoGoBGPupstreamrepository
VPNv4BGProuteforprefixpwithblueRDandbluelabel
p
p
RouteBroker
p
p
p
19
GoBGP:HandlingaRouteReceivedfromanRRoraPE• PreventtheroutefrombeinginstalledinglobalFIB
• ImporttherouteintoappropriateVRFbasedonroutetarget
• InstalltheroutewithlabelinLinuxVRFforwardingtableviaZebra• SendtheroutetoadjacentCE(s)belongingtotheVRF
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
vCEVRF VRF Global
Control-plane
FRRZebra
ZebraAPI
vPE
GoBGP
Data-plane
VRF VRF GlobalRIB
VRF VRF GlobalFIB
VRF VRF GlobalBGPtable
RR
IPv4BGProuteforprefixpp
VPNv4BGProuteforprefixpwithblueRDandbluelabel
p
p
p IPv4BGProuteforprefixpwithbluelabel
p
OurenhancementtoGoBGP
Wefixedthis
RouteBroker
p
p
p
20
SummaryDemonstratedfeasibilityofcreatinganL3-VPNvPEusingOpen
SourceSoftware• Control-plane:– GoBGP,FRR(OSPF,LDPandZebra)• Data-plane:– AT&T-VyattaDPDKbaseddata-plane– AlsoverifiedfeasibilitywithVPPandLinuxdata-planes
RequiredustomakesomeenhancementstoGoBGP1.31• ProperinstallationofroutesintoFIB• AssignMPLSlabelstoVPNv4routes
• ModificationsavailableonGithubat:https://github.com/amanshaikh75/gobgp/tree/zapi_version_5
DANOSURL:https://www.danosproject.org/
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
21
AcknowledgementsAT&T• BillBenson,RamanaChinnapa,KennethDuell,JenniferYates
CumulusNetworks• DavidAhern(forexplaininghowLinuxVRFswork)FRR• DonaldSharp,RenatoWestphal,RussWhite,https://github.com/paulzlabn
GoBGP• IwaseYusukeVPP• MichaelBorokhovich,PierrePfister,JeffShaw
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
23
Open-SourceSoftwareacrosstheFeasibilityTest-bed
Control-plane• GoBGP1.31.1=version1.31+ourenhancements
• FRR5.1-dev=snapshote8f9540WhenLinuxisusedasdata-planeonvPE• vPEOS:Ubuntu16.04.3LTS,Linuxkernel4.14.4-mpls
WhenVPPisusedasdata-planeonvPE• vPEdata-plane:VPPrelease1801+ourenhancementstorouterplug-in
• OS:sameaswhenLinuxisusedasdata-plane
NetworkFunction VNFOS Control-plane Data-plane
vCE Ubuntu16.04.2LTSLinuxKernel4.4.0-64generic
FRR5.1-devBGPandZebra Linux
vPE Debian4.14.62-0Vyatta1+9.1LinuxKernel4.14.0-trunk-vyatta-amd64…(DANOS)
GoBGP1.31.1FRR5.1-devOSPF,LDPandZebra(snapshote8f9540)
AT&T-VyattaDPDK
Corerouter Ubuntu16.04.3LTSLinuxkernel4.14.4-mpls(customconfiguration)
GoBGP1.31.1FRR5.1-devOSPF,LDPandZebra(snapshote8f9540)
Linux
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
24
GoBGP:ConfigurationofVRFandAssociatedCESessionsImportmatchingVPNv4routesintoVRF• RoutesreceivedfromRRsand/orotherPEs
• Outstandingissue;weuseaworkaroundObtainuniquelabelforVRFfromZebra• Zebraactsasacentralagentforlabelassignment
– PreventslabelcollisionbetweendifferentprotocolslikeBGPandLDP
• Pullrequest1587toGoBGPrepositoryInstallanMPLSrouteforthelabelin
Linuxdefaultforwardingtable• AllowsvPEtohandletrafficfromotherPEs
• WeenhancedGoBGPcode
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
VRF VRF Global
Control-plane
FRRZebra
ZebraAPI
vPE
GoBGP
Data-plane
VRF VRF GlobalRIB
VRF VRF GlobalFIB
VRF VRF GlobalBGPtable
ConfigureblueVRFandaddBGPsessiontoCE
ImportrelevantroutesfromglobaltableintoblueVRF
AssignMPLSlabel
# Example of MPLS route installed # in Linux kernel by GoBGP $ ip -f mpls route 144 dev blue proto bgp
RouteBroker
InstallMPLSlabelroute
InstallMPLSlabelroute
InstallMPLSlabelroute
25
GoBGP:InteractingwithZebraGoBGPbydefaultusesAPIversion4for
interactionwithZebra• APIversion4doesnothaveallfeaturestosupportL3-VPN
– Example:lackofsupportformulti-levelrecursivenext-hoplookup
• RequiredustoupgradetoZebraAPIversion5AddedpartialsupportforAPIversion5
inGoBGP• SupportforpartsrequiredforL3-VPN,noteverything
NANOG74:OpenSourcevPEforBGP/MPLSL3-VPN
VRF VRF Global
Control-plane
FRRZebra
ZebraAPIVersion5
vPE
GoBGP
Data-plane
VRF VRF GlobalRIB
VRF VRF GlobalFIB
VRF VRF GlobalBGPtable
RouteBroker
top related