Case Study of a SP Customer running ACI based SDN … · Other LTE /Wi-Fi backend systems OSS & BSS, CDN IBR EPC Enterprise Customer / Med ISP WiFi MAP WiFi MAP ... • Simplified

Case Study of aSP Customer runningACI based SDN Solutionfor Telecom Datacenter

Abhishek Mande, Technical Solution Architect, Cisco

Sonu Khandelwal, Technical Marketing Engineer, Cisco

BRKSPG-3489

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKSPG-3489

• Evolution of Telco Datacenter

• Customer Profile & Technical Requirements

• How ACI solves the challenge

• Looking Ahead

• Conclusion

Agenda


Distributed Services Architecture of Telco Datacenter

5BRKSPG-3489

`

Core and Edge Multi-CloudRemote DC

Near EdgeAccess

Carrier-E /

Transport

Central Data Centers

Edge

Internet / Partner SP Edge

Aggregation

VPN CPE

Cust. Prem

Cust

Prem

vBranch

Analytics

Access

Nothing

is seen

today…

Edge DCs

User Plane, OTT

Caching, Gi-LAN

Central DCs

VPC, Gi-LAN, vIMS,

Biz Services (vMS),

cDVR,

CDN OTT Chahing

Virtualized RR,

Analytics

Co-Lo /

Peering

vCvMS,

DN,

vDDoS,

Cloud Hosted

XaaS

delivered from

the Multi-

Cloud

Peering

DCI

DCI

DCI

DCI

DCI

DCI

Remote DC

Near Edge Co-Lo

Co-Lo

Peering

Peering

CO

vBNG, vOLT, vCMTS,

vPE

Biz Services (vMS),

vRAN,

vCDN, Analytics


• Customer Profile & Technical Requirements• Applications

• Datacenter Fabric


• Looking Ahead

• Conclusion

Agenda


Customer Profile

7BRKSPG-3489

Subscriber

150Mn->400Mn

Services Offered

• Unlimited voice calls

• 1 GB data per day per subscriber

• Unlimited home grown multi-media content

• Many more home grown application planned

Challenges

• Extremely

aggressive timelines

• Brownfield

environment

Data Capacity

1.5EB -> 6.5EB per Month

Scale

• 1M networking elements

• 150K Base Station-> 250K Base Station

• End to End IPv6


Pre-Aggregation(IP/MPLS)

CSR

CSR

CSR

FTTx

WiFi RAP

CSR

Residential

STB

CPE

WiFi RAP

CSR

OLT

OLT OLT

OLT OLT

PGW, SGW, MME,WLC,ISGOther LTE /Wi-Fi backend systems OSS & BSS, CDN

IBR

EPC

Enterprise

Customer /

Med ISP

WiFi MAP

WiFi MAP

Telco DC – 31 Locations IT DC

Internet

IGW

IMS CDN

High Level Network Architecture

Aggregation

(IP/MPLS)

DC-WAN

Domestic

Peering

BRKSPG-3489 8

Core & Super Core

(IP/MPLS)

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9BRKSPG-3489

Telco Applications Requirement

• Caching managed by

OTT Provider

• Self created Media

Content

• Driving 25G & 100G

interfaces

CDN & OTT Caching

• Faster Convergence

• Consistent low latency

• Active/standby

connectivity

IMS

• IPv6 Multicast

eMBMS

• Active/standby

connectivity

• Failover & Redundancy

SGW & PGW

• IPv4 & IPv6 connectivity

PCRF

• TCP Optimizers

• Deep Packet

Inspection (DPI)

• CG-NAT

• Service Chaining

• URL Filtering

Gi-LAN


• Consistent low latency

Voice over Wifi

• IPv4 & IPv6 connectivity

DNS, AAA, DHCP


Datacenter Fabric Requirement

10BRKSPG-3489

• Fabric Automation – Day-0 and Day-1

• Controller based design

• Programmable North bound Interfaces for home grown tools

• Multi-Hypervisor support

• Low and Predictable Latency with minimum Network Hops

• Distributed default gateway

• IPv4, IPv6 and Dual stack connectivity

• Carrier Grade Availability

• Convergence - Less than 100msec


• Simplified Operations & Troubleshooting

• Security & Segmentation


Datacenter Fabric Protocol & Scale Requirement

11BRKSPG-3489

Physical Fabric • 40-100 Leaf Per Physical Fabric

• Multi-Speed Interfaces on Same Leaf & Spine

Traffic Throughput • 1.5-5 Tbps

• Linerate

Protocol • BGPv4, BGPv6

• Static route

• BFD

• IPv6 Multicast

• SNMP

• NTP

• TACACS

Carrier Grade Scale • 500 Vlans

• 50 VRFs

• 1K Mac, ARP & ND per switch

• 10-50K end hosts

• 10-20K IPv4 & IPv6 LPM addresses


Solution Approach

ACI Fabric

SGWEMB

MSSGWSGW PGWPGW PCRF

HYPERVISOR HYPERVISOR Physical Appliance

OSS & BSS Systems

Gi

LAN

• Fabric Automation – Day-0 and Day-1

• Controller based design

• Programmable North bound Interfaces for home grown tools

• Multi-Hypervisor support

• Low and Predictable Latency with minimum Network Hops

• Distributed default gateway

• IPv4, IPv6 and Dual stack connectivity

• Carrier Grade Availability

• Convergence - Less than 100msec


• Simplified Operations & Troubleshooting

• Security & Segmentation

BRKSPG-3489 12




• Looking Ahead

• Conclusion

Agenda


Application to Fabric Connectivity

TCP

Optimizer-1TCP

Optimizer-N

Out of Band

Network

Internet

ASR9K

Spine

N9508

N9732C-EX Service Leaf

N93180YC-EX

Border Leaf

N93180LC-EX

Service Leaf

N93180YC-EX

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

DPI (L1 Device)

BRKSPG-3489 14

IP/MPLS

OSS & BSS, NTP, Syslog,

TACACS+

CG-NAT

Service


Application to Fabric Connectivity

TCP

Optimizer-1TCP

Optimizer-N

Active LB Standby LB

Out of Band

Network

Internet

ASR9K

Active/Active

Firewall Cluster

Spine

N9508


N93180YC-EX

Border Leaf

N93180LC-EX

Service Leaf

N93180YC-EX

N93180LC-EX

Server Leaf

N93180YC-EX

Service Leaf

N93180YC-EXService Leaf

N93180YC-EX

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

DPI (L1 Device)

IMS, MME, PCRF, Voice over wifi, OTT Applications and

Other Telco Appliances

BRKSPG-3489 15

IP/MPLS


TACACS+

CG-NAT

Service


Application to Fabric ConnectivityDNS-N

TCP

Optimizer-1TCP

Optimizer-N

DNS-1

Active LB Standby LB

Out of Band

Network

Internet

ASR9K

Active/Active

Firewall Cluster

Spine

N9508


N93180YC-EX

Border Leaf

N93180LC-EX

Service Leaf

N93180YC-EX

N93180LC-EX

Server Leaf

N93180YC-EX

Service Leaf

N93180YC-EXService Leaf

N93180YC-EX

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

DPI (L1 Device)

IMS, MME, PCRF, Voice over wifi, OTT Applications and

Other Telco Appliances

BRKSPG-3489 16

IP/MPLS


TACACS+

CG-NAT

Service

40G/100G Links




• Automation & Scale• Services Integration

• Migration & Operational Simplification


• OSS & BSS Integration

• Looking Ahead

• Conclusion

Agenda


Fully Automated Provisioning of ACI Fabric

Connect switches in Spine Leaf

topology

Connect APICs to Leaf Pair

Power on APIC and switches

Input simple details like Fabric

Subnet , APIC Out of Band

Management IP & Login

credential on APIC CIMC

Login to APIC and register

switches

Fabric is up and running

BRKSPG-3489 18


Topology View


Automatic Policy deployment

100.1.1.100

Vlan 100

Host A

100.1.1.102

Vlan 100

Host C

100.1.1.1100.1.1.1 100.1.1.1 100.1.1.1

100.1.1.101

Vlan 100

Host B

100.1.1.1 100.1.1.1

Automatic deployment of Tenant, VRF, BD

(Gateway IP) when End Point is detectedGateway- 100.1.1.1

BRKSPG-3489 20



100.1.1.100

Vlan 100

Host A

100.1.1.102

Vlan 100

Host C

100.1.1.1100.1.1.1 100.1.1.1 100.1.1.1

100.1.1.101

Vlan 100

Host B

100.1.1.1 100.1.1.1


(Gateway IP) when End Point is detected

End Point Table

100.1.1.100(Host A)

100.1.1.101(Host B)

End Point Table

100.1.1.100(Host A)

100.1.1.101(Host B)

Gateway- 100.1.1.1

BRKSPG-3489 21



100.1.1.100

Vlan 100

Host A

100.1.1.102

Vlan 100

Host C

100.1.1.1100.1.1.1 100.1.1.1 100.1.1.1

100.1.1.101

Vlan 100

Host B

100.1.1.1 100.1.1.1


(Gateway IP) when End Point is detected

End Point Table

100.1.1.100(Host A)

100.1.1.101(Host B)

End Point Table

100.1.1.100(Host A)

100.1.1.101(Host B)

End host information into hardware

tables even with same vlan is

updated based on communication for

better scale

Gateway- 100.1.1.1

100.1.1.102(Host C)

End Point Table

100.1.1.101(Host B)

100.1.1.102(Host C)

BRKSPG-3489 22

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKSPG-3489

Host to ACI Fabric Connectivity

100.1.1.100

Vlan 100

Active/Standby

links from serverActive/Active (vPC)

links from server100.1.1.200

Vlan 100

100.1.1.1100.1.1.1 100.1.1.1 100.1.1.1Gateway- 100.1.1.1

23

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco PublicBRKSPG-3489

Host to ACI Fabric Connectivity

100.1.1.100

Vlan 100

Active/Standby

links from serverActive/Active (vPC)

links from server100.1.1.200

Vlan 100

Fabric Automation – Day-0 and Day-1

Controller based design

Multi-Hypervisor support

Low and Predictable Latency with minimum Network Hops

Distributed default gateway

100.1.1.1100.1.1.1 100.1.1.1 100.1.1.1Gateway- 100.1.1.1

24



• How ACI solves the challenge• Automation & Scale

• Services Integration• Migration & Operational Simplification



• Looking Ahead

• Conclusion

Agenda

Gi-LAN Services

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27

1.1.1.0/8 & 2000::/64

Subscriber Pool

IP/MPLS

ASR9K

2.2.2.0/8 & 2001::/64

Subscriber Pool

SAE GW-1

(SGW & PGW)

Active Standby

SAE GW-2

(SGW & PGW)

ActiveStandby

eBGP connection from ACI

BorderLeaf to ASR9K

SAE GW Connectivity to ACI Leaf

BRKSPG-3489

CG-NAT

Service

Internet


1.1.1.0/8 & 2000::/64

Subscriber Pool

IP/MPLS

ASR9K

2.2.2.0/8 & 2001::/64

Subscriber Pool

1.1.1.0/8 2000::/64

2.2.2.0/8 2001::/64

SAE GW-1

(SGW & PGW)

Active Standby

SAE GW-2

(SGW & PGW)

ActiveStandby

Advertise subscriber pool through static route

1.1.1.0/8 -> SAE GW-1 IPv4 address

2000::1/64-> SAE GW-1 IPv6 address

2.2.2.0/8 -> SAE GW-2 IPv4 address

2001::1/64-> SAE GW-2 IPv6 address


BorderLeaf to ASR9K

SAE GW Connectivity to ACI Leaf

BRKSPG-3489

CG-NAT

Service

Internet


ASR9K

VLAN100

100.1.1.1.1

VLAN101

100.1.1.1.1


BorderLeaf to ASR9K

1.1.1.0/8 & 2000::/64

Subscriber Pool

2.2.2.0/8 & 2001::/64

Subscriber Pool

SAE GW-1

(SGW & PGW)

Active Standby

SAE GW-2

(SGW & PGW)

ActiveStandby

No BFD

No Dynamic

Routing Support

SAE GW Connectivity to ACI LeafHandling Failure

BRKSPG-3489 29

IP/MPLS Internet


ASR9K

VLAN100

100.1.1.1.1

VLAN101

100.1.1.1.1


BorderLeaf to ASR9K

1.1.1.0/8 & 2000::/64

Subscriber Pool

2.2.2.0/8 & 2001::/64

Subscriber Pool

SAE GW-1

(SGW & PGW)

Active Standby

SAE GW-2

(SGW & PGW)

ActiveStandby

No BFD

No Dynamic

Routing Support


BRKSPG-3489 30

IP/MPLS Internet


ASR9K 2.2.2.0/8 2001::/64

VLAN100

100.1.1.1.1

VLAN101

100.1.1.1.1

1.1.1.0/8 2000::/64

Static route over SVI should be

removed when both active &

standby link goes down


BorderLeaf to ASR9K

1.1.1.0/8 & 2000::/64

Subscriber Pool

2.2.2.0/8 & 2001::/64

Subscriber Pool

SAE GW-1

(SGW & PGW)

Active Standby

SAE GW-2

(SGW & PGW)

ActiveStandby

No BFD

No Dynamic

Routing Support


BRKSPG-3489 31

IP/MPLS Internet


Handling Failure of SAE GWDelivered through APP in two weeks

Visually monitor externally routed interface states

And next hop add/delete

Monitoring and Troubleshooting

cTrac

• Static route over SVI should be removed when all interfaces in SVI goes down

• APP Infrastructure is available for any customer or partner to develop Apps

• Multiple Free Apps including cTrac for static route monitoring available at https://aciappcenter.cisco.com/

• Easy way to integrate with Eco system partners

Feature also delivered in ACI 3.1(1)

Dynamic routing is being supported by SAE GW

BRKSPG-3489 32

https://aciappcenter.cisco.com/

Deep Packet Inspection (DPI) Services


1.1.1.0/8 & 2000::/64

Subscriber Pool

ASR9K

2.2.2.0/8 & 2001::/64

Subscriber Pool

SAE GW-1

(SGW & PGW)

Active Standby

SAE GW-2

(SGW & PGW)

ActiveStandby

Deep Packet Inspection (DPI) connectivity to ACI Leaf

BRKSPG-3489

DPI (L1 Device)

Inline Layer1 (L1) DPI between

ACI Leaf and SAE GW

Internet

Service Chaining for TCP Optimizers


TCP Optimizer Integration with ACIAutomatic Load-balancing and Symmetry of traffic flow

TCP

OPT 1

N-TCP Optimizers in a group Internet

ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

BRKSPG-3489 36



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Symmetric PBR ensure return traffic choses same TCP optimize

Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

BRKSPG-3489 37



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

BRKSPG-3489 38



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

BRKSPG-3489 39



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

BRKSPG-3489 40



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

BRKSPG-3489 41



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Automatic load-balancing of traffic across different TCP optimizers based on forwarding table hash (Source IP, Destination IP, Source Port, Destination Port)

Subscriber Pool

BRKSPG-3489 42



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)


Subscriber Pool

BRKSPG-3489 43



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)


Subscriber Pool

BRKSPG-3489 44



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)


Subscriber Pool

BRKSPG-3489 45



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N


Flow 1

Flow 2

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)


Subscriber Pool

BRKSPG-3489 46


TCP Optimizer Integration with ACISimplified Configuration

TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Source EPG

All prefixes learnt

from SAE GW

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Source EPG

All prefixes learnt

from SAE GW

Destination EPG

All prefixes learnt

from ASR9K

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Source EPG

All prefixes learnt

from SAE GW

Destination EPG

All prefixes learnt

from ASR9K

Contract (PBR)

TCP = Any or

UDP =443NO

YES

SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool


TCP Optimizer Integration with ACITracking TCP Optimizer Liveliness

TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Outside

2.2.2.1

ICMP & TCP Tracking of

Inside & Outside Interface

Inside

1.1.1.1SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

51BRKSPG-3489



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Outside

2.2.2.1



Inside

1.1.1.1SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Removes whole TCP Optimizer if either

Inside or Outside interface goes down

Subscriber Pool

52BRKSPG-3489



TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Outside

2.2.2.1



Automatic Load-Balancing to remaining

TCP Optimizers after failure

Inside

1.1.1.1SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Removes whole TCP Optimizer if either

Inside or Outside interface goes down

Subscriber Pool

53BRKSPG-3489


TCP Optimizer Integration with ACIBypassing TCP Optimizers to avoid congestion

TCP

OPT 1


ASR9K

TCP

OPT 2

TCP

OPT 3

TCP

OPT N

Outside

2.2.2.1Inside

1.1.1.1SAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Subscriber Pool

Traffic is directly send to internet

when more than defined number

of TCP Optimizers fails

54BRKSPG-3489

Firewall & Load Balancer Connectivity


Firewall Connectivity with ACI

Active/Active

Firewall Cluster

Inside Server

20.1.1.1

Inside VLAN IP

100.1.1.2outside VLAN IP

200.1.1.2

Inside BD & VRF – 100.1.1.1

Outside BD & VRF - 200.1.1.1

ASR9K

eBGP connection to

ASR9K

Packet Flow Internal servers to outside

IP/MPLSInternet



Active/Active

Firewall Cluster

Inside Server

20.1.1.1

Inside VRF

0.0.0.0->100.1.1.2

Inside VLAN IP


200.1.1.2



ASR9K

eBGP connection to

ASR9K


IP/MPLSInternet



Active/Active

Firewall Cluster

Inside Server

20.1.1.1

Inside VRF

0.0.0.0->100.1.1.2

Inside VLAN IP


200.1.1.2

0.0.0.0-> 200.1.1.1



ASR9K

eBGP connection to

ASR9K


IP/MPLSInternet



Active/Active

Firewall Cluster

Inside Server

20.1.1.1

Inside VRF

0.0.0.0->100.1.1.2

Inside VLAN IP


200.1.1.2

0.0.0.0-> 200.1.1.1



ASR9K

eBGP connection to

ASR9K


IP/MPLSInternet


Firewall Connectivity with ACIPacket Flow Outside to Inside Server

Active/Active

Firewall ClusterInside Server

20.1.1.1

Inside VLAN IP


200.1.1.2



ASR9K

eBGP connection to

ASR9K

IP/MPLS Internet



Active/Active


20.1.1.1

Inside VLAN IP


200.1.1.2

Outside VRF

20.1.1.0/24-> 200.1.1.2



ASR9K

eBGP connection to

ASR9K

IP/MPLS Internet



Active/Active


20.1.1.1

Inside VLAN IP


200.1.1.2

Outside VRF

20.1.1.0/24-> 200.1.1.2



ASR9K

eBGP connection to

ASR9K

Inside VRF

20.1.1.0/24-> 100.1.1.1

IP/MPLS Internet


Load-Balancer Connectivity with ACIPacket flow from clients to internal servers

Internal ServersActive LB Standby LB

ASR9K

eBGP connection to

ASR9K

eBGP to

advertise VIPeBGP to

advertise VIP

Clients

IP/MPLS Internet




ASR9K

eBGP connection to

ASR9K

eBGP to


advertise VIP

Clients

clients access Load-

Balancers VIP

IP/MPLS Internet




ASR9K

eBGP connection to

ASR9K

eBGP to


advertise VIP

Clients

clients access Load-

Balancers VIP

Load-Balancer forward

request to Internal

servers

IP/MPLS Internet


Load-Balancer Connectivity with ACI


ASR9K

eBGP connection to

ASR9K

eBGP to


advertise VIP

Clients

Packet flow from internal servers to clients

IP/MPLS Internet

BRKSPG-3489 66




ASR9K

eBGP connection to

ASR9K

eBGP to


advertise VIP

Clients

Internal servers sends the

packet to LB


IP/MPLS Internet

BRKSPG-3489 67




ASR9K

eBGP connection to

ASR9K

eBGP to


advertise VIP

Clients

Load-Balancer sends

packet to clients

Internal servers sends the

packet to LB


IP/MPLS Internet

BRKSPG-3489 68

IPv6 Multicast for eMBMS Service


Evolved Multimedia Broadcast Multicast Service (eMBMS)Multimedia services to end customers through IPv6 Multicast

eMBMS GW

2001::3/64

2001::1/64

IP/MPLS

(PIMv6 enabled Mobile

Backhaul)eNodeB Joins mcast group

ff1e::10 using MLDv2

ASR9K

Fabric is L2 for eMBMS Vlan (EPG)

Multicast Source IP is eMBMS GW IP – 2001::3

Multicast IP – ffe1::10

Default GW for Multicast service is on ASR9K- 2001::1

Internet Connectivity


ASR9K

Internet

72BRKSPG-3489

External Connectivity from ACI Fabric to ASR9K


BorderLeaf to ASR9KSAE GW

(SGW & PGW)

SAE GW

(SGW & PGW)

Multiple BorderLeaf for high

speed internet connectivity

Multiple Links to reach to

BorderLeaf

IP/MPLS




• Services Integration

• Migration & Operational Simplification• Faster Convergence


• Looking Ahead

• Conclusion

Agenda

Migration


Migration Process

1. Collected Existing

configuration

2. Mapped Existing

Configuration to ACI

Constructs such as Tenant,

VRF, EPG, BD, L3out etc.

3. Created simple python

scripts to develop XML

based ACI configuration

4. Built ACI Fabric

5. Uploaded XML

configuration using

POSTMAN tool

6. Migrated 2500+

Physical ports, 30

VRF, 300+ Vlans,

200+ Static Routes,

50+ BGP neighbors

7. Troubleshooting

using ACI Operations

tool


Migration ProcessConvert vlan and default GW to ACI EPG and BD

interface Vlan133

vrf member SUBSCRIBER

no ip redirects

ip address 10.1.1.1.100

ipv6 address 2001::100/64

hsrp version 2

hsrp 100

authentication md5 key-chain HSRP-KEY

preempt delay minimum 60 reload 300

priority 254 forwarding-threshold lower 1 upper 254

ip 10.1.1.254

hsrp 100 ipv6

authentication md5 key-chain HSRP-KEY

preempt delay minimum 60 reload 300

priority 254 forwarding-threshold lower 1 upper 254

timers 5 15

ip 2001::1

no shutdown

Created XML configuration for

• Tenant and VRF

• EPG for each Vlan

• Add interfaces into EPG

• BD for each SVI

• No need to configure HSRP since ACI fabric is active/active with default GW on every leaf


Migration ProcessConvert Interface configuration to Interface policies

interface port-channel1

description ## Port Channel-1 for SGI LENA0 ##

switchport access vlan 161

spanning-tree port type edge

interface Ethernet1/1



channel-group 1

interface Ethernet1/2



channel-group 1

vpc domain 100

peer-switch

role priority 90

peer-keepalive destination 1.1.1.2 source 1.1.1.1

delay restore 90

auto-recovery

ip arp synchronize


• ACI Interface policies - LACP Policy, Speed, LLDP, CDP etc.

• Interface policy group – Groups all the above policies for Regular Physical interface, PC or vPC

• Vlan Pool – Vlans that are required for the Fabric

• Interface & switch profile – defines the switch & interface where the policy will be applied


Migration ProcessConvert external routed neighbors to ACI L3out

vrf context SUBSCRIBER

ip route 10.0.0.0/14 10.71.225.132 track 160 name SAEGW-01

ipv6 route 2002::/64 2001::1 track 160 name LTE_SAEGW-01

router bgp 65000

vrf SUBSCRIBER

router-id 200.1.1.1

template peer SAR-IPv4

remote-as 65001

address-family ipv4 unicast

send-community

maximum-prefix 40000 warning-only

template peer SAR-IPv6

remote-as 65001


send-community both

maximum-prefix 40000 warning-only

neighbor 2405:200:801:c00::7b

inherit peer SAR-IPv6


route-map SAR-IPv6-OUT out

neighbor 172.16.24.123

inherit peer SAR-IPv4


route-map SAR-IPv4-OUT out


• Static Route (L3out)

• Static route tracking was needed to track L2 nodes from L2/L3 aggregation router that was two Hop away. ACI Leaf is directly connected to these L2 nodes hence no need for static route tracking.

• BGP neighbors, route-maps, allowed prefixes (L3 out)


Migration ProcessOther Configuration Migration


• vZany Contract that permits communication within VRF

• Default policy is not to allow communication that reduced the ACL requirement

• NTP, SNMP, Syslog, TACACS, PBR


Migration ProcessPOST Using POSTMAN

Any Operating Model

CLI GUI API


Automation for ACI Fabric in ProductionSelf developed Provisioning tool for pushing Configuration

Provisioning Tool

API Calls to

provisioning Tenant,

VRF, BD, EPG etc.

Operations Simplification


Operations Tools

Health Score Card

Troubleshooting Wizard End Point Tracker

Faults Capacity DashBoard Link Statistics

Topology Dashboard

Traffic Map Upgrade/Downgrade


Troubleshooting with Health Score CardDrill Down from Dashboard


Troubleshooting with Health Score Card

BRKSPG-3489 85


End Point Tracker

BRKSPG-3489 86


Capacity Dashboard


Fabric wide Traffic


Easy Upgrade Process for whole Fabric

BRKSPG-3489 89






• Faster Convergence • OSS & BSS Integration

• Looking Ahead

• Conclusion

Agenda


ACI Fabric Convergence

91BRKSPG-3489

ASR9K

Active/Active

ServerInternetIP/MPLS



92BRKSPG-3489

ASR9K

Active/Active

Server

Controller Failure - No Loss

InternetIP/MPLS



93BRKSPG-3489

ASR9K

Active/Active

Server

Fabric Failure - 5 to 10msec

Convergence happens within ASIC


InternetIP/MPLS



94BRKSPG-3489

ASR9K

Active/Active

Server

Fabric Failure - 5 to 10msec

Convergence happens within ASIC

External Connectivity Failure - within 100msec

ACI Fabric to external connectivity failure

Access Failure - within 100msec

ACI Fabric to vPC connected host

failure


InternetIP/MPLS








• Looking Ahead

• Conclusion

Agenda


OSS & BSS Integration

OOB

Network

Syslog Servers

TACACS Servers

TRAP Aggregator

Each ACI Switch sends TRAP

Telecom Network Management platform

based on SNMP

BRKSPG-3489 96


OSS & BSS Integration

OOB

Network

Syslog Servers

TACACS Servers

BRKSPG-3489 97

APIC aggregates traps of

switches in ACI 3.1

Telecom Network Management platform

based on SNMP


ACI Integration with Splunk

98BRKSPG-3489

Splunk APP

for ACI

https://splunkbase.splunk.com/app/1896/


Dashboard

Syslog

APIC SDK






• Automation & Scale





• Looking Ahead

• Conclusion

Agenda


Management of Multiple Sites with ACI Multi-Site Solution

Separate ACI Fabrics with independent APIC clusters

ACI Multi-Site pushes cross-fabric configuration to multiple APIC clusters providing scoping of all

configuration changes

End-to-end policy definition and enforcement

Availability Zone ‘A’

IP Network

REST

API GUI

Site 1 Site 2

Availability Zone ‘B’

BRKSPG-3489 100


All local traffic is switched directly between

endpoints, both virtual and bare metal

Any traffic that requires use of the Spine

Proxy will be forwarded to the primary site(s)

IP Network

Bare-

MetalHypervisorvSwitch

Remote Datacenter

Main Datacenter

Architecture for Distributed DatacentersACI Remote Physical Leaf

BRKSPG-3489 101


Questions?

BRKSPG-3489 102




• Automation & Scale





• Looking Ahead

• Conclusion

Agenda


Conclusion

ACI

• Massive Scale

• Time to Market

• Simplicity

Requirement Solution

Looking Ahead

• Automation

• Scale

• Simplified Operation

• Choices of Integration with North-bound and South-bound devices

Consistent Policy & Management across

Geography

BRKSPG-3489 104


Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKSPG-3489


• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Complete Your Online Session Evaluation

http://ciscolive.com/Online

http://www.ciscolive.com/global/on-demand-library/


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Related sessions

107BRKSPG-3489

Thank you

Case Study of a SP Customer running ACI based SDN … · Other LTE /Wi-Fi backend systems OSS & BSS, CDN IBR EPC Enterprise Customer / Med ISP WiFi MAP WiFi MAP ... • Simplified

Documents