Next generation transport network - Carrier Ethernet - SANOG

Next generation transport network - Carrier Ethernet Mobile Backhaul / Consumer Broadband / Enterprise Services

Ramakrishnan Subramanian

Cisco Systems

© 2014 Cisco and/or its affiliates. All rights reserved.

Agenda

Evolution and Trends in Mobile Networks

Mobile Backhaul with Unified MPLS • Unified MPLS introduction and architectures

• 2G/3G, LTE Backhaul Services

• Synchronization

• Fast Convergence

Consumer Broadband Services

Enterprise Services

Summary

Cisco Confidential 3 © 2011 Cisco and/or its affiliates. All rights reserved.

Backhaul Network Challenges

•Support of Multi-Technology over Transport Network. Mobile (2G, 3G, LTE),

Enterprise, Consumer Services

•SDH based network is having lack of scalability for growing BW needs with low

Capex. High Cost of backhaul Network

•Large Scale

•Fast Convergence

•Quality of Service

•Frequency Synchronization and Phase Synchronization

•Backhaul Network capacity is limiting the growth/expansion


Service Node Category

Bandwidth per Node

2G BTS Larger Cities 2-4 E1. Smaller Town: 1 – 2 E1

3G NB Voice: 1 E1 Data: 42MB in Large City 14-20MB in smaller Town

4G / LTE eNB 100Mbps

SP Wifi AP 4MB

Consumer Broadband

DSLAM / OLT 1G / 10G

Enterprise Service

WiMax 7MB Per Sector. No. of Sector: 3 to 4 Per Location

Microwave Hybrid Backhaul

Vary from 25 or 50 or 400Mbps

Mobile Backhaul with Unified MPLS

RFC

3107 BGP

filtering

LFA

R-LFA

BGP

PIC

IGP/LDP

Domain isolation

E2E

OAM

Classical MPLS

Scalability Security Simplification Multi-Service

Unified MPLS

Architecture

Flex

Access L2/IGP/BGP/MPLS-

TP/LDP DoD

‟Unified MPLS…classical MPLS with a few additions”


Routing + MPLS Design ‘Divide & Conquer’ – Game Plan

Disconnect & Isolate IGP domains

– No more end-to-end IGP view

Leverage BGP for infrastructure (i.e. PE) routes

– Also for infrastructure (i.e. PE) labels (e.g. RFC3107)

Backbone Aggregation

.

Access Region 2

.

PE31

R

PE21

Access .

Region1

.

Aggregation

PE11

PE21

ISIS Level 2

Or

OSPF Area 0

ISIS Level 1

Or

OSPF Area X

ISIS Level 1

Or

OSPF Area Y

Isolated IGP & LDP Isolated IGP & LDP Isolated IGP & LDP

BGP for Infrastructure Prefixes

BGP for Services (e.g. L2, L3)

BGP

(+Label)

BGP

(+Label)


Unified MPLS Architecture (RFC 3107)

Core Network

Aggregation Network

Aggregation Node Core ABR

IGP/LDP IGP/LDP

iBGP/eBGP Pre-Aggregation

Node Access

Network

IGP/LDP

EPC Gateway Access Node

Access Node Centralised RR

IGP/LDP Label

BGP3107 Label Service Label

U-MPLS Classical

MPLS


Sample E2E Unified MPLS Architecture Routing Isolation and Label Stack for LSP between Pre-Agg. Node Loopbacks

No IGP route is propagated from Aggregation to the Core. IGP area has routes for that area only plus routes to core ABRs. Only the core ABR’s are propagated from L2 to L1

• LDP labels are used to traverse each domain and reach core ABRs

• BGP labels are used by Labeled BGP PEs & ABRs to reach Labeled BGP PEs in remote areas

• Service (e.g. PW) labels are used by Label BGP PEs

Core Network

Core ABR

(Inline RR)

Pre-Agg.

Node

Access

Network

Agg. Node MPC

Gateway

Access

Node Centralised RR Agg. Node

Core ABR

(Inline RR)

Access

Node

Pre-Agg.

Node

Core ABR

(Inline RR)

Core ABR

(Inline RR)

Access

Network

Agg. Node

Agg. Node

Aggregation

Network

Aggregation

Network

ISIS Level 1/OSPF x ISIS Level 1/OSPF x ISIS Level 2/OSPF 0 L2 L2

IGP/LDP Label

BGP3107 Label

Service Label

Push

Push

Swap Pop Push Swap Pop

Swap Swap Swap Pop

LDP LSP LDP LSP LDP LSP

BGP LSP


Unified MPLS Model 1 MPLS support in the Core, Aggregation with TDM, uWave or L2 in the access

• The Mobile Core and Aggregation Networks enable Unified MPLS Transport

• The Core and Aggregation Networks are organized as independent IGP/LDP domains

• The network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP IPv4+labels. Intra domain connectivity is based on LDP LSPs

• The Aggregation Node enable Mobile and Wire line Services. The Mobile RAN Access is based on TDM, Packet Microwave or pt-to-pt L2 connectivity

Core Network

Core ABR

(Inline RR)

Pre-Agg.

Node

Access

Network

Agg. Node MPC

Gateway

Centralised RR Agg. Node

Core ABR

(Inline RR) Pre-Agg.

Node

Core ABR

(Inline RR)

Core ABR

(Inline RR)

Access

Network

iBGP Hierarchical LSP


Agg. Node

Agg. Node

Aggregation

Network

Aggregation

Network

L2/TDM/uWave L2/TDM/uWave

L2


Unified MPLS Model 2 MPLS support in the Core, Aggregation and Access Network

• The Mobile Core, Aggregation, Access Network enable Unified MPLS Transport

• The Core, Aggregation, Access are organized as independent IGP/LDP domains

• The network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP IPv4+labels. Intra domain connectivity is based on LDP LSPs

• The Access Network Nodes learn only the required labelled BGP FECs, with selective distribution of the MPC and potentially neighbouring RAN labelled BGP communities

Core Network

Core ABR

(Inline RR)

Pre-Agg.

Node

Access

Network

Agg. Node MPC

Gateway Access

Node

Access


Core ABR

(Inline RR) Access

Node

Access

Node

Pre-Agg.

Node

Core ABR

(Inline RR)

Core ABR

(Inline RR)

Access

Network



Agg. Node

Agg. Node

Aggregation

Network

Aggregation

Network

LDP LSP LDP LSP


Unified MPLS Model 3 MPLS in the Core, Aggregation with IGP/LDP in the access

• The Core and Aggregation are organized as distinct IGP/LDP domains

• Inter domain hierarchical LSPs based on RFC 3107, BGP IPv4+labels which are extended out to the Pre-aggregation

• Intra domain LSPs based on LDP

• The inter domain Core/Aggregation LSPs are extended in the Access Networks by distributing the RAN IGP into the inter domain iBGP and distribute the necessary labelled iBGP prefixes (MPC gateway) into RAN IGP (via BGP communities)

Core Network

Core ABR

(Inline RR)

Pre-Agg.

Node

Access

Network

Agg. Node MPC

Gateway Access

Node

Access


Core ABR

(Inline RR) Access

Node

Access

Node

Pre-Agg.

Node

Core ABR

(Inline RR)

Core ABR

(Inline RR)

Access

Network



Agg. Node

Agg. Node

Aggregation

Network

Aggregation

Network

LDP LSP LDP LSP

Redistribute MPC

iBGP community

into RAN Access IGP

Redistribute

CSN Loopbacks

into iBGP

Unified MPLS Service Infrastructure



LTE MPLS VPN Service Scale Control for S1 and X2 communication

Aggregation Domain Aggregation Domain

VRF VRF

Export: RAN W RT, Common RT

Import RAN W RT, MPC RT

Export: RAN X RT, Common RT

Import RAN X RT, MPC RT

Export: RAN Y RT, Common RT

Import RAN Y RT, MPC RT

Export: MPC RT

Import: MPC RT, Common RT Export: RAN Z RT, Common RT

Import RAN Z RT, MPC RT

MTG

MTG

MTG

VRF

VRF

VRF

VRF

VRF

VRF

VRF

MME

VRF

SGW/PGW

VRF

VRF

VRF

VRF

VRF

SGW/PGW

LTE Transport

MPLS VPNv4/v6 VRF

• Unified MPLS transport with a common MPLS VPN for LTE S1 from all CSGs and X2 per LTE region

• Mobile Transport GWs import all RAN & MPC Route Targets, and export prefixes with MPC Route Target

• CSGs (and Pre-Aggregation Node) in a RAN region import the MPC and neighboring RAN Route Targets: Enables S1 control and user plane with any MPC locations in the core

Enables X2 across CSGs in the RAN region

Core Domain

LTE


Unified MPLS with Microwave Access Integration with Microwave Adaptive Code Modulation (ACM)

The IP/MPLS Access Network adapts intelligently to the Microwave Capacity drops

Microwave Adaptive Code Modulation changes due to fading events are signaled through an Y.1731 VSM to the MPLS Access Node

The MPLS Access Node adapts the IGP metric of the link to the new capacity, triggering optimized SPFs that account for the capacity drops

– Degraded Link Cost = [n +1- n*CB/NB] * Original Link Cost Where: CB = Current BW, NB = Nominal BW, n = nodes in the ring

In addition the Access Node can change the Hierarchical QOS policy on the interface with the microwave system allowing EF traffic to survive despite of the capacity drop.

Aggregation Node

Aggregation Node

Microwave Fading

Y.1731

VSM

Signals the

Microwave

link speed

IP/MPLS

interface

Policy Logic that updates

the IGP metric on the

IP/MPLS interface

Network Synchronization


Synchronization Needs for different applications

18

Technology Frequency

Read: better than…

Phase or Time Synchronization Read: less than…

GSM Macro BS: ±50 ppb

Pico BS: ±100 ppb N/A

WCDMA (and LTE) FDD

WideArea BS: ±50 ppb

Medium/LocalArea BS: ±100 ppb

Home BS: ±250 ppb

OBSAI: ±16 ppb

N/A

WCDMA TDD WideArea BS: ±50 ppb

LocalArea BS: ±100 ppb ± 2.5 µs between base stations

TD-SCDMA WideArea BS: ±50 ppb

LocalArea BS: ±100 ppb ± 3 µs between base stations

LTE TDD WideArea BS: ±50 ppb

LocalArea BS: ±100 ppb

± 3 µs between base stations

May range from ±0.5µs to ±50µs

CDMA2K Macro Cell BS: ±50 ppb

Pico Cell BS and Femto Cell: ±100 ppb

ToD (UTC) sync should be less than 3 μs and

shall be less than 10 μs

WiMAX Mobile Up to ± 1 ppb

Average target : ± 15 ppb Usual values between ± 0.5µs and ± 5µs

LTE-Advanced Services ±5 ppb (CoMP) CoMP, relaying function, carrier aggregation

± 0.5 µs [± 1 µs]

Multi-Media Bcast SFN Service ± 50ppb ± 1 µs

DVB SFN Up to ± 1 ppb General agreement : ± 1 µs

TDM transmission G.823/G.824/G.8261 N/A

Network Monitoring N/A ± 1 to 100 µs ToD synchronization

for 10 µs to 1 ms measurement accuracy


Clocking Mechanism Advantages Disadvantages

GPS Reliable PRC

Relatively cheap

Frequency and phase

Antenna required

US Govt owned

PRC/BITS Reliable PRC

Generally Available

No Phase

Need to maintain TDM in all Ethernet deployment

1588-2008 Packet Based

(Frequency and Phase)

Requires Master w/ PRC

Performance influenced by network

Undefined Profiles in SP environments

SyncE/ESMC Physical layer

(Frequency)

No Phase

Every node in chain needs to support

NTPv4 Packet Based

(Frequency and Phase)

Not as robust as 1588-2008

Open standard

Some proprietary implementations

Synchronisation Requirements Clocking Mechanisms Comparison


UMMT Synchronization Distribution

Pre-Aggregation Node

ME-3800X, 3600X, ASR-903

DWDM, Fiber Rings, Mesh Topology DWDM, Fiber Rings, H&S, Hierarchical Topology Fiber or uWave Link, Ring

Core Network Mobile Access Network

Aggregation Network

Core Node

CRS-3, ASR-9000

IP/MPLS Transport

IP/MPLS Transport

Core Node

CRS-3, ASR-9000

Cell Site Gateway (CSG)

ASR-901, 2941

IP/MPLS Transport

Mobile Transport Gateway (MTG) ASR-9000

Aggregation Node

ASR-9000

Mobile Packet Core Network

IP/MPLS Transport Network

BSC, ATM RNC

Ethernet Fiber

Non-SyncE aware

SyncE, ESMC

1588 BC

PRC/PRS

External Synchronization

Interface (Frequency)

Global Navigation Satellite System (e.g. GPS, GLONASS, GALILEO)- PRTC, Primary Reference

Time Clock

TDM(SDH)

1588 PMC Packet Master Clock

Phase

External Synchronization

Interface (ToD and Phase)

1588 Phase

TDM(SDH)

SyncE

1588 PTP


A

C

E

B

D

F

2 2

10

2

1

8 4

C1

C3

C5

A2 A1

C2 C4

A1

C1

C2

C3

E1

C4

A2

Backbone

Access Region

C5

TE-FRR

Backup tunnel

NH protection

Remote-LFA

tunnel to

PQ node

http://tools.ietf.org/html/draft-shand-remote-lfa

• Simple, Minimum Configuration

• No need for additional protocols overhead like (RSVP TE)

• Simpler for capacity planning then TE-FRR RFC-5286 defines the baseline LFA-FRR










Remote LFA FRR - Protection

C2’s LIB

– C1’s label for FEC A1 = 20

– C3’s label for FEC C5 = 99

– C5’s label for FEC A1 = 21

On failure, C2 sends A1-destined traffic onto an LSP destined to C5

– Swap per-prefix label 20 with 21 that is expected by C5 for that prefix, and push label 99

When C5 receives the traffic, the top label 21 is the one that it expects for that prefix and hence it forwards it onto the destination using the shortest-path avoiding the link C1-C2.

A1

C1

C2

C3

E1

C4

A2

Backbone

Access Region

C5 Directed LDP

session

21

20

99

21 99

21 X

21


What Is PIC or BGP FRR?

Core

1

10

100

1000

10000

100000

1

25000

50000

75000

100000

125000

150000

175000

200000

225000

250000

275000

300000

325000

350000

Prefix

Lo

C (

ms)

PIC

no PIC

1

10

100

1000

10000

100000

1000000

0

5000

0

1000

00

1500

00

2000

00

2500

00

3000

00

3500

00

4000

00

4500

00

5000

00

Prefix

msec

250k PIC

250k no PIC

500k PIC

500k no PIC

Wholesale Consumer Broadband and Enterprise Services


Consumer Broadband Services

Residential

Corporate

Wireless

Cell Phones

Cable/CMTS

DSL/DSLAM

Fiber / OLT

Ethernet

Agg Switch

Mobility

Agg Switch

Agg Switch Agg Switch

MPLS/IP

Network

MPLS/IP

Network

Edge Router

Edge Router

Subscriber Access Aggregation Network Core Network

Internet

Dynamic/Controlled/Accounting Stability/Performance

25

• Centralized BNG

• Distributed BNG


Enterprise VPN and Internet Service (Option-1)

Residential

Corporate

Wireless

Cell Phones

Cable/CMTS

DSL/DSLAM

Fiber / OLT

Ethernet

Agg Switch

Mobility

Agg Switch

Agg Switch Agg Switch

MPLS/IP

Network

MPLS/IP

Network

Enterprise

Edge Router

Enterprise

Edge Router

Subscriber Access Aggregation Network Core Network

Internet

L2/L3 VLAN Pseudo-wire 802.1Q


Pseudo-wire Headend Architecture benefits:

Supports Seamless MPLS end-to-end Architecture: Flexible Edge placement

Simpler resiliency between L3 PE and aggregating network

Easy-to-operate service High-Availability through MPLS based network convergence

Eliminates operationally cumbersome VLAN hand-off

LDP Core / Internet Core

Access PE (A-PE)

e.g.: DSLAM, OLT,

U-PE

Service PE (S-PE)

e.g: PW-HE-L3-PE

PW-HE-BNG

CE

CE L3PE

Internet

Peering

Business L3

VPNs

Aggregation LDP domain


Summary

Metro Ethernet requirements fundamentally change with LTE/LTE-A and Converged Backhaul.

One Network & Many Services

– Mobile (2G/3G/LTE), Enterprise and consumer

Large Scale

– Can support 100K+ Devices

Fast Convergence

– IGP FC: Simple, sub-second, always required in all areas

– LFA FRR and Remote LFA FRR: simple <50ms Link and Node

– BGP PIC : innovation enabling BGP to scale the IGP with simplicity

Next generation transport network - Carrier Ethernet - SANOG

Documents