Top Banner
Cisco Public © 2011 Cisco and/or its affiliates. All rights reserved. 1 Štefan Kollár Consulting System Engineer, CCIE #10668 18. október 2012 Cisco Next Generation Carrier Ethernet System Technológie, produkty, architektúry a rôzne modely implementácií
58

Cisco Next Generation Carrier Ethernet System Technológie ...

May 08, 2023

Download

Documents

Khang Minh
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public © 2011 Cisco and/or its affiliates. All rights reserved. 1

Štefan Kollár Consulting System Engineer, CCIE #10668 18. október 2012

Cisco Next Generation Carrier Ethernet System Technológie, produkty, architektúry a rôzne modely implementácií

Page 2: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

The NGN Carrier Ethernet System Agenda

2  

  NGN Carrier Ethernet System Architecture Overview   The Context of Broadband Forum’s TR-101 & MEF   NGN Carrier Ethernet System Architecture Details ‒ Building Blocks and Variants ‒ Service Delivery Models ‒ Network-based High Availability ‒ Scaling with Unified MPLS

  Summary   Q and A

Page 3: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Circuit to Packet Migration

  Massive change in SP traffic make-up in next 5 years*   SP revenue shifting from circuits to packet services** ‒ 5 yrs ~80% revenue derived from packet services

‒ Packet traffic increasing at 34% CAGR*** *ACG  Research  2011,  **  Cisco  Research  2010,  ***Cisco  VNI  2011  

90+%    IP  Traffic  

Private  Line  TDM/OTN  Traffic  

Private/Public  IP Traffic

2011  

~30-­‐50%

~50-­‐70%*  

2013   2016  

Private  Line  TDM/OTN  Traffic  

Private  Line  TDM/OTN  Traffic  

20-­‐30%   0─10%

Private/Public  IP  Traffic  

Private/Public IP Traffic

70-­‐80%   90+%  Legacy  TDM  

Traffic  

3  

Page 4: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Subscriber  

Business  

Corporate  

ResidenKal  

ATM  AggregaKon  

Edge   Core  Access  

Portal   Subscriber    Database  

Monitoring   Policy    Billing  

Policy  and  Service  Control  Plane  (per  subscriber)  

Mobility  

SDH  

Mobile  

OpKcal  

L2SW  

L2SW   L2SW  

L2SW  

L2SW  

L2SW  

L1SW  

SW  

L1SW  

L1SW  

L2SE  

Ethernet

OLT

DSLAM L2SE  

L2SE  

L2SE  

L2SE   L3SE  

L3SE  

BNG  

L2SE  

L0SW   L0SW   L0SW  L0  W  

Aggregation Edge Boundary

Access Aggregation Boundary

Ethernet  AggregaKon  

MPLS Ethernet Aggregation

ATM/FR  networks    capped  and  to  be  closed  

 SONET/SDH  evolving  to    MPLS  Ethernet  and  OTN  

 Access  and  Edge  opPmized  for  MPLS  

Ethernet  

Service Provider Networks Evolution to Ethernet and MPLS

4  

Page 5: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Carrier Ethernet Transport Architecture Technical Innovations

5  

   OAM  Subsystem  

IPoDWDM    OpPcal  Network  

AggregaPon    Node  

AggregaPon  Network  MPLS/IP  

Carrier Ethernet Aggregation!Access! Edge!

AggregaPon  Node  

AggregaPon  Node  

Ethernet  Node  

STB  

VoD  

Content  Network  

TV   SIP  

PON  Node  

DSLNode    

Core    Nodes  

VoD  

Content  Network  

TV   SIP  

EMS   NMS  Portal  

AAA   Service  and  Performance  Mgmt  DHCP,  DNS  

Multiservice Core!

Core  Network  IP  /  MPLS    DistribuPon    

Node  

Corporate  Business  

Corporate  Business  

ResidenPal  

STB  

ResidenPal  

AggregaPon    Node  

DistribuPon    Node  

Mobile  

2G/3G/4G  Node  RAN  Access  Network  

MPLS/IP  

Corporate  Business  

BSC/RNC  

BSC/RNC  

 MPLS-based transport with MPLS-TP option  Cisco ASR9000, ASR1000, ASR 903, ASR 901, ME3600X/ME3800X/ME3600CX  Cisco Carrier Packet Transport , CPT50, -200, -600

 Flexible Ethernet Edge (EVC infrastructure code)

 Ubiquitous Ethernet UNI across different product lines and OSs

 Flexible Options for Subscriber Awareness  Distributed Edge, Centralized Edge, ISG for IPv4/IPv6

 Intelligent transport of video

  PIM Optimizations, MoFRR, TI-MoFRR, integrated video caching, Video Quality Monitoring

Page 6: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

L2 MPLS Transport

 Two technologies for L2 transport over MPLS: ‒ Ethernet over MPLS (EoMPLS)

Used for L2 point-to-point link over MPLS cloud No MAC learning involved

‒ Virtual Private LAN Services (VPLS) Used for multipoint L2 connections Collection of pseudowires tied together by a Virtual Forwarding Interface (VFI) MAC addresses learned on VFI Traffic forwarding based on destination MAC addresses H-VPLS, an extension of VPLS

  Ethernet VPN is a new technology for NGN L2VPN Services (not described in this breakout!)

6  

Page 7: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

MPLS

EoMPLS Overview

7  

  MPLS in the aggregation network and core   Targeted LDP session between PEs to exchange VC label   Tunnel label is used to forward packet from PE to P to PE   VC label is used to identify L2VPN circuit   Attachment Circuit (AC) can be port-based or VLAN-based (or Ethernet Flow Point

based, see later)

Pseudowire

Aggregation Node

P Aggregation

Node Access Node

FTTB CPE Access Node

FTTB CPE

LDP LDP

Targeted LDP

Attachment Circuit Attachment Circuit

P

Tunnel label

Ethernet PDU VC label

Ethernet PDU

Ethernet PDU

Page 8: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

VPLS (Virtual Private LAN Services)

8  

  Attachment Circuit (AC)—Connection to Aggregation using an Ethernet VLAN

  Virtual Circuit (Pseudowire)—EoMPLS tunnel between PEs using a full mesh

  Virtual Forwarding Instance (VFI)—A virtual L2 bridge instance that connects ACs to VCs (PWs); VFI=VLAN=broadcast domain

  RFC4761 (BGP-signalled) and RFC4762 (LDP Signalled)

  Enhanced with BGP based Autodiscovery (RFC6074)

  Scalability issues almost solved via H-VPLS and state-of-the-art NPU technology (2M MAC address/chip)

Aggregation Node

MPLS

Core VFI

VFI

VFI

Attachment Circuit

Ethernet Port or VLAN

Virtual Forwarding Instance

Eompls Virtual Circuit (Pseudowire)

Aggregation Node

Aggregation Node

Access Node Access Node

Page 9: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

IETF MPLS-TP

9  

  MPLS Transport Profile ‒  T-MPLS requirements feeding into IETF MPLS-TP enhancements:

MPLS-TP differs technologically from T-MPLS. ITU stopped work on T-MPLS. ‒  Effort to address Pt-to-Pt ATM-like transport centric networks (like ATM PVCs) ‒  Focused on connection-oriented (CO-PS) services

  Data plane—based on IETF MPLS, with restricted options ‒  No ECMP, no PHP, no LSP merging

  Control plane—static and/or dynamic ‒  Static provisioning with NMS, with standardized common functions ‒  Dynamic control plane based on GMPLS or IP/MPLS

  Key OAM enhancements ‒  GE-ACH—Generic Associated Channel to support FCAPS functions alongside transport MPLS LSP ‒  GAL—Generic-ACH Label as generic exception mechanism for LSP OAM

Provisioning and Management

CE CE PE2 PE1

Page 10: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

AggregaPon  Network  MPLS/IP  

Dark  Fibre  /  CWDM  /  DWDM  and  ROADM  

Carrier Ethernet Aggregation"

BNG  

Business  PE  

Access" Edge"

AggregaKon  Node  

DSL    

Ethernet    

Core    

VoD  

Content  Network  

TV   SIP  

EMS   NMS  Portal  

AAA   Service  and  Performance  Mgmt  DHCP,DNS  

OAM  Subsystem  

Multiservice Core"

Core  Network  IP  /  MPLS    

DistribuKon    Node  

STB  

Corporate  

STB  

STB  

ResidenKal  

Corporate  

Corporate  

Business  

Business  

Business  

ResidenKal  

ResidenKal  

2G/3G  Node  

PON  

Architecture  variants:  IP/MPLS  ETHERNET  

IP/MPLS  

IP/MPLS  MPLS-­‐TP  

MPLS-­‐TP  

Cisco  supports  the  MPLS-­‐TP  opKon  now  (CPT  Product  Line)  

NGN Carrier Ethernet System Evolution

10  

Page 11: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

NGN Carrier Ethernet Transport Direction

11  

Characteristic SONET /

SDH

Optical OTN

(ROADMs)

Electrical OTN

PBB-TE MPLS-TP IP/MPLS

Ethernet

Eline (10GE)

Eline (sub 10GE)

E-Tree

E-LAN

Legacy

F/R

ATM

TDM

IP

L3VPN

L3 Unicast

L3 Multicast

Content

General

Traffic Engineering

50ms restoration

Multiplexing Technology Time Division

Wave Division Time Division Statistical Statistical Statistical

UNI processing Limited None None Typically rich Typically rich Typically rich

Granularity VC-4 Lambda ODU Variable Variable Variable

Technology Maturity

 Cisco focuses on IP/MPLS for the Carrier Ethernet Transport architecture.

 Cisco targets MPLS-TP for the POTS and Access Networks while supporting already Ethernet Bridged Access

 Cisco also addresses MPLS to the access with Unified MPLS

Page 12: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

VLAN    -­‐802.1q  -­‐QinQ  

L3/VRF  L2,  Bridged  

VPLS  

L2,  Point  to  Point  

EoMPLS  

 H-­‐QOS  per    VLAN  

 

Flexible    VLAN  Trans-­‐  laKon  1:1  2:2  1:2  

Security  

ResidenPal  

STB  

Business  

Corporate  

ResidenPal  

STB  

Business  

Corporate  

ISG  Subscriber  Session  

H-­‐QOS  per    

Session    

Flexible  Mapping  of  subscriber  VLANs  to  services  (L2,  L3,  MPLS,  

ISG)  

VLAN  translaPon  capabiliPes  for  single  and  doubled  tagged  VLANs  

 Business  VPN  L2/L3  Bitstream  wholesale  

services      

ResidenPal  Subscriber  Sessions  with  RADIUS  based  zero-­‐touch  

provisioning  

Flexible Ethernet Services Mapping Enabling Multiservice Aggregation

12  

Page 13: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Domain Managers

Provide core information for devices and technologies Automated discovery and configuration management Network visibility

Cisco Prime Integrated Suite for Experience Lifecycle Management

13  

Optimized resource management

Design

Intelligent fulfillment Fulfill

Automated diagnostic workflows

Analyze Automated service assurance

Assure

Page 14: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 14 © 2011 Cisco and/or its affiliates. All rights reserved.

  “Migration to Ethernet-based Broadband Aggregation”

14  

Page 15: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

TR-101 Scope and Content Migration from ATM to Ethernet Broadband Aggregation

15  

  VLAN architecture   Multicast considerations   Use of a video-optimised Service Router (next to ‘traditional’ TR-59 type

BRAS)   Resilience in the Ethernet Aggregation Network   QoS in the Ethernet Aggregation Network   Ethernet OAM   Support for PPPoA and IPoA (aka interworking between XoA and XoE)

Note: TR-101 introduces the term Broadband Network Gateway (BNG) to differentiate from the legacy ‘BRAS’ term

Page 16: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco’s TR-101 Architecture From Discrete Elements

16  

Video BNG

BNG BRAS

Aggregation Node: Carrier Ethernet Switch/

Service Router with Aggregation Function

Aggregation Node: Carrier Ethernet Switch/

Service Router with Aggregation Function

BNG/BRAS Extremely Important for PPP Services/Migration/Legacy

ATM Support

Business

Residential

STB

IP/MPLS Core L2 Aggregation

with IGMP Snooping

Page 17: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco’s TR-101 Architecture Via Video Optimization

17  

Video BNG

IP/MPLS Core

BNG BRAS

Aggregation Node: Carrier Ethernet Switch/

Service Router with Aggregation Function

Aggregation Node: Carrier Ethernet Switch/

Service Router with Aggregation Function

BNG/BRAS Extremely Important for PPP Services/Migration/Legacy

ATM Support

Business

Residential

STB

L2 Aggregation

+ L3 IP/PIM-SSM

Page 18: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco’s TR-101 Architecture To Integrated Network Elements

18  

BNG BRAS

Carrier Ethernet Service Router (L1, L2, L3) Video BNG (L3 IP/PIM-SSM) + L2 Aggregation

Option to Virtualize L2 Aggregation (IP Control Layer, MPLS Techniques)

BNG/BRAS Extremely Important for PPP Services/Migration/Legacy

ATM Support

Business

Residential

STB

IP/MPLS Core L2 Aggregation

+ L3 IP/PIM-SSM SiSi SiSi

Page 19: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco’s TR-101 Architecture With Distributed Edge

19  

Carrier Ethernet Service Router (L1, L2, L3) Video BNG (L3 IP/PIM-SSM) + L2 Aggregation

Option to Virtualize L2 Aggregation (IP Control Layer, MPLS Techniques)

Business

Residential

STB

IP/MPLS Core L2 Aggregation

+ L3 IP/PIM-SSM SiSi SiSi

ISG  

Subscriber Control is integrated into the Carrier Ethernet node for PPP and IP (IPv4/IPv6) sessions

Page 20: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 20 © 2011 Cisco and/or its affiliates. All rights reserved. 20  

Page 21: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Carrier Ethernet Services Metro Ethernet Forum (MEF) Service Visualization

E-­‐LINE:  Ethernet  Private  Line  (EPL)   E-­‐LAN:  Ethernet  Private  LAN  (EP-­‐LAN)  

E-­‐LINE:  Ethernet  Virtual  Private  Line  (EVPL)   E-­‐LAN:  Ethernet  Virtual  Private  LAN  (EVP-­‐LAN)  

Page 22: Cisco Next Generation Carrier Ethernet System Technológie ...

  Broadcast - Deliver to all UNIs in the EVC but the ingress UNI

  Multicast - Typically delivered to all UNIs in the EVC but the ingress UNI

  Unicast (unlearned and learned) ‒ Typically delivered to all UNIs in the EVC but the ingress UNI if not learned

‒ Otherwise, deliver to the UNI learned for the destination MAC address

‒ Learning is important for Multipoint-to-Multipoint EVCs

  Layer 2 Control (e.g., BPDU) - Discard, peer, or tunnel

From MEF 6.1,10 technical specification

UNI   EVC1  

CE-­‐VLAN  CoS  6   Ingress Bandwidth Profile Per CoS ID 6 CE-­‐VLAN  CoS  4  

CE-­‐VLAN  CoS  2  

Ingress Bandwidth Profile Per CoS ID 4 Ingress Bandwidth Profile Per CoS ID 2

EVC2  

Port/VLAN/CoS-based

Page 23: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 23 © 2011 Cisco and/or its affiliates. All rights reserved.

  Architecture Details

23  

Page 24: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Super    Head    End    (SHE)  (NGVI)  

DCM  

Encoder   Encoder  

DCM  

L2/L3 Ethernet

or TDM

Transport Wireline

& Cable

Access and Aggregation

L2/L3 Ethernet

or TDM

Transport Wireline

& Cable

Access and Aggregation

B2B

B2DC DC2DC RA2B

B2I R2I

VPLS-VPWS

mVPN-V4/v6 IP-V4/v6

VPN-V4/v6

RR NMS

Business Edge PoP

Business Office

Business Branch

Internet

Business Edge PoP

SME / HO / Residence

Business Office

Business Branch

SME / HO / Residence

Video    Hub    Office  (VHO)  /  Regional  Head    End    (RHE)  (NGVI)  

Encoder  

DCM/VQE  

NGN System Scope

Data Center

Any to Any L2/L3 Business and Residential services Based on end to end IPNGN Architecture

Data Center

Page 25: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Architecture Components & Overview

25  

Access!

Ethernet  Node  

Ethernet    Node  

DSL  Node    

PON  Node  

Access! Carrier Ethernet Aggregation!

AggregaPon    Node  

DistribuPon    Node  

DistribuPon    Node  

AggregaPon    Node  

Core  Network  IP  /  MPLS    

Content  Network  

TV   SIP  

Content  Network  

TV   SIP  

IP Edge! Multiservice Core!

MPLS/IPoDWDM  OpPcal  Network  

Core  Node  

Core  Node  

Business  

Corporate  

Business  

Corporate  

2G/3G  RBS  

ResidenPal  

STB  

ResidenPal  

STB  

ResidenPal  

STB  

Business  

Corporate  

AggregaPon  Node  

AggregaPon  Node  

MPLS/IP/Ethernet  

BSC/RNC  

BSC/RNC  Cell  Site  Gateway  

MPLS/IP  Transport    

Transport  Deployment:    VPWS,  VPLS  

 Service  Aware  Deployment:  VPWS,  VPLS,    MPLS  VPN/IP    

HSI  Service  Edge  Node  

OpPonal  Video    Service  Edge  Node  

OpPonal  Business    Service  Edge  Node  

NMS   Service  Management  SEF   EMS  ACS   Performance  Management  AAA,  DHCP,  DNS,  

OAM  Subsystem  

Carrier  Ethernet  AggregaPon   Core  and  Edge  

CPE DSL:

•  Residential: Linksys WAG-310G •  Business: ISR x900 Ethernet:

•  Residential: Genexis •  Business: ISR x900, ME3400E, ME3600X PON:

•  Residential, Business: Wave 7 ONTs

Access ADSL2+ :

•  Alcatel-Lucent ISAM 7302 Ethernet FTTX:

•  ME3400E, ME3600X, Catalyst 4500/4900 series PON:

•  Wave7 Trident G-PON OLT Mobile RAN :

•  ASR 901

Aggregation / Distribution

Cisco ASR9k

•  RSP 440

•  Typhoon LCs: 24 x 10GE, MOD80, MOD160, 2 x 100 GE

•  “Legacy” LCs: 40xGE, 4x10GE, 8x10GE

•  ASR 9001

•  Clustering and Satellite

•  Distributed BNG Services

Cisco CPT200, -600

Cisco ME3800X, Cisco ASR 903

Multiservice Edge Business SEN:

•  ASR9k: 4x10GE, 40xGE, 24x10GE, MOD80, MOD160

HSI-SEN :

•  ASR1k: RP2, ESP-20, ESP-40

Video SEN:

•  Cisco 7609S: RSP-720, ES+

• ONS15454 MSTP with WSON

• Xponders for direct Ethernet connectivity

OpPcal  IntegraPon  

Multiservice Core

• Cisco CRS-1/3

• Cisco Prime 3.8, Activation,Monitoring and Fault Management systems.

• Cisco Access Registrar, Cisco Network Registrar • CNS-Config Engine r3.0, BAC 3.5 • 3rd Party platforms from BroadHop, InfoVista VIN-ANA.

Network  &  Service  Management,  OAM  

Page 26: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

IP NGN Services All Validated in Release 1.8

26  

Market Services Access SLA Type SLA Example Residential Internet Access Ethernet, DSL,

PON Transport Dynamic access bandwidth, session/idle timeout, advertisements, post paid/prepaid (time and volume)

VoIP Telephony Ethernet, DSL, PON Application The number of VoIP appliances, SIP URLs/PST Phone numbers, active calls, VoIP call

quality VoD Ethernet, DSL,

PON Application The number of STBs, stream quality, content flavours, charging models TV Ethernet, DSL,

PON Application The number of STBs, type of TV packages, SD vs HD content and delivery quality Business L3 VPN Ethernet, DSL,

PON Transport Access bandwidth, differentiated services support, L3 VPN topology, managed services (unicast and multicast)

E-Line Ethernet, DSL, PON Transport Access bandwidth, differentiated services support, transparency

E-LAN Ethernet, DSL, PON Transport Access bandwidth, differentiated services support, multipoint transport, transparency

Transport Mobile RAN 2G, 3G R99, 3G R5, R8 Transport Guaranteed bandwidth, delay and jitter synchronization (frequency and phase)

accuracy inline with Mobile Radio technology

HSI Wholesale Ethernet,

DSL, PON Transport Aggregated bandwidth on ISP level, differentiated services support, with subscriber management at ISP, with L2TP or MPLS VPN transport

Triple Play Wholesale Ethernet,

DSL, PON Transport Aggregated bandwidth on ISP level, differentiated services support, transparent P2P Ethernet transport for unicast services, P2MP Ethernet transport for IPTV

Contribution Video Ethernet, Video HD-SDI Transport Guaranteed bandwidth, delay, jitter , and close to zero or zero loss

Page 27: Cisco Next Generation Carrier Ethernet System Technológie ...

Video Service Edge •  Implemented on Aggregation Node •  Layer-3 MPLS/IP unicast VoD and multicast IPTV

transport for video service distribution

HSI/VoIP Services Edge •  Implemented on Centralized BNG •  IPoE and PPPoE service transport over 802.1Q and QinQ

interfaces enabled by per subscriber ISG sessions 27  

Hybrid (Centralized) Service Edge

DSL    Access  Node    

Access!

PON  Access  Node  

Ethernet    Access  Node  

AggregaPon  Network  MPLS/IP  

Carrier Ethernet Aggregation!

DistribuPon  Node  

DistribuPon  Node  

AggregaPon    Node  

AggregaPon  Node  

AggregaPon  Node  

Core  Network  IP  /  MPLS    

VoD  

Content  Network  

TV   SIP  

Multiservice Core!

Business  

Corporate  

ResidenPal  

STB  

ResidenPal  

STB  

Business  

Corporate  

Business  

Corporate  ResidenPal  

STB  

Business  

Corporate  

MPLS/IPoDWDM  OpPcal  Network  

BNG  

Ethernet    Access  Node  

BNG  

Core  

Core  

Internet  Peering  

Core  

BSC/RNC  

BSC/RNC  

HSI/VoIP Service Edge

IP Edge!

Video Service Edge

ASR1000 series:

Up to 64k sessions H-QoS FW, DPI,CGN

Page 28: Cisco Next Generation Carrier Ethernet System Technológie ...

  MPLS/IP Packet Aggregation for 3play Service Delivery

28  

Video Service Edge

  Implemented on Centralized Video-BNG

  Layer-2 VPLS transport of unicast VoD and multicast IPTV for video service distribution

Centralized Service Edge (with L2 Aggr.)

DSL    Access  Node    

Access!

PON  Access  Node  

Ethernet    Access  Node  

Carrier Ethernet Aggregation!

DistribuPon  Node  

DistribuPon  Node  

AggregaPon    Node  

AggregaPon  Node  

AggregaPon  Node  

Core  Network  IP  /  MPLS    

VoD  

Content  Network  

TV   SIP  

Multiservice Core!

MPLS/IPoDWDM  OpPcal  Network  

HSI-­‐BNG  

Ethernet    Access  Node  

Video-­‐BNG  

Core  

Core  

Internet  Peering  

Core  

BSC/RNC  

BSC/RNC  

VFI!

Video Service Edge

HSI/VoIP Service Edge

IP Edge!

Business  

Corporate  

ResidenPal  

STB  

ResidenPal  

STB  

Business  

Corporate  

Business  

Corporate  ResidenPal  

STB  

Business  

Corporate  

VFI!VFI!

VFI!

HSI/VoIP Services Edge   Implemented on Centralized HSI-BNG   IPoE and PPPoE service transport over

802.1Q and QinQ interfaces enabled by per subscriber ISG sessions

Page 29: Cisco Next Generation Carrier Ethernet System Technológie ...

Distributed Service Edge

3Play Service Edge   Implemented on Integrated Edge Node   Unicast services (HSI/VoIP/VoD) enabled by IPoE or PPPoE per subscriber ISG sessions   Multicast services (IPTV) coexist with ISG sessions   Aggregation network implements MPLS/IP for unicast and IP multicast for service transport

  MPLS/IP Packet Aggregation for 3play Service Delivery

29  

DSL    Access  Node    

Access!

PON  Access  Node  

Ethernet    Access  Node  

AggregaPon  Network  MPLS/IP  

DistribuPon  Node  

DistribuPon  Node  

Integrated  Edge  Node  

Integrated  Edge  Node  

Integrated  Edge  Node  

Core  Network  IP  /  MPLS    

VoD  

Content  Network  

TV   SIP  

Optional L3VPN Edge!

Multiservice Core!

MPLS/IPoDWDM  OpPcal  Network  

Ethernet    Access  Node  

Core  

Core  

Internet  Peering  

Core  

BSC/RNC  

BSC/RNC  

Video/HSI/VoIP Integrated Service Edge

Business  

Corporate  

ResidenPal  

STB  

ResidenPal  

STB  

Business  

Corporate  

Business  

Corporate  ResidenPal  

STB  

Business  

Corporate  

Carrier Ethernet Aggregation!

C7600

Based on ES+ Up to 48K sessions Limited IPv6 roadmap

ASR9000: Up to 128K+ Sessions Full IPv6 feature set Very good scalability in combined BNG +MSE+CE apps. IOS-XR

Page 30: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Architecture Comparisons Which one to choose?

30  

The architectures options can be evaluated against the following criteria •  Capital Expenditures •  Scalability (Bandwidth / Subscriber, Transport, Policy Control)

•  Operational Complexity (Troubleshooting, QoS) •  Reuse of existing Operations procedures

•  Availability •  Traffic Patterns •  Economically serving areas of differing subscriber density

•  Service Flexibility •  Operational Flexibility

Page 31: Cisco Next Generation Carrier Ethernet System Technológie ...

31  

Residential Services HSI,  VoIP  VLAN(s)  

EoMPLS  Pseudowire  EoMPLS  PW  

VoD+IPTV  ,  VoIP  VLAN  

802.1Q    QinQ  

N:1  VLAN  

Non/Trunk  UNI,  N:1  or  1:1  VLAN  

MPLS/IP,  IP  MulPcast,  IP  LFA,  MoFRR  

MPLS/IP  MPLS/MulPcast  VPN  

ISG  Sessions  

Enables PPPoE to IPoE migration, usage based services with service and session control, DPI and SBC

May include service supporting functions; Content Cache, FCC, RET, VoD CAC

Retail 3Play Hybrid Edge Deployment

HSI,  VoIP  VLAN(s)  EoMPLS  Pseudowire  EoMPLS  PW  

VoD+IPTV  VLAN  

802.1Q    QinQ  

N:1  VLAN  

Trunk  UNI,  N:1  or  1:1  VLAN  

MPLS/IP  MPLS/MulPcast  VPN  

ISG  Sessions  Retail or Wholesale 3Play Centralized Edge deployment

802.1Q    QinQ  

IP,  PIM  

HSI  SEN  

Video  SEN  

HSI  SEN  

Mul$service  Core  Network  

AggregaPon  Node  ASR9k,  7600,  ME3800X   Video  SEN,  7600  

PPP,  IP,  MPLS   MPLS  802.1ad  NNI,  MPLS/IP  Transport  DSL,  PON,  Ethernet  

Access  Node  

HSI  SEN,  ASR1k  

DistribuPon  Node  ASR9k,  7600  

Large  Scale  AggregaPon  Network  

Intelligent  Services  Edge  

Efficient    Access  Network  

Ethernet  UN

I    

 Ethernet/MPLS  NNI    

Core  Node  CRS-­‐1/3  

Service  Aware  or  Transport  VPWS,  VPLS,  MPLS/IP  

H-­‐VPLS,  IGMP  Snooping,  CAC  

Page 32: Cisco Next Generation Carrier Ethernet System Technológie ...

Gateway for VOD/IPTV

 EoMPLS  PW  

EoMPLS  PW  

VFI  

VFI  

EoMPLS  PW  

Active/Backup PW

EoMPLS  PW  

EoMPLS    PW  

EoMPLS    PW  

VRRP   HSI  SEN  PPPoE/IPoE      Session  load-­‐sharing  

SVI/BVI  

SVI/BVI  

AcKve/Standby  Link  redundancy  Flexlink  or  LACP  

I  C  C  P  

MPLS/IP Aggregation Network

Access Aggregation Distribution HSI SEN

4500 Access

§ Dual-homed with LACP

§ DHCP snooping, Opt-82

§ IPSG, DAI

§ IGMP snooping

ASR 9000 Aggregation

§ mLACP access redundancy

§ IRB/BVI for Video Edge

§ ICCP IGMP sync

§ L2 IGMP CAC

ASR 1000 Internet-BNG

§ ISG IPoE sessions

§ ISG PPPoE sessions

§ Split-subnet redundancy

mLACP

Page 33: Cisco Next Generation Carrier Ethernet System Technológie ...

4500 Access

§ Dual-homed with LACP

§ DHCP snooping, Opt-82

§ IPSG, DAI

§ IGMP snooping

ASR 9000 Aggregation

§ mLACP access redundancy

§ Sub-interface coupled-mode for Video Edge

ASR 1000 Internet-BNG

§ ISG IPoE sessions

§ ISG PPPoE sessions

§ Split-subnet redundancy

Active GW for VOD/IPTV

LACP Active/Standby

VFI  

mLACP

Standby GW for VOD/IPTV

I  C  C  P  

HSI  SEN  PPPoE/IPoE      Session  load-­‐sharing  

Sub  I/F  

Sub  I/F  

VFI  

VFI  

VFI  

EoMPLS  PW  

EoMPLS  PW  

MPLS/IP Aggregation Network

Access Aggregation Distribution HSI SEN

Page 34: Cisco Next Generation Carrier Ethernet System Technológie ...

Business Services Architecture

34  

E-LINE

E-LAN H-­‐VPLS  or  VPLS  

EoMPLS  

Port,  1q,  QinQ    

Port,  1Q,  QinQ  or  .1ad  

Port,  1Q,  QInQ  or  .1ad    

L3 VPN

Ethernet      

QinQ  

Port,  1Q,  QInQ  

MPLS  VPN  

VPLS    

MPLS  VPN/MulPcast  VPN  (GRE)  

H-­‐VPLS  or  VPLS  

MPLS  VPN  

Centralized Edge Deployment

L3 VPN

L2, L3 VPNs SONET/SDH Access

SONET/SDH  Access  

STM4  OC12  

Ethernet    MPLS  VPN  

VPWS  (FR,  IP)  MSE  

E-­‐MSE  

Mul$service  Core  Network  

AggregaPon  Node  ASR9k,  7600,  ME3800X   Video  SEN,  7600  

PPP,  IP,  MPLS   MPLS  802.1ad  NNI,  MPLS/IP  Transport  DSL,  PON,  Ethernet  

Access  Node  

HSI  SEN,  ASR1k  

DistribuPon  Node  ASR9k,  7600  

Large  Scale  AggregaPon  Network  

Intelligent  Services  Edge  

Efficient    Access  Network  

Ethernet  UN

I    

 Ethernet/MPLS  NNI    

Core  Node  CRS-­‐1/3  

Service  Aware  or  Transport  VPWS,  VPLS,  MPLS/IP  

Ch  E1/T1  E3/T3,  

MLPPP/FR  

Distributed Edge Deployment

Page 35: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 35 © 2011 Cisco and/or its affiliates. All rights reserved. 35  

Page 36: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Network Availability Mechanisms Baseline

36  

  Access Mechanisms

‒  Multiple Spanning Tree (MST) or MST Access Gateway

‒  Resilient Ethernet Protocol (REP)

‒  Multi-Chassis LACP

  MPLS Services: ‒  VPLS mac-address withdrawal; MST/REP

and VPLS interworking ‒  Pseudowire redundancy including

pseudowire status bit support

Large  Scale  AggregaPon  

Intelligent  Edge  

DistribuPon  Node    

BNG  

MPLS  PP,  IP,  MPLS  MPLS-­‐TP/MPLS/IP  

AggregaPon  Node    

BNG  

Access  Node  

Efficient  Access  

DSL,  Ethernet  

     

MulPservice  Core  

  IP Services and MPLS IGP:

‒  IP Fast Convergence

‒  LFA / IP FRR

‒  Multicast Fast Convergence, MoFRR

Page 37: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 37 © 2011 Cisco and/or its affiliates. All rights reserved.

  Layer 2 Mechanisms

37  

Page 38: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Ethernet Access Topologies Ring and Hub and Spoke

38  

  Ethernet Access Rings Multiple Spanning Tree

  Convergence Dependant on Type of failure (e.g. root vs. link)

  Often non-deterministic

  Hub and Spoke FlexLink or Link Aggregation

  Fast Convergence independent of VLANs/MAC-addresses

IP/MPLS IP/MPLS

Page 39: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

VFI

VFI

VFI

VFI

I’m the second-best root

I’m just in a normal STP ring

I’m the root

MST Access Gateway Operation and Benefits

39  

  Operation ‒  Top PE sends “pre-configured” BPDUs (best root) into L2 access network ‒  Access network runs normal MSTP, MSTP is terminated locally on the PE access ports ‒  MSTP TCNs trigger VPLS MAC Flush + Withdraw

‒  MST instances have per port local significance – greatly improves scalability ‒  Only subset of functionality needed for REP Access Gateway

  Benefits ‒  Seamless integration with any L2 access network or node running MSTP, full standard compliance

‒  Inherent scalability and faster L2 convergence due to local Rapid STP behaviour

Page 40: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

  A new protocol designed to provide a solution for fast and predicable Layer 2 convergence for Carrier Ethernet networks

  Fast and predictable convergence ‒ Convergence time: 50 to 250ms ‒ Fast failure notification even in large rings

  Limit the scope of Spanning Tree ‒ STP is deactivated on REP interfaces ‒ STP TCN sent away from the segment if segment fails

  Allows VLAN load balancing for optimal bandwidth utilization   Cisco proprietary (future alignment and interworking with ITU-T G.8032)

40  

What Is Resilient Ethernet Protocol (REP) ?

Page 41: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

REP A Segment Protocol

41  

  REP guarantees there is no connectivity between two edge ports on a segment

  A REP segment is a chain of ports connected to each other and configured with a segment ID

  When all interfaces in the segment are UP, the alternate port is blocking   When a link or switch failure occurs on the segment, then blocked port

goes forwarding

REP Segment

Blocked Open Alternate Port Link

Failure Edge Port Edge Port

Page 42: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

  Enhancement to REP introduced in latest Ethernet Access Node releases   Allows interconnection of REP segments with STP/VPLS domains or boxes

with no REP support(EVC)

42  

REP Edge No Neighbour

REP Segment

Blocked Open Alternate Port

Link Failure

Edge Port Edge Port

Non REP Domain STP TCN

Page 43: Cisco Next Generation Carrier Ethernet System Technológie ...

Multi-Chassis Link Aggregation   Inter Chassis Control Protocol

43  

  MC-LAG & ICCP enable a switch/router to use standard Ethernet Link Aggregation for device dual-homing, with active/standby redundancy

  Dual-homed Device (DHD) operates as if it is connected to single virtual device and runs IEEE std. 802.1AX-2008 (LACP)

  Point of Attachment (PoA) nodes run Inter-chassis Communication Protocol (ICCP) to synchronize state & form a Redundancy Group (RG

Inter-­‐chassis  CommunicaKon  Protocol  (ICCP)  

Redundancy  Group  (RG)  

DHD  

Standby  PoA  

AcPve  PoA  

MC-­‐LAG  

LACP  

Page 44: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Network-based HA Example 1/3 Two-Way P2P PW Redundancy with MC-LAG

44  

  Both sides must run MC-LAG   Bundle member port state decide PW redundancy state   Active POA send active PW status to remote Router. Standby POA send

standby PW status. PW become active ONLY if local and remote Routers are both active. The rest of 3 PWs are in standby mode

S S

A A

LACP LACP ICCP ICCP

Standby POA-2

Active POA-3 Active POA-1

Standby POA-4

Active PW

Standby PW

Page 45: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Network-based HA Example 2/3 H-VPLS Spoke (P2P PW) – coupled & “one-way

45  

  The remote VFI Routers don’t have to run MC-LAG. If it run MC-LAG, it need to be in “decouple mode”

  Bundle/POA status decide the PW status. On active POA, it will send active PW status on its primary PW and standby status on its backup PW. On the standby POA, it will send standby PW status on both of its primary and backup PW

  The spoke PW is P2P PW

S

A

LACP ICCP

Standby POA

Active POA

Active PW

Standby PW

MPLS

VFI  

VFI  

Page 46: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Network-based HA Example 3/3 L3 Service – IRB/BVI(asr9k) or SVI(7600) decoupled mode

46  

  Configure L2 sub-interface 2 PW between two POA. Both L2 sub-interface and L2 PW are in the same bridge-domain. Configure IRB/BVI for the bridge-domain for the L3 service

  L3 features like HSRP, VRRP, routing, etc are configured under BVI interface

  BVI interfaces are up on both POA regardless of the bundle status

  Bundle failover only impact the bundle itself. BVI and related L3 topology is not aware fast L3 convergence

  Asr9k - IRB/BVI feature supported from 4.0.1 release

  On the bundle, and then configure LDHD configuration option 1: DHD can have default IP gateway pointing to HSRP/VRRP virtual IP address. POA need to configure HSRP/VRRP under BVI interface

  Option 2: DHD can also run IGP with both POA. Routing session will be up with both POAs

  Bundle/POA failover won’t cause the L3 topology change

LACP ICCP

Standby POA

Active POA

MPLS/IP

BD  

BD  

BVI  

BVI  

Page 47: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 47 © 2011 Cisco and/or its affiliates. All rights reserved.

  Layer 3 Mechanisms

47  

Page 48: Cisco Next Generation Carrier Ethernet System Technológie ...

  “Simplicity is prerequisite for reliability” Edsger Dijkstra

  "Simplicity is the ultimate sophistication" Leonardo da Vinci

  Kiss: Keep It Simple Straighforward

48  

IP FRR: The Principle of Simplicity

Gains  

Page 49: Cisco Next Generation Carrier Ethernet System Technológie ...

IP Fast ReRoute   Theory & Terminology

49  

  Path: Outgoing interface and next hop   Backup: an outgoing interface/nhop which is used to replace another one that went

down. It can be: ‒  another primary ECMP nhop ‒  a secondary LFA routing path

  LFA: Loop-Free Alternate ‒  N is an LFA for S’s primary path to D via F if ND < NS + SD ‒  Node-protecting LFA if: ND < NF + FD ‒  Downstream LFA if: ND < SD

  Computation of LFA occurs after calculating the primary path, therefore IGP FC performance is not affected

  Integrated with LDP   Because LFA is precomputed and installed in the FIB, it provides deterministic

protection(<50ms)

Page 50: Cisco Next Generation Carrier Ethernet System Technológie ...

IP Fast ReRoute   Per-Prefix LFA Algorithm

50  

  For IGP route D1, S’s primary path is link SF.   S checks for each neighbor N (<>F) whether ND1 < NS + SD1 (Eq1) ‒  “does the path from the neighbor to D1 avoid me?” ‒  If so, it is a loop-free alternate (LFA) to my primary path to D1 ‒  C is an LFA for D1, E is an LFA for D2

S   F  

C  

E  

D1  

D2  

Page 51: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Introducing Multicast Only Fast ReRoute MoFRR

51  

  PIM Pre-Signalling of two independent joins ‒  router is connected to the source via

two disjoint branches (requires two plane design)

  Upon failure detection, switch-over from primary to backup branch ‒  IGP detection: order of x00msec ‒  local detection or passive heartbeat:

50msec ‒  RTP sequence monitoring: zeroloss

IPTV source

Pop1

Pop2 PopN

Page 52: Cisco Next Generation Carrier Ethernet System Technológie ...

Cisco Public 52 © 2011 Cisco and/or its affiliates. All rights reserved. 52  

Page 53: Cisco Next Generation Carrier Ethernet System Technológie ...

  Scale - Interconnect 100k Access nodes through an MPLS domain   Resilience - < 50msec convergence as often as possible   Simplicity - Operation of big MPLS networks is often considered difficult

53  

Problem Statement

1k  Nodes  /  Core  

10k  Nodes  /  AggregaPon  

100k  Nodes  /  Access  

Reference  Model  

IGP2   IGP1   IGP3  

DSLAM1  

PE11  

PE12  

ABR11  

ABR12  

ABR21  

ABR22  

PE21  

PE22  

DSLAM2  

Core  and  Edge  DistribuPon  /  AggregaPon  

DistribuPon  /  AggregaPon  

Page 54: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Unified MPLS Solution

54  

  Layer of hierarchy to scale ‒  RFC 3107-based hierarchical LSPs over IGP ‒  IGP/LDP inter-area summarization

  ABRs are BGP speakers (next-hop-self)   ABRs are Route Reflectors.   Further RR hierarchy can be used to avoid full mesh iBGP connectivity

among ABRs   BGP’s applicability to scale PE’s reachability with was made possible by

two key innovations: ‒ BGP Prefix Independent Convergence (BGP-PIC): ‒ BGP additional-path

Page 55: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Unified MPLS BGP Routing and Features

55  

  BGP Add-Path ‒  PE11 learns two paths to PE21: via ABR11 and ABR12

  BGP 3107 RR with next-hop-self ‒  ABR21 reflecting the path to D2 ‒  ABR11 reflecting the path to D2

iBGP3107 PE21 and D2 via ABR21

iBGP3107 PE21 and D2 via ABR11

L1   L2   L1  

D1  

PE11  

PE12  

ABR11  

ABR12  

ABR21  

ABR22  

PE21  

PE22  

D2  

Page 56: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

Unified MPLS Label Stacks and Label Allocation

56  

  Each IGP area has routes for that area only plus routes to core ABRs ( ~1k prefixes)   LDP labels are used to traverse each area and reach core ABRs   BGP labels are used by PEs and ABRs to reach PEs in remote areas   Service (e.g. PW) labels are used by PEs

IGP/LDP  Label  

BGP3107  Label  

Service  Label  

iBGP3107 PE21 and D2 via ABR21

iBGP3107 PE21 and D2 via ABR11

L1   L2   L1  

D1  

PE11  

PE12  

ABR11  

ABR12  

ABR21  

ABR22  

PE21  

PE22  

D2  

NH: ABR21

Label: L1

NH: ABR11

Label: L2

L2   L2   L1   L1  

Page 57: Cisco Next Generation Carrier Ethernet System Technológie ...

  BGP Fast Reroute (BGP FRR)—enables BGP to use alternate paths within sub-seconds after a failure of the primary or active paths

  PIC or FRR dependent routing protocols (e.g. BGP) install backup paths

  Without backup paths

Convergence is driven from the routing protocols updating the RIB and FIB one prefix at a time - Convergence times directly proportional to the number of affected prefixes

  With backup paths

Paths in RIB/FIB available for immediate use

Predictable and constant convergence time independent of number of prefixes

Page 58: Cisco Next Generation Carrier Ethernet System Technológie ...

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public

UMMT High Availability Overview

AggregaKon  Node  

AggregaKon  Node  

AggregaKon    Node  

AggregaKon  Node  

AggregaKon  Node  

AggregaKon    Node  

 

 

 

 

 

 

RAN  IGP  Process  OSPF/  ISIS  

Core  

Core  

Core  

Core  

LDP LSP ! LDP LSP ! LDP LSP ! LDP LSP ! LDP LSP !

iBGP Hierarchical LSP!

AggregaKon  Domain    (OSPFx/ISIS1)  

RAN    Access  

Core  Domain  OSPF0/ISIS2  

iBGP  

AggregaPon    BGP  Community  

iBGP  

AggregaPon    BGP  Community  

iBGP  

iBGP  IPv4+Label  

 RR  

AggregaKon  Domain    (OSPFx/ISIS1)  

RAN  IGP  Process  OSPF/  ISIS  

RAN    Access   Core  

Redistribute MPC iBGP community

into RAN Access IGP

Redistribute CSN Loopbacks into 3107 iBGP

MPC    PE  

LFA  L3  convergence  <  50ms  

BGP  PIC  Core  L3  convergence  <  100ms  BGP  PIC  Edge  L3  convergence  <  100ms