Cat4500 QOS CENIC

Cisco Catalyst 4500 Quality of ServiceJohn Bartlomiejczyk [email protected] Gigabit Systems Business Unit

QoS Technical Update June 2004

2002, Cisco Systems, Inc. All rights reserved

1

Cisco Catalyst 4500 IOS Supervisor OptionsSupervisor V Optimized for Large Networks (Premium HW and SW Services) Support for Higher Port Densities (Catalyst 4510R) Advanced Layer 3 Switching/Routing (OSPF, EIGRP, IS:IS) Highly Scalable Layer 2/3/4 Services Supports Up to 10 Active Slots96Gbps + 72Mpps Redundancy Support in 4507R and 4510R Chassis Catalyst 4503, 4506, 4507R, 4510R, and 4006 Chassis Cisco IOS-Based Optimized for Medium Networks Advanced Layer 3 Switching/Routing (OSPF, EIGRP, IS:IS) Scalable Layer 2/3/4 Services Supports Up to 5 Active Slots64Gbps + 48Mpps Redundancy Support in 4507R Chassis Advanced Layer 3 Switching Catalyst 4503, 4506, 4507R and 4006 Chassis Cisco IOS-Based Optimized for Smaller Networks Basic Layer 3 Switching/Routing (RIP and Static) Layer 2/3/4 Intelligent Services Supports Up to 5 Active Slots64Gbps + 48Mpps Redundancy Support in 4507R Chassis Catalyst 4503, 4506, 4507R and 4006 Chassis Cisco IOS-Based 2002, Cisco Systems, Inc. All rights reserved

Catalyst 4500 Series Cisco IOS-Based Supervisors

Optional NetFlow Daughter Card

Supervisor IV

Optional NetFlow Daughter Card

Supervisor II-Plus


2

Cisco Catalyst 4500 Series Chassis Specs

Catalyst 4503 Sup Redundancy Slots Ports (max) Dimensions (RUs) Chassis/19 rack Power Supplies Supervisors n/a 3 96+2 7 6 1+1 Sup II and higher

Catalyst 4506 n/a 6 240+2 10 4 1+1 Sup II and higher

Catalyst 4507R Catalyst 4510R Yes 7 240+4 (SupV) 11 4 1+1 Sup II+/IV/V Yes 10 336+6 (SupV) 14 3 1+1 Sup V3



Is QoS needed in the Campus ?

throw more bandwidth Justat it. That will solve the problem!Maybe, maybe not. Campus congestion is a buffer management issue.

4



Anatomy of a Campus DesignTCP Traffic Burst + VoIPCoreSi Si

Instantaneous Interface Congestion

Distribution

Si

Si

Access



5

Result

Even though the average link utilization is below 100%, buffers may still fill up and packets droppedQoS Technical Update June 2004

Packets that made it through. Rest are dropped BuffersLink Utilization 60%

Example: 100 Mbps Link

Packets from different Applications


6

Conclusions

Buffers can congest in LANs QoS required when there is congestionin buffers

Buffer Management can help reduce loss Buffering reduces loss but delaysensitive application could be negatively impacted



7

QoS Terminology QoS labels are used to prioritize trafficCOS, TOS, DSCP

Classification is selection of traffic based on labels, policy Marking is application of QoS labels to traffic Policing is process by which the switch limits the bandwidth consumed by a flow of traffic Queuing is placing of traffic in different transmit queues Scheduling is process of emptying the transmit queuesRST-3508 9805_05_2004_c1 2004 Cisco Systems, Inc. All rights reserved.

8

Layer 2 and 3 Traffic ClassificationLayer 2 802.1Q/pPREAM. SFD DA SA Typ e TAG 4 Bytes PT DATA FCS

Three Bits Used for CoS (802.1D User Priority) PRI CFI VLAN ID

Layer 3 IPV4Version Length ToS 1 Byte 6 IP Precedence Len ID Offset TTL Proto FCS IP-SA IPIP-DA IPData

7

5

4

3

2

1

0

Unused Bits; Flow Control for DSCP DSCP Standard IPV4: Three MSB Called IP Precedence (DiffServ May Use Six D.S. Bits Plus Two for Flow Control) 2002, Cisco Systems, Inc. All rights reserved


9

DiffServ Behaviors (RFCs: 2474, 2475, 2597 & 2598)Per-Hop Behaviours (PHB)Expedited Forwarding Assured ForwardingClass Selector (CS) 1 Class Selector (CS) 2 Class Selector (CS) 3

DiffServ Code Points (DSCP)101110

EF

Low Drop Pref

Med Drop Pref

High Drop Pref

AF11 AF21 AF31 AF41

AF12 AF22 AF32 AF42

AF13 AF23 AF33 AF43

001010 001100 001110 010010 010100 010110 011010 011100 011110 100010 100100 100110 000000

Class Selector (CS) 4

Best EffortQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

10

S2

Catalyst 4500 QoS CapabilitiesSupervisor II Layer 2 only System-wide QoS Dual Tx Queues per PortQueue 1

Cisco IOS-Based Supervisors IOSLayer 2, 3, or 4 QoS Per-port QoS Four Tx queues per port Strict priority queue Dynamic queue memory allocation Packet classification and marking Policing/bursting Shaping/sharing Queue 4 Queue 3

Queue 2

Queue 2 Queue 1QoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

11

Predictable Performance 48 Mpps/64 Gbps L2/L3/L4 switching Wirespeed High Touch Services on every port with no performance hit: 32K QoS ACL entries* 32K Security ACL entries* 2,048 Policers * (16Kbps-1Gbps per port) 4 Queues per port 128K uni/multi-cast addresses* 4,096 802.1Q/ISL VLANs* * Lower on Supervisor II-PlusQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

12

Cisco IOS-Based Supervisor QoS Flow SummaryClassification based on: Default DSCP port setting Port Trusted COS or DSCP Layer 2/3/4 ACLs Policing via ACLs Police Action: Mark Drop Based: Byte rate Burst (Token Bucket)

Sharing and Shaping and Strict Priority Q3 to Schedule between Output queues

Queue 1

RX

Shared Memory

Classify

Police

DBL

Rewrite Info

Queue 2 Queue 3 Queue 4 Sched TX

In-coming Encapsulation Can be 802.1Q, 802.1p, ISL, or none

Dynamic Buffer Limiting (Supervisor II Plus Supervisor IV Supervisor V ) Congestion Avoidance

Rewrites TOS Field in IP Header and 802.1p/ISL CoS Field

Out-going Encapsulation Can be 802.1Q, 802.1p, ISL, or none



13

Example Queue Configuration Queue 3 voice / router controlneeds low latency

Queue 2 important TCP Queue 1 less important TCP Queue 4 videoguaranteed bandwidth

Multiple queues must share the linkQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

14

QoS Examples Rewrite all packets from a specific host with high-priority DSCP Trust DSCP of VOIP packets and place in Strict Priority Tx Queue Police all multicast to 30 mbps Mark down vlan 7 traffic beyond 50mbps Put video in Tx queue 2 and shape to 20mbpsQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

15

QoS Syntax : Modular QoS CLI Various Criteria to match packets (ACL, IP Prec, IP-DSCP) If a packet matches criteria, it is a member of this class. Associates a Class with a particular function or policy For Catalyst 4500 IOS Supervisor, a policy is used to trust traffic, mark traffic, or police traffic Applies the service policy to a particular interface Effectively commits the policy to the hardware16

Define Class

Create Policy

Assign Interface



Modular QoS Configuration Example Match all video on Gig2/1 Set its DSCP to 5 Police it to 100mbps and 120 kbps burst Markdown traffic exceeding policed rateQoS Technical Update June 2004

access-list 101 udp permit any any range 19000 19010 class-map video match access-group 101 policy-map vid-server class video set ip precedence 5 police 100m 15K exceed-action policed-dscp-transmit interface Gig2/1 service-policy input vidserver17


Cisco Catalyst 4500 Architecture 3-slot, 6-slot chassisone supervisor with two or five module slots 7-slot chassisone or two supervisors with five module slots 10-slot chassisone or two supervisors (Supervisor V only) with eight module slots Cisco IOS supervisors provide:Central forwarding engine (Fast Forwarding Engine, FFE) Buffering and 64 Gbps fabric (Packet Processing Engine, PPE)96 Gbps fabric with Supervisor V and PPE2

Forwarding Engine (FFE) Shared Memory Fabric (PPE)12 Gbps 12 Gbps 12 Gbps 12 Gbps 12 Gbps

Switching Module

Switching Module

Switching Module

12 Gbps bandwidth to each module Auto MDIX on 10/100/1000 Ports Modules are transparent:Contain simple stub ASICs, PHYs No buffering or local switchingRST-3508 9805_05_2004_c1 2004 Cisco Systems, Inc. All rights reserved.

Switching Module

Switching Module

Note: Supervisor Engine V Supports 3 Additional Line Card Slots18

Catalyst 4000/4500 Linecards 6 full-duplex GbE connections to switch fabric TransparentNo local forwardingall packets go to supervisor

GbE connections from switch fabric straight to front-panel port or connect to stubs6 Full-duplex Gbps Connections to Supervisor Switch Fabric



19

Stub ASIC Overview Fans out GigE ports from switch fabric Up to 8 front-panel ports; 10/100, 1000-only, or 10/100/1000 Flow control on gigabit interfaces Ports can be used in an EtherChannel Not always oversubscribed e.g. 10/100 2004 Cisco Systems, Inc. All rights reserved.

GbE to/from Switch Fabric

Up to 8 Front-Panel Ports, 10/100/100020

RST-3508 9805_05_2004_c1

Blocking and Non-Blocking PortsNon-Blocking Gigabit Line Cards Blocking Gigabit Line Card All Ports on the WSX4424-GB-RJ45 All Ports on the WSX4448-GB-RJ45 WS-X4548-GB-RJ45V All Ports on the WSX4448-GB-LX Last 16 Ports on the WS-X4418-GB 1000 BT Ports on the WS-X4412-2GB-TX All ports on the WSX4424-GB-RJ45 Oversubscripti on Ratio for Blocking Line Cards 4:1 8:1 8:1 8:1 4:1 4:1 4:1

Supervisor Uplink Ports WS-X4306-GBAll Ports Two 1000 Base-X Ports on the WS-X4232-GB-RJ First Two Ports on WS-X4418GB WS-X4302-GBBoth Ports Oversubscribed GbE modules are ideal for deployments that are more bursty in nature such as Gigabit to the Desktop and Servers These interfaces are not recommended for uplinks or sustained connectionsRST-3508 9805_05_2004_c1 2004 Cisco Systems, Inc. All rights reserved.

21

Transmit Queue Sizes, Buffers

packet buffers 240 / 1920 entries For Supervisor Engines IV and II-Plus 240 packet queue depth per 10/100 or blocking Gigabit Port 1920 packet queue depth on non-blocking ports Input Queuing Not Needed



22

New w/ Sup5

PPE2 increases all transmit queue buffersAll system ports benefit ; DBL congestion avoidance works on these queues to isolate belligerent flowsTransmit queues increased to 1368 queues/system (336 * 4 + 6*4) Packets/queue independent of the incoming packet sizeSupervisor Engine V// Non blocking gigabit ports 2336 packets/queue or 9344 packets/port 22% increase

Supervisor Engine II-Plus, IV// Non blocking gigabit ports 1920 packets/queue or 7680 packets/port // Sub ports 240 packets/queue or 960 packets/queue Command: show qos int

// Sub ports 292 packets/queue or 1168 packets/port Command: show qos int



23

QoS Scheduling on Cisco IOS-Based SupervisorsTraffic Sharing Specifies the minimum bandwidth for a Queue Used to implement a prioritized scheduling mechanism Only available on non-blocking Gigabit ports( SupIV & II-+)Available on all ports for Supervisor V

Traffic Shaping Configuring the maximum bandwidth for a Queue Available on every port and queue with an IOS Supervisor Policing vs. Shaping on the Catalyst 4500 IOS Supervisor PolicingTakes place at forwarding engine; if you exceed limit, then packets are dropped (or marked down) ShapingTakes place between queue and physical wire; if you exceed limit, then continue to buffer and try again later



24

QoS Policing vs. ShapingTraffic Traffic Rate Traffic

Data LostTraffic Rate

Policing

Time Traffic Rate

Time

Traffic

Traffic

Data PreservedTraffic Rate

Shaping

TimeQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

Time25

Scheduling: Shaping Max rate (10K to 1 Gbps)Shaped queue like a virtual wire Packets clock out exactly at shaped rate

Hold packets in queue when rate exceeded Example use:Shape a bursty application to 1 Mbps to smooth it

Supported on all ports, typically used with strict priority queueShaper (Specifies Max BW) TX Port Q



26

Uses for Policing and Shaping

Regulate Traffic Flows Business Models (Policing or Shaping)a pay-by-the-megabit model

Smoothing Traffic (Shaping)can reduce buffers needed downstream



27

Scheduling: Sharing Minimum rate (32 Kbps to 1 Gbps)Rate is guaranteed minimum

Scheduling algorithm:If below share rate, queue is high priority High priority queues serviced first

Sharing only on non-blocking gigabit ports in Supervisor IV and II-Plus Supported on ALL ports on Supervisor Engine VShaper (Specifies Max BW) Non-Blocking Port TX Port Q Sharer (Specifies Min Guaranteed BW)



28

Uses for Sharing

Want to guarantee bandwidth to an application Data/Voice/Video example:On a 100 mbps ETTH link video guaranteed 80 mbps voice gets 10mbps high priority data guaranteed 20 mbps (but can use more if no video)



29

Scheduling: Strict Priority Strict priority queue is always checked 1st(subject to shaping) ensures low delay: intended for voice/control

Queue 3 on all ports (default voice queue) Caveat: On blocking ports..strict priority can prevent other queues from being servicedShape the strict queue to avoid this! Reserve it for voice and controlQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

30

QoS Scheduling differences Supervisor IV versus Supervisor V

New w/ Sup5

Sharing is available on all the ports with Supervisor V Auto QoS macro automatically does sharing on subports-- auto qos voip {cisco-phone| trust} Traffic Sharing Specifies the minimum bandwidth for a Queue ( similar to WRR) Used to implement a prioritized scheduling mechanism e.g. bandwidth over video servers Available on all physical ports Supervisor IV supported sharing only on non-blocking gigabit ports Traffic Shaping Configuring the maximum bandwidth for a Queue on a physical port Limited shaping is supported on Supervisor VQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

31

Sharing for Sub ports (GTTD)

New w/ Sup5

1 Gb/sec is mapped into 4 front panel GE ports on a WS-X4424

Per-Queue-Share = 1 Gbps / ( Num-Ports * Tx-Queues-Per-Port) For WS-X4424-RJ45 (24 port 10/100/1000), 1 Gbps / ( 4 Sub-ports * 4 tx-queueus-per-subport )4510R#sh qos int Gi8/24 QoS is enabled globally Tx-Queue Bandwidth ShapeRate (bps) (bps) 1 62500000 disabled 2 62500000 disabled 3 62500000 disabled 4 62500000 disabledQoS Technical Update June 2004

The bandwidth compares to the 4:1 mapping since 4 GTTD ports map to a 1GE port (STUB) Priority N/A N/A high N/A QueueSize (packets) 292 292 292 292

62.5Mbps * 4 = 250 Mbps per 4424 port 32


Shaping and Sharing on Port Txqueuescat4507R#show qos interface gig6/4 QoS is enabled globally Port QoS is enabled Port Trust State: 'dscp' Default DSCP: 0 Default CoS: 0 Appliance trust: none Tx-Queue Bandwidth (bps) 1 2 3 4 250000000 250000000 250000000 250000000 ShapeRate (bps) disabled disabled 50000000 disabled N/A N/A high N/A Priority QueueSize (packets) 2336 2336 2336 2336

Above output is from a Supervisor Engine V



33

Cisco IOS-Based Supervisor Traffic Shaping/Sharing exampleConfigure ACL, policy-map using IOS MQC, then apply to the interface qos interface GigabitEthernet1/1 no switchport ip address 11.0.11.1 255.255.255.0 ip pim sparse-dense-mode service-policy output video-servers tx-queue 2 bandwidth 200m tx-queue 3 priority high bandwidth 5m shape 20m // //Give Voice guaranteed minimum share of 5 Mb Shape the strict priority queue to no more than 20 Mbps

//

Give Multicast share of 200m (min bandwidth guarantee)



34

Rewrite: DSCP and CoS Rewrite

DSCP, CoS (header) rewrite

Ethernet MAC

DSCP and CoS are rewritten on transmit Using the global tos-to-cos map DSCP value is the internal DSCPQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

35

DSCP to Queue MappingCheck DSCP to TX Queue Mappingscat4500(config)# qos map dscp 50 to tx-queue 2 cat4500# sh qos maps dscp tx-queue DSCP-TxQueue Mapping Table (dscp = d1d2) d1 : d2 0 1 2 3 4 5 6 7 8 9 For DSCP of 50 TX Queue Is 2

------------------------------------0 : 1 : 2 : 3 : 4 : 5 : 6 : 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 04 04 02 04 04 04 04 04 04 04 04 04 04 04 04 04



36

Tx Queue SchedulingshapingQ1 Q2 DSCP to queue map Q3 Q4 Queue selection based on internal DSCPDefault DSCP on Port Trust CoS/DSCP Via Service Policies

sharing, strict priority

Ethernet MAC

Switch-wide DSCP to Tx Queue map, not per-port! Shaping: max rate per queue Sharing: min rate per queue Strict priority on queue 3 All in hardware at wire rateQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

37

Classification/TOS Re-Write Summary

Determine the internal DSCP valuethis depends on the policymap and port trust configuration If a packet encounters both input and output classification policy:Output policy has precedence If no output policy then input policy has precedence If no output/input policy then RX port trust is usedQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

38

Whats in the Queues ?r3_4507R_S4#sh int gig5/1 count detail(truncated) Port Gi5/1 Port Gi5/1 Port Gi5/1 Port Gi5/1 Port Gi5/1 Port Gi5/1 Port Gi5/1 Port Gi5/1 InBytes 3133178 OutBytes 1470897765 InPkts 64 16079 InPkts 128-255 0 InPkts 512-1023 0 InUcastPkts 0 OutUcastPkts 28 OutPkts 64 259966 OutPkts 128-255 0 OutPkts 512-1023 0 InMcastPkts 24110 OutMcastPkts 1291706 InPkts 65-127 0 InPkts 256-511 8031 InBcastPkts 0 OutBcastPkts 20170536 OutPkts 65-127 21189826 OutPkts 256-511 8153

InPkts 1024-1518 OutPkts 1024-1518 InPkts 1519-1548 OutPkts 1519-1548 0 4325 0 0 Tx-Bytes-Queue-1 1377824448 Tx-Drops-Queue-1 0 Tx-Bytes-Queue-2 Tx-Bytes-Queue-3 0 1904 Tx-Drops-Queue-2 Tx-Drops-Queue-3 0 0 Tx-Bytes-Queue-4 93071413 Tx-Drops-Queue-4 0



39

Extras: QoS on the CPU Port0:ESMP Packets to the CPU 9-10: L3 Rx (telnet/SNMP) 15:MTU Fail/Invalid 1:Control 2: Host Learning 3-5: L3 Forwarding

CPU queuesProtects important traffic when CPU usage is high BPDUs/routing updates get priority Can still telnet or SNMP query when CPU is highQoS Technical Update June 2004 2002, Cisco Systems, Inc. All rights reserved

40

Policing on the Catalyst 4500 IOS Based Supervisors Two Types of PolicersIndividual: acts on each of the applied ports/VLAN Aggregate: acts on all of the applied ports/VLAN

Two policer parameters: rate and burstrate from 32kbps to 32gbps, burst in bytes

Two actionsexceed-action: drop, transmit, markdown conform-action: drop, transmit

Input and output policing on every packet1020 input, 1020 output policers, sharable



41

Policing IssuesMake Sure the Correct Type of Policer Is UsedCat4500# show policy-map interface Gig1/1 GigabitEthernet1/1 service-policy input: p1 class-map: c1 (match-all) 3435 packets match: access-group 100 police: Per-interface

Cat4500 QOS CENIC

Documents

cisco catalyst

cisco systems

port qos

rights reserved catalyst

chassis cisco ios

plus qos technical update

impacted qos technical

qos terminology qos