Understand And Troubleshoot Input Discards For Nexus 5600/6000 Contents Introduction Prerequisites Requirements Components Used Background Information Unicast Traffic Flow And Buffering Multicast Traffic Flow And Buffering What Causes Input Discards? Troubleshooting Scenarios Scenerio 1. Input Discards Step 1. Identify Ports With Input Discards Step 2. ASIC Identification Step 3. Identify Egress Congested Port Scenerio 2. Input Discards With HOLB HOLB Mitigation: Enable VOQ Limit HOLB Mitigation: Traffic Classification Related Information Introduction This document describes how to troubleshoot input discards on the Cisco Nexus 5600/6000 series switches. Prerequisites Requirements Cisco recommends that you have basic knowledge of Cisco Nexus 6000 Series configuration. Components Used The information in this document is based on these software and hardware versions: Cisco Nexus 6001 ● 7.1(3)N1(1) ● The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
13
Embed
Understand And Troubleshoot Input Discards For Nexus 5600/6000 · Troubleshooting Scenarios Scenerio 1. Input Discards Lab Setup: Line rate traffic egressing e1/3 and possible over-subscription:
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Understand And Troubleshoot Input DiscardsFor Nexus 5600/6000 Contents
IntroductionPrerequisitesRequirementsComponents UsedBackground InformationUnicast Traffic Flow And BufferingMulticast Traffic Flow And BufferingWhat Causes Input Discards?Troubleshooting ScenariosScenerio 1. Input DiscardsStep 1. Identify Ports With Input DiscardsStep 2. ASIC IdentificationStep 3. Identify Egress Congested PortScenerio 2. Input Discards With HOLBHOLB Mitigation: Enable VOQ LimitHOLB Mitigation: Traffic ClassificationRelated Information
Introduction
This document describes how to troubleshoot input discards on the Cisco Nexus 5600/6000 seriesswitches.
Prerequisites
Requirements
Cisco recommends that you have basic knowledge of Cisco Nexus 6000 Series configuration.
Components Used
The information in this document is based on these software and hardware versions:
Cisco Nexus 6001●
7.1(3)N1(1)●
The information in this document was created from the devices in a specific lab environment. All ofthe devices used in this document started with a cleared (default) configuration. If your network islive, ensure that you understand the potential impact of any command.
Background Information
Input discards are an indication of an oversubscribed egress port. It also means that you are likelydropping unicast traffic on that specific port. This article helps you to understand how unicast andmulticast traffic is buffered on this platform and how input discards could occur along with themitigation steps.
Unicast Traffic Flow And Buffering
Unicast traffic is queued at egress buffer pool first and then ingress buffer after egress queue isfull as shown in the image.
There is 16MB ingress shared buffer and 9MB egress shared buffer. The buffers are sharedbetween 12 x 10 gig ports or 3 x 40 gig ports. Shared buffer is good for burst absorption.
Here is a visual depiction of the memory allocation for your reference (Bigsur is the name of theASIC/Unified Port Controller) as shown in the image.
Multicast Traffic Flow And Buffering
Multicast packets are buffered and dropped at egress●
Drop the multicast packet close to congestion point in order to avoid Head of Line Blocking(HOLB)
●
Maintain lossless fabric for unicast as shown in the image.●
In majority of the cases, egress drops is always due to multicast/broadcast/Unknown unicasttraffic.
What Causes Input Discards?
A congested egress port causes the egress buffers in order to fill up first and then it causes theback pressure on the ingress. This is only for unicast traffic. Once the ingress buffers are full thenyou could potentially drop traffic on ingress which results in input discards.
This explanation is at a very high level and easy to digest but there is a bit more to it especiallywhen you look at different class of traffic, queues etc. There is a concept of Virtual Output Queue(VOQ) which is frequently used on the Nexus platform. VOQ is an allocation of ingress buffers forevery IEEE 802.1p Class of Service (CoS) per egress port. So there is 8 VOQ per egress port.
Congestion on one egress port in one CoS eventually bleeds into the congestion of itscorresponding VOQ on the ingress port. Once the limit is reached then traffic gets dropped. Ithowever, does not affect traffic destined for other CoSs or other egress interfaces, thus avoidingHOLB, which would otherwise cause congestion to spread. The traffic flow from the ingress toegress port and the various blocks in play is as shown in the image.
Troubleshooting Scenarios
Scenerio 1. Input Discards
Lab Setup:
Line rate traffic egressing e1/3 and possible over-subscription:
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause
In a simulated setup as here, you know the cause of the oversubscription but in a production setupwhere traffic profile burst and it can be a challenge to spot out the congested egress ports thoughthese commands.
The steps listed here helps you to identify the congested egress ports.
Step 1. Identify Ports With Input Discards
Input discards seen on port e1/4:
nexus6001# sh int e1/4 | in i disc
0 input with dribble 3024 input discard
0 lost carrier 0 no carrier 0 babble 0 output discard
The flow that must be affected is towards 10.10.10.50. The flow between 10.10.10.101 and10.10.10.102 must be clean.
This is however not the case. A stuck or slow-draining egress port can cause all buffers on one ormore ingress ports that sends traffic to the egress port to be exhausted which thereby affects alltraffic on these ingress ports. This is the classic HOLB problem.
Spirent traffic generators shows that the flows are dropped. The port numbers are Spirent portnumbers is as shown in the image.
HOLB Mitigation: Enable VOQ Limit
In order to avoid this scenario, the VOQs (only for unicast traffic) can be configured with a setthreshold.
nexus6001# sh plat so qd info counters voq asic-num 1 <snip>
After the configuration, the flows towards non congested ports is not affected.
Spirent Traffic generator view after the VOQ limit configuration is as shown in the image.
Though this configuration shows a clear advantage in order to prevent drops due to HOLB. Why isthis not the default config?
Typically, traffic in a production environment could burst in nature. By the disablement of the VOQthreshold, you allow the ingress buffers to absorb a traffic micro burst without the need to getdropped.
Unless the situation warrants the need to enable VOQ limit, it is recommended to use the defaultwhich is to leave it disabled.
HOLB Mitigation: Traffic Classification
There is another method to mitigate HOLB with the use of QoS configuration. Since ingressdiscards only affects a specific VOQ which in turn is a specific QOS class, you can map theaffected traffic to non-congested port to a different QOS group. From this output, the ingressdiscards affects QOS Group 0 class.