Cisco ASR 1000 Series Testing Results and Analysis

RA-070112-03 Cisco ASR 1000 Ser ies Test Plan and Resul ts July 2012

Copyright © 2012 Router Analysis 1"

Temporary Text

Cisco ASR1000 Series Router Test Plan and

Results

Report RA-070112-03 July 2012

RA-070112-03 Cisco ASR 1000 Ser ies Test Plan and Resul ts July 2012

Copyright © 2012 Router Analysis 2"

"

Don’t Just Test. Analyze. "

Cisco ASR1004 Router Test Plan and Results

!REPORT RA-070112-03

Quick Summary: The ASR1004 with ESP40 and RP2 Works As Expected When Configured With Features. Fabric Throughput Issues Do Not Impact QoS. Router Supports NetFlow On A Few Interfaces Without Impacting Performance.

In this report Router Analysis tests the Cisco

ASR1000 Series, specifically the ASR1004 along with the ASR1006 in non-redundant mode.

With the latest RP2 processor and ESP40 Fabric,

how does the ASR1004 perform under stress? Router Analysis has created and executed a test

plan to confirm Cisco’s claims about performance, features and QoS. We also check for head-of-line blocking issues.

In general we had no issues with the ASR and its

performance. The performance numbers mostly stayed above the ones quoted by Cisco.

Read on to f ind out more.

Devices covered in this report: Cisco: ASR1004 RP2 / ESP40 ARR1006 RP2 / ESP40 How we test: Utilizing state-of-the-art testing technology from vendors such as IXIA and MU Dynamics (now Spirent), Router Analysis has a world class testing lab.

RA-070112-03 Cisco ASR1000 Ser ies Test Plan and Resul ts July 2012 !

Copyright © 2012 Router Analysis !3

Table of Contents

Table of Contents ................................................................................ 3!Results Summary ................................................................................. 4!Test Lab Setup .................................................................................... 5!Tests Carried Out ................................................................................. 5!Testing Results Reviewed .................................................................... 6!Appendix A – Test Cases ................................................................... 17!Appendix B – Test Result Data Tables ............................................... 21!About Router Analysis ........................................................................ 22!

Table of Figures Figure 1 - Cisco ASR1004 IPv4 and IPv6 Forwarding ........................... 4 Figure 2 – Cisco ASR1004 ................................................................... 5 Figure 3 - Cisco ASR1004 IPv4 and IPv6 Forwarding ........................... 6 Figure 4 - ACL + uRPF Results ............................................................ 8 Figure 5 - At < 50% Per Port We Saw No Issues ................................. 9 Figure 6 - QoS Loss On Tos 0 at 57% Line Rate .................................. 9 Figure 7 - QoS Loss on TOS 0 at 57% Line Rate ............................... 10 Figure 8 - At 69% TOS 4 is affected ................................................... 11 Figure 9 - 39% Loss On TOS 4 .......................................................... 11 Figure 10 - 88.52% Caused issues With All Streams .......................... 12 Figure 11 - Background Traffic ........................................................... 12 Figure 12 - Background Traffic 10GE ................................................. 13 Figure 13 - QoS + BG Traffic .............................................................. 13 Figure 14 - QoS + BG Traffic - Highlighted ......................................... 13 Figure 15 - HOLB Testing .................................................................. 14 Figure 16 - OSPF Neighbors .............................................................. 15 Figure 17 - RSVP Tunnels .................................................................. 15 Figure 18 - MPLS TE Traffic ............................................................... 16



Results Summary The Cisco ASR1004 performed within Cisco’s claimed specifications when

tested at the Router Analysis test lab. Featureless IPv4 forwarding performed better than advertised: forwarding at

27 Million packets per second (Mpps) versus the claimed 23Mpps. Featureless IPv6 forwarding was tested at 21Mpps. Enabling features such as ACLs and uRPF dropped the routers forwarding

performance down to 23.4Mpps for IPv4 and 11.7Mpps for IPv6. Both numbers are above the 10.4Mpps minimum forwarding rate of the system advertised by Cisco.

QoS functioned as expected, protecting streams as configured even with

fabric congestion. No head-of-line blocking was observed.

Figure 1 - Cisco ASR1004 IPv4 and IPv6 Forwarding

!

0

5000000

10000000

15000000

20000000

25000000

30000000

IPv4 Base IPv4 ACL IPv6 Base IPv6 ACL

Claimed

Observed

w uRPF



Test Lab Setup The Router Analysis test lab utilizes the latest in IXIA testing hardware. The

tests in this report were performed using an IXIA testing chassis with 22 Gigabit Ethernet ports, two 10 Gigabit Ethernet ports along with IXIAs IxNetwork, IxLoad and IxAutomate software.

The ASR1004 was configured with one RP2, one ESP40 and two SIP40

carrier cards. Each carrier card had three five port Gigabit Ethernet v2 SPAs and one Single Port 10 Gigabit Ethernet v2 SPA.

Figure 2 – Cisco ASR1004

Tests Carried Out

• IP forwarding rate o Minimum Packet Size (64 bytes)

! With ACLs (permit any any) ! With uRPF ! With QoS ! With NetFlow

o Packet IMIX + Features o 1500 Byte Packets (Fabric Throughput) + Features

• Oversubscription o One to Many (HOLB Test) o Many to One o QoS

• MPLS o RSVP-TE Scale and Performance



Testing Results Reviewed IPv4 and IPv6 Forwarding Results with Features

Figure 3 - Cisco ASR1004 IPv4 and IPv6 Forwarding

The main takeaway from the results is that the ASR1004 performed above

the quoted specifications for featureless IPv4 based forwarding. IPv4 + uRPF also came in above the stated 23Mpps specification. IPv4 forwarding with ACLs was 21.3Mpps (about the same as IPv6 base which is 21Mpps).

While testing the ASR1004, the lowest performance appeared while

executing the “IPv6 forwarding with ACLs, uRPF, QoS and NetFlow” test case. By the time I had NetFlow configured on all of the interfaces the ASR1004 performance was down below 1Mpps.

Most if not all companies would never apply NetFlow inbound and outbound

on ALL interfaces. Running NetFlow on a few interfaces on the ASR1004 does not cause any issues.

Both the IMIX and Fabric Bandwidth tests were above the minimum

threshold and able to be done at full fabric bandwidth (40G). The Fabric numbers are uni-directional, if traffic is counted both directions we exceeded 80G across the system.

0

5000000

10000000

15000000

20000000

25000000

30000000

IPv4 Base IPv4 ACL IPv6 Base IPv6 ACL

Claimed

Observed

w uRPF



IPv4/IPv6 Forwarding + Features Tests Step 1: Configure 40G of ports to send traffic in a mesh Result: No Loss Seen Step 2: Configure a simple permit all ACL on the router: Step 3: Configure the ACL inbound on one interface Result: No Loss Seen Step 4: Configure the ACL outbound on one interface Result: No Loss Seen Step 4: Configure the ACL inbound and outbound on all interfaces Result: Traffic forwarding rate drops to 21Mpps Step 5: Configure traffic to run at about 21Mpps Step 6: Configure uRPF on one interface Result: No Loss Seen (Hard to get the traffic perfect) Step 7: Configure uRPF on a second interface Result: Drop in performance of 70Kpps Step 8: Configure uRPF on the rest of the interfaces checking the stats as

you go along. Result: uRPF appears to cause a 25Kpps impact per GE port and a

250Kpps impact per 10GE port.



!Figure 4 - ACL + uRPF Results

QoS Testing and Results: From our tests we have determined that QoS works as expected on the

system. In a simple test involving three GbE links (a two to one oversubscription). Three different TOS bits were set (0, 4 and 7) and traffic was sent balanced (33.3% per TOS) from each GbE towards a single GbE port outbound.

We configured the router so that TOS 0 was protected up to 1% of line rate,

TOS 4 was protected up to 40% of line rate and TOS 7 was protected up to 70% of line rate.

When running the test we saw the following results: < 100% all streams went through without issue > 100% TOS 0 started to be dropped ~ 140% TOS 4 started to be dropped ~ 190% TOS 7 started to be dropped Step 1: Send less than 50% line-rate from each of the two ports



Figure 5 - At < 50% Per Port We Saw No Issues

Result: Up to almost 50% line-rate from the 2 ports (< 100% outbound on

other interface) we saw no loss. Above 50% we saw loss for the TOS 0 stream but not the TOS 4 or 7 streams.

Step 2: Send 57% line-rate from each the two ports (114% total)

Figure 6 - QoS Loss On Tos 0 at 57% Line Rate



Result: At about 57% we start to see TOS 0 traffic drop as shown in the magnified section below:

Figure 7 - QoS Loss on TOS 0 at 57% Line Rate

! Step 3: Up the traffic to about 69% line-rate per port (138% total)



Figure 8 - At 69% TOS 4 is affected

As shown in this magnified section we see about 38% Loss on TOS 4 when sending 69% line-rate per port (138% line-rate outbound)

Figure 9 - 39% Loss On TOS 4



Step 4: Raise the traffic to about 88.5% we start seeing traffic loss on all streams.

Figure 10 - 88.52% Caused issues With All Streams

!! Result: We see loss on all 3 QoS traffic profiles. Final Result: From this basic test we have determined that QoS works as expected. !

QoS + Background Traff ic + Fabric Loss/Forwarding Issues We then did the same QoS test with background traffic between all other

ports on the system.

Figure 11 - Background Traffic

!



The background traffic saw packet loss as seen above and magnified below:

Figure 12 - Background Traffic 10GE

Here you can see we are sending 14Mpps and only getting 4Mpps on the

10GE ports. And QoS traffic continued to function as expected:

Figure 13 - QoS + BG Traffic

Figure 14 - QoS + BG Traffic - Highlighted

As you can see there are no drops on the TOS 4 or TOS 7 traffic.



ASR1004 Pol icy Configuration Here is the router configuration: Policy Map ixia-test2 Class prec_0 bandwidth remaining ratio 10 police cir percent 1 conform-action transmit exceed-action transmit violate-action drop Class prec_4 police cir percent 40 conform-action transmit exceed-action transmit violate-action drop bandwidth remaining ratio 10 Class prec_7 police cir percent 70 conform-action transmit exceed-action transmit bandwidth remaining ratio 10 Head-of-Line Blocking Testing: For HOLB Testing we setup a test case where we sent traffic from three GE

ports to one GE port, then from another GE port to both the over-loaded GE port and an empty GE port. The red * indicates that the IXIA did not have real time information at the time I took the screen shot.

Figure 15 - HOLB Testing

!! !



MPLS Testing and Results Step 1: Enable OSPF with MPLS Traffic Engineering

Cisco Example: router ospf 1

mpls traffic-eng area 0 Step 2: Enable MPLS Traffic Engineering on the router and interfaces Cisco Example interface GigabitEthernet1/0/2 description to IXIA G3_2 ip address 198.18.8.1 255.255.255.0 ip ospf 1 area 0 mpls traffic-eng tunnels Step 3: Configure the IXIA to do OSPF with the interfaces on the router

!Figure 16 - OSPF Neighbors

Step 4: Configure the IXIA to generate 100 fully meshed tunnels on each

interface (1200 tunnels total).

!Figure 17 - RSVP Tunnels



Step 5: Setup fully meshed traffic between all ports and run traffic to verify functionality.

!Figure 18 - MPLS TE Traffic

Step 6: Increase the number of tunnels until you can no longer forward traffic, you see failures on the system or any functionality is compromised. Result: Tested up to 4800 TE Tunnels without issue. !



Appendix A – Test Cases

The following two test cases represent a skeleton format of the set of tests performed in the Router Analysis lab. Test Case One – IP Forwarding Synopsis: When testing systems which have a set PPS or Fabric Bandwidth which is lower then the combined interfaces available; it is necessary to validate the best-case base forwarding rate. Once the rate has been determined, it is possible to judge other test cases against the best-case forwarding rate. Goal of Test: To confirm best-case IPv4 or IPv6 forwarding rate using the minimum and maximum packet size accepted by the system. Procedure: Configure the minimum number of links between the tester and the system under test (SUT) to provide either a.) More than the maximum PPS or b.) More than the maximum fabric bandwidth as claimed by the vendor. For example: The ASR1004 ESP40 claims a maximum of 23Mpps or 40Gbps. To test 23Mpps, you could use two 10GE interfaces running 64 byte packets at line rate (14.8Mpps x 2). To test the maximum fabric bandwidth you run large packets such as 1500 bytes and would need at least 42G of interfaces such as four 10GE and two GE (due to packet overhead which would be stripped before it reached the fabric). Data to Col lect: Maximum PPS for IPv4, IPv6 and a 50/50 mix of IPv4 and IPv6 Maximum fabric bandwidth for IPv4, IPv6 and a 50/50 mix of IPv4 and IPv6



Test Case Two – IP Forwarding with Features Synopsis: Once the best-case forwarding rate for IP has been determined, it is important to start enabling features to see what impact they may have on the forwarding rate. Goal of Test: To determine which features if any impact the forwarding rate of the SUT. Procedure: ACLs: Using your best-case setup from test one, apply a simple “allow all” access list to one interface inbound. Once you have recorded the data, swap the access list to outbound on the same interface. Next apply the access list both in and out on the same interface. The final steps are to apply the access list in and out on the other interfaces observing if the impact is the same or greater. When you have completed the testing you should have the access list applied to all interfaces inbound and outbound. (If no impact is seen even with all interfaces running ACLs, you may need to configure a bigger ACL using multiple Access List Entries (ACEs) with random source/destination addresses and ports blocked before an allow all statement.) If ACLs have no impact on forwarding performance it is possible that the SUT does all access list actions in hardware and no impact will be seen. Consult with your vendor to discuss. uRPF: For uRPF follow the same concept as with ACLs: apply uRPF to one interface at a time, log the results and then continue.



NetFlow: For NetFlow follow the same concept, a simple capture all NetFlow configuration applied inbound and outbound QoS: For QoS, build a simple policer and add it to each interface inbound/outbound and see if there is any impact. Data to Col lect: Screen captures and if possible csv data for each of the tests you execute. !



Test Case Three – QoS Testing Synopsis: !This is a simple QoS test based on information from the IXIA BlackBook. We have modified it slightly to utilize more than 2 ports. Goal of Test: Determine if QoS functions as expected by creating three different TOS valued streams, 0, 4 and 7. On the router configure a traffic policy to prefer 7 over 4 and 4 over 0. Procedure: Using IxNetwork create three streams with TOS levels of 0, 4 and 7 on two interfaces going towards a single interface. On the outbound interface configure a QoS policy to prefer TOS 7 over TOS 4 and TOS 4 over TOS 0. Run traffic and slowly raise it until you see packets drop for 0, then 4 and then 7. Addit ive Tests: Add in background traffic from your earlier test where you saw packet loss due to features. Confirm that the QoS policy is still working as expected. Data to Col lect: Packet loss data for each stream at different traffic levels. Screen Captures of the IxNetwork screen during the tests. !



Appendix B – Test Result Data Tables The following tables represent a sample of the data collected during testing in the Router Analysis Lab. IPv4 Forwarding Results Data

IPv4 + uRPF Forwarding Results Data

IPv6 Forwarding Results Data

Test Claimed Test Result Pass/Fail IPv6 PPS no ACL 10,400,000 21,000,000 Pass IPv6 PPS with ACL 10,400,000 14,900,000 Pass IPv6 Fabric BW 40G 40G Pass IPv6 Fabric BW with ACL 40G 40G Pass

IPv6 + uRPF Forwarding Results Data

Test Claimed Test Result Pass/Fail IPv6 + uRPF 10,400,000 17,500,000 Pass IPv6 ACL + uRPF 10,400,000 14,900,000 Pass IPv6 Fabric + uRPF 40G 40G Pass IPv6 Fabric ACL + uRPF 40G 40G Pass

!

Test Claimed Test Results Pass/Fail IPv4 PPS no ACL 23,000,000 26,600,000 Pass IPv4 PPS with ACL 10,400,000 21,774,000 Pass IPv4 Fabric BW 40G 40G Pass IPv4 Fabric BW ACL 40G 40G Pass

Test Claimed Test Results Pass/Fail IPv4 + uRPF 10,400,000 23,450,000 Pass IPv4 ACL + uRPF 10,400,000 18,140,800 Pass IPv4 Fabric + uRPF 40G 40G Pass IPv4 Fabric ACL + uRPF 40G 40G Pass



About Router Analysis

With a world-class fully vendor independent testing lab, Router Analysis provides unbiased information to its clients.

Router Analysis provides the following services: Router and Switch Testing Test Plan Creation On-Site Testing Support Router, Switch and Tester Training Network Design Consultation, Verification and Installation … And More. Contact Router Analysis by phone at +1 888 SONN COM (766-6266) or by

email at [email protected] for a quote, more information or any questions.

Cisco ASR 1000 Series Testing Results and Analysis

Technology

router analysis test

cisco asr1004 ipv4

cisco asr1000 series

report router analysis

table of figures figure

test cases

test lab setup

testing results