Network Test: EX8200 Virtual Chassis Performance and Scale

Juniper Networks

EX8200 Virtual Chassis

Performance and Scale

May 2012

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 2

Executive Summary Juniper Networks commissioned Network Test to evaluate its Virtual Chassis technology in

Juniper EX8200 modular switches. In this first installment of a two-part project, the focus is

Virtual Chassis system performance and scale. A second report will assess the Virtual

Chassis technology’s resiliency and high-availability features.

Among the highlights of performance testing:

EX8200 Virtual Chassis configurations, when comprised of four member

switches, deliver throughput of up to 2.55 Tbit/s. There is no penalty in moving

between Layer 2 switching and Layer 3 routing, either with IPv4 or IPv6 traffic.

EX8200 Virtual Chassis configurations introduce remarkably consistent

latency across frame sizes, a key consideration in determining enterprise

application performance.

EX8200 Virtual Chassis configurations offer high control- and data-plane

scalability for IP multicast traffic. In these tests, the Virtual Chassis system

forwarded traffic to up to 4,000 multicast groups without dropping a single frame.

EX8200 Virtual Chassis configurations ease the migration path as networks

grow in size without added complexity. Tests show no impact on existing traffic

as additional EX8200 member switches are added to form a larger Virtual Chassis

system.

Introducing EX8200 Virtual Chassis Technology Virtual Chassis technology allows multiple EX8200 switches to be interconnected to form

one logical entity.

This unified approach has many advantages:

Virtual Chassis technology doubles available bandwidth by using active/active

redundancy instead of the active/passive model used by the spanning tree protocol.

Virtual Chassis technology enhances scalability by adding capacity as needed. A

Virtual Chassis configuration requires just two EX8200 chassis to get started;

network architects can then add chassis as the network grows. There’s no

disruption to existing Virtual Chassis components, and the newly expanded Virtual

Chassis system will continue to appear as one entity to the rest of the network.

Virtual Chassis technology simplifies network management by using just one

configuration file for all EX8200 chassis. This reduces the number of network

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 3

elements seen by external monitoring and management tools, easing the

management workload.

Virtual Chassis member switches can be deployed across geographically

dispersed locations and still be managed as a single entity. In the second phase of

this project, we plan to demonstrate this capability.

Virtual Chassis technology allows “rightsizing” by combining switches with

different port densities. In all performance tests described here, engineers

combined smaller EX8208 and larger EX8216 switches to form a single logical entity.

Virtual Chassis Terminology To understand the benefits of Virtual Chassis technology, it helps to begin with key terms:

External Routing Engine (XRE). Juniper EX8200 Virtual Chassis configurations employ an

external routing engine, the XRE200, to handle control-plane tasks such as storing the

Virtual Chassis configuration file; building the Ethernet switching, IPv4 ARP, and IPv6 ND

tables; storing IGMP snooping entries; and building PIM routing tables. A redundant Virtual

Chassis system includes master and backup XREs, which in turn are connected to master

and backup LCC-REs (defined below).

Line Card Chassis (LCC). A Juniper EX8200 chassis, along with its line cards, becomes an

LCC when it joins a Virtual Chassis configuration.

LCC Routing Engines (LCC-REs). When a Juniper EX8200 chassis joins a Virtual Chassis

configuration, its routing engines become LCC-REs. In a fully redundant configuration,

master and backup XREs attach to master and backup LCC-REs in all LCC members of the

Virtual Chassis configuration.

Virtual Chassis Port (VCP). The connection between the XRE and LCC, called a Virtual

Chassis Port, carries Layer 2 and Layer 3 control-plane traffic (noted above in the XRE

definition) as well as Virtual Chassis Control Protocol (VCCP) frames. VCPs carry only

control-plane traffic.

Virtual Chassis Port extension (VCPe). A VCPe is a fabric interconnect between LCCs. It

can carry all the same control-plane traffic as a VCP, and also can forward data-plane traffic.

As described below in the “Methodology and Results” section, engineers did not use VCPe

interconnects in this project in the interest of supplying maximum data-plane bandwidth

for attached devices.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 4

Methodology and Results Figure 1 shows the test bed used for most performance benchmarks described in this

document. The Virtual Chassis system comprised four EX8200 chassis, each equipped with

8-port EX8200-8XS modules, for a total of 256 10-Gbit/s Ethernet switch ports. Redundant

EX8200-XRE200 external routing engines handled control-plane tasks.

The Spirent TestCenter traffic generator/analyzer served as the primary test instrument in

this project. To showcase support for IEEE 802.3ad link aggregation, the Spirent test ports

formed four-member link aggregation groups with Virtual Chassis switch ports. Engineers

configured the system with one Spirent port attached to each of the four EX8200 chassis.

Figure 1: The Juniper EX8200 Virtual Chassis Test Bed

Unicast performance The EX8200 Virtual Chassis configuration never dropped a frame in any of the unicast

benchmarks. Engineers repeated these unicast throughput and latency tests in three

configurations, using Layer 2 switching, Layer 3 IPv4 static routing, and Layer 3 IPv6 static

routing. In all three cases, throughput was identical.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 5

In these tests, the Spirent traffic generator offered frames at rates of up to 2.55 Tbit/s, using

four fully meshed 64-port traffic patterns. For N-way link aggregation connections to a

Virtual Chassis system, Juniper’s recommended configuration is to physically attach one link

aggregation group member to each of N switches. That is the configuration used in these

tests: Each server emulated by the Spirent instrument had 4-way redundancy via links to

four physical switch chassis.

A key objective of this project was to determine the maximum bandwidth available from a

Virtual Chassis system. While it’s possible to construct a traffic pattern that spans multiple

EX8200 switches, such a configuration would be blocking due to oversubscription of links

between switches. Thus, test engineers used the Juniper-recommended configuration with

one connection to each EX8200 comprising the Virtual Chassis system.

As recommended in RFC 2544, the IETF’s foundation methodology for network device

benchmarking, engineers measured throughput using seven standard frame lengths,

ranging from the Ethernet minimum of 64 bytes to the maximum of 1,518 bytes. Engineers

also used two additional lengths often seen in enterprise data centers: 2,176-byte frames

for storage traffic and 9,216-byte jumbo frames for bulk data transfer. To account for minor

clocking differences between the test instrument and the Virtual Chassis system, engineers

configured the Spirent instrument to offer traffic at 99.99 percent of Ethernet line rate.

Figure 2 summarizes results from the unicast throughput tests, comparing actual results

with the theoretical maximum.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 6

Figure 2: Juniper EX8200 Virtual Chassis Unicast Throughput

It’s important to note that throughput with Virtual Chassis technology is double the

amount possible with spanning tree protocol. Because spanning tree uses an

active/passive approach to redundancy and loop prevention, one of every two switch ports

is in blocked state and cannot forward traffic. In contrast, Virtual Chassis technology

employs an active/active approach, where all switch ports can forward traffic while still

providing redundancy and preventing traffic loops.

Engineers also measured latency in the unicast performance tests. As specified in RFC 2544,

latency is measured at the throughput rate. Thus, these latency measurements represent

delay under the most stressful possible conditions. If anything, delay under lighter loads

and/or with less stressful traffic patterns (using port pairs instead of fully meshed patterns,

for example) would likely result in lower latency.

Even so, latency is remarkably consistent across frame sizes and test cases. Average

latency is typically 15 microseconds or less in most tests, and never exceeds 20

microseconds except with jumbo frames (where it still remains below 30 microseconds).

Figure 3 presents unicast latency measurements.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 7

Figure 3: Juniper EX8200 Virtual Chassis Average Latency

Multicast performance The EX8200 Virtual Chassis configuration achieved high control- and data-plane

scalability in the multicast tests. In Layer 2 tests, the switches learned 4,000 multicast

groups using IGMPv3 snooping and forwarded traffic to all groups without loss. In Layer 3

tests, the system used a combination of Protocol Independent Multicast-Sparse Mode (PIM-

SM) and IGMPv3, again forwarding all traffic without loss.

A key goal of the multicast performance tests was to demonstrate the same high throughput

in both Layer 2 and Layer 3 scenarios. As shown in Figure 4, the EX8200 Virtual Chassis

system achieved that goal. This figure also compares actual throughput with the theoretical

maximum.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 8

Figure 4: Juniper EX8200 Virtual Chassis Multicast Throughput

For the multicast tests, engineers constructed traffic patterns that consisted of four sets of

IGMPv3 multicast source, group (s,g) trees. The Spirent test instrument represented one

multicast transmitter and 63 receiver ports on each EX8200 switch. As noted in the unicast

performance discussion, this is consistent with Juniper’s design recommendation of

attaching one member of each link aggregation group to each physical chassis within the

Virtual Chassis system. Also similar to the unicast tests, engineers configured the Spirent

instrument to offer traffic at 99.99 percent of Ethernet line rate to account for minor

clocking differences between the test instrument and the Virtual Chassis system.

One difference between the Layer 2 and Layer 3 scenarios involved the number of multicast

group addresses involved. In the Layer 2 tests, the Virtual Chassis system learned 4,000

multicast groups via IGMPv3 snooping. In the Layer 3 tests, engineers used 512 multicast

groups while concurrently running switching and routing protocols. The Virtual Chassis

system forwarded traffic at the same rate in both Layer 2 and Layer 3 scenarios.

Engineers also measured multicast latency. Here, as recommended in RFC 3918, the IETF’s

methodology for IP multicast benchmarking, engineers measured latency at the throughput

rate. Latency may be lower under lighter loads and/or with less stressful traffic patterns.

Figure 5 presents multicast latency measurements. Latency is slightly higher in the Layer 3

test cases, but only by around 2 microseconds at most, due to the longer code path when

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 9

routing multicast traffic. In the vast majority of enterprise networks, a 2-microsecond

difference in latency will not have a meaningful impact on application performance.

Figure 5: Juniper EX8200 Virtual Chassis Multicast Latency

Ease of Migration As networks grow, more switch ports can be added to a Virtual Chassis system with

no disruption to existing traffic and no added configuration complexity. Network Test

validated the ability to add components to a Virtual Chassis system by running “before” and

“after” tests involving two and four EX8200 chassis. This test had three objectives: First, to

determine the effect on system throughput by the expansion of the Virtual Chassis; second,

to determine what effect, if any, an expansion would have on traffic latency; and finally, to

determine whether configuration complexity increased when adding ports.

In the “before” scenario, the Virtual Chassis system comprised two Juniper EX8216 chassis,

each with 64 10-Gbit/s Ethernet ports. Here, engineers configured Spirent TestCenter to

emulate servers connected to each chassis in two-port link aggregation groups.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 1

0

In this scenario, engineers ran throughput tests similar to those described in the “Unicast

performance” section, offering 64-byte frames in two fully meshed traffic patterns (one per

EX8200 chassis).

As shown in Figure 6, engineers next added two additional EX8208 systems to the Virtual

Chassis configuration, expanding the system size from 128 to 256 10-Gbit/s Ethernet ports,

and repeated the throughput test. This time, traffic consisted of four 64-port fully meshed

patterns, the same setup used in the “Unicast performance” section above. Here, the Spirent

test instrument modeled servers attached with four-member link aggregation groups.

Figure 6: Juniper EX8200 Virtual Chassis Migration

Figure 7 compares bandwidth before and after adding switches to the Virtual Chassis

configuration. Note that bandwidth is exactly double when Virtual Chassis capacity

increases. Further, adding components caused no packet loss or other disruption to

existing flows.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 1

1

Figure 7: Virtual Chassis Ease of Migration: Throughput and Latency

Another consideration beyond throughput is whether the change in network topology

would have any impact on latency.

Figure 7 also compares average latency for 64-byte frames before and after adding switch

ports to the Virtual Chassis system. In the latter scenario, latency actually decreased

slightly, by about 100 nanoseconds, even though the Virtual Chassis system handled

more traffic post-migration. Thus, migrating to a larger Virtual Chassis configuration

did not have an adverse impact on latency. Finally, adding components to a Virtual

Chassis system required only minimal configuration changes, and the entire system

continued to operate as a single logical entity.

Conclusion These tests validated the high performance and ease of migration of EX8200 Virtual Chassis

technology when used with Juniper EX8200 modular switches. Unicast performance tests

showed zero frame loss and consistent latency, both in Layer 2 switched and Layer 3 routed

scenarios. The same is true for IP multicast traffic, except that the multicast tests also

showcased high control-plane scalability. Finally, migration tests demonstrated that adding

switch ports to an existing Virtual Chassis configuration boosts bandwidth with no adverse

impact on latency, and without adding configuration complexity.

Juniper EX8200 Virtual Chassis Performance Assessment

Pag

e 1

2

Appendix A: About Network Test

Network Test is an independent third-party test lab and engineering services consultancy.

Our core competencies are performance, security, and conformance assessment of

networking equipment and live networks. Our clients include equipment manufacturers,

large enterprises, service providers, industry consortia, and trade publications.

Appendix B: Hardware and Software Releases Tested

This appendix describes the software versions used on the test bed. All tests were conducted in April 2012 at Juniper’s headquarters facility in Sunnyvale, CA, USA.

Component Version

Juniper EX8208, Juniper EX8216, Juniper EX8200-XRE200

Junos 12.1R1.9 (all tests except Layer 3 multicast); Junos 12.1B1-FS.2 (Layer 3 multicast)

Spirent TestCenter 3.90.0686.0000

Appendix C: Disclaimer Network Test Inc. has made every attempt to ensure that all test procedures were conducted with the utmost precision and accuracy, but acknowledges that errors do occur. Network Test Inc. shall not be held liable for damages which may result for the use of information contained in this document. All trademarks mentioned in this document are property of their respective owners.

Version 2012050301. Copyright © 2012 Network Test Inc. All rights reserved.

Network Test Inc. 31324 Via Colinas, Suite 113 Westlake Village, CA 91362-6761 USA +1-818-889-0011 http://networktest.com [email protected]