This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Foundry 18FastIron/BigIron Series 18EdgeIron 24G 19FastIronEdge FES12GC 19
HP 20ProCurve Series 20
Nortel Networks 21Passport 8300 21
APPLICATION NOTE:
Third-Party Switch Link-Aggregation ConfigurationAbstract: The implementation of Link-Aggregation on the Panasas Gigabit Ethernet Switch is described, and examples of configuration commands for several switches that might be encountered in a customer’s network are given.
The four Network Ports on the back panel of the Panasas Gigabit Ethernet Switch are fixed in a link-aggregated mode, and are treated as a single connection to the customer's network. The Panasas Switch does not support Link-Aggregation Control Protocol (LACP), where ports automatically enter into an aggregated configuration, therefore, the external third-party switch ports that connect to the Panasas Switch ports must be forced to link-aggregated mode as well. The following pages show how to configure switches from several different vendors for this "forced-on" aggregated mode.
Terminology
It is important to note the different terminology between vendors for aggregated ports:
Sometimes the terms “link-aggregation” and “trunking” are used interchangeably, but it should be noted that what Cisco calls a “trunk” is actually a single port or port-channel that supports multiple VLANs.
The ports on both ends of the aggregated link must be configured for the same speed, duplex mode, and flow control settings, as well as VLAN tagging, QoS priority, and CoS parameters, if any.
When only two of the four available aggregated ports on the Panasas Gigabit Ethernet Switch are used, either Ports 1 & 3 or Ports 2 & 4 should be used, for even traffic distribution over the two links.
Three-port aggregated links are not supported, only 2 and 4-port configurations are valid. A single-port connection is valid as well, to any one of the four ports, or to the Downlink port on the Panasas Switch.
The aggregated link configuration should be completed on the Third-Party Switch before connecting the cables to the Panasas ports, to prevent error conditions and reports being generated on the customer’s network.
In the following examples, the commands to be entered by the user are denoted in bold type.
In switches and routers available today, most high-density Gigabit Ethernet modules (24 to 48 ports per module) are designed to be over-subscribed – that is, some number of ports (2 – 8) form a group that shares a connection to the switch fabric that is less than the combined bandwidth of the ports in this group (see Figure 1). This is a cost-effective method used to give users the capability of bursting to Gigabit Ethernet data rates through these ports, without providing the full bandwidth of a GE link.
When a customer is interested in maximizing the performance of the Panasas ActiveScale system, the Link-Aggregated ports from the shelf should not be connected to ports within the same over-subscribed group on a module (as shown at the top of Figure 1).
Some vendors allow Link-Aggregated ports to span over-subscribed groups, but still the potential exists for limited available bandwidth in this configuration, due to the traffic from the other connections within the group (as shown in the center of Figure 1).
The preferred connection from the Panasas shelf to a third-party switch is to ports that are non-blocking, each with the full Gigabit Ethernet link bandwidth to the Third-Party Switch fabric (as shown at the bottom of Figure 1).
1. From the Port Trunks Available listbox, select a port trunk.
2. Click the Select button. The Available Ports listbox displays the ports that are available to be placed in the port trunk. The Trunk Members listbox displays the ports that belong to the port trunk.
Placing Ports in a Port Trunk
1. From the Port Trunks Available listbox, select a port trunk.
2. Click the Select button.
3. Click a port in the Available Ports listbox.
4. Click the Add >> button. The port is assigned to the port trunk, and the port is displayed in the Trunk Members listbox.
To verify the trunk configuration, use the commands listed above in the “Display the Ports that Belong to Each Port Trunk” section.
Notes: AT9410-GB is non-blocking, no restrictions. These commands have not been tested.
AT9410-GB
1. From the Main Menu, type A to select Advanced Switch Configuration.2. From the Advanced Switch Configuration Menu, type T to select Trunk
Configuration.3. Type A to select Add Trunk Member.4. Enter the trunk group number to configure and press Enter.5. Enter the ports that will constitute the port trunk and press Enter. Up
to a maximum of 4 ports can be specified, either individually (e.g., 1, 2, 3, 4) or as a range (e.g., 7-10).
6. To set trunk status, type S to display the status of the trunk.7. Enter the trunk group number to be set and press Enter.8. Type E to enable the new port trunk.9. Repeat steps 3 - 8 (using unique trunk-group numbers in step 7) to set
Notes: 6 GE links per slot to SupervisorIII’s K2 ASIC over the backplane WS-X4424 24-Port 10/100/1000BaseT card is 4:1 over-subscribed. It has six
ASICs, each with 1 Gbps of backplane bandwidth – use every fourth port for a trunk for maximum bandwidth to the backplane
WS-X4448 48-Port 10/100/1000BaseT card is 8:1 over-subscribed. It has six ASICs, each with 1 Gbps of backplane bandwidth – use every eighth port for a trunk for maximum bandwidth to the backplane
Notes: The following commands create two groups of aggregated ports, group 1 with
ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2), on a 24-port module in Slot 2 of the Cisco. Ports 9-12 of this module are added (for clients) with the aggregated groups to vlan 101.
login:password:
>enable>password:
Create vlan 101:
# vlan database(vlan)# vlan 101(vlan)# exit
Setup the aggregated-port load-balancing algorithm:
(From the Cisco GUI, the equivalent of the above “channel-group xx mode on” command is selecting “Manual” trunking mode for the selected ports in the group.)
Setup the aggregated-port group (the “port-channel”) in the vlan:
Display parameters to check configuration:# show running-config interface port-channel 1# show etherchannel 1 port-channel # show etherchannel 1 detail # show running-config interface gig 2/1..# show running-config interface gig 2/4# show interfaces gig 2/1 etherchannel..# show interfaces gig 2/4 etherchannel# show vlan id 101# show ip interfaces# clear counters
# show interface counters module 2
Save this configuration:# copy running-config startup-config
Cisco CatOS
Notes: The following commands create two groups of aggregated ports, group 1 with
ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2), on a 24-port module in Slot 2 of the Cisco. Ports 9-12 of this module are added (for clients) with the aggregated groups to vlan 101.
Notes: These commands have not been tested. The following commands create two groups of aggregated ports, group 1 with
ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2). Ports 9-12 are added (for clients) with the aggregated groups to vlan 101.
Matrix E1 Series(Firmware Version 2.05.xx)Notes:
Ports can only be assigned to one trunk; ports in a trunk must belong to the same port group, and must be of the same port type.
Only one trunk can be configured per port group. None of the ports in a trunk can be configured as a mirror source port or mirror
target port. All the ports in a trunk have to be treated as a whole when moved from/to, added
or deleted from a VLAN. Before removing a port trunk via CLI commands, all network cables must be
removed, to prevent loops. To disable a single link within a port trunk, first remove the network cable, and
then disable both ends of the link. This allows the traffic passing across that link to be automatically distributed to the other links in that trunk, without losing any significant amount of traffic.
Matrix> set trunk algorithm machashing
Create channel groups and add ports:
Matrix> set trunk trunk1 createMatrix> set trunk trunk1 enableMatrix> set trunk trunk2 createMatrix> set trunk trunk2 enableMatrix> set trunk port trunk1 ge.1.1-4Matrix> set trunk port trunk2 ge.1.5-8
Create vlan 101 and add ports:
Matrix> set vlan create 101Matrix> set port vlan ge.1.1-12 101
1. Select the Device Control Menu from the Main Menu.2. Select SmartTrunking Configuration.3. Enter a SmartTrunk Group ID number from 1 to 6 to identify the trunk.4. Select from two to eight ports to configure as one trunk. The ports used for each
trunk must all be on the same internal switch chip, which is synonymous with the SmartTrunk Group ID.
Notes: Summit Series switches are non-blocking – no bandwidth limitations between
ports. Link-Aggregated ports should be grouped on same side (left or right) of the 7i Tested on the 7i, the following commands create two groups of aggregated ports,
group 1 with ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2). Ports 9-12 are added (for clients) with the aggregated groups to vlan 101.
Notes: 8 GE backplane links per slot for all chassis sizes (6804, 6808, 6816) G8Ti 8-port blade is non-blocking, no restrictions on port groups. G24T3 24-port blade is 4:1 oversubscribed. Groups of 4 ports share a GE link to
local switch module fabric. LinkAgg should spread across groups (e.g. 1,5,9,13).
Alpine
Notes: 4 GE backplane links per slot for all chassis sizes (3802, 3804, 3808) GM4Ti 4-port blade is non-blocking, no restrictions. Trunks can span modules,
no restrictions. GM16T3 16-port blade is 4:1 oversubscribed. Groups of 4 ports share 1 GE link
Notes: These commands have not been tested. The following commands create two groups of aggregated ports, group 1 with
ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2).
Force 10 E-Series
# configure
(conf)# interface range 2/1-4(conf-if-range)# no switchport(conf-if-range)# exit(conf)# interface range 2/5-8(conf-if-range)# no switchport(conf-if-range)# exit
Notes: These commands have not been tested. 8.4 Gbps per slot over the backplane Jumbo frame support on JetCore ASIC-based modules A port cannot be configured as a member of a trunk group if 802.3ad Link
Aggregation (LACP) is enabled on the port. A trunk group must start with a primary port – the first in every 4 ports: 1, 5, 9, 13,
17, 21, etc. Port assignment must be contiguous: ports 1-4, for example. Any changes to port parameters must be done on the primary port, and will be
automatically applied to the other ports in the trunk group. 8-port Mini-GBIC module: non-blocking, supports copper mini-GBIC 16-port Copper (RJ45) 10/100/1000 module: non-blocking on-board, 2:1
oversubscribed to backplane (8.4Gbps is dynamically allocated to active ports) In the “trunk” command, the Load-Balancing method of “server” distributes traffic
based on source and destination IP addresses. (vs. “switch” which is based only on destination IP addresses).
FastIron# configFastIron(config)# trunk server ethernet 1/1 to 1/4FastIron(config)# trunk server ethernet 1/5 to 1/8
Notes: These commands have not been tested. The 4108GL Chassis advertises a 36.6Gbps central switching fabric, but each
module has only two full-duplex Gigabit Ethernet connections to this fabric.
1. From the Switch Console Main Menu, select:3. Switch Configuration5. Advanced Features2. Load Balancing (Meshing, FEC, Trunks)Press E (for Edit) to access the load balancing parameters.
2. In the Group column, move the cursor to the port to be configured.3. Use the Spacebar (or type the trunk name, such as trk5) to choose a trunk
assignment for the selected port.4. Move the cursor to the Type column for the selected port and use the Spacebar to
select the trunk type:– Trunk (Source Address/Destination Address trunk; the default
5. When finished assigning ports to trunks, press Enter, then S (for Save) and return to the Main Menu. (It is not necessary to reboot the switch.)
Notes: Nortel calls groups of aggregated ports “multilink trunks” (mlt). These are
identified by multilink IDs, or MIDs. For 8348TX 10/100 modules, allowable MIDs are 1-7. For 8324GT 10/100/1000 modules, allowable MIDs are 1-31. With dual 8393SF fabric modules, each module will have a 20gig (or 40gig full
duplex) dedicated connection to the switch fabrics. The following commands create two multilink trunks, MID 1 with ports 1-4 (for
Shelf 1) and MID 2 with ports 5-8 (for Shelf 2), on a 24-port module in Slot 5. Ports 9-12 of this module are added (for clients) with the multilink trunks to (port-based) vlan 101.
config vlan 101 create byport 1 (where 1 is the default Spanning-Tree Group ID)
config vlan 101 ports add 5/9-5/12
config mlt 1 create
config mlt 1 add ports 5/1-5/4
config mlt 1 add vlan 101
config mlt 2 create
config mlt 2 add ports 5/5-5/8
config mlt 2 add vlan 101
To view the settings, statistics, and errors for the mlts: