Configuring Nexus 7000 Virtualization
LTRCRT-2605
Barry Gursky CCIE#7208
[email protected] Sr. Data Center Architect
www.fireflyeducate.com
Dr. Peter J. Welcher
Principal Consultant
www.netcraftsmen.net
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Agenda
Describe the Cisco Nexus 7x00 Series Switch Hardware and Software Features
Discuss VDC Design Practices and Configuration
Review vPC Design and Configuration
Hands-on Lab Deploying VDC
Hands-on Lab Configuring Double-Sided vPC
Hands-on Lab Establishing L3 Communication
3
Cisco Nexus 7x00 Data Center Switch
4
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Objectives
Discuss the Cisco Nexus 7000
Describe the architecture of VDCs
Describe the fabric module capacity and redundancy capability
5
Cisco Nexus 7x00
6
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Nexus 7004 Nexus 7009 Nexus 7010 Nexus 7018
Slots 2 I/O + 2 Sup 7 I/O + 2 sup 8 I/O + 2 sup 16 I/O + 2 sup
Height 7 RU 14 RU 21 RU 25 RU
BW / Slot Fab 1 N/A N/A 230 Gig / slot 230 Gig / slot
BW / Slot Fab 2 1.92Tbs Fixed 550 Gig / Slot 550 Gig / slot 550 Gig / slot
1.92 - 15+ Tb/s System
DCB and FCoE Ready
Modular OS
Device Virtualization
Continuous Operations
Cisco Nexus 7000 Platform
7
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7004 Chassis
Supervisor Slots (1-2)
I/O Slots (2-3)
Side-to-back airflow
1.92Tb/s in 4RU modular chassis 2 I/O module slots, supporting 1, 10, 40 and 100 Gb M-series and
F-series
The chassis does not have fabric modules, the I/O modules connect directly through the backplane
Side-to-back air flow
8
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Power Supplies
Optional Front Doors
Summary LEDs
Integrated Cable
Management
Supervisor Slots (1-2)
Crossbar Fabric
Modules
Side-to-side airflow
Locking Ejector Levers
I/O Slots (3-9)
Fan Tray
Cisco Nexus 7009 Chassis
Front Rear
N7K-C7009
9
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7010 Chassis
Optional Locking
Front Doors
Front Rear
System Status
LEDs
Integrated Cable
Management with cover
Supervisor
Slots (5-6)
I/O Module Slots
(1-4, 7-10)
Air Intake with
Optional Filter
Air Exhaust
Crossbar Fabric
Modules
System Fan Trays
Power Supplies
Fabric Fan Trays
21RU
ID LEDs on
all FRUs Front-to-back
airflow
Locking Ejector
Levers
Common Equipment
Removes from Rear
Two Chassis
per 7 Rack
N7K-C7010 10
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7018 Chassis
Optional front
door
Front Rear
System status
LEDs Integrated cable
management
Supervisor
slots (9-10)
Power supply
air intake
Crossbar
fabric
modules
Power supplies
(2 - 4)
25RU
ID LEDs on all
FRUs
Side-to-side
airflow
Locking
ejector levers
Common equipment
removes from rear
System
fan trays
Payload slots
(1-8, 11-18)
11
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7706 Chassis
9RU
8 Payload Slots
(1.3T/slot) Redundant Supervisor
Engines
Front-to-Back Airflow
Up to 8x 3kW
AC/DC Power Supplies
Front
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7710 Chassis
Redundant Supervisor
Engines
8 payload slots
(1.3T/slot)
Up to 8x 3kW
AC/DC power supplies
14RU
3 fan trays
6 fabric modules
(behind fan trays)
Rear Front
Front-to-back airflow
34
13
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7718 Chassis
16 3000W AC/DC
power supplies
3 fan trays
6 fabric modules
(behind fan trays)
Rear
26RU
Front
Redundant Supervisor
Engines
Front-to-back airflow
16 payload slots
(1.3T/slot)
35
14
Cisco Nexus 7x00 Switch Line Modules
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Supervisor Engine 2/2E
N77-SUP2E
N7K-SUP2
N7K-SUP2E
Beacon LED
Console
AUX Management
Ethernet
Compact Flash USB Ports CMP Ethernet
N7K-SUP1
16
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Management Ethernet Interface
10/100/1000 interface
Belongs to dedicated management VRF
Supports IEEE 802.1ae LinkSec encryption
17
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 10GE M1 I/O Modules
8-port 10G with X2 transceivers
80G full-duplex fabric connectivity
Two integrated forwarding engines (120Mpps): Support for XL forwarding tables (licensed
feature)
8 ports wire-rate L3 multicast replication
802.1AE LinkSec
N7K-M108X2-12L
32-port 10G with SFP+ transceivers
80G full-duplex fabric connectivity
Integrated 60Mpps forwarding engine:
XL forwarding engine on L version
Oversubscription option for higher density (up to 4:1)
8 ports wire-rate L3 multicast replication
802.1AE LinkSec
N7K-M132XP-12L
18
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 48-Port 1G M1 I/O Modules
Two 1G I/O module options: 48 1G SFP ports with XL forwarding
engine (N7K-M148GS-11L)
48 10/100/1000 RJ-45 ports with XL forwarding engine (N7K-M148GT-11L)
Integrated 60Mpps forwarding engine
46G full duplex fabric connectivity: Line rate on 48-ports with some
local switching
48 ports wire-rate L3 multicast replication
802.1AE LinkSec
Cisco Public
N7K-M148GS-11L
N7K-M148GT-11L
19
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 F1 Series I/O Module SFP+ 10G I/O module
1G/10G dual-speed system-on-chip (SoC) design
Layer 2 forwarding with L3/L4 services (ACL/QoS)
Multi-protocol Classic Ethernet, FabricPath, DCB
High performance: 230Gb/s fabric connectivity
20 line-rate ports per slot over fabric at 64 bytes
32 line-rate ports per slot with local switching
320512 line-rate ports per system (7018 chassis)
Low latency unicast/multicast 5sec module-to-module at 64 bytes
N7K-F132XP-15
sometimes called switch-on-chip 20
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 Enhanced F2-Series I/O Module 1/10Gb SFP+/1/10GBaseT
48 SFP/SFP+ 1/10G I/O module (N7K-F248XP-25E)
48 port 1/10GBASE-T (N7K-F248XT-25E)
1G/10G dual-speed system-on-chip (SoC) design
L2/L3 forwarding with L3/L4 services (ACL/QoS)
Multi-protocol Classic Ethernet, FabricPath, FCoE, DCB
Support for the Nexus 2000 Series Fabric Extenders
High performance: 480Gb/s fabric connectivity 48 line-rate ports per slot Up to 768 line-rate ports per system (7018
chassis)
Less that 7.5W per port (N7K-F248XP-25E) Less that 9W per port (N7K-F248XT-25E)
N7K-F248XP-25E N7K-F248XT-25E
21
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 M2 Series 24-port 10Gb I/O Module
10G Line-rate forwarding
Compatible with Fab1 or Fab2
L2/L3 switching functionality
Support for Nexus 2000 Fabric Extender
High performance: 24 non-blocking ports per slot Up to 384 line-rate ports per system (7018
chassis)
N7K-M224XP-23L
22
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 M2 Series 6-port 40Gb I/O Module
40G/10G dual-speed
Common QSFP interface for 40G and 4x10G
Compatible with Fab1 or Fab2
L2/L3 switching functionality
High performance: 550Gb/s fabric connectivity 16 non-blocking ports per slot Up to 96 line-rate ports per system (7018
chassis)
N7K-M206FQ-23L
23
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 M2 2-Port 100Gb I/O Module
Dual speed capability: Flexible 40GbE / 100GbE capability
Common CFP Interfaces for 100G and 40G
Optics for Single Mode:
24
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 F3-Series 12-Port 40Gb Module
12 Ports line-rate 40Gbps
QSFP+ modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV, MPLS, and VPLS.
N7K-F312FQ-25
25
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 F3-Series 6-Port 100Gb Module
6 Ports line-rate 100Gbps
CPAK modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV, MPLS, and VPLS.
N7K-F306CK-25
CPAK-100G-LR4
26
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Nexus 7700 F2E 48-Port 1G/10G Module
48-port 1G/10G SFP/SFP+ module
Based on F2E ASIC technology *Same exact functionalities of the F2E on Nexus 7000
Wire-rate L2/L3 IPv4/IPv6 480 Gbps/slot
720 Mpps/slot
Multi-protocol Classic Ethernet, FabricPath, DCB/FCoE
32K FIB TCAM/16K adjacency table
16K MAC address table
Nexus 2000 (FEX) support
VOQ Buffering: 72MB per module
N77-F248XP-23E
27
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7700 F3-Series 24-Port 40Gb Module
24 Ports line-rate 40Gbps
QSFP+ modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV, MPLS, and VPLS.
N77-F324FQ-25
28
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7700 F3-Series 12-Port 100Gb Module
12 Ports line-rate 100Gbps
CPAK modular transceivers
Highlights include L2 and L3 features : Fabricpath, FCoE, VXLAN, OTV, MPLS, and VPLS.
N77-F312CK-26
29
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
The Cisco Nexus 2000 Series Fabric Extender Fabric Extender
24 or 48TX 100/1000M host interfaces; 2 or 4x 10GE uplink interfaces
32 or 48 SFP/SPF+ 1/10G host interfaces with 8x 10GE or 4x 40G uplinks
32 port 1/10G TX host interfaces with 8x 10GE
Host port-channel support up to 24 port-channels per FEX
SPAN source/destination support
48 100/1000 RJ45 Downlinks
4 10GE SFP+ Uplinks
FET-10G
Fabric Extender
transceiver
For FEX links only
2 10GE SFP+ Uplinks
8 10GE SFP+ Uplinks
32 1/10G SFP/SFP+ Downlinks
24 100/1000 RJ45 Downlinks
48 1/10G SFP/SFP+ Downlinks
4 40GE QSFP+ Uplinks
32 1/10GTX Downlinks
8 10GE SFP+ Uplinks
30
Cisco Nexus 7x00 Fabric Modules
31
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Fabric Module
N7K-C7010-FAB-1
N7K-C7018-FAB-1
N7K-C7010-FAB-2
N7K-C7018-FAB-2 N7K-C7009-FAB-2
32
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Nexus 7700 Fabric-2 Modules
N77-C7710-FAB-2
N77-C7718-FAB-2 Consistent Nexus 7700 Fabric Architecture:
1.32 Tbps per slot with 6 Fabric modules
Multilevel redundancy with all modules All
modules share the total fabric bandwidth
helping to ensure a lossless forwarding during
failover
VoQ provides a QoS aware lossless fabric
Arbitrated Cross for Unicast
33
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
N7000
46 Gb/s
110 Gb/s
Fabric Capacity and Redundancy
Fabrics
Module Slots
40 G
1 G Module
480 G
F2 10 G Module N7700 220 Gb/s
34
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Fabric Capacity and Redundancy (Cont.)
Module Slots
1G Module 10G Module
40 G 480 G
Fabrics
N7000
230 Gb/s
550 Gb/s
N7700 1.32 Tb/s (6 Fabric)
35
Cisco Nexus 7x00 Power Supplies
36
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
System Power
N7K-AC-6.0 KW N7K-AC-7.5 KW
Load-sharing
Hot-swappable
37
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
DC System Power
Load-sharing
Hot-swappable
N7K-DC-6.0KW
N7K-DC-PIU
38
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Nexus 7700 Power Supplies 3000W AC
3000W DC
90+% power supply efficiency above 50% load
Typical power draw values up to 30% less due to optimized cooling design
Redundancy modes for power supply or grid failure
AC power supply accepts 110v or 220v inputs at 20A with 10 different power cables.
DC power supply accepts 47v or 60v inputs at 40A
N77-AC-3KW
N77-DC-3KW
39
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
CB 10 Slot
Why Use Many Smaller Power Supplies?
More flexibility for provisioning power redundancy
Provide grid redundancy for smaller configurations
Pay-as-you-grow power
Headroom for future growth
40
Grid 1 Grid 2
CB 18 Slot
Minimum bootup power:* CB 18 ~4.5kW (2 PSUs) CB 10 ~3kW (1 PSU) Fully loaded w/F2E: CB 18 ~12.5 kW (6 PSUs) CB 10 ~7kW (4PSUs) Fully loaded, grid redundant: CB 18 ~24kW (10 PSUs) CB 10 ~14kW (6 PSUs)
* Chassis w/ 2 sups, 6 fabs, 3 fans
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Rear Accessible Variable Speed Fan Tray
- Dynamically adjusted based on system temperature to reduce power
NEW Independent Fan Speed Control
- Further Optimizes System Cooling
- Fans can be off when slots are empty avoiding cooling of open slots
- Reduces fan speeds by slot
Power Reporting of Fan Tray Power Draw
- Visibility into system power usage
Nexus 7009 Cooling System
41
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
System Cooling for 7010/7018 Redundant system fan trays provide cooling of I/O modules and supervisor
engines.
Redundant fabric fans provide cooling of crossbar fabric modules.
N7K-C7010-FAN-F N7K-C7010-FAN-S
Dual Connectors
Fabric Fans
Single Fan
Protection against any single fan, controller or connector failure
Variable fan speed allowing speed reduction for lower power usage in well-controlled environments
Two system fan trays top and bottom: Full redundancy for single fan failure Fully redundant fan controllers
42
Cisco Nexus 7x00 supervisor Redundancy
43
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Supervisor Synchronization
Linux Kernel O
SP
F
BG
P
PIM
etc
.
HA Manager
NX7K Data Plane
Linux Kernel
HA Manager
Request snapshot
OS
PF
BG
P
PIM
etc
.
Provide snapshot Start services in standby and notify
Provide event-driven sync messages
State synchronized
Data plane streams
Active Standby
44
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Linux Kernel O
SP
F
BG
P
PIM
etc
.
HA Manager
NX7K Data Plane
Linux Kernel
HA Manager
Active
OS
PF
BG
P
PIM
etc
.
Standby
Supervisor Failure
Active Standby
Data plane streams
Active Standby Active
45
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Linux Kernel O
SP
F
BG
P
PIM
etc
.
HA Manager
NX7K Data Plane
Linux Kernel
HA Manager
Switchover
OS
PF
BG
P
PIM
etc
.
Reload
Go Active
Supervisor Switchover
Active
Data plane streams
46
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Linux Kernel O
SP
F
BG
P
PIM
etc
.
HA Manager
NX7K Data Plane
Linux Kernel
HA Manager
OS
PF
BG
P
PIM
etc
.
Supervisor Reactivation
Data plane streams
Provide snapshot
Start services in standby and notify Provide event driven sync messages
State synchronized
Request snapshot Run boot diags
Active Standby
47
Nexus 7x00 Licensing
48
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 License Summary Description Part Number
Cisco NX-OS Enterprise LAN License N7K-LAN1K9
Cisco NX-OS Advanced LAN License N7K-ADV1K9
VDC license Supports 8 VDCs on Sup2E N7K-VDC1K9
Cisco NX-Cisco NX-OS Transport Services License* N7K-TRS1K9
Cisco NX-OS Enhanced Layer 2 License N7K-EL21K9
Cisco Nexus 7000 MPLS License ** N7K-MPLS1K9
Cisco FCoE License for Nexus 7000 32-port 10G SFP+ (F1) N7K-FCOEF132XP
Cisco Nexus 7000 SAN Enterprise License N7K-SAN1K9
Cisco FCoE License for Nexus 7000 48 port 10G SFP/SFP+ (F2) N7K-FCOEF248XP
Cisco Nexus 7004 Scalable Feature License N7K-C7004-XL
Cisco Nexus 7009 Scalable Feature License N7K-C7009-XL
Cisco Nexus 7010 Scalable Feature License N7K-C7010-XL
Cisco Nexus 7018 Scalable Feature License N7K-C7018-XL
Notes: * For OTV deployment, Enterprise and Advanced packages are required.
** For MPLS deployment Enterprise package is required.
49
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco NX-OS Licensing (7700 specific)
Description Part Number
Cisco NX-OS Enterprise LAN License N77-LAN1K9
VDC license Supports 8 VDCs on Sup2E N77-VDC1K9
Cisco NX-OS Enhanced Layer 2 License N77-EL21K9
Cisco Nexus 7000 SAN Enterprise License N77-SAN1K9
Cisco FCoE License for Nexus 7700 48 port 10G SFP+ (F2e) N77-FCOEF248XP
50
Nexus 7x00 Virtualization
51
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Hypervisor Model
Various Degrees of Virtualization
Data/Control Plane
Data/Control Plane +
Management Plane
Data/Control Plane +
Management Plane +
Resources +
Operating Environment
52
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Introduction to the VDC Architecture
Linux 2.6 Kernel
Infrastructure
Physical Switch
Protocol Stack (IPv4 / IPv6 / L2)
VDC1
L2 Protocols
VLAN Mgr UDLD
VLAN Mgr UDLD
LACP CTS
IGMP 802.1x
MAC Table
L3 Protocols
OSPF GLBP
BGP HSRP
EIGRP VRRP
PIM SNMP
RIB
VDCn
Protocol Stack (IPv4 / IPv6 / L2)
L2 Protocols
VLAN Mgr UDLD
VLAN Mgr UDLD
LACP CTS
IGMP 802.1x
MAC Table
L3 Protocols
OSPF GLBP
BGP HSRP
EIGRP VRRP
PIM SNMP
RIB
53
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Scalability:
4096 VLANs/VDC
1000 VRFs/VDC
Admin + 4 VDCs Sup2
Admin + 8 VDCs Sup 2E
Virtualization Hierarchy
N7K VDC VLAN VLAN VLAN VRF VRF VRF VLAN VLAN VLAN VRF VRF VRF
VLAN VLAN VLAN VRF VRF VRF
VDC VLAN VLAN VLAN VRF VRF VRF VLAN VLAN VLAN VRF VRF VRF
VLAN VLAN VLAN VRF VRF VRF
VDC VLAN VLAN VLAN VRF VRF VRF VLAN VLAN VLAN VRF VRF VRF
VLAN VLAN VLAN VRF VRF VRF
54
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Virtual Device Contexts
Consolidates physical network by virtualizing onto common data center networking infrastructure
Secures traffic between user departments
Allows departmental administration
Provides testing capability with no impact on production systems
VDC
Extranet
VDC
Prod
VDC
DMZ
55
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
VDC Use Case Examples Vertical Consolidation
Objective: Consolidate vertical infrastructure that delivers orthogonal roles to the same administrative or operational domain
Benefits: Reduced power and space requirements, can maximize density of the platform, provides smooth growth path, easy migration to physical separation in future
Considerations: Number of VDCs (4), Four VDCs != Four CPU Intra-Nexus 7000 cabling needed for connectivity between layers
core1 core2
agg4 agg3
accY accN accY accN
core VDC
agg VDC
Core VDCs
Aggregation VDCs
Core Devices
Aggregation Devices
core VDC
agg VDC
56
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
VDC Use Case Examples Vertical & Horizontal Consolidation
Combined vertical & horizontal consolidation in small to medium designs (2 aggregation blocks or less)
Power, cooling and real estate optimization for multiple layers
Maximize the benefits of a high-density platform
Simplified growth migration path
core1 core2
agg2 agg1
acc2 acc1
agg4 agg3
accY accN acc2 acc1 accY accN
core core
agg agg
Core VDCs
Aggregation VDCs
Core Devices
Aggregation Devices
57
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
The Default VDC
Physical Switch
Linux 2.6 Kernel
Infrastructure
VDC1 is the default VDC for Sup1, This is replaced with the Admin VDC on Sup2/2E
The default VDC has several Unique features:
Has all ports are assigned initially assigned
Enabled when the system is activated
Cannot be deleted
Is responsible for other VDC administration not necessarily configuration
Handles all Software Installation
Controls Systemwide parameters such as Licensing, VDC Resources, CoPP, NTP
Protocol Stack (IPv4 / IPv6 / L2)
VDC X
L2 Protocols
VLAN Mgr UDLD
VLAN Mgr UDLD
LACP CTS
IGMP 802.1x
MAC Table
L3 Protocols
OSPF GLBP
BGP HSRP
EIGRP VRRP
PIM SNMP
RIB
Default
VDC
58
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
VDC Fault Domain
Each VDC is a separate fault domain
A process crashes in any VDC
Processes in the other VDCs are not affected and continue to run unimpeded
59
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
VDC Administration
Super User
VDC Administrator
VDC User
60
Nexus 7x00 Features
61
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
L2
L3
L3
L2
IP Cloud
vPC
Core
Aggregation
Access
Servers
vPC
vPC
STP+ vPC Cisco FapricPath (L2MP)
STP Enhancements
Bridge Assurance
NIC Teaming
Simplified loop-free trees
2x Multi-pathing
Inter-POD Connectivity across L3
(Failure Boundary Preservation)
16x ECMP
Low Latency / Lossless
MAC Scaling
Operational Flexibility
L2MP
Feature Overview & Terminology Intelligent L2 Domains POD Evolution
Failure
Boundary
62
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
vPC and VSS Comparison
vPC (Virtual Port Channels)
VSS (Virtual Switching System)
Multi-Chassis Port Channel Yes Yes
Loop-free Topology (no blocking ports)
Yes Yes
STP as a fail-safe protocol only Yes Yes
Switch Control Plane Two Independent Nodes, both
active Single Logical Node
Switch Redundancy (sup failover) Intra-chassis Inter-chassis
Control Plane Protocols Instances per Node Single instance
Switch Configuration Separate Configs
(w/ consistency checker) Combined Configs
Maximum Physical Nodes 2 2
ISSU Support Yes 12.2(33)SXI
Inter-switch Link Hardware 32 Port 10GE Module PFC3C mode, Sup 70 10G,
6708, 6716
Virtual Port Channel (vPC) is a version of VSS for the Nexus.
63
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
vPC Features
vPC
Allow a single device to use a port channel across two upstream switches
Eliminate STP blocked ports
Uses all available uplink bandwidth
Dual-homed server operate in active-active mode
Provide fast convergence upon link/device failure
Reduce CAPEX and OPEX
64
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
vPC Terminology
A virtual port channel (vPC) allows multiple links that are physically connected to two different Cisco Nexus 7000 Series to appear as a single port channel to a third device.
Layer 3
Cloud
Peer
Link
vPC Domain
vPC
Normal
Port Channel
vPC Peer
Keepalive Link
Orphan
Port
vPC Member
Port
CFS
Orphan
Device
vPC
Peer
65
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
vPC is supported on both the Cisco Nexus 5000 and Cisco Nexus 7000 Series Switches.
vPC can be deployed in multiple layers of the data center simultaneously: Server to access Access to aggregation
Double-sided vPC enables a unique 16-way port channel: Can be scaled to 32-way port channels with
F-series modules
Max 16 Ports
vPC
Domain 1
vPC
Domain 2
Double-Sided vPC
66
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
FabricPath brings Layer 3 routing benefits to flexible Layer 2 bridged Ethernet networks.
Easy Configuration Plug & Play Provisioning Flexibility
Multi-pathing (ECMP) Fast Convergence Highly Scalable
Switching Routing
FabricPath
Introducing Cisco FabricPath
Control plane: IS-IS
Load balancing: ECMP and multi-topology
Frame format: MAC-in-MAC
Interoperability with Classical Ethernet
67
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
FabricPath Operation
FabricPath Port
S42 L1, L2, L3,
L4
L1
L2 L3 L4
A B C
L8
L5 L6 L7
L9
L10 L11 L12
S1 S2 S3 S4
S11 S12 S42
1/1 3/1
S11 S42 A C
L2MP Fabric
A 1/1
C S42
MA
C
IF C 3/1
L12
MAC IF
C 3/1
A S11
CE Port
Control Plane:
L2 IS-IS is running in the L2MP Core network no STP
Data Plane:
L2MP Core: Ethernet frames are encapsulated with MAC-in-MAC
(MiM) header and forwarded based on switch table derived from L2
IS-IS
Forwarding of Multicast is through distinct SPF Trees.
Switch IF
S42
Switch IF
MAC IF
68
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Loop Mitigation with FabricPath
STP Domain Root
L2 Fabric
S1
S10
S2
TTL=3
TTL=2 TTL=1
TTL=0
TTL is part of FabricPath header
Decrement by 1 at each hop
Frames are discarded when TTL=0
RPF check for multicast based on tree info
Root
Block redundant paths to ensure loop-free topology
Frames loop indefinitely if STP failed
Could result in complete network melt-down as the result of flooding
69
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Cisco Nexus 7000 OTV Topology
Communication between Server 1(site 1) and Server 2(site 2)
Server 1 Server 2
OTV OTV
IP A IP B
Encap Decap
Ethernet Traffic IP Packet Ethernet Traffic Ethernet Traffic
Ethernet traffic between sites is tunneled in IP Packets
Allows simple Ethernet connectivity across an IP network
Provides simplicity of Ethernet with the feature rich characteristics of IP
70
Nexus 7x00 Switch Configuration
71
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Configuration Steps: Switch Mode
1. Configure basic connectivity and administrative access
2. Provision VDCs
3. Configure Ethernet interface
4. Configure IP routing protocols
5. Validate interface configurations
6. Validate routing configuration
72
Configuring Basic Connectivity and Administrative Access
73
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Initial Switch Configuration
Do you want to enforce secure password standard (yes/no): yes
Enter the password for "admin": 1234Qwer
Confirm the password for "admin":1234Qwer
You will be prompted for secure password configuration on a Nexus 7000 switch with no previous
configuration
If a password is weak (short, easy-to-decipher), your password configuration is rejected.
Passwords are case-sensitive.
Password must be at least 8 characters with a mix of letters, numbers and capitals
Must not contain dollar signs ($) or spaces anywhere in the password.
Cannot include quotation marks (" or '),vertical bars (|), or right angle brackets (>). at the beginning
of the password
74
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Basic System Configuration
---- Basic System Configuration Dialog VDC: V ----
This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.
Please register Cisco Nexus7000 Family devices promptly with your
supplier. Failure to register may affect response times for initial
service calls. Nexus7000 devices must be registered to receive
entitled support services.
Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
to skip the remaining dialogs.
Would you like to enter the basic configuration dialog (yes/no): yes
The Nexus 7000 includes a CLI setup script
This script will appear in three cases:
Upon initial configuration of a new switch
After a write erase reload
Upon typing setup from the command line
75
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Basic Manual Configuration
Assign the switch hostname and domain
Assign the switch IP address and gateway ( ensure in management vrf)
Define additional usernames if required with network-admin credentials
N7K-1 # conf
N7K-1(config)# hostname N7K-P
N7K-1(config)# ip domain-name pod1.com
N7K-1(config)# vrf context management
N7K-1(config-vrf)# ip route 0.0.0.0/0 10.1.1.1
N7K-1(config)# username tarzan password Jane123 role network-admin
N7K-1#(config)# show user-account
...
user:tarzan
this user account has no expiry date
roles:network-admin
76
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Management Administrative Access
Configure RSA keys and enable the SSH server process
Verify that the SSH server is running
N7K-1(config)# ssh key rsa 1024 force
deleting old rsa key.....
generating rsa key(1024 bits).....
generated rsa key
N7K-1(config)# show ssh server
ssh is enabled
version 2 enabledN7K-1(config)# feature ssh
Ensure that you verify management access with ping from mgmt vrf
N7K-1# ping 10.x.1.1 vrf management (Where x is your pod number.)
PING 10.1.1.1 (10.1.1.1): 56 data bytes
Request 0 timed out
64 bytes from 10.1.1.1: icmp_seq=1 ttl=254 time=1.28 ms
64 bytes from 10.1.1.1: icmp_seq=2 ttl=254 time=1.197 ms
64 bytes from 10.1.1.1: icmp_seq=3 ttl=254 time=1.094 ms
77
Nexus VDC Configuration
78
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
N7K-1(config)# vdc engineering
N7K-1(config-vdc)#
N7K-1(config-vdc)# show vdc
vdc_id vdc_name state mac
------ -------- ----- ----------
1 N7K-1 active 00:22:55:79:1d:41
2 engineering active 00:22:55:79:1d:42
Nondefault VDCs are created from within the default VDC global configuration context:
N7K-1# config t
N7K-1(config)# no vdc engineering
Deleting this vdc will remove its config. Continue deleting this vdc? [no]
yes
Note: VDC deletion is a time consuming process, please wait until the
command completes
Nondefault VDCs are removed from within the default VDC global configuration context:
VDC Configuration
79
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
N7K-1(config-vdc)# show run vdc
vdc N7K-7 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 96 maximum 96
limit-resource u6route-mem minimum 24 maximum 24
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
vdc engineering id 2
boot-order 1
limit-resource vlan minimum 16 maximum 4094
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 8 maximum 8
limit-resource u6route-mem minimum 4 maximum 4
limit-resource m4route-mem minimum 8 maximum 8
limit-resource m6route-mem minimum 2 maximum 2
VDC Resource Assignment
80
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
N7K-1(config)# vdc engineering
N7K-1(config-vdc)# limit-resource vlan minimum 32 maximum 100
N7K-1(config-vdc)# show run | begin vdc
vdc N7K-7 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 96 maximum 96
limit-resource u6route-mem minimum 24 maximum 24
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
vdc engineering id 2
boot-order 1
limit-resource vlan minimum 32 maximum 100
limit-resource monitor-session minimum 0 maximum 2
limit-resource vrf minimum 2 maximum 1000
-More--
Configuring Resource Assignment
81
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
N7K-1# config t
N7K-1(config)# vdc engineering
N7K-1(config-vdc)# allocate interface ethernet 2/47
Moving ports will cause all config associated to them in source vdc to
be removed. Are you sure you want to move the ports? [yes] yes
N7K-1# config t
N7K-1(config)# vdc engineering
N7K-1(config-vdc)# allocate interface ethernet 2/1-2, e2/5
Moving ports will cause all config associated to them in source vdc to
be removed. Are you sure you want to move the ports? [yes] yes
VDC Interface Allocation
Allocating a single Ethernet interface to a VDC:
Allocating a range of Ethernet interfaces to a VDC:
82
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Validating VDC Interfaces
Display VDC interface information from within the default VDC:
N7K-1# show vdc membership
vdc_id: 1 vdc_name: N7K-1 interfaces:
Ethernet2/1 Ethernet2/2 Ethernet2/3 Ethernet2/4 Ethernet2/5 Ethernet2/6
Ethernet2/7 Ethernet2/8 Ethernet2/9 Ethernet2/10 Ethernet2/11 Ethernet2/12
Ethernet2/13 Ethernet2/14 Ethernet2/15 Ethernet2/16 Ethernet2/17 Ethernet2/18
Ethernet2/19 Ethernet2/20 Ethernet2/21 Ethernet2/22 Ethernet2/23 Ethernet2/24
Ethernet2/25 Ethernet2/26 Ethernet2/27 Ethernet2/28 Ethernet2/29 Ethernet2/30
Ethernet2/31 Ethernet2/32 Ethernet2/33 Ethernet2/34 Ethernet2/35 Ethernet2/36
Ethernet2/37 Ethernet2/38 Ethernet2/39 Ethernet2/40 Ethernet2/41 Ethernet2/42
Ethernet2/43 Ethernet2/44 Ethernet2/45 Ethernet2/48
vdc_id: 2 vdc_name: engineering interfaces:
Ethernet2/47
83
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
VDC Navigation
Navigating between the default and nondefault VDCs:
N7K-1# switchto vdc engineering
TAC support: http://www.cisco.com/tac Copyright (c) 2002-2008, Cisco Systems, Inc.
All rights reserved. The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under license. Certain
components of this software are licensed under the GNU General Public License (GPL)
version 2.0 or the GNU Lesser General Public License (LGPL) Version 2.1. A copy of
each such license is available at http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
N7K-1-engineering#
N7K-1-engineering# switchback
Switch from a nondefault VDC back to default VDC:
84
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
VDC Running Configuration
Copy the running configuration for all VDCs on the physical device to the startup configuration:
N7K-1# copy running-config startup-config vdc-all
N7K-1# show running-config vdc-all
!Running config for default vdc: N7K-7
!Command: show running-config
!Time: Mon Aug 2 03:30:42 2010
version 5.0(3)
license grace-period
feature telnet
username admin password 5 $1$pjCtSd9F$FLCKjyWF9c74BBAhUXOkr. role network-
admin
--Remaining output omitted--
Display the running configurations for all VDCs:
85
Nexus 7x00 Interface Configuration
86
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
N7K-1(config)# interface eth1/1-3
N7K-1(config-if-range)# switchport
N7K-1(config-if-range)# no shut
N7K-1(config-if)# interface eth1/4,e1/7-8
N7K-1(config-if-range)# switchport
N7K-1(config-if-range)# switchport mode trunk
N7K-1(config-if-range)# swichport trunk allowed vlan 10,20
All Cisco Nexus 7000 interfaces are designated
interface ethernet slot/port.
CLI L2 Interface Configuration
87
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
CLI Slash Notation
N7K-1(config)# interface e1/1
N7K-1(config-if)# no switchport
N7K-1(config-if)# ip address 10.1.23.1/24
N7K-1(config-if)# ipv6 add ::abcd:223/120
N7K-1(config)# ip access-list test
N7K-1(config-acl)# permit ip 10.1.1.0/24 any
88
Nexus 7000 vPC Configuration
89
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
vPC Configuration
N7K-1(config)# feature vpc
N7K-1(config)# feature lacp
N7K-1(config)# interface ethernet 7/1, e8/1
N7K-1(config-if)# switchport mode trunk
N7K-1(config-if)# channel-group 20 mode active
N7K-1(config-if)# exit
N7K-1(config)# vpc domain 1
N7K-1(config-vpc-domain)# peer-keepalive source 10.2.2.1 destination 10.2.2.2 vrf keepalive
Step 1: Enable feature vPC and LACP Step 2: Configure interface that you want to be peer link to be a port channel Step 3: Create the vPC domain
90
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
vPC Configuration (Cont.)
N7K-1(config)# interface port-channel 20
N7K-1(config-if)# vpc peer-link
N7K-1(config-if)# exit
N7K-1(config)# interface e3/1
N7K-1(config-if)# channel-group 50 mode active
N7K-1(config-if)# exit
N7K-1(config)# interface port-channel 50
N7K-1(config-if)# description Link To Access N7K-1(config-if)# vpc 50
N7K-1(config-if)# switchport
N7K-1(config-if)# switchport mode trunk
Step 4: Configure the vPC peer link Step 5: Configure interface that connects to the vPC device as Layer 2 LACP port channel Step 6: Add the port channels that connect to the downstream device to the vPC
91
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Verifying vPC Domain Status
To verify the status of the vPC peer relationship use the show vpc brief command:
N7K-1# show vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
Type-2 consistency reason : Consistency Check Not Performed
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Disabled
Dual-active excluded VLANs : -
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po20 up 100-105
92
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Verifying vPC Consistency
To check for potential vPC configuration consistency problems use the show vpc consistency-parameters command:
N7K-1# show vpc consistency-parameters vpc Legend:
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
------------- ---- ---------------------- ----------------STP Port Type
1 Default Default
STP Port Guard 1 None None
STP MST Simulate PVST 1 Default Default
lag-id 1 [(7f9b, [(7f9b,
0-23-4-ee-be-a, 8007, 0-23-4-ee-be-a, 8007,
0, 0), (8000, 0, 0), (8000,
0-5-9b-1f-89-fc, 0, 0, 0-5-9b-1f-89-fc, 0,0,
0)] 0)]
mode 1 active active
Speed 1 10 Gb/s 10 Gb/s
Duplex 1 full full
Port Mode 1 trunk trunk
Native Vlan 1 1 1
MTU 1 1500 1500
Allowed VLANs - 1-3967,4048-4093 1-3967,4048-4093
Local suspended VLANs - 1,10 -
93
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Summary
Data Center Aggregation and Core layers typically consist of highly available redundantly interconnected switches providing advanced services.
The Cisco Nexus 7000 is typically deployed either as a Core switch or as an Aggregation switch, in parallel with Catalyst Series Switches.
The Cisco Nexus 7000 integrated core provides high density 10 GE ports alongside Catalyst series services.
When the VDC is created, a default resource allocation is made for this VDC.
Networks and interfaces are configured on a interface basis per VDC from interface configuration mode.
94
Lab
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Continue Your Education
Demos in the Cisco Campus
Walk-in Self-Paced Labs
Table Topics
Meet the Engineer 1:1 meetings
96
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
DC/V Related Official Cisco Training Offerings
Course Description Cisco Certification
Implement Cisco Data Center
Unified Fabric/Unified Computing
(DCUFI and DCUCI)
Learn how to deploy complex virtualized Data Center Fabric
and Computing environments with Nexus and UCS families
CCNP Data Center
Cisco Data Center CCIE Unified
Fabric/Computing Workshop
(DCXUF and DCXUC)
Prepare for your CCIE Data Center Practical exam with
hands on lab exercises running on a dedicated
comprehensive topology
CCIE Data Center
Configuring Cisco MDS 9000
switches
Deep dive into the MDS Storage Networking Product family:
hands on lab exercises, newest features and platforms
explained
Introducing Cisco Data Center
Networking and Technologies
(DCICN and DCICT)
Start your career in Data Center learning all the different
technologies contributing to build a Data Center
infrastructure
CCNA Data Center
For more details please visit : http://learningnetwork.cisco.com
Questions: Visit the Learning@Cisco Booth
97
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Participate in the My Favorite Speaker Contest
Promote your favorite speaker through Twitter and you could win $200 of Cisco Press products (@CiscoPress)
Send a tweet and include Your favorite speakers Twitter handle
Two hashtags: #CLUS #MyFavoriteSpeaker
You can submit an entry for more than one of your favorite speakers
Dont forget to follow @CiscoLive and @CiscoPress
View the official rules at http://bit.ly/CLUSwin
Promote Your Favorite Speaker and You Could be a Winner
98
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Complete Your Online Session Evaluation
Give us your feedback and you could win fabulous prizes. Winners announced daily.
Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center.
Dont forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
99
2014 Cisco and/or its affiliates. All rights reserved. LTRCRT-2605 Cisco Public
Continue Your Education
Demos
Labs
Lunch
Topics
Final copy TBD
100