© 2010 Cisco Systems, Inc. and VMware 1 Cisco and VMware: Virtualizing the Data Center VMware User Group forum Feb 2011 John Schaper Technical Solutions Architect Cisco Systems UCS in a VMworld UCS technical Overview
Dec 14, 2015
© 2010 Cisco Systems, Inc. and VMware 1
Cisco and VMware: Virtualizing the Data Center
VMware User Group forum Feb 2011
John SchaperTechnical Solutions ArchitectCisco Systems
UCS in a VMworld
UCS technical Overview
2© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Cisco Unified Computing System
2© 2008 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
A single system architecture Compute: Industry standard x86 Network: Unified fabric (Data Center Bridge) Virtualization: Control, scale, performance Storage Access: Wire once for SAN, NAS, iSCSI
Embedded management (single point) Increase scalability without added complexity Dynamic resource provisioning Ability to integrate with broad partner ecosystem
Highly efficient Fewer servers, switches, adapters, cables Lower power and cooling requirements Fewer people to deploy and manage Augments the VMware environment
3© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
UCS ManagerEmbedded – manages entire system
UCS Fabric Interconnect20 Port 10Gb FCoE - DCBx40 Port 10Gb FCoE - DCBx
UCS Fabric ExtenderRemote line card to fabric interconnect
UCS Blade Server ChassisFlexible bay configurations
UCS Compute Options Industry-standard x 86 architecture
UCS Virtual AdaptersChoice of multiple adapters (including virtualised)
Cisco UCS Modular Building Blocks
4© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
B200 M22 Socket Intel 5600, 2 SFF Disk, 12 DIMM
B440 M1 4 Socket Intel 7500, 4 SFF Disk, 32 DIMM
C200 M22 Socket Intel 5600, 4 Disks, 12 DIMM, 2 PCIe 1U
C210 M22 Socket Intel 5600, 16 Disks, 12 DIMM, 5 PCIe 2U
C250 M22 Socket Intel 5600, 8 Disks, 48 DIMM, 5 PCIe 2U
C460 M1 4 Socket Intel 7500, 12 Disks, 64 DIMM, 10 PCIe 4U
UCS Computing Options
B250 M22 Socket Intel 5600, 2 SFF Disk, 48 DIMM
B230 M1 2 Socket Intel 6500/7500, 2 SSD Disk, 32 DIMM
5© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
UCS Distributed Architecture using DCBX
• Wire once for bandwidth, not connectivity
• Unified Fabric – DCBx (extensions to Ethernet)
• All DCBx links are be active all the time
• Integrates as a single system into your data center
20Gb/s 40Gb/s 80Gb/s
LAN/SAN Uplinks
6© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Cisco UCS density - Single Rack example
48 Blade single rack solution using dual socket 8 core B230-M1 half width blades
Excellent VM density all 10GE DCBx Balanced across all dimensions
Processing: 768 Intel EX coresMemory: 12,288GBI/O: 960Gb
7© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Cisco UCS Solves VMware Data Center Management Complexity for the underlying hardware
Full visibility from one console–From software configuration settings all the way down to the BIOS: across servers, network, storage, VMs, and software
Service Profile Templates automate software configuration set up vs. one-off manual effort – stateless computing hardware
Simplify and control system admin tasks with role-based access controls
Automate movement of workloads to meet changing demand patterns and performance spikes
Automate discovery and population of configuration information to CMDB(s) via XML interface
Event reporting and incident diagnosis through built-in Cisco management software
Integration with leading service management tools
Network
Compute
VMware
Storage
Turnkey solutions that is pre-built, tested, and serviced as an integrated product
ie vBlock or Flexpod with VMware
8© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Introducing UCS Service Profiles (stateles computing)
UCS Service Profiles are very much like virtual machine definitions in a VMware environment
Configuration files store server characteristics:
Boot parameters NIC and HBA configuration (MAC, WWN, etc.) UUID Various policies (what happens when a link
fails?) All those items can be grouped in resource
pools Those service profiles are then mapped to
physical servers to very quickly provision one or more appropriate servers
PHYSICAL SERVERS
Service Profiles
Server NameUUIDMACWWNBoot infoLAN ConfigSAN Config
Server NameUUIDMACWWNBoot infoLAN ConfigSAN Config
9© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Embedded Unified Management - Service Profiles
• Unified Management Domain• Automatic discovery• Dynamic Provisioning
• Building Block for Dynamic Data Center• Simplify management of infrastructure for
ESX clusters and datacenters• One-click configuration of LAN, SAN
and firmware parameters
TightlyCoupledPartnerMgmtTools
ExistingCustomer
MgmtTools
XML API TraditionalAPIs
Service Profile: HR-VM-ESXiNetwork: HR-VLANNetwork QoS: HighMAC: 08:00:69:02:01:FC-EWWN: 5080020000075740-3BIOS: Version 1.03Boot Order: SAN, LAN
OSApp
Firmware
Network
10© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Name: UCS 2104Class: FEXID: 234222-33IOM 1: UCS 2104IOM 2: UCS 2104Blade slot occupied: 8Fans: 8
Name: UCS 5108Class: ChassisID: 234222-33IOM 1: UCS 2104IOM 2: UCS 2104Blade slot occupied: 8Fans: 8
Zero Touch IntegrationDecouple Complexity & Scale
• Increase capacity, not complexity• New equipment self integrates
Physical InventoryName: UCS 12Class: SystemID: 77449-32Chassis: 1- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8Chassis: 2- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8Chassis: 3- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8Chassis: 4- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8
Chassis: 5- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8
11© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Physical InventoryName: UCS 12Class: SystemID: 77449-32Chassis: 1- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8Chassis: 2- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8Chassis: 3- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8Chassis: 4- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8
Chassis: 5- IOM 1: UCS 2104- IOM 2: UCS 2104- Blade slots occupied:
8
Policy InventoryService Profile: ESXiService Profile: HR-VM-ESXi
Zero Touch IntegrationDecouple Complexity & Scale for hardware additions to VM farms
• Increase capacity, not complexity• New equipment self integrates• Inventory & status updated
12© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Zero Touch IntegrationDecouple Complexity & Scale
• Increase capacity, not complexity• New equipment self integrates• Inventory & status updated• Immediately apply existing policies
Service Policy InventoryService Profile: ESXiService Profile: HR-App1
13© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
UCS Manager Topology View
Backplane
4 link chassisdiscover policy
Uplink ports FC ports
IOMs
14© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
C-Series servers connectivity– Management connectivity through FEX to FI– Data connectivity directly to Fabric Interconnect
Stateless computing– Service profiles extended to C-series– Migration among compatible B & C series servers
All UCSM management services– Automated discovery– Fault and monitoring– Firmware updates
UCSM C-Series Rack server Integration
Unified Management across entire UCS portfolio
Advanced capabilities extended to rack servers
Customer benefits
Feature details UCS 6100 UCS 6100
Nexus 2248Nexus 2248
UCSManager
Data Connection
Mgmt Connection
15© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
UCSM XML API
Programmatic Infrastructure to north boundDevelop With The Infrastructure, Not Just On The Infrastructure
• Comprehensive XML API, standards-based interfaces• Bi-Directional access to physical & logical internals
System StatusPhysical InventoryLogical InventorySNMP GET
Direct UCS CLI UCS GUI Customer
Self Serve portals
Management Tools
Auditing Tools
3rd Party
16© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Virtualization Scalability, VMware benefits summary
Cisco Value Add HyperVisor Bypass(save those CPU cycles)
Cisco Value Add Memory extension up to
384Gb per blade (B250) High density racks Validated Vmware
designs vBlock, Flexpod….
CPU
I/O
Mem
ory
VM VMVM
VM
VMVM
VM
VMVM
VM
Cisco Value Add VN-Link in hardware (future 802.1QBH) 10GE scalability for VMotion and VM traffic with L2 extension Cisco Fabric Path OTV – L2 Data Center Interconnect
• Virtualizes more apps
• Increase VM density
• Increase visibility and control
17© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Update on
UCS Performance benchmark testing
Sept 21, 2011
VMmark performance
18© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data CenterSAVBU UCS Performance Benchmarks
at a glance – FY2010
Q4 FY09/Q1 FY10 Q2 FY10 Q3 FY10 Q4 FY10
May ‘09 – Oct ‘09 Nov ‘09 - Jan ’10 Feb ‘10 – Apr ’10 May ‘10 – Jul ‘10
SPECint_rate2006 B200 M1 – X5570,50,40,20C200 M1 and C210 M1 –
X5570, 50, 40, 20B200 M2 – X5680, 70, 50, 40
C460 M1 – X7560
C200 M2, C210 M2 – X5670, 50, 40; B440 M1 – X7560, 50,
40; C460 M1 – X7550, 40
SPECfp_rate2006 B200 M1 – X5570,50,40,20C200 M1 and C210 M1 –
X5570, 50, 40, 20B200 M2 – X5680, 70, 50, 40
C460 M1 – X7560
C200 M2, C210 M2 – X5670, 50, 40; B440 M1 – X7560, 50,
40; C460 M1 – X7550, 40
SPECjAppServer2004 C250 M2 (Single Node)
SPECjbb2005 B200 M1 – X5570, 50, 40, 20C200 M1 X5570, C210 M1
X5570, B250 M1 X5570B200 M2 – X5680C460 M1 – X7560
B440 M1 – X7560
VMmark B200 M1 – X5570 B200 M1 – X5570B250 M2 – X5680, C460 M1 – X7560
B440 M1 – X7560C460 M1 – X7560
SAP-SD 2-Tier B200 M1 – X5570 B200 M2 – X5680
SPEC OMP2001 B200 M1 – X5570 (M and L)C200 M1 and C210 M1 –
X5570B200 M2 – X5680 (M and L)C460 M1 – X7560 (M and L)
C200 M2, C210 M2 – X5670, 50, 40; B440 M1 – X7560, 50,
40; C460 M1 – X7550, 40
SPECpower_ssj2008 B200 M1 – X5570,50,40,20C200 M1 X5570, C210 M1
X5570, B250 M1 X5570B200 M2 – X5680C460 M1 – X7560
B440 M1 – X7560
Prime95/mPrime B200 M1 – X5570,50,40,20C200 M1 X5570, C210 M1
X5570, B250 M1 X5570B200 M2 – X5680C460 M1 – X7560
B440 M1 – X7560
Linpack B200 M1 – X5570C200 M1 X5570C210 M1 X5570
B200 M2 – X5680, 70, 50, 40C460 M1 – X7560
C200 M2, C210 M2 – X5670, 50, 40; B440 M1 – X7560, 50,
40; C460 M1 – X7550, 40
LS-Dyna C460 M1 – X7560 (3 Cars, Car2Car, Neon_refined)
Stream (diff mem cfg) B200 M1 – X5570,50,40,20C200 M1 X5570, C210 M1
X5570, B250 M1 X5570B200 M2 – X5680C460 M1 – X7560
B440 M1 – X7560
One or more new world records Featured in Press Releases/Keynotes
UCS platforms set 25+ new world records on highly competitive industry std benchmarks in FY2010
19© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Highest Ever VMmark World Record on UCS C460 M1
© 2010 Cisco Systems, Inc. and VMware 20
Cisco and VMware: Virtualizing the Data Center
VMware User Group forum Feb 2011
Data Center Network Virtualisation ArchitectureNexus NX-OS Overview
21© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Nexus 1000V VSM
Cisco Nexus 1000V Architecture for VMware
Nexus 1000V VSM
vCenter
Virtual Supervisor Module (VSM) Virtual or Physical appliance running
Cisco NXOS (supports HA) Performs management, monitoring, &
configuration Tight integration with VMware vCenter
Virtual Supervisor Module (VSM) Virtual or Physical appliance running
Cisco NXOS (supports HA) Performs management, monitoring, &
configuration Tight integration with VMware vCenter
Virtual Ethernet Module (VEM) Enables advanced networking
capability on the hypervisor Provides each VM with dedicated
“switch port” Collection of VEMs = 1 vNetwork
Distributed Switch
Virtual Ethernet Module (VEM) Enables advanced networking
capability on the hypervisor Provides each VM with dedicated
“switch port” Collection of VEMs = 1 vNetwork
Distributed Switch
Cisco Nexus 1000V Installation ESX & ESXi VUM & Manual Installation VEM is installed/upgraded like an ESX
patch
Cisco Nexus 1000V Installation ESX & ESXi VUM & Manual Installation VEM is installed/upgraded like an ESX
patch
vSphere
Nexus1000V VEM
vSphere vSphere
Nexus1000V VEM
Nexus1000V VEM
VM VM VM VM VM VM VM VM VM VM VM VM
22© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Features benefits of the Nexus 1000V
Switching L2 Switching, 802.1Q Tagging, VLAN Segmentation, Rate Limiting (TX)
IGMP Snooping, QoS Marking (COS & DSCP), Class-based WFQ*
Private VLAN’s for VM segmentation
Security Policy Mobility, Private VLANs w/ local PVLAN Enforcement
Access Control Lists (L2–4 w/ Redirect), Port Security
Dynamic ARP inspection, IP Source Guard, DHCP Snooping
Provisioning Automated vSwitch Config, Port Profiles, Virtual Center Integration
Optimized NIC Teaming with Virtual Port Channel – Host Mode
Visibility VMotion Tracking, NetFlow v.9 w/ NDE, CDP v.2
VM-Level Interface Statistics
Policy-based SPAN & ERSPAN
Management Virtual Center VM Provisioning, Cisco Network Provisioning, CiscoWorks
Cisco CLI, Radius, TACACs, Syslog, SNMP (v.1, 2, 3)
Hitless upgrade
*In 1.4 Release, 4Q CY2010
23© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Network Interface Virtualization
vNIC vNIC vNIC
vHBA vHBA
NIV allows a single physical PCIe device to be virtualized into multiple different PCIe devices
Provides true traffic segregation in hardware without need for VLANs or QinQ tags
Virtual Interfaces can be Ethernet NICs or Fibre Channel HBAs and presented as individual PCIe devices at the host level
Enforces the value proposition of FCoE, but can support iSCSI and NFS services
Driven to standards through IEEE 802.1Qbh working group
VN-Tag& VIC Protocol
24© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
OTV an innovation for vCloud & DCIOTV with VMware
L2 Domain Elasticity:vPC, L2MP/TRILLOTV
Workload Mobility
OTV
VN-link notifications
IP localization:
VM-awareness:VN-linkPort Profiles
Storage Elasticity:FCIP, IO AccelerationFCoE, Inter-VSAN routing
Device Virtualization:VDCs, VRF enhancements
OTV
Domain Elasticity
OTV
OTV
Compute resources are part of the cloud, location is transparent to the user
25© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Layer 3 strengths
Leverage bandwidth Fast convergence Highly scalable
Introducing Cisco FabricPathAn NX-OS Innovation for Layer 2 Networks
Simplicity Flexibility Bandwidth Availability Cost
Layer 2 strengths Simple configuration Flexible provisioning Low cost
Perform
ance Scale
Sim
plic
ityResilience
Fle
xibi
lityFabric
Path
"The FabricPath capability within Cisco's NX-OS offers dramatic increases in network scalability and resiliency for our service delivery data center. FabricPath extends the benefits of the Nexus 7000 in our network, allowing us to leverage a common platform, simplify operations, and reduce operational costs.”
Mr. Klaus Schmid, Head of DC Network & Operating, T-Systems International GmbH
26© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Architecture Flexibility Through NX-OS
Spanning-Tree vPC FabricPath
PodBandwidth
Active Paths
Up to 10 Tbps Up to 20 Tbps Up to 160 Tbps
Single Dual 16 Way
Infrastructure Virtualization and Capacity
Layer 2 Scalability
16Switches
© 2010 Cisco Systems, Inc. and VMware 27
Cisco and VMware: Virtualizing the Data Center
Thank You
28© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Nexus 1000V on vCenter
Nexus 1000V is a Distributed Virtual Switch
Physical SideVirtual Side
The
Sw
itch
29© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Nexus 1000V on vCenter
Nexus 1000V is a Distributed Virtual Switch
Physical Side
Virtual Side
30© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Port Profile: Network Admin View
n1000v# show port-profile name WebProfile port-profile WebProfile description: status: enabled capability uplink: no system vlans: port-group: WebProfile config attributes: switchport mode access switchport access vlan 110 no shutdown evaluated config attributes: switchport mode access switchport access vlan 110 no shutdown assigned interfaces: Veth10
Support Commands Include:
Port management VLAN PVLAN Port-channel ACL Netflow Port Security QoS
Support Commands Include:
Port management VLAN PVLAN Port-channel ACL Netflow Port Security QoS
31© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Port Profile: Server Admin View
32© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Nexus 1000V on the Host
Nexus 1000V is an ESX Host Software Packageesx-host # esxupdate --vib-view query--------------------VIB ID------------------- Package Statecross_cisco-vem-v120-esx_4.0.4.1.3.0.0-1.9.16 installed
esx-host # vmkload_mod -bName Size Used vmkernel 2713065 51vem-v120-l2device 24576 5 vem-v120-n1kv 77824 3 vem-v120-vssnet 14901248 3 vem-v120-stun 90112 1
esx-host # ps | grep vemdpa33959 33959 vemdpa
VEM Software Package
Data Path AgentCommunicates with VSM
Hypervisor “Drivers” – Packet Switching
33© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
NAM Virtual Blade on Nexus 1010 Optimize Application Performance and Network Resources
Application Performance Monitoring
Traffic Analysis and ReportingApplications, Host, Conversations, VLAN, QoS, etc.
Per-application, per-user traffic analysis
View VM-level Interface Statistics
Packet Capture and Decodes
Historical Reporting and Trending
ERSPAN
Nexus 1000V VSM
vSphere
Nexus1000V VEM
vCenter
NetFlow
NAM Virtual
Blade on Nexus 1010
VM VM VM VM
34© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
Cisco VN-Link Solution
A virtual network link between the switch and the VM Extends the network to the virtualization layer Enables:
– Policy-Based VM Connectivity– Mobility of Network & Security Properties– Non-Disruptive Operational Model
SWITCHVMVM
35© 2010 Cisco and VMware
Cisco and VMware: Virtualizing the Data Center
The scope of the VN-Link
Within the server (Hypervisor Switch)
Extending to physical upstream switch
SWITCHVMVM
UCS SERVER
SWITCHVMVM
SERVERNETWORK DEVICE
Network Interface Virtualization(VNTAG TechnologyIEEE 802.1Qbh pre-standard)
Nexus 1000V• IEEE 802.1Q
standard-based• Rich NX-OS
features