VMworld 2014: vSphere Distributed Switch

vSphere Distributed Switch: Technical Deep Dive

NET2745

Jason Nash, Varrow Chris Wahl, AHEAD

NET2745

Senior Solutions Architect @ AHEAD VCDX #104, vExpert Blogger – WahlNetwork.com Author – Networking for VMware Administrators Author – Pluralsight IT Pro training

CCNA Data Center, vCloud Director Host – VUPaaS and IT Engine Builders Podcasts

Chief Technology Officer @ Varrow VCDX #49, vExpert Blogger – JasonNash.com Author – Pluralsight IT Pro training

XtremIO, Cisco UCS, Nexus 1000v, vC Ops, + more

•  Overview •  Requirements •  Construction •  Alternatives •  Tips •  Real World Use Cases

ü  Enterprise Plus licensing ü  VMware skillset ü  ESXi host versions ≥ to VDS version

o  Lives in vCenter o  All 5.1+ features are Web Client only L o  VDS is a Data Center level object o  VDS database syncs with each ESXi host

o  Lives on the ESXi host o  We suggest •  Use Elastic ports •  Connect uplinks (vmnics) to a single network segment o  If you need multiple network segments •  Possible, but requires workarounds

Servers Segment

DMZ Segment

vmnic0

vmnic1

vmnic2

vmnic3

Port Group

VM1

VM2

VM3

VM4

VM5

Servers Segment

DMZ Segment

vmnic0

vmnic1

vmnic2

vmnic3

Port Group

VM1

VM3

VM5

VM2

VM4

Port Group

vmnic2

vmnic3

vmnic0

vmnic1

o  vSphere Standard Switch (VSS) o  Cisco Nexus 1000v o  IBM 5000V aka “Chupacabra”

ü Use 802.1Q tags for port groups ü At least 2 vmnics (uplinks) per VDS ü A 2x 10 GbE configuration can work fine ü  Put QoS tagging in VDS or physical, not both ü Use descriptive naming everywhere o  No one knows what “dvPortGroup-1” does

Migration VSS to VDS

Mixing 1Gb and 10Gb

Hosts

Handling vMotion

Saturation

vSphere Replication Bandwidth

Quality of Service Tagging

Load Based Teaming vs

Link Aggregation

Triggers: Ø  Licensing (purchased Enterprise Plus) Ø Consume features found only in VDS Ø Reduce operational overhead Ø Separate control planes and related responsibilities

Tips and Advice: Ø Have a detailed plan in place Ø Test the process on a single host with non-prod

VMs Ø Test network convergence time and ping drops Ø Become comfortable with the steps Ø  Put in a change control Ø  Execute change during maintenance window

Standard Switch

Distributed Switch

vmnic0

vmnic1

VM Port Group VM

VMKs VMK Port Group

VM Port Group

VMK Port Group

Standard Switch

Distributed Switch

vmnic0 VM Port Group VM

VMKs VMK Port Group

VM Port Group

VMK Port Group vmnic1

Standard Switch

Distributed Switch

vmnic0 VM Port Group VM

VMK Port Group

VM Port Group

VMK Port Group vmnic1 VMKs

Standard Switch

Distributed Switch

vmnic0 VM Port Group

VMK Port Group

VM Port Group


VM

Standard Switch

Distributed Switch

VM Port Group

VMK Port Group

VM Port Group


VM vmnic0

Distributed Switch

VM Port Group


VM vmnic0

Triggers: Ø  Purchase of new server / switch hardware Ø Staged migration to 10 GbE Ø Data Center transformation process

Tips and Advice: Ø Use a single network segment Ø Use a single VDS Ø Hosts should be entirely 1 GbE or 10 GbE Ø VM Traffic can traverse any uplink Ø Control teaming policies on VMK networks

dvUplinks

VM Port Groups VMK 1Gb (Mgmt, vMotion, etc)

ESXi 1 Gb

vmnic1 vmnic2 vmnic3 vmnic4

dvUplink1

1 2 3 4 1 2 3 4

dvUplink2 dvUplink3 dvUplink4

vmnic0 was left off to make the numbers

match

dvUplinks

VM Port Groups

ESXi 1 Gb

vmnic1 vmnic2

dvUplink1

VMK 10Gb (Mgmt, vMotion, etc)

1 2 1 2


dvUplinks

VM Port Groups VMK 1Gb (Mgmt, vMotion, etc)

ESXi 1 Gb

vmnic1 vmnic2 vmnic3 vmnic4

ESXi 1 Gb

vmnic1 vmnic2

dvUplink1

VMK 10Gb (Mgmt, vMotion, etc)

1 2 3 4 1 2 3 4 1 2


dvUplinks

VM Port Groups

ESXi 1 Gb

vmnic1 vmnic2

dvUplink1

1 2 3 4


No Link

Triggers: Ø Multiple hosts migrate VMs to a single host Ø  2+ host maintenance mode Ø DRS migrations Ø DRS affinity and anti-affinity rules

Tips and Advice Ø Know how Ingress vs Egress works in VDS Ø Use NIOC for source-based control Ø Use Traffic Shaping for destination-based control

Triggers: Ø Sharing a WAN pipe with other traffic Ø  Paying for bandwidth at certain % of peak Ø Multiple VR tenants between data centers Ø Contention with other backup or replication jobs

Tips and Advice: Ø Use NIOC with Limits (per vmnic) Ø Use Network Resource pool for VR Ø Alternative is to limit based on VR ports

Ø Run script for different limits during day/night Ø Requires PowerCLI and vCenter service account Short URL = http://goo.gl/dAgqBz

Triggers: Ø  Tag traffic for various SLAs Ø Use L2 Priority Code Point (PCP) Ø Use L3 Differentiated Services Code Point (DSCP) Ø Data Center Bridging extensions in 802.1 Ø  Priority-based Flow Control (PFC) – 802.1Qbb Ø  Enhanced Transmission Selection (ETS) – 802.1Qaz

Tips and Advice: Ø KISS: QoS solves contention problems Ø  Pick a place to tag traffic – virtual or physical Ø  Try not to enforce QoS in too many ways Ø Use clearly defined tagging when needed Ø Avoid hard limits on traffic flows

0 1 2 3 4 5 6 7

Triggers: Ø Network and Server teams not cooperating Ø  Pop out of those silos! Ø  Poor convergence times during link failover Ø  Poor use of uplink throughput Ø  Excessive Topology Change Notifications (TCN) Ø  Excessive vMotion activity

Load Distribution Ø Assigning workloads to uplinks based on identifiers Ø  Example: L2, L3, L4, and VLAN values

Load Balancing Ø Assigning workloads to uplinks based on traffic Ø  Example: “Route based on physical NIC load”

Load Distribution Ø No iSCSI Binding or Multi-NIC vMotion Ø  Potential Layer 2 Path Optimization

Load Balancing Ø  Imbalanced NIC saturation Ø Network can tolerate TCN and MAC table updates

Load Distribution Ø  Link Aggregation Group (LAG) Ø Static (EtherChannel) or Dynamic (LACP)

Load Balancing Ø Set PG to Route based on physical NIC load Ø Also known as Load Based Teaming (LBT)

Chris Wahl @ChrisWahl

WahlNetwork.com

Jason Nash @TheJasonNash JasonNash.com

Thank You

Fill out a survey Every completed survey is entered into a

drawing for a $25 VMware company store gift certificate

vSphere Distributed Switch: Technical Deep Dive

NET2745

Jason Nash, Varrow Chris Wahl, AHEAD