VMUGIT UC 2013 - 04 Duncan Epping

Architecting a virtual infrastructure

Duncan Epping

Principal Architect

Integration Engineering – R&D

Agenda

Method Q/AIntro Design Patterns

Introduction

• Duncan Epping

� Principal Architect

� Integration Engineering – R&D

• Also known as Customer 0

� VCDX 007

� Owner of Yellow-Bricks.com

� Twitter @DuncanYB

What is this session about?

• Architecting a VMware based infrastructure

• Design Decisions

• Real world examples

• Understanding the potential pitfalls

• Feel free to tweet / blog about this session

• Most importantly – Interact!

Agenda

Method Q/AOverview Design Patterns

Vision

Architecture

PlanTransition

Manage

Governance

Requirements Management

Change Mgmt

Method

Articulated vision

Business and Technical Goals

Requirements, Assumptions,

Constraints, Scope, Risks,

Use Case Definition

Gap Analysis

Architecture Definition

(Business, Information

Systems, Technology)

Roadmap definition

Implementation planning

Iteration planning

Implement solution

Validation

Continuous monitoring

Optimization

Gather input

• Most important part of any project

• Identify your stakeholders

• Gather information and document

• Categorize

� Requirements

� Nice to have

� Constraints

� Assumptions

� Risks

• Conceptualize

• Sound like VCDX to Anyone?

What are we trying to achieve?

• Many architectural discussions are about challenges

• What are your current challenges?

• How do you solve these today?

• How would you like to solve these tomorrow?

• Ultimately, what are we trying to achieve?

IT Can Take Weeks to Provision

Server deliveryUnpack

Build and configure processMigrate to dev test center

infrastructure network

storage

facilities

application

Security

Change controlapprovals

Go live/production

User Acceptance Testing

Quality Assurance

Ready for use

CAB/Deploymentplanning meetings

Line of businessselects service

Assesscapacity

Order

serverApprove

purchase order

That could be… building an SDDC

Software-Defined Data Center

All infrastructure is virtualized and

delivered as a service,

and the control of this data center is

entirely automated

by software.

Use cases

• Server consolidation (power and cooling savings, green

computing, lowering TCO)

• OPEX Savings on redundant tasks

� Self Service Provisioning

• Server infrastructure resource optimization (load

balancing, high availability)

• Standardization

• Business Agility (Rapid provisioning)

• Infrastructure as a Service

Business Requirements

B101

B102

B103

B104

B105

B106

B107

B108

B109

B110

System provides separate dedicated environments.

Complete automation of the provisioning process.

System leverages shared infrastructure and resource pooling.

Provisioning based on pre-defined policies.

Lowering time to market

Technical Requirements

T101

T102

T103

T104

T105

T106

T107

T108

T109

T110

Must integrate with existing ticketing system

Leverage thin provisioning for storage efficiency

Centralized LDAP directory to be used

System supports a catalog of standardized vApp templates.

System provides differentiated offerings based on cost.

Orchestrated disaster recovery

Constraints

C101

C102

C103

C104

C105

C106

C107

C108

C109

C110

Dell and AMD have been preselected as the platform of choice

Eight 1GbE ports will be used per server

NetApp’s NAS storage will be used

All Tier 2 NAS volumes are de-duplicated

Physical switches will not be configured for QoS

Existing Cisco TOR environment to be used

Limited VLANs available

Assumptions

A101

A102

A103

A104

A105

A106

A107

A108

A109

A110

Staff trained on VMware technology

Shared storage configured

VLANs and IP address reserved

Sufficient rack space available

Sufficient power and cooling available

Agenda

Method Q/AIntro Design Patterns

Design Patterns

Sizing and Scaling

Hosts

Networking

Storage

Virtual Constructs

Infrastructure

Design

Infrastructure

Design

Basic details

What is the use case?

� Time to Market / Business Agility?

� Business Continuity?

What does the environment look like today?

� How many sites?

� How many potential virtualization candidates?

� Multiple waves?

How will this impact your Design / Project?

� Different Cluster / Datacenter structure

� Within the limits?

Sizing and scaling

Use tools like

� VMware Capacity Planner

� PlateSpin Recon

� Lanamark

Do we really need it?

� Don’t all results just look the same?

� What is important?

What am I designing for?

� Average vs Peak

� Consolidation vs Performance

Compute Considerations

How many eggs in on basket?

� Two sockets vs four sockets

� Impact of 100:1 consolidation ratio?

AMD vs Intel

� AMD supports more cores, while Intel generally is faster

� VMmark can be used to make perf comparisons!

Sweet spot?

� Still seems to be dual socket – 96GB of memory

� Optimal Memory configurations

• DIMM sizes versus price

• Triple channel configurations

• Number of DIMM slots might be different per vendor / model

Network Sizing

Is this ever really a bottleneck?

In most of the Capacity Planner reports we’ve seen…

� Expected average network bandwidth requirement ~ 4Mbpsbased on an average of 20 VMs per ESXi host.

10GbE will lift all (or most) constraints for a very long time!

Use the report to identify anomalies!

Storage Sizing

Not only size but performance matters!

(TOTAL IOps × % READ)+ ((TOTAL IOps × % WRITE) × RAID Penalty)

(42 x 62%) + (( 42 x 38%) x 2)

(26.04) + ((15.96) x 2)

26.04 + 31.92 = 57.96

But what about size?

How does this drive your Storage Considerations?

Storage Considerations

RAID level used � impacts IOps

� IOps penalty can be severe

� Spindle count

RTO � impacts amount of VMs per Datastore

� Backup environment need to be capable of restoring within RTO window

� (RTO * restore speed) / avg VM Size

Adding it up…

� 270 VMs from backup perspective vs 50 VMs from IOps

� What does your customer feel comfortable with?

Going SSD / Hybrid solutions?

Host Design

Design Considerations

Vendor

AMD vs Intel

Blade vs Rack

� Density increases

� Hot spots

� Costs

� Management

Additional considerations

� Is embedded ESXi available?

� How much local SSD (capacity and IOPS) can it handle?

� Does it have built-in 2x 10 GE ports?

� Does the built-in NIC card have hardware iSCSI capability?

� Management integration

ESXi boot considerations

4 methods of booting ESXi

� Local Disk

� Local SD / USB

� SAN Boot

� PXE Boot with Auto-Deploy

Considerations

� USB is cheap

� Local disk usually higher availability than USB

� SAN Boot easy to move identity, but what about costs

� Best of all worlds: Auto-Deploy

• Relatively new and dependencies…

vCenter and Clustering

Design Considerations

How many VMs?

� Do I need a dedicated vCenter Server?

� Can I still use the vCenter Appliance?

� Use the sizing calculators!

Is there a need for the Web Client?

� Can I use the vCenter Appliance for that?

� Use it!

Will there be other products used like SRM / View / vCloud

Director?

� vCenter Heartbeat requirements?

Where do we start?

How many physical Datacenters will there be?

Will each physical DC need a vCenter Server?

For each vCenter, do we need multiple virtual Datacenters?

For each DC, do we need multiple Clusters?

For each Cluster, how many hosts?

Physical DC

vCenter

Datacenter Datacenter

Cluster Cluster

ESXi ESXi ESXi

vCenter

Datacenter

Design Considerations

Separate clusters for DMZ?

� Why not use vShield App or vShield Edge?

Separate clusters for test?

� Might also allow you to test vSphere patches!

vCenter Datacenter object is a vMotion boundary not the

Cluster!

vCenter Datacenter object is a VDS boundary not the Cluster!

Will you be using HA / FT / DRS / DPM?

� Will this impact your design?

Design Considerations

Is 8 the perfect Cluster size?� Primary / secondary nodes (4.1 and prior) vs Master / Slave (5.x)

� Blade environment implication on design?

� Linked Clones restriction lifter in 5.1!

� DRS and DPM love big clusters

What about EVC? Should I turn it on by default…� EVC can only be enabled when all VMs are powered off

� Did you know that DRS requires EVC to be enabled to balance and place FT virtual machines in a cluster?

Is there a need for Resource Pools?� How will you handle shares?

� Reservations / limits?

Storage

Design Considerations

Protocol Wars!

Multiple Tiers?

� Or even Auto-Tiering, what is the impact?

vSphere Storage APIs – Array Integration (VAAI)

� Does it impact sizing?

vSphere Storage APIs – Storage Awareness (VASA)

� Will it impact operations?

Thin provisioning?

� Thin, Thick and Eager Zeroed Thick

� vSphere vs Storage Array!

Design Considerations

Can we use Storage DRS?

� Impact on storage array features?

� Impact on sizing?

� Impact on other VMware products like vCloud Director?

Profile-Driven Storage?

� How does it utilize VASA?

DR Requirements?

� Or possibly in the future?

No more worrying about block sizes with VMFS-5

� When upgrading VMFS-3 to VMFS-5 block size does not change!

Did you know VAAI is T-10 compliant?

� Makes leveraging it easier for lower-end devices

Impact of Features

Storage DRS has constraints…

SRM does not support Storage vMotion / Storage DRS

vCloud Director does support Storage DRS in 5.1!

Feature or Product Initial Placement Migration Recommendations

Array-based replication

(SRDF, MirrorView, SnapMirror, etc )Supported Manual: I/O and Space

Array-based snapshots Supported Manual: I/O and Space

Array-based Dedupe Supported Manual: I/O and Space

Array based thin provisioning SupportedFully Automated: I/O and Space

Array-based auto-tiering

(EMC FAST, Compellent Data Progression, etc)Supported Manual: Space

Array-based I/O balancing

(Dell Equallogic)Supported Manual: Space

Networking

Design Considerations

What type of vSwitch will be used?� VSS vs VDS

What are the pSwitch capabilities?

Requirements for Jumbo Frames?

What type of load balancing will be used?� What type of traffic

� Load Based Teaming vs Virtual Port ID vs IP Hash

How about Network Virtualization?

Design Considerations

Network I/O Control� Even in 1GbE environment NIOC is useful

� Especially when connecting outbound

� Did you know that Limits apply to a NIC pair level?

� Did you know that shares apply on a NIC Port level?

Additional Security Requirements?� vShield App

� vShield App with Data Security

� vShield Edge

3rd party security products?� Hy-trust

• 2-factor authentication• Audit trails• Rigid, Hierarchical Access Controls

Agenda

Method Q/AIntro Design Patterns

And we haven’t even touched on…

� Automation / Orchestration

� Monitoring

Questions?