VMworld 2013: The Story Behind Designing and Building a Distributed Automation Framework for vCloud Hybrid Services

The Story Behind Designing and Building

a Distributed Automation Framework for

vCloud Hybrid Services

Nan Liu, VMware

Nicholas Weaver, VMware

PHC5640

#PHC5640

2

Who Are We?

project zombie

3

About Nan

Nan Liu

(bio + background)

Likes: attempting … to grow beards

Twitter: @sesshin

Linkedin: www.linkedin.com/in/nanliu

Blog: nanliu.github.io

4

About Nick

Nicholas Weaver

(bio + background)

Likes: clouds, clouds, clouds

Twitter: @lynxbat

Linkedin: www.linkedin.com/in/nicholasweaver

Blog: nickapedia.com

5

What Is vCloud Hybrid Service?

6

vCloud Hybrid Service: Extending to the Cloud

VMware vCloud

Hybrid Service

Your Data Center

Any Application… No Changes

Software-Defined

Data Center

VMware vSphere &

vCloud Suite

Existing & New Apps

Seamless Networking

Common Management

One Support Call

IaaS cloud owned and operated by VMware based on VMware software

7

What Do We Do?

8

Automation

9

10

11

12

Effort Evolution

13

Why Is Automation Important for vCHS?

14

What Are We Really Doing?

15

16

17

18

19

The Problem with Success Is…

20

21

Nothing Stays the Same

22

23

24

We Are Not the First

25

26

27

28

29

We Need Automation Special Sauce

30

project zombie

31

Key Principles

32

Scale

33

Extensibility

34

Simplicity

35

Resiliency

36

37

38

What is Project Zombie?

39

40

41

42

43

Rez

44

Distributed Resource Management

• Resources

• State

• Configuration

• Location

• Distributed Locking

• Ownership

• Event History

45

Distributed Resource Management

Razor

Razor

Storage

Manager

Storage

Manager

46

Engine

47

Zombie Engine DSL – ZED

-------------- 1 ---------------Init VPC- Engine - API Call

------------- 3 --------------Install ESXiEngine Broker Plugin - RazorInputs: - Razor IP - Blade - Hostname - Network InformationTasks: - Create Policy - Boot Blade

------------- 3 --------------Prep TLMEngine Broker Plugin - Puppet vCenterInputs: - TLM VCSA IP - TLM VCSA Creds - TLM DVS - Portgroup Name - Portgroup VLAN - Portgroup Settings - Ports: 128 - Load Balancing: IP Hash - Reset at Disconnect: EnabledTasks: - Create PGs - d#p#v#-dmz-pg-## - d#p#v#-esx-pg-## - d#p#v#-pvt-pg-##

----------- 4 ------------Deploy VPC VCSAEngine Broker Plugin - OVFToolInputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - Network - VM Hardware Settings - Memory: 16384Tasks: - Deploy VPC VCSA - d#p#v#-mgmt-vc0 - Update VM Hardware

----------- 4 ------------Deploy VPC VSE Pub PairEngine Broker Plugin - Puppet vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 1 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name - Type - Portgroup - Subnet Settings - Interface 1 - Name - Type - Portgroup - Subnet SettingsTasks: - Create VPC Pub Edge - d#p#v#-mgmt-vse-pub

----------- 4 ------------Deploy VPC VSE Priv PairEngine Broker Plugin - Puppet VMware-vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 2 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name: InterVPC - Type - Portgroup - Subnet Settings - Interface 1 - Name: DMZ - Type - Portgroup - Subnet Settings - Interface 2 - Name: PVT - Type - Portgroup - Subnet Settings - Interface 3 - Name: ESX - Type - Portgroup - Subnet SettingsTasks: - Create VPC Priv Edge - d#p#v#-mgmt-vse-priv

----------- 5 ------------Configure VPC VSE Pub PairEngine Broker Plugin - Puppet vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - DNS Settings - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default Route - SyslogTasks: - Configure VPC VSE Pub Pair

----------- 5 ------------Configure VPC VSE Priv PairEngine Broker Plugin - Puppet VMware-vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings: - Syslog - DNS - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default RouteTasks: - Configure VPC VSE Priv Pair

----------- 7 ------------Init VPC VCSAEngine Broker Plugin - Puppet VMware-VCSAInputs: - Credentials - NTP Settings - Syslog Settings - DB Settings - Sizing: LargeTasks: - Init VPC VCSA

----------- 4 ------------Deploy VPC VSMEngine Broker Plugin - OVFToolInputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - NetworkTasks: - Deploy VPC VSM - d#p#v#-mgmt-vsm0

----------- 9 ------------Configure VPC VSMEngine Broker Plugin - Puppet VMware-vShieldInputs: - VPC VSM IP - VPC VSM Creds - VPC VCSA IP - VPC VCSA Creds - NTP - SyslogTasks: - Configure VPC VSM

----------- 4 ------------Deploy VPC vCloudEngine Broker Plugin - ??Inputs: - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Names - VM Settings - NetworkTasks: - Deploy VPC vCloud Cells - Deploy VPC vCloud NFS - Deploy VPC vCloud DB

-------------- 2 -----------------Reserve VPCEngine Broker Plugin: - RezInputs: - VPC #

----------- 8 ------------Configure VPC vCenterEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP Address - VPC VCSA Credentials - Licenses - vCenter - ESXi - vCloud Net & Sec - Retention Policies - Task: Enabled - Event: Enabled - Datacenter Name - Cluster Settings - Name - DRS Settings - EVC Settings? - DVS Configuration - Name: d#p#v# - Settings - Uplinks: 4 - MTU: 9000 - Enable NIOC - Portgroup settings - Name - VLAN - Settings-Tasks: - Configure VPC VCSA - Configure DVS

----------- 7 ------------Init VPC VSMEngine Broker Plugin - ??Inputs: - VPC VSM Network SettingsTasks: - Init VPC VSM

----------- 6 ------------VPC VCSA Port CheckEngine Broker Plugin - UtilInputs: - VPC VCSA IPTasks: - VPC VCSA Port Check

----------- 6 ------------VPC VSM Port CheckEngine Broker Plugin - UtilInputs: - VPC VSM IPTasks: - VPC VSM Port Check

----------- 6 ------------Verify VPC ESXiEngine Broker Plugin - Puppet Util?Inputs: - VPC ESXi IP - VPC ESXii CredsTasks: - Verify VPC ESXi

----------- 9 ------------Add ESXi to VPC DatacenterEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC ESXi IP - VPC ESXii Creds - VPC VCSA IP - VPC VCSA Creds - VPC Datacenter NameTasks: - Add ESXi to VPC Datacenter

----------- 10 ------------Configure VPC ESXiEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - DNS Settings - NTP Settings - Syslog Settings - VMK Configuration - vMotion - Storage - FT - ???Tasks: - Configure ESXi - Add to DVS - Add VMK Networking

----------- 11 ------------Add Storage to VPC ESXiEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - iSCSI Settings - LUN InformationTasks: - Configure iSCSI - Add LUNs

------------- 3 ---------------Configure VNX VPC Storage GroupEngine Broker Plugin: - Storage ControllerInputs: - VPC # - LUN IDs - ESXi iSCSI informationTasks: - Create VPC Storage Group - Configure VNX Host registrations - Add LUNs to VPC Storage Group

----------- 6 ------------VPC vCloud Cell Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud Cell Port Check

----------- 6 ------------VPC vCloud Cell Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud Cell Port Check

----------- 6 ------------VPC vCloud NFS Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud NFS Port Check

----------- 6 ------------VPC vCloud DB Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud DB Port Check

----------- 7 ------------Configure VPC vCloud NFSEngine Broker Plugin - ??Inputs: - VPC vCloud NFS IP - VPC vCloud NFS Creds - NFS Export Settings - ???Tasks: - Config VPC vCloud NFS

----------- 7 ------------Configure VPC vCloud DBEngine Broker Plugin - ??Inputs: - VPC vCloud DB IP - VPC vCloud DB Creds - vCloud Database Config - ???Tasks: - Configure VPC vCloud DB

----------- 8 ------------Configure VPC vCloud CellEngine Broker Plugin - ??Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud NFS Config - VPC vCloud Installation Responses.properties - VPC vCloud Cell Cert - NTP SettingsTasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service

----------- 9 ------------Configure VPC vCloud CellEngine Broker Plugin - ??Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud Installation Responses.properties - VPC vCloud NFS Config - VPC vCloud Cell Cert - NTP SettingsTasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service

----------- 12 ------------Create VPC VM Storage ProfileEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - Storage Profile NameTasks: - Create VPC VM Storage Profile - Tag VPC Datastores

----------- 13 ------------Add VPC vCenter to vCloudEngine Broker Plugin - Puppet VMware-vCloudInputs: - VPC vCloud IP - VPC vCloud Creds - VPC VCSA IP - VPC VCSA Creds - ???Tasks: - Add VPC vCenter to VPC vCloud

----------- 10 ------------Configure VXLANEngine Broker Plugin - Puppet VMware-vShieldInputs: - VPC VSM IP - VPC VSM Creds - Multicast Information - VPC DVS Information - VPC Cluster - ???Tasks: - Configure VXLAN

----------- 11 ------------Reconfigure VXLAN VMK PortsEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - VXLAN VMK Network SettingsTasks: - Configure VXLAN VMK Ports

----------- 14 ------------Configure VPC vCloudEngine Broker Plugin - Puppet VMware-vCloudInputs: - VPC vCloud IP - VPC vCloud Creds - Provider VDC Name - External Org Network Name - ???Tasks: - Configure VPC vCloud

----------- 12 ------------Configure VPC Cluster HAEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - VPC ESXi Hosts - VPC Cluster - HA Settings - Failure: percentageTasks: - Add VPC ESXi hosts to Cluster- Configure VPC Cluster HA Settings

----------- 5 ------------Prep VPC vCloud CellEngine Broker Plugin - Puppet vCenter - ??Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static RoutesTasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking

----------- 5 ------------Prep VPC VCSAEngine Broker Plugin - ??Inputs: - Credentials - Network SettingsTasks: - Configure Networking

----------- 5 ------------Prep VPC vCloud CellEngine Broker Plugin - Puppet vCenter - ??Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static RoutesTasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking

----------- 5 ------------Prep VPC VSMEngine Broker Plugin - ??Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC VSM name - Network SettingsTasks: - PowerOn VM - Configure Networking

48

Zombie Engine DSL – ZED

Puppet

Apply

MCollective

MCollective

Puppet

Apply

MCollective

API Query

OVFTool

49

Zombie Engine DSL – ZED

Reserve Compute

Install ESXi

Add To Cluster

Add Compute

Reserve Compute

Install ESXi

Add To Cluster Replace Compute

Un-Reserve Compute

Remove From Cluster

50

Distributed and Location-aware Execution

Work to do

Concurrent

Work Concurrent

Work Concurrent

Work

Concurrent

Work Concurrent

Work Concurrent

Work

P

P

P

B

B B

B B

Datacenter: Dallas

Datacenter: Singapore

Work to do

51

VPC Deployment

52

Deployment

Diagram -> Deployment

-------------- 1 ---------------Init VPC- Engine - API Call

------------- 3 --------------Install ESXiEngine Broker Plugin - RazorInputs: - Razor IP - Blade - Hostname - Network InformationTasks: - Create Policy - Boot Blade

------------- 3 --------------Prep TLMEngine Broker Plugin - Puppet vCenterInputs: - TLM VCSA IP - TLM VCSA Creds - TLM DVS - Portgroup Name - Portgroup VLAN - Portgroup Settings - Ports: 128 - Load Balancing: IP Hash - Reset at Disconnect: EnabledTasks: - Create PGs - d#p#v#-dmz-pg-## - d#p#v#-esx-pg-## - d#p#v#-pvt-pg-##

----------- 4 ------------Deploy VPC VCSAEngine Broker Plugin - OVFToolInputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - Network - VM Hardware Settings - Memory: 16384Tasks: - Deploy VPC VCSA - d#p#v#-mgmt-vc0 - Update VM Hardware

----------- 4 ------------Deploy VPC VSE Pub PairEngine Broker Plugin - Puppet vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 1 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name - Type - Portgroup - Subnet Settings - Interface 1 - Name - Type - Portgroup - Subnet SettingsTasks: - Create VPC Pub Edge - d#p#v#-mgmt-vse-pub

----------- 4 ------------Deploy VPC VSE Priv PairEngine Broker Plugin - Puppet VMware-vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 2 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name: InterVPC - Type - Portgroup - Subnet Settings - Interface 1 - Name: DMZ - Type - Portgroup - Subnet Settings - Interface 2 - Name: PVT - Type - Portgroup - Subnet Settings - Interface 3 - Name: ESX - Type - Portgroup - Subnet SettingsTasks: - Create VPC Priv Edge - d#p#v#-mgmt-vse-priv

----------- 5 ------------Configure VPC VSE Pub PairEngine Broker Plugin - Puppet vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - DNS Settings - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default Route - SyslogTasks: - Configure VPC VSE Pub Pair

----------- 5 ------------Configure VPC VSE Priv PairEngine Broker Plugin - Puppet VMware-vShieldInputs: - TLM VSM IP - TLM VSM Creds - Edge Settings: - Syslog - DNS - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default RouteTasks: - Configure VPC VSE Priv Pair

----------- 7 ------------Init VPC VCSAEngine Broker Plugin - Puppet VMware-VCSAInputs: - Credentials - NTP Settings - Syslog Settings - DB Settings - Sizing: LargeTasks: - Init VPC VCSA

----------- 4 ------------Deploy VPC VSMEngine Broker Plugin - OVFToolInputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - NetworkTasks: - Deploy VPC VSM - d#p#v#-mgmt-vsm0

----------- 9 ------------Configure VPC VSMEngine Broker Plugin - Puppet VMware-vShieldInputs: - VPC VSM IP - VPC VSM Creds - VPC VCSA IP - VPC VCSA Creds - NTP - SyslogTasks: - Configure VPC VSM

----------- 4 ------------Deploy VPC vCloudEngine Broker Plugin - ??Inputs: - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Names - VM Settings - NetworkTasks: - Deploy VPC vCloud Cells - Deploy VPC vCloud NFS - Deploy VPC vCloud DB

-------------- 2 -----------------Reserve VPCEngine Broker Plugin: - RezInputs: - VPC #

----------- 8 ------------Configure VPC vCenterEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP Address - VPC VCSA Credentials - Licenses - vCenter - ESXi - vCloud Net & Sec - Retention Policies - Task: Enabled - Event: Enabled - Datacenter Name - Cluster Settings - Name - DRS Settings - EVC Settings? - DVS Configuration - Name: d#p#v# - Settings - Uplinks: 4 - MTU: 9000 - Enable NIOC - Portgroup settings - Name - VLAN - Settings-Tasks: - Configure VPC VCSA - Configure DVS

----------- 7 ------------Init VPC VSMEngine Broker Plugin - ??Inputs: - VPC VSM Network SettingsTasks: - Init VPC VSM

----------- 6 ------------VPC VCSA Port CheckEngine Broker Plugin - UtilInputs: - VPC VCSA IPTasks: - VPC VCSA Port Check

----------- 6 ------------VPC VSM Port CheckEngine Broker Plugin - UtilInputs: - VPC VSM IPTasks: - VPC VSM Port Check

----------- 6 ------------Verify VPC ESXiEngine Broker Plugin - Puppet Util?Inputs: - VPC ESXi IP - VPC ESXii CredsTasks: - Verify VPC ESXi

----------- 9 ------------Add ESXi to VPC DatacenterEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC ESXi IP - VPC ESXii Creds - VPC VCSA IP - VPC VCSA Creds - VPC Datacenter NameTasks: - Add ESXi to VPC Datacenter

----------- 10 ------------Configure VPC ESXiEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - DNS Settings - NTP Settings - Syslog Settings - VMK Configuration - vMotion - Storage - FT - ???Tasks: - Configure ESXi - Add to DVS - Add VMK Networking

----------- 11 ------------Add Storage to VPC ESXiEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - iSCSI Settings - LUN InformationTasks: - Configure iSCSI - Add LUNs

------------- 3 ---------------Configure VNX VPC Storage GroupEngine Broker Plugin: - Storage ControllerInputs: - VPC # - LUN IDs - ESXi iSCSI informationTasks: - Create VPC Storage Group - Configure VNX Host registrations - Add LUNs to VPC Storage Group

----------- 6 ------------VPC vCloud Cell Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud Cell Port Check

----------- 6 ------------VPC vCloud Cell Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud Cell Port Check

----------- 6 ------------VPC vCloud NFS Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud NFS Port Check

----------- 6 ------------VPC vCloud DB Port CheckEngine Broker Plugin - UtilInputs: - VPC vCloud Cell IPTasks: - VPC vCloud DB Port Check

----------- 7 ------------Configure VPC vCloud NFSEngine Broker Plugin - ??Inputs: - VPC vCloud NFS IP - VPC vCloud NFS Creds - NFS Export Settings - ???Tasks: - Config VPC vCloud NFS

----------- 7 ------------Configure VPC vCloud DBEngine Broker Plugin - ??Inputs: - VPC vCloud DB IP - VPC vCloud DB Creds - vCloud Database Config - ???Tasks: - Configure VPC vCloud DB

----------- 8 ------------Configure VPC vCloud CellEngine Broker Plugin - ??Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud NFS Config - VPC vCloud Installation Responses.properties - VPC vCloud Cell Cert - NTP SettingsTasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service

----------- 9 ------------Configure VPC vCloud CellEngine Broker Plugin - ??Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud Installation Responses.properties - VPC vCloud NFS Config - VPC vCloud Cell Cert - NTP SettingsTasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service

----------- 12 ------------Create VPC VM Storage ProfileEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - Storage Profile NameTasks: - Create VPC VM Storage Profile - Tag VPC Datastores

----------- 13 ------------Add VPC vCenter to vCloudEngine Broker Plugin - Puppet VMware-vCloudInputs: - VPC vCloud IP - VPC vCloud Creds - VPC VCSA IP - VPC VCSA Creds - ???Tasks: - Add VPC vCenter to VPC vCloud

----------- 10 ------------Configure VXLANEngine Broker Plugin - Puppet VMware-vShieldInputs: - VPC VSM IP - VPC VSM Creds - Multicast Information - VPC DVS Information - VPC Cluster - ???Tasks: - Configure VXLAN

----------- 11 ------------Reconfigure VXLAN VMK PortsEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - VXLAN VMK Network SettingsTasks: - Configure VXLAN VMK Ports

----------- 14 ------------Configure VPC vCloudEngine Broker Plugin - Puppet VMware-vCloudInputs: - VPC vCloud IP - VPC vCloud Creds - Provider VDC Name - External Org Network Name - ???Tasks: - Configure VPC vCloud

----------- 12 ------------Configure VPC Cluster HAEngine Broker Plugin - Puppet VMware-vCenterInputs: - VPC VCSA IP - VPC VCSA Creds - VPC ESXi Hosts - VPC Cluster - HA Settings - Failure: percentageTasks: - Add VPC ESXi hosts to Cluster- Configure VPC Cluster HA Settings

----------- 5 ------------Prep VPC vCloud CellEngine Broker Plugin - Puppet vCenter - ??Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static RoutesTasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking

----------- 5 ------------Prep VPC VCSAEngine Broker Plugin - ??Inputs: - Credentials - Network SettingsTasks: - Configure Networking

----------- 5 ------------Prep VPC vCloud CellEngine Broker Plugin - Puppet vCenter - ??Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static RoutesTasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking

----------- 5 ------------Prep VPC VSMEngine Broker Plugin - ??Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC VSM name - Network SettingsTasks: - PowerOn VM - Configure Networking

53

Documentation Fail

54

Click Fail

Add VPC Hosts to VCSA

Continuing from the vCenter view of the vSphere Web Client

In the main content pane, click Related Objects > Clusters

Click on VPC on the list

Click the second plus icon to add a host

• Host name: Enter first host FQDN

• Location: Confirm the location is set

• Click NEXT

…

Repeat Manually?

55

Script Fail

Easy from clean slate

Repeat?

Resume?

Modular?

Different Environment?

56

Puppet Configuration

Idempotent

Declarative

Small Components

Compose Infrastructure

57

Puppet Example

58

Mass Production

59

With Customization

60

Mass Customization

Data

+

Configuration Modules

61

Data

Information about environment

62

Configuration Module

How to deploy a specific service

63

Mass Customization

Data > Configuration Modules

# Compute * How to deploy ESX

64

Beyond Operating Systems

Managing Services though APIs:

• vCenter: vSphere API (SOAP)

• vCNS: vShield API (REST/XML)

• vCD: vCD API (REST)

65

Managing Appliances and Services

Puppet Management Server

VMware vCenter

SSH

Puppet Management Server

VMware vCenter

vSphere API

Datacenter

Folders

Datacenter

ESX

66

Managing vSphere/vShield/vCD APIs

Puppet Management Server

VMware vCenter

VMware vShield

Datacenter

ESXESX

ESX

Datacenter

ESXESX

vSphere API

vShield API

Edge

67

No Manual Interaction

68

VPC Initialization

Software Defined Networking deployment

• Network Load Balancers

• Firewall

• Routing

Compute (ESX)

Storage

vCenter + vCNS

vCD

VM/OS Deployment and Configuration

Application Deployment and Configuration

69

Does It Work?

70

71

Cloud Build A

Maximum capacity per week:

Man hours: 72

2

Time to delivery: 6 days

72

Maximum capacity per week:

Man hours: 1.5

336 x Zombie Engine instance count

Time to delivery: 2.5 hours*

Cloud Build A

73

Details (per job):

• ~120 individual tasks (plugin calls)

• ~2700 discrete configuration points

• ~1400 managed resources

• Dynamically sized (pick the # of compute and storage)

• Controls: vCloud Director, vCenter, ESXi, EMC VNX,

Razor, vShield Manager, vShield Edge, Linux and

Windows operating systems

Cloud Build A

74

Cloud Build A

75

So Yeah… It Works

76

What’s Next?

77

SDN

Big Data

Datacenter Automation

Faster, Faster, Faster

project zombie

78

VMware Puppet Modules

Puppet Enterprise Users:

http://forge.puppetlabs.com/vmware

Puppet Developers:

http://github.com/vmware/vmware-vmware_lib

http://github.com/vmware/vmware-vcsa

http://github.com/vmware/vmware-vcenter

http://github.com/vmware/vmware-vshield

79

Coming Soon…

vCloud Director

• PVDC Management

• External networks

• Roles

• Users

• Standard switch to distributed switch migration

80

Bring Your Own Licenses

Bring Your Own VMs

Bring Your Own Tools

Customer Components

VMware vCloud

Hybrid Service

Hybrid Service Delivery Components

Infrastructure Hardware & Facilities

Compute Storage Networking &

Security

OS Catalogs

Application Catalogs

Infrastructure Management

Web Console vCloud API

81

project zombie

Q & A

82

Other VMware Activities Related to This Session

HOL:

HOL-HBD-1303

vCloud Hybrid Service - Manage Your Cloud

PHC5640

THANK YOU

The Story Behind Designing and Building

a Distributed Automation Framework for

vCloud Hybrid Services

Nan Liu, VMware

Nicholas Weaver, VMware

PHC5640

#PHC5640