The Grid and Virtualization Orran Krieger Sr. Technical Staff Member Vmware
Mar 27, 2015
The Grid and Virtualization
Orran KriegerSr. Technical Staff Member
Vmware
2
• A level of indirection between hardware and software
• Virtual Machines: a full abstraction of a physical machine• Safely and efficiently multiplex virtual hardware on physical hardware
• Benefits: resource sharing, portability, compatibility
Physical machine
Virtual machine
What is Virtualization?
3
Before Server Virtualization: Single OS image per machine Software and hardware tightly coupled Running multiple applications on same
machine often creates conflict Underutilized resources Fault/security vulnerability
Server Virtualization
4
Before Server Virtualization: Single OS image per machine Software and hardware tightly coupled Running multiple applications on same
machine often creates conflict Underutilized resources Fault/security vulnerability
After Server Virtualization: Virtual machines (VMs) break 1-to-1
dependency between OS and HW Manage OS and application as single
unit by encapsulating them into VMs Hardware-independent: they can be
provisioned anywhere Strong isolation between VMs
Server Virtualization
5
VMMVMM VMMVMM
VM Encapsulation and Mobility
Suspend
Resume
Snapshot
Clone
Migrate
Record
Replay
etc.
Virtual Machines are more versatile than Physical MachinesVirtual Machines are more versatile than Physical Machines
LogLog
6
VMkernelVMkernel
HAL and Device DriversHAL and Device Drivers
Resource ManagementResource Management
NetworkingNetworkingStorageStorage
RHEL3-basedRHEL3-basedService ConsoleService Console
HelpersHelpers VMMVMM
AgentAgent AgentAgent……
VMMVMM
Disk Footprint:Disk Footprint: 2 GB2 GB32 MB32 MB
RPMRPM
VMMVMM
Disk Footprint:Disk Footprint:
Traditional ESX Server
Percent of PatchesPercent of Patches >50%>50%
98%98% 2%2%
7
ESX Server 3i: Thin Virtualization!
VMkernelVMkernel
Resource ManagementResource Management
NetworkingNetworkingStorageStorage
RHEL3-basedRHEL3-basedService ConsoleService Console
VMMVMM
AgentAgent AgentAgent……
VMMVMM
RPMRPM
VMMVMMHelpersHelpers
HAL and Device DriversHAL and Device Drivers
Disk Footprint:Disk Footprint: 2 GB2 GB32 MB32 MBDisk Footprint:Disk Footprint:
Percent of PatchesPercent of Patches >50%>50%
98%98% 2%2%
Small footprint with OS-independenceVirtualization should have the reliability, security, and performance of hardware
Less code and fewer interfaces:Compact 32MB footprint
CIM- and remote CLI-based management
Aids both security and reliability (no legacy)
Integrated in server systems:Hardware is certified and ready-to-run
Simple start up experience
Launched with IBM, HP, Dell, FSC, NEC.
In the next few years, virtualization will be pervasive embedded in all HW
8
Grid and grid-like technologies are the future
• Web applications being developed with scale-out frameworks
• MapReduce and related frameworks…, Data Intensive Supercomputing
• Utilities inside companies • Hosting companies like Rackspace• Shared utilities like Amazon’s EC2• The rise of SAAS like salesforce.com • HPC capacity and capability systems
Are both technologies really going to be pervasive?Are both technologies really going to be pervasive?Grid and Virtualization are complementary:Grid and Virtualization are complementary:•Grid: motivated by analogy of power gridGrid: motivated by analogy of power grid•Virtualization: converts computation into a fungible Virtualization: converts computation into a fungible commodity; hugely simplifies realization of grid commodity; hugely simplifies realization of grid
9
Traditional Grid Adoption Challenges
• Security• Grid jobs can potentially compromise security on host machine
• Intruders may piggy-back on Grid jobs to attack the system
• Isolation• Grid jobs may “eat” into host machine's resources
• Buggy grid job may crash host system
• OSes often can’t control resource usage between Grid jobs• Forces single job per host models
• OS heterogeneity limits number of Grid nodes available for a job
• Service level guarantees• Enterprise grid jobs need guarantees on completion time
• In many cases, applications must be re-written for Grid
10
Traditional Grid Adoption Challenges (Contd)
• Provisioning• Increased activity on host system can slow down Grid jobs
• Utilization• Dedicated machines for Grid jobs lead to severe under-utilization
• Legacy applications fail to utilize multi-core features
• Management• Grids lead to increase in number of “nodes”
• Management of these nodes is a big challenge
• Infrastructure for application virtualization• Grid middleware lack support from infrastructure layer for on-demand
deployment of application instances
• Grid implementation limited by having to use commodity OSes• Large attack surface, high overhead, …
11
Virtualization in Grid Computing• “Any problem in computer science can be solved
with another level of indirection” (David Wheeler)
• Virtualization provides a natural level of indirection to address Grid Computing challenges• Inherent “cures” to Grid adoption issues
• Same solution applies to broader utility, hosting, enterprise, web … marketplace
12
Benefits of Virtualization in Grids
• Virtual machines (VM) provide robust security
• A compromised VM can be isolated without affecting the host
• VMs provide excellent isolation
• A Virtual Grid node (VGN) crash has no effect on other VGNs or host
• VGNs can guarantee service level times
• VMs can be provisioned with required explicit resource reservation• CPU bandwidth, memory, disk space, disk bandwidth, etc
• Ensures the grid job has the muscle power to complete within the SLA
• Virtualization reduces data centre complexity
• Abstracts out infrastructure heterogeneity
13
Benefits of Virtualization in Grids (Contd)
• No resource usage creep with VMs• Provisioning places limit on resources that can be consumed
• Very useful for “resource scavenging” Grids• Host performance does not deteriorate
• Multiple VGNs on a single physical machine• Increases utilization of Grid boxes
• Enhances workload management capability of the Grid• Additional VGNs can be started and deployed instantly
• On-demand provisioning of VGNs eliminates OS heterogeneity issues
• As many VGNs of a particular OS as needed
14
Benefits of Virtualization in Grids (Contd)• Dynamic resource management
• (1) Vary resource allocated to a VGN based on load/requirement
• (2) Move VGN to a more/less powerful host
• Provisioning additional grid nodes becomes trivial• Maintain repository of pre-created VM images
• Choose and bring-up VGN based on requirement
• Ease of management• Infrastructure can be managed through virtual machine management
solutions
• Maintenance and upgrade of VMs easier than physical servers
• Custom OS support• Special purpose OSes, specialized for Grid applications can be
developed, resulting in major performance, reliability, simplicity advantages.
• Lets the Grid application get much closer to the metal.
15
Grid Computing
GridGrid
MiddlewareMiddleware
GridGrid
MiddlewareMiddleware
Grid JobGrid Job
Grid JobGrid Job
Grid JobGrid Job
Grid JobGrid Job
Grid JobGrid JobGrid JobGrid Job
Jobs IN/OUTJobs IN/OUT
16
Virtualization based Grid
Grid jobGrid job Other JobOther Job
Grid MiddlewareGrid MiddlewareGrid MiddlewareGrid MiddlewareJobs IN/OUTJobs IN/OUT
•Grid middleware can focus on “core” Grid issues.
• Virtual infrastructure provides simplified abstraction used by Grid middleware and other applications.
17
VMware’s GRID enablement features
• Base features: low-overhead, strong isolation…• Integrated management of virtual and physical
infrastructure. • Guaranteed quality of service:
• VMotion to move tasks between physical machines• Distributed resource scheduling
• Reliability:• Snapshotting for checkpointing of long running jobs • High availability
• Cheap creating of VMs• memory overcommit• templates & clones
• Special purpose appliances
18
Towards Distributed Virtualization
RESOURCE RESOURCE POOLPOOL
RESOURCE RESOURCE POOLPOOL
RP 1BRP 1B
Machine VirtualizationMachine VirtualizationPhysical MachinesPhysical Machines Distributed VirtualizationDistributed Virtualization
RP1ARP1A
OSOS OSOS
19
VMotion™
• Migrate running VMs from one server to another• Encapsulation and HW independence are the key enablers
• To minimize service disruption:• Iteratively pre-copy VM memory from source to destination• Store virtual disks on networked storage: FC, iSCSI, NAS
• VGNs moved across ESX hosts with zero downtime; No need to restart Grid jobs• Allows efficient usage of hardware resources and improves overall hardware
utilization• Similar VGNs with can be grouped together for better performance
20
Storage VMotion
• Storage independent migration of virtual machine disks
• Zero downtime to virtual machines
• LUN independent
• Supported for Fibre channel SANs
21
Automatic load balancing across hosts
Distributed ResourceDistributed Resource
Scheduling (DRS)Scheduling (DRS)Dynamic BalancingDynamic Balancing
Continuous OptimizationContinuous Optimization
22
Adding and removing hosts
Hot-plug machinesHot-plug machinesAdd/remove capacity on demandAdd/remove capacity on demandDistributed power optimizationDistributed power optimizationImprove application availabilityImprove application availability
DRSDRS
XX
23
HWHW HWHW HWHW HWHW HWHW
HWHW HWHW HWHW HWHW HWHW
HWHW HWHW
HWHW HWHWHWHW
HWHW
HWHWHWHW
Transforming capacity management
VIRTUALIZEDVIRTUALIZEDPHYSICALPHYSICAL POOLEDPOOLED
RRPP11
RRPP22
Logical Resource Pooling (RP)Logical Resource Pooling (RP)Distributed Resource Scheduler (DRS)Distributed Resource Scheduler (DRS)
24
Firewall SW
Linux
FIREWALLVirtual Appliance
Apache
Linux
Linux
LinuxTomcat
mySQLCRMVirtual Appliance
What is a Virtual Appliance
• Pre-built, pre-configured and ready-to-run software application packaged with the OS inside a Virtual Machine.
• Or packaged inside multiple Virtual Machines
25
•Appliance OSes can be simplified or customized
• Don’t need complex hardware management
• Don’t need broad application support
• Highly-customizable, application-specific OS
• Include only what application needs.
• Look at hardware appliance operating systems for examples
Hardware Mgmt.
Application Support
ApplicationApplication
Operating System
Virtualization Layer
Virtual Appliance OS
26
• Virtualization and Grid will both be pervasive technologies
• Virtualization will increasingly be embedded on all server platforms.• Various forms of utility computing are increasingly dominating many markets.
• Virtualization and Grid are complementary.• Its time to start working together:
• What is the right interface between Grid middleware and virtual infrastructure?• What are the requirements of the Grid? Determinism for collective operations? Fast
start up of VM? Fast Clone…
• Its time to write Grid appliances/OSes. • How tiny can they be? Plan9?• Does a Library OS model make sense? Libra/Prose? • Is there any paravirtualization support that would be useful?
Orran Krieger <[email protected]>
Concluding remarks
27
Thanks
• Hariprasad Nellitheertha
• Chinmay Albal
• Radhika Kulkarni
• Larry Rudolph; Steve Strassman; Scott Davis; Jack Lo
28
29
VMware Memory Overcommit
• Unique VMware Feature
• Host system memory is usually the resource bottleneck
• VMware memory management technology allows an average of 2:1 memory overcommit (and oftentimes greater)
Transparent Page SharingTransparent Page Sharing Memory ballooningMemory ballooning
30
HW Independence & Transparency
Run a virtual machine on any PC without modification. Break
dependence between OS & HW
Partitioning
Run multiple virtual machines simultaneously on a single PC
Isolation
The virtual machine is isolated from Host OS, other VMs on the
same PC
Encapsulation
Entire virtual machine is saved in files. Can be moved and copied
like any other file.
Key Properties of Virtualization