Cloud Services Innovation Platform (CSIP)
Dynamic Scaling for Service Oriented Applications: Implications
of Virtual Machine Placement on Infrastructure-as-a-Service
CloudsWes Lloyd, Shrideep Pallickara, Olaf David, Mazdak Arabi, Ken
Rojas
March 13, 2014
Colorado State University, Fort Collins, Colorado USAIC2E 2014:
IEEE International Conference on Cloud Engineering
1OutlineBackgroundResearch ProblemResearch QuestionsExperimental
SetupExperimental ResultsConclusions2March 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS Clouds2Background3Dynamic Scaling forService
Oriented Applications
4WHENHot Spot DetectionVM Launch LatencyFuture Load
PredictionPre-provisioning
WHAT
WHEREMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsDynamic Scaling forService Oriented Applications
5WHATSizeVertical ScalingQuantityHorizontal ScalingLoad
BalancingVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVM
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsDynamic Scaling forService Oriented Applications
6WHEREPhysical HostPhysical HostPhysical HostPhysical
HostVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMLaunch RequestsVM
SchedulingMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsResearch Problem7IaaS Cloud: VM PlacementIn practice there
are two predominant VM placement schemesGreedy fill first:
consolidateRound-robin spread first: load balanceCommonly provided
byEucalyptus, OpenStack, OpenNebula, Apache CloudStackWhat are the
performance implications for dynamic scaling?For SOAs, are they
sufficient? 8Physical HostPhysical HostPhysical
HostVMVMVMVMVMVMMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for
Service Oriented Applications: Implications of VM Placement on IaaS
CloudsResearch Questions9Research QuestionsWhat performance
implications result from VM placement location when dynamically
scaling service oriented applications?
How important is VM placement for scaling in response to
increasing service demand?
How do resource costs (# of VMs) vary when dynamically scaling
service oriented applications as a result of VM placement
location?
10RQ1:
RQ2:March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsExperimental Setup11
ModelingServicesUSDA-NRCS - Soil erosion modelsRUSLE2: soil loss
from rainfall and runoffWEPS: soil loss from wind12
OMS3RUSLE2/WEPSPOSTGRESQLPOSTGIS~6 million shapes340k XML
filesMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsVM-Scaler13VM-ScalerInfrastructure Mgmt RequestsApplication
Service RequestProxy ServicesCloud Mgmt ServicesElastic Compute
Cloud API Physical HostPhysical HostPhysical HostPhysical
HostPhysical HostPhysical HostVMVMVMVMVMVMVMVMVMVMVMVMVMVM
Physical HostPhysical HostPhysical HostPhysical HostPhysical
HostPhysical HostVMVMVMVMVMVMVMVMVMVMVMVMVMVM
futureLeast-Busy VM SchedulerMarch 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS CloudsLeast-Busy VM PlacementRU-sensors
collect Virtual/Physical machine data @ 15s/intervalsBusy-Metric
used to calculate aggregate load at each physical machineFlexible
metric designObjective not to design perfect metric / VM
scheduler
Resource Utilization Data14Network- Network bytes sent (NBR)-
Network bytes received (NBS)CPU- Total CPU time weighted 2X
Disk- Disk sector reads (DSR)- Disk sector writes
(DSW)Virtualization- Total VM count per hostMarch 13, 2014 IEEE
IC2E 2014 Dynamic Scaling for Service Oriented Applications:
Implications of VM Placement on IaaS CloudsDynamic Scaling
TestsShared cluster load simulation
Modeling workloads
15
Rusle2:7,000 runs/test
WEPS:300 runs/test Initial load before scaling Simulates
multi-tenant cloud environments
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsExperimental Results16Application Performance Improvement vs.
Round-Robin VM Placement17
Normalized %Performance ImprovementStatistical significance
Average Performance Improvement:
~16.1% RUSLE2~11.6% WEPS_~14% aggregateMarch 13, 2014 IEEE IC2E
2014 Dynamic Scaling for Service Oriented Applications:
Implications of VM Placement on IaaS CloudsResource Cost Savingsvs.
Round-Robin VM Placement
18Resource Cost Savings% Fewer VMsAverage Savings:
~2.7% fewer VMs~14.7 fewer CPU coresMarch 13, 2014 IEEE IC2E
2014 Dynamic Scaling for Service Oriented Applications:
Implications of VM Placement on IaaS
CloudsConclusions19ConclusionsLeast-Busy VM placement
enabledperformance improvements up to:29% (RUSLE2), 19% (WEPS) When
dynamically scaling In the presence of a shared load
These performance improvements were realized using slightly
fewer (2.7 %) virtual machines.
20RQ1:
RQ2:
Abstraction of physical hardware using IaaS clouds leads to the
simplistic view:(1) Resources are homogeneous (2) Scaling
infinitely provides linear performance increases
Our results demonstrate:(1) The importance of fine grained
resource management for supporting infrastructure elasticity (2)
Where hardware is not infiniteMarch 13, 2014 IEEE IC2E 2014 Dynamic
Scaling for Service Oriented Applications: Implications of VM
Placement on IaaS CloudsQuestions21
Extra Slides22
Gaps in Related WorkPrior work investigates:WHEN to scale hot
spot detectionWHAT to scale size and number of VMsWHERE to
scaleTask/job schedulingVM placement/migration across nodesNo
studies have investigated implications of VM placement for dynamic
scaling application infrastructureMarch 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS CloudsAverage Model Execution Time
24March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsLeastBusy VM PlacementLeastBusy VM placementBusy Metric rates
resource utilization of physical hosts and virtual machinesResource
Identifies Parallel launch: launch N VMs per scaling
eventDouble-schedule MAX BusyMetric thresholdOnly if min distance
(BusyMetric) to nearest neighbor
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
Clouds
Eucalyptus 3 Private Cloud(9) Sun X6270 blade serversDual Intel
Xeon 4-core 2.8 GHz CPUs24 GB ram, 146 GB 15k rpm HDDsCentOS 5.6
x86_64 (host OS)Ubuntu 9.10 x86_64 (guest OS)Eucalytpus 3.1Amazon
EC2 API support8 Nodes (NC), 1 Cloud Controller (CLC, CC,
SC)Managed mode networking with private VLANsXEN hypervisor v 4.1,
paravirtualization26March 13, 2014 IEEE IC2E 2014 Dynamic Scaling
for Service Oriented Applications: Implications of VM Placement on
IaaS CloudsRUSLE2 ComponentsVirtual MachineDescriptionMModelApache
Tomcat 6.0.20, Wine 1.0.1, RUSLE2 Model, Object Modeling System
(OMS 3.0)DDatabasePostgresql-8.4, and PostGIS 1.4.0-2. soil data:
1.7 million shapes, 167 million pointsmanagement data: 98 shapes,
489k pointsclimate data: 31k shapes, 3 million points4.6 GB for the
state of TNFFile Servernginx http server 0.7.62 57,185 XML files
consisting of 305MB.LLoggerCodebeamer 5.5 running 32-bit
ApacheTomcat 6.0Custom REST/JSON logging service as wrapper.
27March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
Clouds27SC2M DF LSC4M D
FLSC7LMD
FSC3M D
F LSC5MDF LSC6MD FLSC8MD
F LSC9MD LFSC10M FD LSC11M FDLSC12M LD FSC13M LDFSC14M DLFSC15M
LFDSC1M DF L28(15) Tested Component Deployments
Each VM deployed to separate physical machinesAll components
installed on composite imageScript enabled/disabled components to
achieve configsMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for
Service Oriented Applications: Implications of VM Placement on IaaS
CloudsTested Resource Utilization Variablesc29Network- Network
bytes sent (nbr)- Network bytes received (nbs)CPUCPU time
Disk- Disk sector reads (dsr)- Disk sector reads completed
(dsreads)March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsAmazon Web Services: White Paper on Application
DeploymentAmazon white paper suggests:bundling the logical
construct of a component into an Amazon Machine Image so that it
can be deployed more often.J. Varia, Architecting for the Cloud:
Best Practices, Amazon Web Services White Paper, 2010,
https://jineshvaria.s3.amazonaws.com/public/
cloudbestpractices-jvaria.pdfTo support application scaling
30March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsService Isolation Advantages
31tomcat7nginxPostgreSQLMemcacheDBMySQLMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBSCALEEnablesHorizontal
scalingFault tolerance
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsService Isolation Overhead32Isolation requiresSeparate
operating system instancesMore network traffic
tomcat7nginxPostgreSQLMarch 13, 2014 IEEE IC2E 2014 Dynamic
Scaling for Service Oriented Applications: Implications of VM
Placement on IaaS CloudsProvisioning VariationProblems &
Challenges 33VMPhysical HostPhysical HostPhysical HostPhysical
HostPhysical HostPhysical
HostVMVMVMAmbiguousMappingVMVMVMVMVMVMVMVMVMVMVMVMVMVMRequest(s) to
launch VMsVMs ReservePM Memory BlocksVMs Share PMCPU / Disk /
Network
PERFORMANCEMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for
Service Oriented Applications: Implications of VM Placement on IaaS
CloudsApplicationServersLoad BalancerLoad BalancerService
RequestsnoSQL data storesrDBMSdistributed cacheInfrastructure
ManagementProblems & Challenges 34Scale Services
Tune Application Parameters
Tune Virtualization ParametersMarch 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS CloudsApplication Profiling
VariablesPredictive Power35
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsApplication Deployment ChallengesVM image compositionService
isolation vs. scalabilityResource contention among
componentsProvisioning variation Across physical hardware
36
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsResource Utilization VariablesStatisticDescription P/VCPU
timeCPU time in msP/Vcpu usrCPU time in user mode in msP/Vcpu
krnCPU time in kernel mode in msP/Vcpu_idleCPU idle time in
msP/VcontextswNumber of context switchesP/Vcpu_io_waitCPU time
waiting for I/O to completeP/Vcpu_sint_timeCPU time servicing soft
interruptsVdsrDisk sector reads (1 sector = 512
bytes)VdsreadsNumber of completed disk readsVdrmNumber of adjacent
disk reads mergedVreadtimeTime in ms spent reading from
diskVdswDisk sector writes (1 sector = 512 bytes)VdswritesNumber of
completed disk writesVdwmNumber of adjacent disk writes
mergedVwritetimeTime in ms spent writing to diskP/VnbrNetwork bytes
sentP/VnbsNetwork bytes receivedP/VloadavgAvg # of running
processes in last 60 sec37March 13, 2014 IEEE IC2E 2014 Dynamic
Scaling for Service Oriented Applications: Implications of VM
Placement on IaaS CloudsExperimental DataScript captured resource
utilization statsVirtual machinesPhysical MachinesTraining data:
first complete run20 different ensembles of 100 model runs 15
component configurations30,000 model runs Test data: second
complete run30,000 model runs38March 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS Cloudsn=# components; k=# components per
set
Permutations
Combinations
But neither describes partitions of a set!Application
Deployments
39
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
Clouds
Bells Number 40ModelComponentDeploymentn =
#componentsApplication StackVM deployments# of
ConfigurationsDatabaseFile ServerLog Server. . .k= #configsconfig
1MDFLconfig 2MFLconfig nMLFD1 VM : 1..n
componentsnk4155526203787784,140921,147n. . .DNumber of ways a set
of n elements can be partitioned into non-empty subsets
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsXEN Mbound vs Dbound Performance Same Ensemble41
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsXEN 10 GB VMs42
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsKVM Mbound vs Dbound PerformanceSame Ensemble
43March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsKVM 10GB PerformanceSame Ensemble
44March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsKVM 10 GB Performance ChangeSame Ensemble
45March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsKVM Performance ComparisonDifferent Ensembles
46March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsKVM Performance Change From Service Isolation
47March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsService Configuration TestingBig VMsAll application services
installed on single VMScripts enable/disable services to achieve
configurations for testingEach VM deployed on separate
hostProvisioning Variation (PV) TestingKVM used15 total service
configurations46 possible deployments48March 13, 2014 IEEE IC2E
2014 Dynamic Scaling for Service Oriented Applications:
Implications of VM Placement on IaaS CloudsPV: Performance
Difference vs. Physical Isolation
49March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsService Configuration Testing - 2Big VMs used in physical
isolation were effective at identifying fastest service
configurationsFastest configurations isolate L service on separate
physical host; and VMsSome provisioning variations fasterOther SC
provisioning variations remained slowSC4A-D, SC9C-D Only SCs w/ avg
ensemble performance < 30 seconds50March 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS Clouds
Can Resource Utilization Statistics
51
Model Application Performance? March 13, 2014 IEEE IC2E 2014
Dynamic Scaling for Service Oriented Applications: Implications of
VM Placement on IaaS CloudsRQ1 Which are the best predictors? PM
Variables52
CPUNetwork I/OMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for
Service Oriented Applications: Implications of VM Placement on IaaS
CloudsRQ2 How should VM resource utilization data be used by
performance models?Combination: RUdata=RUM+RUD+RUF+RUL
Used Individually: RUdata={RUM; RUD; RUF; RUL;}
53March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
Clouds
RQ2 How should VM resource utilization data be used by
performance models?54D-bound separateD-bound combinedM-bound
separateM-bound combinedTreating VM data separately for D-bound was
better !RUM or RUMDFLfor M-bound was better !Note the larger
RMSEfor D-bound RUMDFL!March 13, 2014 IEEE IC2E 2014 Dynamic
Scaling for Service Oriented Applications: Implications of VM
Placement on IaaS CloudsRQ3 Which modeling techniques were
best?Multiple Linear Regression (MLR)Stepwise Multiple Linear
Regression (MLR-step)Multivariate Adaptive Regression Splines
(MARS)Artificial Neural Network (ANNs)55March 13, 2014 IEEE IC2E
2014 Dynamic Scaling for Service Oriented Applications:
Implications of VM Placement on IaaS Clouds
RQ3 Which modeling techniques were
best?56MultipleLinearRegressionStepwiseMLRMultivariateAdaptiveRegresionSplinesArtificalNeuralNetworkRUMDFL
data used tocompare models.
Had high RMSEtest error for D-Bound (32% avg)Model performance
did not vary much
Best vs. Worst
D-BoundM-Bound .11% RMSEtrain.08% .89% RMSEtest.08% .40 rank
err.66
March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service
Oriented Applications: Implications of VM Placement on IaaS
CloudsResource Utilization Statisticsc57CPU- CPU time- CPU time in
user mode- CPU time in kernel mode- CPU idle time- # of context
switches- CPU time waiting for I/O- CPU time serving soft
interrupts- Load average (# proc / 60 secs)
Disk- Disk sector reads- Disk sector reads completed- Merged
adjacent disk reads- Time spent reading from disk- Disk sector
writes- Disk sector writes completed- Merged adjacent disk writes-
Time spent writing to diskNetwork- Network bytes sent- Network
bytes receivedPMVMVMPMVMVMVMMarch 13, 2014 IEEE IC2E 2014 Dynamic
Scaling for Service Oriented Applications: Implications of VM
Placement on IaaS Clouds