Cloud Services Innovation Platform (CSIP)
Service Isolation vs. Consolidation: Implications for IaaS Cloud
Application DeploymentWes Lloyd, Shrideep Pallickara, Olaf David,
James Lyon, Mazdak Arabi, Ken Rojas
March 26, 2013
Colorado State University, Fort Collins, Colorado USAIC2E 2013:
IEEE International Conference on Cloud Engineering
1OutlineBackgroundResearch ProblemResearch QuestionsExperimental
SetupExperimental ResultsConclusions2March 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application Deployment2Background3Traditional Application
Deployment4Object StorePhysical Server(s)March 26, 2013 IEEE IC2E
2013 Service Isolation vs. Consolidation: Implications for IaaS
Clouds Application DeploymentIaaS Component Deployment 5App
ServerComponentDeploymentApplicationComponentsApplication
StackVirtual Machine (VM) Images
PERFORMANCErDBMS r/oFile ServerLog ServerLoad BalancerImage
2rDBMS write. . .Image 1App ServerFile ServerLog ServerrDBMS
writeImage nrDBMS r/oLoad BalancerDist. cacheMarch 26, 2013 IEEE
IC2E 2013 Service Isolation vs. Consolidation: Implication for IaaS
Cloud Application DeploymentResearch Problem6Amazon Web Services:
White Paper on Application DeploymentAmazon white paper
suggests:bundling the logical construct of a component into an
Amazon Machine Image so that it can be deployed more often.J.
Varia, Architecting for the Cloud: Best Practices, Amazon Web
Services White Paper, 2010,
https://jineshvaria.s3.amazonaws.com/public/
cloudbestpractices-jvaria.pdfTo support application scaling
7March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentService Isolation Advantages
8tomcat7nginxPostgreSQLMemcacheDBMySQLMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBSCALEEnablesHorizontal
scalingFault tolerance
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implication for IaaS Cloud Application
DeploymentService Isolation Overhead9Isolation requiresSeparate
operating system instancesMore network traffic
tomcat7nginxPostgreSQLMarch 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentProvisioning VariationProblems &
Challenges 10VMPhysical HostPhysical HostPhysical HostPhysical
HostPhysical HostPhysical
HostVMVMVMAmbiguousMappingVMVMVMVMVMVMVMVMVMVMVMVMVMVMRequest(s) to
launch VMsVMs ReservePM Memory BlocksVMs Share PMCPU / Disk /
Network
PERFORMANCEMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentResearch Questions11Research QuestionsWhat performance
and resource utilization implications result based on how
application components are deployed? How does increasing VM memory
impact performance?
How much overhead results from VM service isolation?
Can resource utilization data be used to build models to predict
performance of component deployments?
12RQ1:
RQ2:
RQ3:March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
Deployment
Gaps in Related WorkPrior work investigates:Virtualization
performanceIsolation properties of hypervisorsAutonomic scaling of
application infrastructurePerformance variation fromProvisioning
variationShared cluster/cloud loadsNo studies have investigated
implications of how the application stack is deployedMarch 26, 2013
IEEE IC2E 2013 Service Isolation vs. Consolidation: Implications
for IaaS Clouds Application DeploymentExperimental Setup14
RUSLE2 ModelRevised Universal Soil Loss EquationCombines
empirical and process-based sciencePrediction of rill and interrill
soil erosion resulting from rainfall and runoffUSDA-NRCS agency
standard modelUsed by 3,000+ field officesHelps inventory erosion
ratesSediment delivery estimationConservation planning tool15March
26, 2013 IEEE IC2E 2013 Service Isolation vs. Consolidation:
Implications for IaaS Clouds Application DeploymentRUSLE2 Web
ServiceMulti-tier client/server applicationRESTful, JAX-RS/Java
using JSON objectsSurrogate for common architectures
16OMS3RUSLE2POSTGRESQLPOSTGIS1.7+ million shapes57k XML files,
305MbMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
Deployment
Eucalyptus 2.0 Private Cloud(9) Sun X6270 blade serversDual
Intel Xeon 4-core 2.8 GHz CPUs24 GB ram, 146 GB 15k rpm HDDsCentOS
5.6 x86_64 (host OS)Ubuntu 9.10 x86_64 (guest OS)Eucalytpus
2.0Amazon EC2 API support8 Nodes (NC), 1 Cloud Controller (CLC, CC,
SC)Managed mode networking with private VLANsXEN hypervisor v
3.4.3, paravirtualization17March 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentRUSLE2 ComponentsVirtual
MachineDescriptionMModelApache Tomcat 6.0.20, Wine 1.0.1, RUSLE2
Model, Object Modeling System (OMS 3.0)DDatabasePostgresql-8.4, and
PostGIS 1.4.0-2. soil data: 1.7 million shapes, 167 million
pointsmanagement data: 98 shapes, 489k pointsclimate data: 31k
shapes, 3 million points4.6 GB for the state of TNFFile Servernginx
http server 0.7.62 57,185 XML files consisting of
305MB.LLoggerCodebeamer 5.5 running 32-bit ApacheTomcat 6.0Custom
REST/JSON logging service as wrapper. 18March 26, 2013 IEEE IC2E
2013 Service Isolation vs. Consolidation: Implications for IaaS
Clouds Application Deployment18SC2M DF LSC4M D
FLSC7LMD
FSC3M D
F LSC5MDF LSC6MD FLSC8MD
F LSC9MD LFSC10M FD LSC11M FDLSC12M LD FSC13M LDFSC14M DLFSC15M
LFDSC1M DF L19(15) Tested Component Deployments
Each VM deployed to separate physical machinesAll components
installed on composite imageScript enabled/disabled components to
achieve configsMarch 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentTested Resource Utilization Variablesc20Network- Network
bytes sent (nbr)- Network bytes received (nbs)CPUCPU time
Disk- Disk sector reads (dsr)- Disk sector reads completed
(dsreads)March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
Deployment
RUSLE2 Application Profiles21D-bound:join w/ a nested
queryM-bound:standard modelMarch 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application Deployment21Experimental Results22
Slow group:ah3, ah5, ah8*Middle group:ah4, ah9*, ah10*, ah13Fast
group:ah1, ah2*, ah6*, ah7, ah11, ah12*, ah14, ah15Test: 2
identical runs, 4GB VMs, 15 component deployments, 10 ensemble runs
of 100 model runs each
Performance was reproduced.
Strong correlationp=0.000000000809050298* - indicates same group
membership as DBoundReproducibility of tests Conclusion:
Service Composition of VMs mattered.Performance is different and
can bemeasured with reproducible results.
24
SC15SC14SC13SC12SC11SC10SC9SC8SC7SC6SC5SC4SC3SC2SC1 CPU time
disk sector reads disk sector writes net bytes rcvd net bytes
sentRQ1: Resource utilization implications from component
deployments
Boxes represent absolute deviation from mean (m-bound)Magnitude
of variance for deployments
Resource Utilization ChangeMin to Max Utilization
m-bound d-boundCPU time:6.5%5.5%Disk sector
reads:14.8%819.6%Disk sector writes:21.8%111.1%Network bytes
received:144.9%145%Network bytes sent:143.7%143.9%
RQ1: Performance implications from component deployments
25Slower deploymentsFaster deployments Performance Change:Min to
max performance
M-bound:14%D-bound:25.7%March 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentRQ1: How does increasing VM memory allocation
impact performance?
26In some casesmore memory lead toslower performanceMore
memoryFaster performanceMarch 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentRQ2: How much overhead results from VM
service isolation?27
1.2 %.3 %2.4 %Performance Overhead
Xen:~1% averageKVM:~2.4% averageMarch 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application Deployment28100 random runsJSON object20x Ensembles100
random runs100 random runs100 random runs100 random runs100 random
runs100 random runs100 random runs100 random runsSC5MDF LSC8MD
F LSC11M FDLSC14M DLFSC1M DF L(15) RUSLE2deployments
Resource UtilizationDatascript captureData used to build
multiple linear regressionperformance model
1st run training dataset2nd run test datasetMarch 26, 2013 IEEE
IC2E 2013 Service Isolation vs. Consolidation: Implications for
IaaS Clouds Application Deployment
RQ3: Can resource utilization data be used to build models to
predict performance of component deployments?CPUDisk I/ONetwork
I/O29# VMs.71.37.14.007.008.04Multiple Linear Regression
Performance Model
For the test dataset:
Combined R2:.8416Mean absolute error: 324ms (test
dataset)Average rank error:2 unitsFastest deployment predicted
accurately
Explained 84% of the varianceMarch 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentConclusions30ConclusionsComponent deployments
led to:25% performance variationNetwork and disk resource
utilization most affected. VM memory did not always improve
performance
Up to 2.4% performance overhead from service isolation
Our MLR-model accounted for 84% of the variance when predicting
deployment performance
31RQ1:
RQ2:
RQ3:March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentQuestions32
Extra Slides33ApplicationServersLoad BalancerLoad
BalancerService RequestsnoSQL data storesrDBMSdistributed
cacheInfrastructure ManagementProblems & Challenges 34Scale
Services
Tune Application Parameters
Tune Virtualization ParametersMarch 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentApplication Profiling VariablesPredictive
Power35
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentApplication Deployment ChallengesVM image
compositionService isolation vs. scalabilityResource contention
among componentsProvisioning variation Across physical hardware
36
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentResource Utilization VariablesStatisticDescription P/VCPU
timeCPU time in msP/Vcpu usrCPU time in user mode in msP/Vcpu
krnCPU time in kernel mode in msP/Vcpu_idleCPU idle time in
msP/VcontextswNumber of context switchesP/Vcpu_io_waitCPU time
waiting for I/O to completeP/Vcpu_sint_timeCPU time servicing soft
interruptsVdsrDisk sector reads (1 sector = 512
bytes)VdsreadsNumber of completed disk readsVdrmNumber of adjacent
disk reads mergedVreadtimeTime in ms spent reading from
diskVdswDisk sector writes (1 sector = 512 bytes)VdswritesNumber of
completed disk writesVdwmNumber of adjacent disk writes
mergedVwritetimeTime in ms spent writing to diskP/VnbrNetwork bytes
sentP/VnbsNetwork bytes receivedP/VloadavgAvg # of running
processes in last 60 sec37March 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentExperimental DataScript captured resource
utilization statsVirtual machinesPhysical MachinesTraining data:
first complete run20 different ensembles of 100 model runs 15
component configurations30,000 model runs Test data: second
complete run30,000 model runs38March 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application Deploymentn=# components; k=# components per set
Permutations
Combinations
But neither describes partitions of a set!Application
Deployments
39
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
Deployment
Bells Number 40ModelComponentDeploymentn =
#componentsApplication StackVM deployments# of
ConfigurationsDatabaseFile ServerLog Server. . .k= #configsconfig
1MDFLconfig 2MFLconfig nMLFD1 VM : 1..n
componentsnk4155526203787784,140921,147n. . .DNumber of ways a set
of n elements can be partitioned into non-empty subsets
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implication for IaaS Cloud Application DeploymentXEN
Mbound vs Dbound Performance Same Ensemble41
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentXEN 10 GB VMs42
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentKVM Mbound vs Dbound PerformanceSame Ensemble
43March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentKVM 10GB PerformanceSame Ensemble
44March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentKVM 10 GB Performance ChangeSame Ensemble
45March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentKVM Performance ComparisonDifferent Ensembles
46March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentKVM Performance Change From Service Isolation
47March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentService Configuration TestingBig VMsAll application
services installed on single VMScripts enable/disable services to
achieve configurations for testingEach VM deployed on separate
hostProvisioning Variation (PV) TestingKVM used15 total service
configurations46 possible deployments48March 26, 2013 IEEE IC2E
2013 Service Isolation vs. Consolidation: Implications for IaaS
Clouds Application DeploymentPV: Performance Difference vs.
Physical Isolation
49March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentService Configuration Testing - 2Big VMs used in physical
isolation were effective at identifying fastest service
configurationsFastest configurations isolate L service on separate
physical host; and VMsSome provisioning variations fasterOther SC
provisioning variations remained slowSC4A-D, SC9C-D Only SCs w/ avg
ensemble performance < 30 seconds50March 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application Deployment
Can Resource Utilization Statistics
51
Model Application Performance? March 26, 2013 IEEE IC2E 2013
Service Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentRQ1 Which are the best predictors? PM
Variables52
CPUNetwork I/OMarch 26, 2013 IEEE IC2E 2013 Service Isolation
vs. Consolidation: Implications for IaaS Clouds Application
DeploymentRQ2 How should VM resource utilization data be used by
performance models?Combination: RUdata=RUM+RUD+RUF+RUL
Used Individually: RUdata={RUM; RUD; RUF; RUL;}
53March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
Deployment
RQ2 How should VM resource utilization data be used by
performance models?54D-bound separateD-bound combinedM-bound
separateM-bound combinedTreating VM data separately for D-bound was
better !RUM or RUMDFLfor M-bound was better !Note the larger
RMSEfor D-bound RUMDFL!March 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application DeploymentRQ3 Which modeling techniques were
best?Multiple Linear Regression (MLR)Stepwise Multiple Linear
Regression (MLR-step)Multivariate Adaptive Regression Splines
(MARS)Artificial Neural Network (ANNs)55March 26, 2013 IEEE IC2E
2013 Service Isolation vs. Consolidation: Implications for IaaS
Clouds Application Deployment
RQ3 Which modeling techniques were
best?56MultipleLinearRegressionStepwiseMLRMultivariateAdaptiveRegresionSplinesArtificalNeuralNetworkRUMDFL
data used tocompare models.
Had high RMSEtest error for D-Bound (32% avg)Model performance
did not vary much
Best vs. Worst
D-BoundM-Bound .11% RMSEtrain.08% .89% RMSEtest.08% .40 rank
err.66
March 26, 2013 IEEE IC2E 2013 Service Isolation vs.
Consolidation: Implications for IaaS Clouds Application
DeploymentResource Utilization Statisticsc57CPU- CPU time- CPU time
in user mode- CPU time in kernel mode- CPU idle time- # of context
switches- CPU time waiting for I/O- CPU time serving soft
interrupts- Load average (# proc / 60 secs)
Disk- Disk sector reads- Disk sector reads completed- Merged
adjacent disk reads- Time spent reading from disk- Disk sector
writes- Disk sector writes completed- Merged adjacent disk writes-
Time spent writing to diskNetwork- Network bytes sent- Network
bytes receivedPMVMVMPMVMVMVMMarch 26, 2013 IEEE IC2E 2013 Service
Isolation vs. Consolidation: Implications for IaaS Clouds
Application Deployment