Automated Configuration and Deployment of Applications in
Heterogeneous Cloud EnvironmentsChris Bunch
Ph.D. DefenseNovember 30, 2012
Public Cloud Computing
• Utility-oriented approach to computing
• Pay for only resources that you use
• Rent resources from large datacenters maintained by Amazon, Microsoft, Google
• Don’t maintain a rack in your office - just use somebody else’s rack
Using the Cloud for Apps
• Cloud services have seen uptake in:
• Web services domain
• High performance computing
• General-purpose applications
Challenges in Cloud Adoption
• Primary barriers to entry:
• Wide array of services
• Varying cost models
• Many technologies providing APIs
Plethora of Services
• Storage Services
• Queue Services
• Compute Services
• Fully Managed Software Stacks
• Web services only
• MapReduce only
Varying Cost Models
• Unlimited usage per-hour (EC2)
• Unlimited usage per-wall-clock-hour (Azure)
• First 15-minutes, then charge per-minute (App Engine)
• Meter per API call (SQS, App Engine)
Accessing Services via APIs
• Need an API to connect your application to the cloud service
• First-party native libraries, per-language
• Typically only for “popular” languages
• Cross-language serialization services
• Convert from your language to “popular” language
Thesis Question
• How can we enable applications to be executed on cloud systems, by automatically configuring and deploying applications across cloud offerings that vary based on the type of service offered, cost model employed, and APIs via which services are exposed?
Our Solution
• Provide research tools to execute computationally intensive applications
•Automatically configure and deploy applications for use with cloud services
• Programming language support, to facilitate expressive workflows
Design SpaceDomain Language / Platform
Support
Web ServicesAppScale (IEEE
CLOUD10)
High Performance Computing
Neptune (ScienceCloud10,
DataCloud12)
General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*
Design SpaceDomain Language / Platform
Support
Web ServicesAppScale (IEEE
CLOUD10)
High Performance Computing
Neptune (ScienceCloud10,
DataCloud12)
General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*
Cloud Computing
• Three tiers of abstraction:
• Infrastructure: Scalable hardware
• Platform: Scalable software stack
• Software: Scalable applications
PaaS for Science
• Need a cloud that is extensible to:
• Services from competing cloud vendors
• Differing cost models from each cloud
• Varying APIs offered by cloud vendors
•And it must be open source!
Introducing AppScale
• An open source implementation of the Google App Engine APIs
• Deploys over Amazon EC2 or Eucalyptus
•Configures and deploys automatically
• User only needs to specify the number of nodes to run over
One-Button Deployment
Limitations
• “Recipes” are statically defined
• Limited to three-tier web applications
• Runtime environment is restricted to enable autoscaling
• Not cost-aware
Design SpaceDomain Language / Platform
Support
Web ServicesAppScale (IEEE
CLOUD10)
High Performance Computing
Neptune (ScienceCloud10,
DataCloud12)
General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*
HPC in the Cloud
• Easy access to vast resources
• Hard to automatically configure and deploy libraries
• Requires in-depth knowledge of each technology required
• Hard to get performance on opaque cloud
• Wide range of APIs for similar services
Introducing Neptune
• A domain specific language for running HPC applications
• Supports MPI, UPC, X10 programs
•Configures and deploys automatically
• Scientists need only specify the number of nodes to execute over
One-Button Deployment
Language-Based Deployment
neptune :type => :mpi,
:code => “/home/user/cpi”,
:nodes_to_use => 32,
:output => “/output/cpi-32”
Automated Application Execution
• Calls to neptune() are translated into SOAP messages, dispatched to AppScale
• AppScale pulls in library support that details how to run each type of job
• Acquires nodes, runs job, saves output
•Cost awareness for VMs
Limitations
• “Recipes” for each framework are static
• Must be pre-defined by an expert user
• Software must be pre-installed on VMs
• Metadata not easily accessible
• Limited by underlying hardware
Design SpaceDomain Language / Platform
Support
Web ServicesAppScale (IEEE
CLOUD10)
High Performance Computing
Neptune (ScienceCloud10,
DataCloud12)
General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*
Problem Domain
• Easy access to vast resources
• Hard to automatically configure and deploy
• Hard to evaluate services b/c of:
• The abstractions they expose
• The cost model they charge with
• Varying APIs for each language
Introducing MEDEA
• Extends Neptune to provide an execution model for applications
• Abstract away compute, storage, queue services via a common interface
• Automatically manage cost for the user
• Automatically connect competing APIs
High-Level Design
• Scripting language support
• Maximizes flexibility and interoperability with other code
• Deployment engine (PaaS layer)
• Automatically configure and deploy applications over cloud services
System Design
Scripting Language Support
• Extends the Neptune DSL
• Adds a function call, medea()
• Users specify code, inputs, services to use
• (M)essages the Deployment Service with this data, called a “task”
n-body in AWSresult = medea(
:executable => “python”,
:code => “/home/user/nbody.py”,
:compute => “ec2”,
:storage => “s3”,
:queue => “sqs”)
puts results.stdout
n-body in Azureresult = medea(
:executable => “python”,
:code => “/home/user/nbody.py”,
:compute => “azure-compute”,
:storage => “azure-storage”,
:queue => “azure-queue”)
puts results.stdout
Deployment Engine
• Extends the AppScale PaaS
• Consists of two new services:
• Task Manager: Manages workers, tasks
• Task Worker: Executes tasks
• Receives task (M)essages from Scripting Language Support, code, inputs to execute
(E, D) Queue services
• Task Manager (E)nqueues the task
• Task Workers (D)equeues tasks
• Both use a QueueInterface, requiring:
• FIFO queue: Push / Pop / Queue Length
• Supports Amazon, Google, Microsoft, VMWare Queues
(E) Compute services
• Task Manager spawns Task Workers in the specified clouds
• Task Workers (E)xecute tasks
• Follows ComputeInterface, requiring:
• dispatch_task / get_task_status / get_task_results
• Supports EC2, Eucalyptus, App Engine, Azure
(A) Storage services
• Task Workers store the following outputs:
• Standard output of job
• Standard error of job
• Metadata
• User’s script (A)ccesses result of job
• Supports S3, App Engine, Azure, and AppScale datastores (HBase, Cassandra, etc.)
Use Cases
• Execute scientific apps and share the results
• Execute quickly (but expensively)
• Execute inexpensively (but slowly)
• Community cloud for benchmarking programming language performance
Scientific Use Cases
• Computational systems biology application
• Simulates conditions found in yeast
• Written in Python, Java
• Deploy to EC2, App Engine, Azure
• All values are the average of five runs
Scientific App Execution
Scientific App Execution
Polyglot Science
• Implementations of the n-body application in eleven programming languages
• Execute with Amazon EC2, SQS, and S3
• Measure time taken to execute, cost
• All values are the average of ten runs
n-body benchmark
n-body in Amazon EC2Language Per-Second Cost
C $0.0069
Java $0.0075
Python $0.5876
Ruby $2.1944
Scala $0.0075
n-body across cloudsCloud Cost To Execute
Amazon EC2 $0.32
App Engine (Java) $0.0013
App Engine (Python)
$0.0049
Microsoft Azure $0.02
Related Work
• Pegasus / Swift (WORKS ’11)
• YCSB (SOCC ‘10), YCSB++ (SOCC ‘11)
• Elastisizer (SOCC ’11)
• Condor / StratUm (BIOINFORMATICS ‘12)
• AME (WORKS ’11)
• Google App Engine Pipeline API
Review
•MEDEA automatically configures and deploys applications, over multiple clouds
• Abstracts away cloud compute, storage, and queue services from the user
• Extensible to support other clouds
• Programming language support to enable Turing-complete workflows
Limitations
• Does not intelligently schedule
• Many different hardware profiles offered by compute services
• Hard to use them effectively b/c of:
• Opaque pricing models
• Lack of Cost APIs
Introducing Exodus
• An Application Programming Interface (API)
• Determines how to “optimally” execute tasks, when “optimal” means:
• Minimizing cost
• Minimizing total execution time
• User-defined functions
System Design
API Support
• Adds a Neptune function call, exodus()
• Users specify :optimize_for:
• Cost, performance, or a user’s Function
• Profiles code locally or remotely
• Estimates time and cost to use each instance type at each number of machines
• Constructs and executes tasks via MEDEA
example
results = exodus(
:clouds_to_use => [:AmazonEC2],
:code => “/home/user/map.py”,
:num_tasks => 1000,
:optimize_for => :cost
)
exampleresults = exodus
:clouds_to_use => [:AmazonEC2],
:code => “/home/user/map.py”,
:num_tasks => 1000,
:optimize_for => float func(t,c) {
...
}
Cost-Aware Science
• Same app as evaluated with MEDEA
• Computational systems biology application
• Written in C
• Try to optimize cost, performance, or a weighted average of the two
• All values are the average of five runs
Time v. Cost
Related Work
• RO-BURST (CCGrid 2012)
• Cannot schedule a priori
• Bicer, Chiu, and Agrawal (CCGrid 2012)
• Cost-aware middleware for MapReduce
• Java apps only, can budget based on time or cost
Review
• Exodus automatically optimizes application deployment over multiple clouds
• Extensible to support evolving use cases
• Programming language support to enable Turing-complete problem descriptions
Contributions
• AppScale cloud platform
•Neptune programming language
•MEDEA extensions to Neptune
• Exodus extensions to MEDEA
• In combination
Impact
• Publications in peer-reviewed conferences
• Best Paper award for Neptune at HPDC’s ScienceCloud
• All work done released as open source
• >10,000 downloads of AppScale / Neptune
Future Work
• Autoscaling in conjunction with IaaS
• Adaptive profiling for app execution
• Cost-aware fault tolerance
• Budgeting and deadlines for entire Exodus programs, across invocations to exodus()
Thanks
• To my advisor, Chandra Krintz
• To my committee, Amr El Abbadi and John Gilbert
• To the AppScale team, especially co-lead Navraj Chohan
• To my family for their continued support
• To all of you for coming!