Ph.D. Defense

Automated Configuration and Deployment of Applications in

Heterogeneous Cloud EnvironmentsChris Bunch

Ph.D. DefenseNovember 30, 2012

Public Cloud Computing

• Utility-oriented approach to computing

• Pay for only resources that you use

• Rent resources from large datacenters maintained by Amazon, Microsoft, Google

• Don’t maintain a rack in your office - just use somebody else’s rack

Using the Cloud for Apps

• Cloud services have seen uptake in:

• Web services domain

• High performance computing

• General-purpose applications

Challenges in Cloud Adoption

• Primary barriers to entry:

• Wide array of services

• Varying cost models

• Many technologies providing APIs

Plethora of Services

• Storage Services

• Queue Services

• Compute Services

• Fully Managed Software Stacks

• Web services only

• MapReduce only

Varying Cost Models

• Unlimited usage per-hour (EC2)

• Unlimited usage per-wall-clock-hour (Azure)

• First 15-minutes, then charge per-minute (App Engine)

• Meter per API call (SQS, App Engine)

Accessing Services via APIs

• Need an API to connect your application to the cloud service

• First-party native libraries, per-language

• Typically only for “popular” languages

• Cross-language serialization services

• Convert from your language to “popular” language

Thesis Question

• How can we enable applications to be executed on cloud systems, by automatically configuring and deploying applications across cloud offerings that vary based on the type of service offered, cost model employed, and APIs via which services are exposed?

Our Solution

• Provide research tools to execute computationally intensive applications

•Automatically configure and deploy applications for use with cloud services

• Programming language support, to facilitate expressive workflows

Design SpaceDomain Language / Platform

Support

Web ServicesAppScale (IEEE

CLOUD10)

High Performance Computing

Neptune (ScienceCloud10,

DataCloud12)

General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*

Design SpaceDomain Language / Platform

Support

Web ServicesAppScale (IEEE

CLOUD10)

High Performance Computing

Neptune (ScienceCloud10,

DataCloud12)

General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*

Cloud Computing

• Three tiers of abstraction:

• Infrastructure: Scalable hardware

• Platform: Scalable software stack

• Software: Scalable applications

PaaS for Science

• Need a cloud that is extensible to:

• Services from competing cloud vendors

• Differing cost models from each cloud

• Varying APIs offered by cloud vendors

•And it must be open source!

Introducing AppScale

• An open source implementation of the Google App Engine APIs

• Deploys over Amazon EC2 or Eucalyptus

•Configures and deploys automatically

• User only needs to specify the number of nodes to run over

One-Button Deployment

Limitations

• “Recipes” are statically defined

• Limited to three-tier web applications

• Runtime environment is restricted to enable autoscaling

• Not cost-aware

Design SpaceDomain Language / Platform

Support

Web ServicesAppScale (IEEE

CLOUD10)

High Performance Computing

Neptune (ScienceCloud10,

DataCloud12)

General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*

HPC in the Cloud

• Easy access to vast resources

• Hard to automatically configure and deploy libraries

• Requires in-depth knowledge of each technology required

• Hard to get performance on opaque cloud

• Wide range of APIs for similar services

Introducing Neptune

• A domain specific language for running HPC applications

• Supports MPI, UPC, X10 programs

•Configures and deploys automatically

• Scientists need only specify the number of nodes to execute over

One-Button Deployment

Language-Based Deployment

neptune :type => :mpi,

:code => “/home/user/cpi”,

:nodes_to_use => 32,

:output => “/output/cpi-32”

Automated Application Execution

• Calls to neptune() are translated into SOAP messages, dispatched to AppScale

• AppScale pulls in library support that details how to run each type of job

• Acquires nodes, runs job, saves output

•Cost awareness for VMs

Limitations

• “Recipes” for each framework are static

• Must be pre-defined by an expert user

• Software must be pre-installed on VMs

• Metadata not easily accessible

• Limited by underlying hardware

Design SpaceDomain Language / Platform

Support

Web ServicesAppScale (IEEE

CLOUD10)

High Performance Computing

Neptune (ScienceCloud10,

DataCloud12)

General PurposeMEDEA (IPDPS13)*, Exodus (CCGrid13)*

Problem Domain

• Easy access to vast resources

• Hard to automatically configure and deploy

• Hard to evaluate services b/c of:

• The abstractions they expose

• The cost model they charge with

• Varying APIs for each language

Introducing MEDEA

• Extends Neptune to provide an execution model for applications

• Abstract away compute, storage, queue services via a common interface

• Automatically manage cost for the user

• Automatically connect competing APIs

High-Level Design

• Scripting language support

• Maximizes flexibility and interoperability with other code

• Deployment engine (PaaS layer)

• Automatically configure and deploy applications over cloud services

System Design

Scripting Language Support

• Extends the Neptune DSL

• Adds a function call, medea()

• Users specify code, inputs, services to use

• (M)essages the Deployment Service with this data, called a “task”

n-body in AWSresult = medea(

:executable => “python”,

:code => “/home/user/nbody.py”,

:compute => “ec2”,

:storage => “s3”,

:queue => “sqs”)

puts results.stdout

n-body in Azureresult = medea(

:executable => “python”,

:code => “/home/user/nbody.py”,

:compute => “azure-compute”,

:storage => “azure-storage”,

:queue => “azure-queue”)

puts results.stdout

Deployment Engine

• Extends the AppScale PaaS

• Consists of two new services:

• Task Manager: Manages workers, tasks

• Task Worker: Executes tasks

• Receives task (M)essages from Scripting Language Support, code, inputs to execute

(E, D) Queue services

• Task Manager (E)nqueues the task

• Task Workers (D)equeues tasks

• Both use a QueueInterface, requiring:

• FIFO queue: Push / Pop / Queue Length

• Supports Amazon, Google, Microsoft, VMWare Queues

(E) Compute services

• Task Manager spawns Task Workers in the specified clouds

• Task Workers (E)xecute tasks

• Follows ComputeInterface, requiring:

• dispatch_task / get_task_status / get_task_results

• Supports EC2, Eucalyptus, App Engine, Azure

(A) Storage services

• Task Workers store the following outputs:

• Standard output of job

• Standard error of job

• Metadata

• User’s script (A)ccesses result of job

• Supports S3, App Engine, Azure, and AppScale datastores (HBase, Cassandra, etc.)

Use Cases

• Execute scientific apps and share the results

• Execute quickly (but expensively)

• Execute inexpensively (but slowly)

• Community cloud for benchmarking programming language performance

Scientific Use Cases

• Computational systems biology application

• Simulates conditions found in yeast

• Written in Python, Java

• Deploy to EC2, App Engine, Azure

• All values are the average of five runs

Scientific App Execution

Scientific App Execution

Polyglot Science

• Implementations of the n-body application in eleven programming languages

• Execute with Amazon EC2, SQS, and S3

• Measure time taken to execute, cost

• All values are the average of ten runs

n-body benchmark

n-body in Amazon EC2Language Per-Second Cost

C $0.0069

Java $0.0075

Python $0.5876

Ruby $2.1944

Scala $0.0075

n-body across cloudsCloud Cost To Execute

Amazon EC2 $0.32

App Engine (Java) $0.0013

App Engine (Python)

$0.0049

Microsoft Azure $0.02

Related Work

• Pegasus / Swift (WORKS ’11)

• YCSB (SOCC ‘10), YCSB++ (SOCC ‘11)

• Elastisizer (SOCC ’11)

• Condor / StratUm (BIOINFORMATICS ‘12)

• AME (WORKS ’11)

• Google App Engine Pipeline API

Review

•MEDEA automatically configures and deploys applications, over multiple clouds

• Abstracts away cloud compute, storage, and queue services from the user

• Extensible to support other clouds

• Programming language support to enable Turing-complete workflows

Limitations

• Does not intelligently schedule

• Many different hardware profiles offered by compute services

• Hard to use them effectively b/c of:

• Opaque pricing models

• Lack of Cost APIs

Introducing Exodus

• An Application Programming Interface (API)

• Determines how to “optimally” execute tasks, when “optimal” means:

• Minimizing cost

• Minimizing total execution time

• User-defined functions

System Design

API Support

• Adds a Neptune function call, exodus()

• Users specify :optimize_for:

• Cost, performance, or a user’s Function

• Profiles code locally or remotely

• Estimates time and cost to use each instance type at each number of machines

• Constructs and executes tasks via MEDEA

example

results = exodus(

:clouds_to_use => [:AmazonEC2],

:code => “/home/user/map.py”,

:num_tasks => 1000,

:optimize_for => :cost

)

exampleresults = exodus

:clouds_to_use => [:AmazonEC2],

:code => “/home/user/map.py”,

:num_tasks => 1000,

:optimize_for => float func(t,c) {

...

}

Cost-Aware Science

• Same app as evaluated with MEDEA

• Computational systems biology application

• Written in C

• Try to optimize cost, performance, or a weighted average of the two

• All values are the average of five runs

Time v. Cost

Related Work

• RO-BURST (CCGrid 2012)

• Cannot schedule a priori

• Bicer, Chiu, and Agrawal (CCGrid 2012)

• Cost-aware middleware for MapReduce

• Java apps only, can budget based on time or cost

Review

• Exodus automatically optimizes application deployment over multiple clouds

• Extensible to support evolving use cases

• Programming language support to enable Turing-complete problem descriptions

Contributions

• AppScale cloud platform

•Neptune programming language

•MEDEA extensions to Neptune

• Exodus extensions to MEDEA

• In combination

Impact

• Publications in peer-reviewed conferences

• Best Paper award for Neptune at HPDC’s ScienceCloud

• All work done released as open source

• >10,000 downloads of AppScale / Neptune

Future Work

• Autoscaling in conjunction with IaaS

• Adaptive profiling for app execution

• Cost-aware fault tolerance

• Budgeting and deadlines for entire Exodus programs, across invocations to exodus()

Thanks

• To my advisor, Chandra Krintz

• To my committee, Amr El Abbadi and John Gilbert

• To the AppScale team, especially co-lead Navraj Chohan

• To my family for their continued support

• To all of you for coming!