Managing Your Cloud Assets with AWS

Managing your cloud assets

Ryan Shuttleworth – Technical Evangelist @ryanAWS

Agenda

Setting the scene – a spectrum of system definition and control

A shrink wrapped approach – Elastic Beanstalk

Fine grained control – CloudFormation

Complementary uses – CloudFormation & Beanstalk working together

Your feedback is important

Tell us:

What’s good, what’s not

What you want to see at these events

What you want AWS to deliver for

you

Cloud assets are growing quickly

Defined architectural patterns More applications

More data More services

aws.amazon.com/architecture

Greater agility = faster moving environments

How do we maintain control over what is deployed in the cloud?

Two ends of the spectrum…

Controlled chaos

Meticulous management

Controlled chaos


Prototype an app Deploy to an instance

Test and develop cycles Go to production

Deploy more instances Deploy RDS

Add load balancers Add autoscaling

Audit what you have?

Quickly becomes DevOps nightmare

Controlled chaos







Quickly becomes DevOps nightmare!

Prototype an app Plan infrastructure

Build standards Deploy environment

Manage through change control Formal processes

Formal tools Embed configuration in

chef/puppet

Tightly controlled, but lots of effort to achieve

Controlled chaos


People Processes Controls

Tools






Quickly becomes DevOps nightmare!

Prototype an app Plan infrastructure

Build standards Deploy environment

Manage through change control Formal processes

Formal tools Embed configuration in

chef/puppet

Tightly controlled, but lots of effort to achieve

High level, shrink

wrapped

Fine grained, declarative

control

Tools

“I just want my app to run, and I only want

access to machines if I really need to”

“I want to source control my entire datacenter

deployment like I would my apache config”

High level, shrink

wrapped


control

Elastic Beanstalk

CloudFormation

Tools





High level, shrink

wrapped


control

Elastic Beanstalk

CloudFormation

Tools





Quickly deploy and manage apps in AWS…

Elastic Beanstalk

CloudFormation

Tools

…into a range of containers

Elastic Beanstalk

CloudFormation

Tools

.Net PHP

Java Python

…into a range of containers

Elastic Beanstalk

CloudFormation

Tools

http://aws.typepad.com

Keep updated on new containers:

Application package is deployed into Beanstalk

Elastic Beanstalk

CloudFormation

Tools

User Application

Application Service

HTTP Service

Language Interpreter

Operating System

Host

Which creates the container in EC2

Elastic Beanstalk

CloudFormation

Tools

Beanstalk takes care of the environment…

Elastic Beanstalk

CloudFormation

Tools

…adding Elastic Load Balancer…

Elastic Beanstalk

CloudFormation

Tools

…Auto Scaling groups

Elastic Beanstalk

CloudFormation

Tools

…and launching instances

Elastic Beanstalk

CloudFormation

Tools

…all wired into the Beanstalk framework

Elastic Beanstalk

CloudFormation

Tools

…and published under a CNAME

Elastic Beanstalk

CloudFormation

Tools

…with logs and app versions held in S3

Elastic Beanstalk

CloudFormation

Tools

How do you create an app for AWS ElasticBeanstalk?

Like any other Java / .Net / PHP / Python

app

No dependencies on AWS inherent for Beanstalk deployment

Elastic Beanstalk

CloudFormation

Tools

Source uploaded and executed in AWS

Source downloaded

and executed locally

Package up as normal…

Elastic Beanstalk

CloudFormation

Tools

Java .war file

Microsoft Web

Deploy package

PHP .zip file

Python .zip file

Git integration IDE plugins

Package up as normal…

Elastic Beanstalk

CloudFormation

Tools

Java .war file

Microsoft Web

Deploy package

PHP .zip file

Python .zip file

Git integration IDE plugins

Console deployments and version updates

Elastic Beanstalk

CloudFormation

Tools

Command line tools and wizards

Elastic Beanstalk

CloudFormation

Tools

CheckDNSAvailability elastic-beanstalk-check-dns-availability

CreateApplication elastic-beanstalk-create-application

CreateApplictaionVersion

elastic-beanstalk-create-application-version

CreateEnvironment

elastic-beanstalk-create-environment

eb init wizard to initialize an application

eb start/stop start/stop an application

eb update

update application version

eb status

get status of a running application

Eclipse integration

Elastic Beanstalk

CloudFormation

Tools

Eclipse integration

Elastic Beanstalk

CloudFormation

Tools

Eclipse integration

Elastic Beanstalk

CloudFormation

Tools

Eclipse integration

Elastic Beanstalk

CloudFormation

Tools

Visual Studio integration

Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools

Git integration

Elastic Beanstalk

CloudFormation

Tools

Container configuration

Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools


Elastic Beanstalk

CloudFormation

Tools

Full admin access on underlying resources EC2 instances, AMIs, AutoScaling groups…

Application prototype

One-click deployment

Production ready AWS infrastructure

Idea App

Idea App Heavy lifting

70%

30%

Idea App Infrastructure

services

Idea App Elastic Beanstalk

Elastic Beanstalk

Idea App

Company launching product into new geography

Online application & marketing site deployed to support launch

Site crashed under exceptional demand - DoS existing datacenter

Needed fast deployment mechanism with no interest in underlying infrastructure

Company launching product into new geography

Online application & marketing site deployed to support launch

Site crashed under exceptional demand - DoS existing datacenter

Needed fast deployment mechanism with no interest in underlying infrastructure

Turned to Beanstalk - deployed & scaled seamlessly, restoring website

High level, shrink

wrapped


control

Elastic Beanstalk

CloudFormation

Tools





What happens under the covers of Beanstalk?

Elastic Beanstalk

CloudFormation

Tools

Elastic Beanstalk

AWS

Create security group

Create auto-scaling policy

Create elastic load balancer

Add instance

Install software

Consolidate logs

What happens under the covers of Beanstalk?

Elastic Beanstalk

CloudFormation

Tools

AWS

Create security group

Create auto-scaling policy

Create elastic load balancer

Add instance

Install software

Consolidate logs

A series of calls into AWS to marshal resource

creation/configuration

CloudFormation – underlying automation framework

Elastic Beanstalk

CloudFormation

Tools

Security Scaling

Database

Networking Monitoring

Messaging

Workflow

DNS

Load Balancing

Backup CDN

Compute

Storage

Elastic Beanstalk

CloudFormation

Tools

CloudFormation

A consistent scripting interface for creating/configuring AWS

resources

A framework for lifecycle management of resources

created using scripts

Security Scaling

Database

Networking Monitoring

Messaging

Workflow

DNS

Load Balancing

Backup CDN

Compute

Storage

CloudFormation – underlying automation framework

CloudFormation components & terminology

Elastic Beanstalk

CloudFormation

Tools

Template CloudFormation

Stack

JSON formatted file

Parameter definition

Resource creation

Configuration actions

Configured AWS services

Comprehensive service support

Service event aware

Customisable

Framework

Stack creation

Stack updates

Error detection and rollback

Template examples

Elastic Beanstalk

CloudFormation

Tools

Template

JSON formatted file


Resource creation


{

"Parameters" : {

"KeyName" : {

"Description" : "Name of an existing EC2 KeyPair to enable SSH

access to the instance",

"Type" : "String"

}

},

}

Snippet: Taking user defined parameters

Template examples

Elastic Beanstalk

CloudFormation

Tools

Template

JSON formatted file


Resource creation


{

"AWSTemplateFormatVersion" : "2010-09-09",

"Description" : "Create RDS with fixed username and password",

"Resources" : {

"MyDB" : {

"Type" : "AWS::RDS::DBInstance",

"Properties" : {

"AllocatedStorage" : "5",

"DBInstanceClass" : "db.m1.small",

"Engine" : "MySQL",

"EngineVersion" : "5.5",

"MasterUsername" : "MyName",

"MasterUserPassword" : "MyPassword"

}

}

}

}

Snippet: Creating RDS instance

Template examples

Elastic Beanstalk

CloudFormation

Tools

Template

JSON formatted file


Resource creation


"AWS::CloudFormation::Init" : {

"config" : {

"packages" : {

"yum" : {

"mysql" : [],

"mysql-server" : [],

"httpd" : [],

"php" : [],

"php-mysql" : []

}

},

"sources" : {

"/var/www/html" : "https://s3.amazonaws.com/

cloudformation-examples/

CloudFormationPHPSample.zip"

},

Snippet: Installing software

The benefit of templated environments

Elastic Beanstalk

CloudFormation

Tools

Template

Procedural definition Create it programmatically


Elastic Beanstalk

CloudFormation

Tools

Template


Parameter driven Dynamic and user-driven

templates


Elastic Beanstalk

CloudFormation

Tools

Template



templates

Collaboration Share templates with ease as just files


Elastic Beanstalk

CloudFormation

Tools

Template


Known configuration Store stack configuration in source control


templates

Collaboration Share templates with ease as just files

NASA TV & Curiosity Route 53 DNS Multi-region & AZ

Elastic Loadbalancing Amazon Live Streaming

Cloud Front Cloud Formation




Template ELBs to front secondary cache

~100 Nginx secondary cache servers

2-3 Nginx mid-tier cache servers

Stack

CloudFormation template defined video caching

infrastructure




Availability Zone

As demand increased new stacks added to an availability

zone using template

One-click complex

rollout

Where to get more information?

Elastic Beanstalk

CloudFormation

Tools

http://aws.amazon.com/cloudformation/aws-cloudformation-templates/

Server templates High availability configurations

RDS configurations Open source stacks

Snippets

+ Cloud Former (prototype)

Where to get more information?

Elastic Beanstalk

CloudFormation

Tools

http://aws.amazon.com/cloudformation/aws-cloudformation-templates/

Server templates High availability configurations

RDS configurations Open source stacks

Snippets

+ Cloud Former (prototype)


Elastic Beanstalk

CloudFormation

Tools

Amazon EC2 Instances (On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Elastic Block Store (EBS) Volumes Elastic Load Balancers Elastic IP Addresses Amazon EC2 Security Groups Auto Scaling Groups Amazon Relational Database Service (RDS) Instances Amazon RDS Security Groups Amazon ElastiCache Clusters Amazon ElastiCache Security Groups Amazon ElastiCache Parameter Groups

Amazon CloudWatch Alarms Amazon Simple Storage Service (S3) Buckets

Amazon Simple DB Domains Amazon Simple Queue Service (SQS) Queues

Amazon Simple Notification Service (SNS) Topics Amazon SNS Subscriptions

Amazon Route 53 DNS Records Amazon CloudFront Distributions and Multi-Origin support

AWS Identity and Access Management users and groups AWS Identify and Access Management policies

Amazon Virtual Private Cloud VPCs, Subnets, Gateways Amazon DynamoDB Tables


Elastic Beanstalk

CloudFormation

Tools

Amazon EC2 Instances (On-Demand Instances, Spot Instances, and Reserved Instances) Amazon Elastic Block Store (EBS) Volumes Elastic Load Balancers Elastic IP Addresses Amazon EC2 Security Groups Auto Scaling Groups Amazon Relational Database Service (RDS) Instances Amazon RDS Security Groups Amazon ElastiCache Clusters Amazon ElastiCache Security Groups Amazon ElastiCache Parameter Groups

Amazon CloudWatch Alarms Amazon Simple Storage Service (S3) Buckets

Amazon Simple DB Domains Amazon Simple Queue Service (SQS) Queues

Amazon Simple Notification Service (SNS) Topics Amazon SNS Subscriptions

Amazon Route 53 DNS Records Amazon CloudFront Distributions and Multi-Origin support

AWS Identity and Access Management users and groups AWS Identify and Access Management policies

Amazon Virtual Private Cloud VPCs, Subnets, Gateways Amazon DynamoDB Tables

+ AWS Elastic Beanstalk

High level, shrink

wrapped


control

Elastic Beanstalk

CloudFormation

Tools





High level, shrink

wrapped


control

Elastic Beanstalk

CloudFormation

+

+

High level, shrink

wrapped


control

Elastic Beanstalk

CloudFormation

Developer sandboxes with production ready

infrastructure

+

+

Comprehensive control of all aspects of infrastructure

Get started fast with Beanstalk (control the chaos)

Take fine grained control with CloudFormation

(auditable environment specs)

http://aws.amazon.com/elasticbeanstalk/

http://aws.amazon.com/cloudformation/

Find out more…

aws.amazon.com

get started with the free tier

Managing Your Cloud Assets with AWS

Technology

cloudformation beanstalk

beanstalk framework

app toi

application startstop

running application

app versions

control run

app foraws elasticbeanstalk