AWS Webcast - Build Agile Applications in AWS Cloud for Government
Post on 16-Jul-2015
559 Views
Preview:
Transcript
Building Agile Applications using
Continuous Integration & Deployment
on AWS
Leo Zhadanovsky
Senior Solutions Architect
What is Continuous Integration?
Changes to code automatically deployed to mainline branch• After passing unit and mock tests
Makes changes to code, and deployments iterative, not monolithic
Bugs are detected quickly
Allows rapid development
Helps automate deployments
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
CODE QUALITY
TESTS
TEST
RESULTS
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
DOCS
BINARIES
& PACKAGES
DEV FACING
NOTIFICATIONS
New AWS Managed Options
CodeDeploy• Deploy your code to hundreds or thousands of instances
CodeCommit• AWS Managed Git Hosting
CodePipelines• AWS Managed Continuous Integration System
Cloud Software Development Lifecycle
10/13/14 19
MonitorProvisionDeployTestBuildCode
Elastic Beanstalk
OpsWorks
Cloud
Watch
Cloud
Formation
?
Why?
Customers are asking
Share Amazon tools and processes with customers
Showcase and integrate AWS partner tools
9/25/14 20
Source ControlPrivate Git repositories hosted on Amazon S3
Full Git compatibility (use with existing tools)
All the benefits of the cloud(scalable, durable, reliable, low pay as you go pricing)
No size limits on repositories (store binary files)
Online code tools with browse, edit, diff
10/13/14 21
Continuous DeliveryCustomizable release automation, with integrated build and test
Model and visualize custom release workflow(source build beta gamma prod)
Automate builds, tests, and deployments
Enforce custom rules, approvals, and gates
Integrate with third-party and custom tools
10/13/14 23
DeploymentCoordinate software updates to fleets of EC2 instances
Rolling updates for no downtime
Deployment health checks and easy rollback
Auto Scaling integration
Works with any application
Reuse existing setup tools(Bash, Powershell, Chef, Puppet…)
10/13/14 25
Cloud Software Development Lifecycle
10/13/14 27
MonitorProvisionDeployTestBuildCode
Elastic Beanstalk
OpsWorks
Cloud
Watch
Cloud
Formation
Code
Deploy
Code
Commit
Code
Pipeline
PAIN POINTS• UNIT TESTS INCOMPLETE
• MOCK TESTS MAINTENANCE
• EXPENSIVE TEST ENVIRONMENT
• TEST ENVIRONMENT ≠ PRODUCTION
• DEPLOYMENT CYCLES
A lot of options…
Configuration Management Systems• Puppet• Chef• Saltstack
Deployment Frameworks• Elastic Beanstalk• OpsWorks• Ansible• Fabric• Capistrano
Infrastructure Management• CloudFormation
Bake an AMI Configure dynamically
Time consuming configuration (startup time)
Static configurations (less change management)
Bootstrapping
Bake an AMI Configure dynamically
Continuous deployment (latest code)
Environment specific (dev-test-prod)
Bootstrapping
IngredientsUbuntu nginx boundary Unity jQuery SQLServer hbase NewRelic
EC2 node.js Cybersource hive ElasticSearch Ruby Twilio EE S3
ELB boto Magento PHP EMR SES Route53 SimpleDB Campfire
nagios Paypal CentOS CloudSearch levelDB mongoDB python
securitygroups Usahidhi PostgresSQL Github apache bootstrap
SNS OpsView Jekyll RoR EBS FPS VPC Mashery Vertica RDS
Optimizely MySQL puppet tsunamiUDP R asgard cloudwatch
ElastiCache cloudopt SQS cloudinit DirectConnect BSD rsync STS
Objective-C DynamoDB
Infrastructure, Configuration Management & Monitoring
Ubuntu nginx boundary Unity jQuery SQLServer hbase NewRelic
EC2 node.js Cybersource hive ElasticSearch Ruby Twilio EE S3
ELB boto Magento PHP EMR SES Route53 SimpleDB Campfire
nagios Paypal CentOS CloudSearch levelDB mongoDB python
securitygroups Usahidhi PostgresSQL Github apache bootstrap
SNS OpsView Jekyll RoR EBS FPS VPC Mashery Vertica RDS
Optimizely MySQL puppet tsunamiUDP R asgard cloudwatch
ElastiCache cloudopt SQS cloudinit DirectConnect BSD rsync STS
Objective-C DynamoDB
They had this built for the previous 3 months, all on the East Coast.
We built this part in 9 hours to be safe.
AWS +Puppet +
Netflix Asgard + WAN Optimization Software +
DevOps =
Cross-Continent Fault-Tolerance On-Demand
Webmaker.org circa 2012
• Included Apps, non-SOA: Thimble, Popcorn, Goggles
• ~20 pushes of new software in 2012
• Operations and Development interacted mostly through bugzilla
tickets for deploys.
• Hosting in physical datacenter at Mozilla
Webmaker.org circa early 2013
• Deciding to go 12-factor, SOA in app layer
• Weekly pushes of Popcorn on train model
• Operations and Development interacted mostly through bugzilla
tickets for deploys.
April 2013
Webmaker begins rebuilding entire platform
SOA, 12-factor in node.js exclusively
Moving apps into AWS and DevOps / CI
Since April 2013….Openbadges, Webmaker combine for: 1339 Pushes
Pushes Per Day to Staging / Prod
Pushes Per Day (Staging and Prod)
Who?
• ~30 Paid Developers
• Hundreds of Students
• Thousands of Contributors
• One DevOps / Internet Jedi
• Multiple Teams
How?
• Puppet, Jenkins, Fabric
• Tight feedback loops:
Newrelic, Opsview
• Culture Shift
• Staging Envs
• Brave devs iterate,
keeping work in-context
• Visible Ops
• Cross-train developers in
operations
1) Know How You Were Doing Before
2) Know What Changed When
3) Know How You Are Now Doing
=
The confidence to try more things
and try them faster, with minimum
viable planning.
{"Description" : "Create RDS with username and password","Resources" : {
"MyDB" : {"Type" : "AWS::RDS::DBInstance","Properties" : {
"AllocatedStorage" : "500","DBInstanceClass" : "db.m1.small","Engine" : "MySQL","EngineVersion" : "5.5","MasterUsername" : "MyName","MasterUserPassword" : "MyPassword"
}}
}}
"AWS::CloudFormation::Init" : { "config" : {
"packages" : {"yum" : {
"mysql" : [],"mysql-server" : [],"httpd" : [],"php" : [],"php-mysql" : []
}},"sources" : {
"/var/www/html" :"https://s3.amazonaws.com/my-builds/build-v4.zip"
}}
{"Parameters" : {"KeyName" : {
"Description" : "Name of an existing EC2KeyPair to enable SSHaccess to the instance",
"Type" : "String"}
},}
CLOUDFORMATION
TEMPLATE
PROCEDURAL
DEFINITIONCreate it programmatically
KNOWN
CONFIGURATIONStore stack configuration in
source control
PARAMETER
DRIVENDynamic and user-driven
templates
COLLABORATIONShare templates with ease
as just files
11.6s
Mean time
between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean number of
hosts
simultaneously
receiving a
deployment
30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM
top related