Achieving Continuous Delivery: An Automation Story

ACHIEVING CONTINUOUSDELIVERY:

AN AUTOMATION STORY

ABOUT MEJAMES CAMMARATA, DIRECTOR - CORE ENGINEERING

Python developer since 2003

Started contributing to Cobbler in September of 2008, and tookover the project leadership in 2010

Joined Ansible in July, 2013

WHAT IS CONTINUOUSDELIVERY?

Continuous delivery aims to reduce the impact of releasingsoftware by doing it more frequently.

Popularized in the book 'Continuous Delivery: Reliable SoftwareReleases through Build, Test, and Deployment Automation',

written by Jez Humble and David Farley in 2010.

(VIDEO)

WHY USE ANSIBLE FORCONTINUOUS DELIVERY?

SIMPLICITYNo agents, and a clean playbook syntax are undoubtably

Ansible's two greatest strengths.

AUTOMATION OF COMPLEX DEPLOYMENTORCHESTRATION

Rolling reboots of web serversMigrating primary/secondary/tertiary databases like MySQLBuilding and deploying new AMIs/virtual instances orcontainers for immutable infrastructureRemoving/re-enabling monitoring on servers or services beingupdated

REUSABILITY AND CONSISTENCYDevelopers using Vagrant or other setups locally can easilycheck out the exact same playbooks run in staging and/orproduction to ensure consistency.Your CI system can use the exact same playbooks too, ratherthan shell scripting everything.

EXTENSIBILITYAnsible has over 250 modules included, but if you need tointerface with an API we don't support, writing modules toextend the functionality is very easy and straightforward.

OTHERS HAVE ALREADY DONE THISTons of blogs out there already outline how to set all of this upusing Ansible, so most of the hard work has been done for you!

Ansible's user community has exploded, and getting help is easy.

COMMON PATTERNS

CONTINUOUS INTEGRATION SYSTEMJenkins/Hudson, TravisCI, etc.

(source, DevOps Reactions – http://bit.ly/1NkG1ax)

SOURCE CONTROLStores both the application code and your Ansible playbooks

(infrastructure as data, not code).

All commits trigger an automatic deployment to the staging/QAenvironment(s), followed by automated testing.

If the automated testing passes, the deployment to productioncan triggered by a person or automatically.

AUTOMATED TESTINGAbsolutely critical to continuous integration and delivery,

automated testing is what verifies your code is ok to deploy.

Unit testingIntegration testingSelenium, Ghost / CasperJS, etc.

MONITORING (ALL THE THINGS!)Log and graph everything (Graphite and other popular projects

make this nearly trivial). This is how you diagnose problems withany given deployment later.

(source, https://codeascraft.com/2010/12/08/track-every-release/)

An excellent Digital Ocean blog on Graphite+statsd+collectd:http://bit.ly/1Gx4UM2

CHATOPS

BENEFITS OF CHATOPS:Everyone knows what everyone else is doing, leading to anincreased level of transparency across your teams.Because of this transparency, these actions also become self-documenting which helps new hires learn the ropes morequickly, especially if they're remote.Communication is instantaneous and more fluid, as Ops don'thave to flip between email or head off to a 'war room' forissues.Time to execute actions is reduced, as there is now a singleinterface to trigger actions on disparate pieces ofinfrastructure.

CASE STUDIES

ATLASSIAN

Atlassian is a very well known software company, with productssuch as JIRA, HipChat and Confluence to name a few.

Uses Ansible to address what they call the 'last mile problem', orthe final step in continuous deployment.

From their blog:

Developers perform continous testing/integration againstfeature branchesAfter these feature branches are merged into the releasebranch, Bamboo (Atlassian's CI product) is used to create abuild artifact (binary)Ansible is used to deploy this same artifact to each of thestaging, QA and production environments in turn

ATLASSIAN (CONT.)Advantages:

1. Promoting the same build artifact through each environmentensures consistency.

2. Despite using in-house products, their approach is verytechnology agnostic and could easily be adapted to other tools.

Disadvantages: Internal teams are very segregated, with strictseparation of duties (silos) for deployments.

RISINGSTACK

RisingStack is a JavaScript consultancy group, focusing onNodeJS delivery.

Using CodeShip (a 3rd party service) and a custom web hookservice to trigger Docker builds and Ansible deployments to

automate their container-based approach to immutableinfrastructure.

(source: http://blog.risingstack.com/shipping-node-js-applications-with-docker-and-codeship/)

RISINGSTACK (CONT.)From their blog:

Commits to the master branch trigger trigger builds in theCodeShip serviceIf all tests pass, a new docker image is created and uploaded toDocker HubAfter the upload is complete, another web hook is triggered ontheir (custom, in-house) service which executes AnsibleAnsible downloads and deploys the new docker image on eachapplication server

RISINGSTACK (CONT.)Advantages: Completely automated with no intervention

required to deploy.

Disadvantages: Custom service, when an off-the-shelf CI systemcould be used to simplify things.

BIGPANDA

BigPanda is a company centered around automating incidentmanagement, and use ChatOps and Ansible heavily in their day

to day operations

5 Reasons We Love Using Ansible for Continuous Delivery -http://bit.ly/1EkbqEi

Their motto is: 'Make the scariest part of the build the easiest'.

BIGPANDA (CONT.)Use a combination of HipChat and Hubot (called BeanBotinternally) to automate ops tasks (a one-person ops team).

GRASSWIRE

GrassWire is a recent startup built around crowd-sourcing factverification in news reports.

Another ChatOps follower, GrassWire deploys their software viaSlack, which natively supports executing webhooks remotely and

does not require a chat bot.

Use Packer to build new AMIs via Jenkins, and use Ansible todeploy those new images to production along with their

application.

Using Terraform to build initial AWS environments, rather thanCloudFormation or other tools.

GRASSWIRE (CONT.)Their Packer integration also uses Tower, to take advantage of

the provisioning callback feature.

"provisioners": [{ "type": "file", "source": "./{{user ̀gw_env̀}}/setup_callback.sh", "destination": "/home/ec2-user/setup_callback.sh" }, {"type": "shell", "inline": [ "sudo chmod +x /home/ec2-user/setup_callback.sh", "echo sh /home/ec2-user/setup_callback.sh | sudo tee --append /etc/rc.local ]}}]

Since they also use Tower for other tasks, they are able to use thetower-cli command directly in Jenkins to simplify their Jenkins

tasks.

TIPS AND TRICKS

USE ANSIBLE TO BUILD YOUR CONTINUOUS DELIVERYENVIRONMENT

Roles exist (on Galaxy and generally on Github) to build all ofthese components for you now. Use them to get up and running

quickly.

USE ANSIBLE FROM DAY 1If you do anything, do it from Ansible. Don't SSH to a server, just

put whatever it is you need to do in a playbook and run thatinstead.

QUESTIONS?

ATTRIBUTIONS:LEGO Great Ball Contraption (GBC) Layout 2012.9https://www.youtube.com/watch?v=sUtS52lqL5w

DevOps Reactions: Continuous Deliveryhttp://bit.ly/1NkG1ax

Code as Craft Blog: Tracking Every Releasehttps://codeascraft.com/2010/12/08/track-every-release/

Digital Ocean Blog: An Introduction to Tracking Statistics withGraphite, StatsD, and CollectD

http://bit.ly/1Gx4UM2

Atlassian Blog: Practical Continuous Deployment:http://www.slideshare.net/tarkasteve/london-atlassian-user-

group-february-2014

ATTRIBUTIONS (CONT.):RisingStack: Shipping Node.js Applications with Docker and

Codeshiphttp://blog.risingstack.com/shipping-node-js-applications-with-

docker-and-codeship/

BigPanda:#ChatOpsFTW: http://www.slideshare.net/chuparkoff/chatops-

ftw5 Reasons We Love Using Ansible for Continuous Delivery:

http://bit.ly/1EkbqEi

THANKS!twitter: jimi1283 github: jimi-c