Continuous Deployment with Jenkins and Salt

Continuous deployment with Jenkins and Salt

LinuxCon CloudOpen ContainerCon

North America 2015, Seattle

Anirban Saha

About me

Techie…

Traveller…

Thinker…

Author of ‘Salt Cookbook’

Deployment

Should be…

• Simple

• Stable

• Fast (good to have…)

• Reliable

• One click

Problems faced

• Code distribution

• Additional tasks (commands, migrations)

• Latency

• Parallel execution

• Batch deployments

• Ensuring uptime

Methods availableDeployment

Configuration management tools :

• Puppet, Chef, Ansible, Salt

Remote execution tools:

• SSH, Parallel SSH, Rundeck

Packages:

• Code packaged as RPM or DEB files

Code storage

• GIT server (Github, Gitlab, Bitbucket, etc.)

• Object storage (Amazon S3, Swift, etc.)

• Package repositories (YUM/Debian)

• Configuration management repositories

• File servers managed manually

ProblemsDeployment:

• Remote execution tools :

• Time consuming (over SSH)

• Need to maintain inventory of hosts

Code storage:

• GIT repositories:

• Usually centrally located, latency when deployed from GIT

• Package repositories (RPM/DEB):

• Problems in simultaneously updating repo metadata

Rollback:

• Version manipulation is a problem with most configuration

management and remote execution tools

Salt

Provides…

• Agent based communication

• Fast execution

• Orchestration

• Batch execution

• Strong Integration with providers and

services

Salt featuresTargeting deployment nodes :

• By hostname

• ‘myappweb*’

• By specific grains

• -G ‘node_type:webserver’

• By nodegroups

• -N webgroup

Salt featuresTask dependencies :

• Salt requisites to make tasks dependent on each other

get_archive:

module.run:

.

.

deploy_app:

module.run:

.

.

- require:

- module: get_archive

Salt featuresOrchestration :

webgrp2_deploy: salt.state:

- tgt: ' webgrp2' - tgt_type: nodegroup - sls:

- webserver.deploy - require:

- salt: webgrp1_deploy

webgrp1_deploy: salt.state:

- tgt: 'webgrp1'- tgt_type: nodegroup - sls:

- webserver.deploy

Salt features

Batch Execution :

• By number of hosts

• --batch-size 5

• Executes on 5 hosts at a time

• By percentage of hosts

• --batch-size 25%

• Executes on 25% of total target

hosts at a time

Salt features

Salt API:

# curl -H "Accept: application/json"

-d secretkey= "mysupersecretkey"

-k https://10.0.0.2:8080/hook/deploy

{"success": true}

Tools

To be used…

• GIT for code repository

• Jenkins for CI

• Ant for build and tasks

• Amazon S3 for code archive storage

• Salt for deployment

• Shell scripts (good old bash) for

additional tasks

Salt deployment methods

• Salt hosted on the Jenkins server.

Jenkins calls Salt binary

• Salt hosted independently.

Jenkins calls Salt via SSH

• Jenkins calls Salt via Salt API

Objective

Steps• salt-cloud used to launch instances

• Post install actions used to synchronize EC2 grains

• Salt reactor used to run states on new instances ,

perform deployment and register with load balancer

• Jenkins build job used to build new code, create tags

and push new deployment ready archive to Amazon

S3

• Jenkins deploy job used to fetch new code version

and deploy code on target servers

Steps explained• Salt cloud is used to spawn instances

• Following parameter is provided in the profile to push custom EC2 grains,

• sync_after_install: grains

• The node is registered and deregistered from the load balancer with the

following module definition,

register:

module.run:

- name: boto_elb.register_instances

- m_name: mywebapp

- instances:

- {{ grains['ec2']['instance_id'] }}

• Here ['ec2']['instance_id'] is one of the custom grains pushed to the node

Steps explained

Fetching archive:

Salt s3 execution module used to get code archive from

Amazon S3

fetch_app_archive: module.run:

- name: s3.get

- bucket: mywebapp-us

- path: mywebapp-{{ app_version }}.tar

- local_file: /tmp/mywebapp-{{ app_version }}.tar

Steps explained

Setting App version grain:

After every deploy, a grain is set to record the

app version deployed for tracking,

app_version:

grains.present:

- value: {{ app_version }}

- require:

- cmd: deploy_app

Steps explainedDeployment based on App version:

At every deploy, it is checked if the version to

be deployed is already on the node using the

app_version grain,

{% if grains['app_version'] != app_version %}

deregister:

fetch_app_archive:

deploy_app:

register:

{% endif %}

Lets do it then !!!

Demo repository

Get the demo repository at the

following location,

https://github.com/rosesnthornz/

cloudopen-na-2015

Questions ?

Contact

Email :

[email protected]

Twitter : @rosesnthornz

Thank You !!!