Rock Solid Deployment of Web Applications

Pablo Godel @pgodel - codemiami.comJan 12th 2013 - Miami, FL

Rock Solid Deploymentof Web Applications

Who Am I?

⁃ Born in Argentina, living in the US since 1999⁃ PHP & Symfony developer

⁃ Founder of the original PHP mailing list in spanish ⁃ Co-founder of ServerGrove

⁃ Founded ServerGrove Networks in 2005

⁃ Provider of web hosting specialized in PHP, Symfony, ZendFramework, MongoDB and others

⁃ Servers in USA (right here in Miami) and Europe!

ServerGrove!

⁃ Very active open source supporter through codecontributions and usergroups/conference sponsoring

Community is our teacher

Deployment

?

Deployment

Software deployment is all of the activities that makea software system available for use.

http://en.wikipedia.org/wiki/Software_deployment

http://en.wikipedia.org/wiki/Software_system

http://en.wikipedia.org/wiki/Software_system



Deployment

A very important part of the application life-cycle

Deployment

A very important critical part of the application life-cycle

Deployment

It should not be an after thought

Deployment

It should be predictable

Deployment

The more you do it the better it goes

Deployment: Goals

Deployment: Goals

One-click deploys

Continuous deploys

Deployment: Goals

Web AppsDeployment

Web AppsDeployment

Web AppsDeployment

Anytime & Anywhere

Deployment: Goals

Anyone

Deployment: Goals

Reliable

Deployment: Goals

Rollbacks

Deployment: Goals

No downtime

Deployment: Goals

Reusable

Deployment: Goals

Scalable

Deployment: Goals

• One-click / continuous deploys• Anytime & Anywhere• Anyone • No downtime• Predictable & Reliable• Rollbacks• Reusable• Scalable

Deployment: Goals

Deployment Facts

Deployment: Fact #1

Deployment starts with the developer

• Setup development environment to be as close as possible to productions servers

• Setup test/qa/staging servers• Use Vagrant to manage VMs• Use Puppet/Chef to manage OS packages/

configuration

Deployment: Fact #2

Success linked to server OS setup

• Use Puppet/Chef to manage OS packages/configuration

• Create OS packages for 3rd party software• Setup your own package repositories

Deployment: Fact #3

Monitoring is uptime

• Use monitoring tools to know what is going on with your servers (Ganglia, Cacti, Zabbix, etc.)

• Add monitoring and metrics to your app (Graphite, StatsD, New Relic)

• Use your logs wisely (Graylog, Logstash, Kibana)

Deployment Methodologies

Deployment Methodologies

• VIM-style• FTP uploads• rsync• source control (svn, git)• Build tools (ant, phing)• Specialized tools (capistrano, fabric, etc)• Package based (rpm, deb, etc)

Web Apps Deployment: Steps overview

Web Apps Deployment: First time

• Copy files to server(s)• Set server-side configurations• Load DB fixtures• Process and install assets• Warm up cache• “Enable” site

• Copy files to server(s)• Apply DB updates (migrations)• Process and install assets• Warm up cache• “Enable” site

Web Apps Deployment: Subsequent times

Deployment: Challenges


• rsync• git pull• setup git repo on local network to save

bandwidth and avoid issues if git server is down (i.e. github)

Challenge:Fast & reliable copy of files

Solutions:


• use a tool that allows to go from 1 to n servers easily (i.e. capistrano)

• pssh allows to send commands to n servers in parallel

• package your app in OS packages like .rpm/.deb to easily install across n servers

Challenge:Scalable

Solutions:


• test!• tag releases• dedicated branches (master for production)• deploy each release in its own directory

Challenge:Rollbacks

Solutions:


• use ssh based connections• don’t store passwords on source control• store sensitive strings (passwords) in server

environment variables

Challenge:Secure

Solutions:


Challenge:DB migrations

Solutions:• Doctrine Migrations• Consider document oriented DBs like

MongoDB

“The best migrations are the ones you don’t have to do”


Challenge:Static assets

Solutions:• YUICompress shrinks JS and CSS file sizes• Enable web server compression• Add versioning to static assets links (code.js?v=1)• Assetic combines multiple files into one• Run utilities locally or in a staging server, deploy

result


Challenge:Caching

Solutions:• Update one server while others handle load• Group servers and update group at a time• execute commands on “finalize” to clear up APC

cache


Challenge:File permission conflicts

Solutions:• Run Apache/PHP with same user• Use php-fpm instead of mod_php• Create “deploy” user and add web server to the

group• Use setfacl to give write access to multiple users

Web Apps Deployment: Other common pitfalls

Web Apps Deployment: Other common pitfalls

• Case sensitive filesystems• Configuration differences• Outdated 3rd party software• Github down

$ git daemon --base-path=/git/repo/path/ --export-all

$ git clone git://127.0.0.1/repo

http://ozmm.org/posts/when_github_goes_down.html



Web Apps Deployment: Examples


Simplest continuous deployment ever!

<?php

exec(‘/usr/bin/env -i HOME=/var/www git pull’);echo “All done!”;

hook.php

screenshot


Capistrano

• Ruby based• Very extensible• Large number of extensions• Simple client side installation

$ gem install capistrano


Capistrano

set :application, "myapp" # Application nameset :deploy_to, "/var/www/myapp"

set :user, "deployer"set :use_sudo, false # sudo isn't required

set :deploy_via, :remote_cache set :repository, "[email protected]:user/repo.git"

role :web, "server.example.com", “server2.example.com”

mailto:[email protected]



$ cap deploy:setup

Capistrano


|-- releases`-- shared |-- logs `-- uploads

Capistrano


Capistrano

$ cap deploy$ cap deploy:migrations$ cap deploy:rollback


|-- current (symlink to releases/20130112)|-- releases| `-- 20130112`-- shared |-- logs `-- uploads

Capistrano

Web Apps Deployment:Other options

• Fabric• WePloy• Phing• Magallanes• Jenkins

Web Apps Deployment: Tools


App Metrics: StatsD & Graphite


Logging: LogstashShip logs from any source, parse them, get the right timestamp, index them, and search them


Logging: Logstash

Configure Apache to log jsonLogFormat "{ \"@timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \"@fields\": { \"client\": \"%a\", \"duration_usec\": %D, \"status\": %s, \"request\": \"%U%q\", \"method\": \"%m\", \"referrer\": \"%{Referer}i\" } }" logstash_json

# Write our 'logstash_json' logs to logs/access_json.logCustomLog logs/access_json.log logstash_json

{ "@timestamp": "2012-08-22T14:35:19-0700", "client": "127.0.0.1", "duration_usec": 532, "status": 404, "request": "/favicon.ico", "method": "GET", "referrer": "-" }

Result


Logging: Graylog


Logging: KibanaKibana is a user friendly way to view, search and visualize your log data


Packaging: fpm

https://github.com/jordansissel/fpm

Build packages for multiple platforms (deb, rpm, etc) with great ease and sanity.

fpm -s dir -t rpm -n "myapp" -v 1.0 /var/www/myapp

fpm -s dir -t deb -a all -n myapp -v 1.0 /etc/apache2/conf.d/my.conf /var/www/myapp



Web Apps Deployment: Summary

•Stop using FTP

•Stop using FTP•Plan early

•Stop using FTP•Plan early•Practice

•Stop using FTP•Plan early•Practice•Monitor

•Stop using FTP•Plan early•Practice•Monitor•AUTOMATE!

QUESTIONS?

Slides: http://slideshare.net/pgodelTwitter: @pgodel

E-mail: [email protected]

http://slideshare.net/pgodel




Thank you!

Slides: http://slideshare.net/pgodelTwitter: @pgodel

E-mail: [email protected]





Rock Solid Deployment of Web Applications

Documents

web apps deployment

deploymentsoftware deployment

deployment facts

web appsdeployment

web server compression

git server

server os setup use

provider of web hosting