Transcript
Prodution Architecture and
Deploymentwith Fabric
- Andy McCurdy -@andymccurdy
Whiskey Media
Whiskey Sites
Your First Django App
Basic Config (web)•Apache•mod_wsgi•use daemon mode•threads more efficient•processes if you're unsure of thread safety
WSGIDaemonProcess my-site python-path=/home/code/ processes=2 threads=150 maximum-requests=5000WSGIProcessGroup my-siteWSGIScriptAlias / /home/code/my-site/deploy/wsgi/my-site.wsgi
Basic Config (media)•Nginx•Use Nginx to proxy traffic to Apache•Meanwhile Nginx serves media
upstream my-site { server 127.0.0.1:8000;}server { listen 80; location ~ ^/media/ { root /home/code/my-site; expires 30d; } location / { proxy_pass http://my-site; proxy_set_header X-Real-IP $remote_addr; }}
Basic Config (db)•Databases•PostgreSQL
•Frank Wiles @ www.revsys.com
•MySQL•Percona @ www.mysqlperformanceblog.com
Basic Config (cache)•Use Memcached! Even 16mb will significantly help against a Digg or being Slashdot'ed•It's incredibly easy...
# settings.pyMIDDLEWARE_CLASSES = ( 'django.middleware.cache.UpdateCacheMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware')
CACHE_BACKEND = 'memcached://127.0.0.1:11211/'CACHE_MIDDLEWARE_SECONDS = 60*5 # 5 minutesCACHE_MIDDLEWARE_ANONYMOUS_ONLY = True
Basic Deployment•Copy Code & Media (rsync or scp)•Run DB migrations / syncdb•Bounce wsgi daemons
Managing Growth (db)•Move DB to its own server
•As much RAM as possible•Write-heavy (>10%)? Get fast disks•Tune your config file
Managing Growth (web)•Add more web servers
•Use a resource monitoring tool like Munin to understand if your app is CPU or memory bound
Even More Growth•Replicated or sharded Databases•Multiple load balancers for redundancy•Message queues•Crons•Search daemons (Solr, Sphinx)•etc...
(not so) Basic Deployment
image credit:Brad
Fitzpatrick
Deployment Reqs•Copy media to CDN•Maintenance splash page•Run DB migrations•Install/Upgrade Python dependencies•Add a new web server to the cluster•Execute arbritrary commands for sysadmin maintenance tasks
Deployment OptionsCapistrano
+ Out of box support for common use cases+ Hooks to customize tasks+ Source control integration+ Threaded deployment to multiple hosts
- Ruby :(
Deployment OptionsFabric
+ Very simple, tasks are just Python functions+ Easy to chain together tasks to create complex scripts out of bite size pieces
- No source control integration- No out of box support- Some bugs, although fairly easy to work around, and new maintainer is working on fixes
Fabric Basics•sudo easy_install fabric•need a fabfile.py•from fabric.api import *
•be mindful of tasks that may fail•each remote command starts fresh•changing directories
Core Functionality•local() - Run a command locally
•run() - Run a command remotely
•sudo() - Run a command remotely as another user
•put() - Copy a file from local to remote
•get() - Copy a file from remote to local
•many more helper-ish commands
Authentication•Relies on SSH model•Use SSH keys•Control access to root user via sudoers•When you have to revoke access, you just turn off their SSH account
Configuration•Fabric environment (env) -- it's just a dictionary•Hosts and Roles•Code Repositories•Whatever you need
•~/fabricrc•Global settings or all Fabric deployments•SSH username
Example Config# fabfile.pyfrom fabric.api import *
env.roledefs = { 'web' : ['10.1.1.1', '10.1.1.2'], 'db' : ['10.1.1.3'], 'lb' : ['10.1.1.4'], }
env.repositories = {...}
Tasks# fabfile.pydef uptime(): run('uptime')
$> fab uptime -H 10.1.1.3[10.1.1.3] run: uptime[10.1.1.3] out: 05:20:39 up 88 days, 12:00, 0 users, load average: 0.03, 0.03, 0.00
Mapping Roles to Tasks# fabfile.py
@roles('web')def uptime(): run('uptime')
$> fab uptime[10.1.1.1] run: uptime[10.1.1.1] out: 05:20:39 up 88 days... [10.1.1.2] run: uptime[10.1.1.2] out: 05:20:39 up 88 days...
Decorator Problems•Some problems with Fabric's role management•Can't override decorated tasks at command line as docs suggest
def default_roles(*role_list): def selectively_attach(func): if not env.roles and not env.hosts: return roles(*role_list)(func) else: if env.hosts: func = hosts(*env.hosts)(func) if env.roles: func = roles(*env.roles)(func) return func return selectively_attach
All better now#fabfile.py
@default_roles('web', 'db')def uptime(): run('uptime')
$> fab uptime# runs on all hosts in the 'web' and 'db' roles
$> fab uptime --roles lb# runs only on hosts in the 'lb' role
Dealing with Failures•By default Fabric dies if a task fails•Use a context manager when failures are anticipated
# fabfile.pyfrom __future__ import with_statement # py2.5
def symlink_me(): with settings(warn_only=True): run('rm /path/to/symlink') run('ln -s /home/andy /path/to/symlink')
Easy sys-admin•Make an "invoke" command•Great for sys-admin and one-off tasks
# fabfile.py
@default_roles('all')def invoke(command): "Invoke an arbritrary command" sudo(command)
# install new packages on all hosts in one command$> fab invoke:"apt-get install git-core"
Real World Tasks$> fab --list
Available commands:
bounce_wsgi_procs Bounce the WSGI procs by touching the filesdeploy Full deploymentdeploy_media Push media to S3invoke Invoke an arbritrary commandmigrate Run any migrations via Southreload_nginx Update Nginx's running configsplash_off Configure Nginx to serve the sitesplash_on Configure Nginx to serve a downed-site pageupdate_repositories Push code to serversupdate_dependencies Update dependencies to third party libs
Whiskey's Deploymentdef deploy(splash='no'):
"Full deployment" deploy_media() update_cached_repositories() update_dependencies() generate_releases() if splash == 'yes': splash_on() _symlink_code() migrate() bounce_wsgi_procs() if splash == 'yes': splash_off()
$> fab deploy:splash=yes
Questions?
top related