Gunicorn, Django & WSGI

Gunicorn, Django & WSGIBenoît Chesneau

23/05/2010 - djangocon berlin

Tuesday, May 25, 2010

Gunicorn [email protected] web & OpensourceWeb craftmanhttp://www.e-engura.com & http://benoitc.im

benoît chesneau


mailto:[email protected]




http://benoitc.im

http://benoitc.im

http://benoitc.im

http://benoitc.im

• WSGI ?

• Django & WSGI

• Gunicorn


WSGI ?

•Web Server Gateway Interface

• PEP 333

• Web interface between the HTTP server and Python applications.


WEB Server

WSGI

Python app


Serveur web

WSGI

Application Python

App App App


def app(environ, start_response): """Simplest possible application object""" data = 'Hello, World!\n' status = '200 OK' response_headers = [ ('Content-type','text/plain'), ('Content-Length', str(len(data))) ] start_response(status, response_headers) return [data]

simple application


WSGI Middleware

class CustomHeader(object): def __init__(self, app): self.app = app def __call__(self, environ, start_response): environ["HTTP_X_MY_HEADER"] = "1" return self.app(environ, start_response)


Chain WSGI aps

application = CustomHeader(app)


Django & WSGI


a way to deploy


but we can discusswith WSGI

• Why reinvent the wheel

• Common principles: middlewares, request, ...

• intégrate != compatible wsgi


• django-wsgi

• twod.wsgi


twod.wsgi - embed

import osfrom twod.wsgi import DjangoApplication

os.environ['DJANGO_SETTINGS_MODULE'] = "yourpackage.settings"django_app = DjangoApplication()


embed WSGI apps

from twod.wsgi import call_wsgi_appfrom somewhere import wsgi_app

def run_app(request, path_info): response = call_wsgi_app(wsgi_app, request, path_info) response['Server'] = "twod.wsgi 1.0" return response


Deploy

• The real thing

• 2 kindsof deploiements :

• Modules to HTTP servers

• WSGI servers


Modules

• uWSGI (NGINX)

• mod_wsgi (Apache)


WSGI server

• spawning, paster, ...

• cherrypy, ..

• gunicorn




Why

• Unicorn is awesome

• need something stable

• need something simple


Philosophy

• Simple

• Minimal

• Performant

• Unix


gunicorn

• Green unicorn

• WSGI 1.0

• Load balancing via pre-fork and a shared socket

• async, sync worker (slow and fast clients)

• Easy integration with Paster compatible app & ... Django


More...

• HTTP Stream. Decode on the fly HTTP chunks

• Upgrade “à la nginx”

• Eventlet, Gevent, Tornado

• DSL Config


http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi

gunicorn is the winner in my eyes. It's easy to configure and get up and running and certainly fast enough [..] .




blogg.se - swedens largest blog provider

(stats for week 18 '10) * 2.5M unique visitors (source: kiaindex.net) * 800 000 new entries * 850 000 new comments * 16 000 new blogs


http://blogg.se/

http://blogg.se/

http://kiaindex.net/

http://kiaindex.net/

DSL Configbind = "127.0.0.1:8000" daemon = Truedebug = Falseworkers = 3

def when_ready(server): ....


Simple command line

• gunicorn_django -w 3 -d -p /path/to/pidfile /myproject/settings.py

• ./manage.py run_gunicorn -w 3

• gunicorn_django -w 3 -k “egg:gunicorn#eventlet” /myproject/settings.py


Graceful reload

KILL -HUP `head -1 /path/to/pidfile`


QUIT

kill -QUIT `cat /path/to/pidfile`


Increase/ Decreasenumber of workers

• KILL -TTIN PID

• KILL -TTOU PID


use it behind nginx

• proxy

• buffering

• secure your app against DOS


nginx.conf

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;proxy_set_header Host $http_host;

proxy_redirect off;

if (!-f $request_filename) { proxy_pass http://app_server; break;}


upstream app_server { server 192.168.0.7:8080 fail_timeout=0; server 192.168.0.8:8080 fail_timeou=0;}

server { listen 80 default; client_max_body_size 4G; server_name _; keepalive_timeout 5; root /path/to/app/current/public;

location / { proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host;

proxy_redirect off;

if (!-f $request_filename) { proxy_pass http://app_server; break; } }}


async - the trick

proxy_buffering off;


Some tricks

• number of workers = 2xCPUs +1

• preload your project

• pre/before hooks

• when_ready hook


Tune your os

• Increment fd limits : ulimit -n 2048

• Increment the connections queue (somaxcon)

• Play with tcp windows

• More : http://gunicorn.org/tuning.html


DEMO


0.10

• HTTP parser (in C?)

• Increase unitests

• Reload hook

• status


Liens

• http://gunicorn.org

• http://e-engura.org

• http://www.python.org/dev/peps/pep-0333/

• http://bitbucket.org/2degrees/twod.wsgi/

• http://github.com/alex/django-wsgi


http://gunicorn.org

http://gunicorn.org

http://e-engura.org

http://e-engura.org

http://www.python.org/dev/peps/pep-0333/

http://www.python.org/dev/peps/pep-0333/

http://bitbucket.org/2degrees/twod.wsgi




Questions


@benoitc


Cette création est mise à disposition selon le Contrat Paternité 2.0 France disponible en ligne http://

creativecommons.org/licenses/by/2.0/fr/ ou par courrier postal à Creative Commons, 171 Second Street, Suite

300, San Francisco, California 94105, USA.


http://creativecommons.org/licenses/by/2.0/fr/




Gunicorn, Django & WSGI

Documents

wsgi http

django wsgi gunicorn

compatible wsgi

wsgi apache

wsgi server spawning

djangowsgi twod

web server wsgi python

wsgi appsfrom twod