@nzupan Turbo-boosting your Python development
Turbo boosting your python development
Jan 13, 2015
A talk describing how I use services like Travis CI, Coveralls, Papertrail, DataDog and others to make my life easier and more productive on Plone and Pyramid projects.
Personally, I regard time spent fixing preventable problems and doing routine tasks is time wasted stupidly. I'll show you how you can leverage some of the recently available "cloud" services to cut the amount of routine tasks in your day-to-day work.
This talk is somewhat based on the Travis CI talk I had last year on PloneConf in Arnhem, but extended to include other cloud services.
Personally, I regard time spent fixing preventable problems and doing routine tasks is time wasted stupidly. I'll show you how you can leverage some of the recently available "cloud" services to cut the amount of routine tasks in your day-to-day work.
This talk is somewhat based on the Travis CI talk I had last year on PloneConf in Arnhem, but extended to include other cloud services.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
@nzupan
Turbo-boostingyour Python development
Problem
Problem
Pull Request
Problem
Pull Request
Problem
Pull Request
Travis build
Problem
Pull Request
Travis build
Automatic Deployment
Problem
Pull Request
Travis build
Automatic Deployment
Sentry
Problem
Pull Request
Travis build
Automatic Deployment
SentryQA
Problem
Pull Request
Travis build
Automatic Deployment
SentryQA
Datadog
Papertrail
CODE REPOSITORY
Any DVCS is OK, but I prefer git
Main reason: GitHub’s UI
CONTINUOUS INTEGRATION
Run your tests often
Preferably on every commit
In a clean environment
Easy and fun CI for your Python packages
Travis CI
Hosted Continuous Integration for the Open Source Community
+
BUILDS
SINGLE BUILD
SINGLE BUILD (CONT.)
Free for any public GitHub repo
Private repos supported too!
Already ran ~8 mio tests for ~78k open-source projects
Supports:C, C++, Clojure, Erlang, Go, Groovy,
Haskell, Java, JavaScript, Perl, PHP, Python, Ruby, Scala
Preinstalled DBs:MySQL, PostgreSQL, MongoDB,
CouchDB, Redis, Riak, RabbitMQ, Memcached, Cassandra, Neo4J, ElasticSearch, Kestrel, SQLite3
GETTING STARTED
Sign in to travis-ci.org with GitHub account
Enable repository
Add .travis.yml to your repository
Push it
MINIMAL CONFIG
MORE CONTROL
PREINSTALLED SERVICES
HEADLESS TESTING
BRANCHES
BUILD NOTIFICATIONS
+ many more
STATUS IMAGES
STATUS IMAGES
https://travis-ci.org/[YOUR_GITHUB_USERNAME]/[YOUR_PROJECT_NAME].png
PULL REQUESTS TESTING
HEROKU: CLOUD APPLICATION PLATFORM
OUT-OF-THE-BOX
Add to .travis.yml:
AUTO-DEPLOYING PLONE
SENTRY: CLOUD ERROR REPORTING
ERROR AGGREGATION
ERROR CONTEXT
SENTRY SETUP
Datadog: cloud monitoring
Datadog: cloud monitoring
MUNIN + ...
... + OPS TWITTER
OUT-OF-THE-BOX
EVENTS:
Fabric, Chef, Puppet
RSS
GitHub, Pivotal Tracker, Redmine, etc.
Pingdom
Jenkins
HTTP API
METRICS
Nginx & Apache
Amazon AWS & HP Cloud
Cacti & Nagios
PostgreSQL, MySQL, Cassandra, MongoDB, CouchDB, etc.
Haproxy, Varnish, Memcached, RabbitMQ, Solr & ElasticSearch, etc.
CUSTOM METRICS
$ easy_install dogstatsd-python
from statsd import statsd
stasd.gauge(‘users.active’, 50)
statsd.counter(‘base_edit.rendered’)
statsd.histogram: (‘catalog.query.time’, duration)
statsd.set: (‘users.unique’, user_id)
CUSTOM APP METRICS
PAPERTRAIL: CLOUD LOG AGGREGATION
WHO IS DOWNLOADING?
BONUS SLIDE: CODE COVERAGE REPORTS
Problem
Pull Request
Travis build
Automatic Deployment
SentryQA
Datadog
Papertrail
bobtemplates.niteoweb
THE PLONE TEMPLATE
zc.buildout best practices
GenericSetup install profile
Zope 3 browser layer
z3c.jbot overrides folder
static/ resourceDirectory for serving static resources (images, CSS, JS, etc.)
Sphinx documentation
test suite with a solid test coverage
Travis CI integration
THE PYRAMID TEMPLATE
zc.buildout best practices
SQLite for development, PostgreSQL for production
A sample view with a Chameleon template
A sample DB model
static resources filled with Twitter Bootstrap niceties
Sphinx documentation
Exemplary test suite
Travis CI & Coveralls integration
Heroku continuous deployment
Try it out:https://github.com/niteoweb/bobtemplates.niteoweb/
Then, let’s generalize it!
Thanks!
Some images taken from sxc.hu
http://travis-ci.org/
http://coveralls.io/
https://papertrailapp.com/
http://www.datadoghq.com/
http://docs.fabfile.org/
https://pypi.python.org/pypi/githook
https://www.heroku.com/
@nzupan
Related Documents
