Packaging in packaging: dh-virtualenv

Packaging in packaging

dh-virtualenv

@nailor

Jyrki Pulliainen [email protected]

Photo credit: Pontus Persson

What’s in this talk?

•  Look at different, existing deployment strategies

•  What is dh-virtualenv?

•  Packaging sentry with dh-virtualenv

virtualenv

+  Stable +  Non-Python Dependencies

+  Existing infrastructure +  Scripting

-  Out of date -  Complex -  Global state

+  Has the new stuff +  De facto +  Battle tested +  Contained

virtualenv

-  Native dependency handling

-  Source installs -  Random scripts

sh: mysql_config: command not found

What is dh-virtualenv?

Virtualenv inside a Debian package

It’s Open Source! •  GPL’d •  Documented •  Comes with a simple tutorial!

It’s a DebHelper Extension •  Idea stolen adapted from @hynek

* https://hynek.me/articles/python-app-deployment-with-native-packages/

In practice?

Package builder that…

1.  Creates a virtualenv 2. Installs requirements.txt into that virtualenv 3. Installs your project 4. Fixes the virtualenv to match production

Packaging with dh-virtualenv

Case in point: sentry

1.  Install dh-virtualenv apt-get install dh-virtualenv

2.  Create ‘debian’ directory

3.  Insert the minimal boilerplate files in there

Source: sentrySection: pythonPriority: extraMaintainer: Jyrki Pulliainen <[email protected]>Build-Depends: debhelper (>= 9), python, dh-virtualenv, python-devStandards-Version: 3.9.5

Package: sentryArchitecture: anyDepends: ${python:Depends}, ${misc:Depends}Description: dh-virtualenv-packaged sentry Amazing tool, now packaged with dh-virtualenv!

debian/control:

sentry (6.4.4-1) unstable; urgency=low

* Initial packaging with dh-virtualenv

-- Jyrki Pulliainen <[email protected]> Wed, 23 Jul 2014 12:02:20 +0200

debian/changelog:

echo 9 > debian/compat

#!/usr/bin/make -f

%:dh $@ --with python-virtualenv

debian/rules:

5.  Build your package! dpkg-buildpackage

dpkg-buildpackage: source package dh-virtualenvdpkg-buildpackage: source version 0.7-2dpkg-buildpackage: source distribution unstabledpkg-buildpackage: source changed by Jyrki Pulliainen <[email protected]>dpkg-buildpackage: host architecture i386 dpkg-source --before-build dh-virtualenv fakeroot debian/rules cleandh clean --with python2 --with sphinxdoc dh_testdir debian/rules override_dh_auto_cleanmake[1]: Entering directory `/vagrant_src/dh-virtualenv'rm -rf doc/_buildrm -f doc/dh_virtualenv.1rm -rf dh_virtualenv.egg-infodh_auto_cleanrunning clean'build/lib.linux-i686-2.7' does not exist -- can't clean it'build/bdist.linux-i686' does not exist -- can't clean itremoving 'build/scripts-2.7' (and everything under it)make[1]: Leaving directory `/vagrant_src/dh-virtualenv' dh_clean dpkg-source -b dh-virtualenvdpkg-source: error: can't build with source format '3.0 (quilt)': no upstream tarball found at ../dh-virtualenv_0.7.orig.tar.{bz2,gz,lzma,xz}

dh_virtualenvRunning virtualenv with interpreter /usr/bin/python2New python executable in debian/sentry/usr/share/python/sentry/bin/python2Also creating executable in debian/sentry/usr/share/python/sentry/bin/pythonInstalling setuptools, pip...done.Unpacking /vagrant_src/sentry-with-dh-virtualenv Running setup.py (path:/tmp/pip-ttX1Ip-build/setup.py) egg_info for package from file:///vagrant_src/sentry-with-dh-virtualenv

warning: no previously-included files matching '*~' found anywhere in distributionDownloading/unpacking BeautifulSoup>=3.2.1,<3.3.0 (from sentry==6.4.4) Downloading BeautifulSoup-3.2.1.tar.gz Running setup.py (path:/vagrant_src/sentry-with-dh-virtualenv/debian/sentry/usr/share/python/sentry/build/BeautifulSoup/setup.py) egg_info for package BeautifulSoup

Downloading/unpacking celery>=3.0.15,<3.1.0 (from sentry==6.4.4) Downloading celery-3.0.25.tar.gz (1.1MB): 1.1MB downloaded Running setup.py (path:/vagrant_src/sentry-with-dh-virtualenv/debian/sentry/usr/share/python/sentry/build/celery/setup.py) egg_info for package celery Upgrade: no old version found.

no previously-included directories found matching '*.pyc' no previously-included directories found matching '*.sw*’

5.  Install the shiny Debian package in production!

+  Non-Python dependencies +  Existing Infrastructure +  The new hotness +  Contained

dh-virtualenv

-  Can be slow to build -  Requires still digging for requirements -  Build system needs to have the exactly

same Python

dh-virtualenv

Thanks! github.com/spotify/dh-virtualenv dh-virtualenv.readthedocs.org labs.spotify.com/2013/10/10/packaging-in-your-packaging-dh-virtualenv/