Harness the speed of the wheel Xavier Fernandez - @xavierfernandez Polyconseil May 13, 2014 Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 1 / 10
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Harness the speed of the wheel
Xavier Fernandez - @xavierfernandez
Polyconseil
May 13, 2014
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 1 / 10
What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
with a specially formatted file name and the .whl extension{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10
What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
with a specially formatted file name and the .whl extension{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10
What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
with a specially formatted file name and the .whl extension{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10
What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
with a specially formatted file name and the .whl extension{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10
What is Wheel?
Quick overview
a package format
defined in PEP 427 ”The Wheel Binary Package Format 1.0”
doc available at http://wheel.rtfd.org/
ZIP-format archive (with a structure close to PEP376)
with a specially formatted file name and the .whl extension{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 2 / 10
Benchmark
Installation times of pure python modules
Module .tar.gz .whl Ratio
pep8-1.5.0 0.65 sec 0.27 sec x 2.4flake8-2.1.0 1.84 sec 0.37 sec x 5.0mccabe-0.2.1 0.61 sec 0.25 sec x 2.4pyflakes-0.8 0.67 sec 0.29 sec x 2.3Django-1.6.4 10.6 sec 6.76 sec x 1.6
Installation times of python modules needing some compilation
Module .tar.gz .whl Ratio
lxml-3.3.5 60 sec 0.42 sec x 143numpy-1.8.1 208 sec 2.0 sec x 104
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 3 / 10
Benchmark
Installation times of pure python modules
Module .tar.gz .whl Ratio
pep8-1.5.0 0.65 sec 0.27 sec x 2.4flake8-2.1.0 1.84 sec 0.37 sec x 5.0mccabe-0.2.1 0.61 sec 0.25 sec x 2.4pyflakes-0.8 0.67 sec 0.29 sec x 2.3Django-1.6.4 10.6 sec 6.76 sec x 1.6
Installation times of python modules needing some compilation
Module .tar.gz .whl Ratio
lxml-3.3.5 60 sec 0.42 sec x 143numpy-1.8.1 208 sec 2.0 sec x 104
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 3 / 10
So how do we use it?
First
Distribute your own pure python libraries in this format, it’s easy:pip install wheel
setup.py based on setuptools:python setup.py bdist wheel
setup.py based on distutils alone:python -c "import setuptools; file = ’setup.py’;
setup code = open( file ).read().replace(’\r\n’, ’\n’);
exec(compile(setup code, file , ’exec’))" bdist wheel
(so switch to setuptools)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 4 / 10
So how do we use it?
First
Distribute your own pure python libraries in this format, it’s easy:pip install wheel
setup.py based on setuptools:python setup.py bdist wheel
setup.py based on distutils alone:python -c "import setuptools; file = ’setup.py’;
setup code = open( file ).read().replace(’\r\n’, ’\n’);
exec(compile(setup code, file , ’exec’))" bdist wheel
(so switch to setuptools)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 4 / 10
So how do we use it?
First
Distribute your own pure python libraries in this format, it’s easy:pip install wheel
setup.py based on setuptools:python setup.py bdist wheel
setup.py based on distutils alone:python -c "import setuptools; file = ’setup.py’;
setup code = open( file ).read().replace(’\r\n’, ’\n’);
exec(compile(setup code, file , ’exec’))" bdist wheel
(so switch to setuptools)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 4 / 10
So how do we use it?
Second
Precompile all your modules and cache them in a wheelhouse
Disclaimer
This solution is based on pip and virtualenv
Pitfall
You need a fairly recent of pip/setuptools (requires setuptools ≥ 0.8.0).A solution is to add a hook in virtualenvwrapper (postmkvirtualenv):
pip install -U pip setuptools
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 5 / 10
So how do we use it?
Second
Precompile all your modules and cache them in a wheelhouse
Disclaimer
This solution is based on pip and virtualenv
Pitfall
You need a fairly recent of pip/setuptools (requires setuptools ≥ 0.8.0).A solution is to add a hook in virtualenvwrapper (postmkvirtualenv):
pip install -U pip setuptools
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 5 / 10
So how do we use it?
Create your wheel
In your construction venv:pip install wheel
pip wheel my module --wheel-dir=my wheelhouse path
Install your wheel
In the destination venv:pip install my module --find-links=file://wheelhouse path
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 6 / 10
So how do we use it?
Create your wheel
In your construction venv:pip install wheel
pip wheel my module --wheel-dir=my wheelhouse path
Install your wheel
In the destination venv:pip install my module --find-links=file://wheelhouse path
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 6 / 10
Show me the conf!
pip.conf
find-links = file://my wheelhouse path
wheel-dir = my wheelhouse path
Also define a download cache to prevent useless downloads
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 7 / 10
So how do we use it?
Create your wheel
In your construction venv:pip install wheel
pip wheel my module
Install your wheel
In the destination venv:pip install my module
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 8 / 10
So how do we use it?
Create your wheel
In your construction venv:pip install wheel
pip wheel my module
Install your wheel
In the destination venv:pip install my module
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 8 / 10
The future
Work In Progress
Add an option in pip to create a wheel auto-magically at the firstinstallation of a module, in order to speed up the next ones
See also
Check out http://pip2014.com/ to get a script (untested) that’s supposedto configure all this (and apparently some more)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 9 / 10
The future
Work In Progress
Add an option in pip to create a wheel auto-magically at the firstinstallation of a module, in order to speed up the next ones
See also
Check out http://pip2014.com/ to get a script (untested) that’s supposedto configure all this (and apparently some more)
Xavier Fernandez (Polyconseil) Harness the speed of the wheel May 13, 2014 9 / 10
Thanks!
Related Documents
