vbench: lightweight performance testing for Python

vbench: lightweight performance testing

Wes McKinney@wesmckinn

PyCon 2012

Sunday, March 11,

Why do we test?

Sunday, March 11,

Freedom from fear

Sunday, March 11,

Testing for performance?

Sunday, March 11,

Who made my code slower???

Sunday, March 11,

Sunday, March 11,

Who made my code slower???

Sunday, March 11,

Me

Sunday, March 11,

speed.pypy.org is a one-off solution

Sunday, March 11,

Stop. Help is near

• GitHub: wesm/vbench

• Integrates with git: runs benchmarks for each revision in your repo

• Persists results locally in SQLite

• Generates graphs using matplotlib

Sunday, March 11,

Catch performance regressions sooner

Sunday, March 11,

Writing vbenchmarkssetup = common_setup + """values = np.concatenate([np.arange(100000), np.random.randn(100000), np.arange(100000)])s = Series(values)"""

stats_rank_average = Benchmark('s.rank()', setup)

setu

p

Sunday, March 11,

Use them in your workflow

In [2]: stats_rank_average.run()Out[2]: {'loops': 10, 'repeat': 3, 'succeeded': True, 'timing': 33.135390281677246, 'units': 'ms'}

Same code as %timeit in IPython

Sunday, March 11,

Get involved

• A useful weekend hack

• git bisect integration

• More version control systems

• Upload results to codespeed instance

• More setup/teardown control

Sunday, March 11,

GitHub: wesm/vbench

@wesmckinn

Sunday, March 11,