DjangoCon 2013 - How to Write Fast and Efficient Unit Tests in Django

How to Write Fast and Efficient Unit Tests in DjangoCasey KinseyDjangoCon 2013

Monday, September 2, 13

A REAL NEED FOR TEST SPEED


A REAL NEED FOR TEST SPEED

•Made an initial production release of a real product for a national media company

• Test coverage was not great

• Started seeing regressions in subsequent releases

•Decided to aggressively pursue greater test coverage

• Results were successful, but one thing clear : As test coverage increased we had an acute need for faster test suites.


WHY SHOULD I BE CONCERNED WITH UNIT TEST SPEED?

• If you’re serious about testing, you’re serious about at least two things:

•Having lots of tests

• Running those tests frequently


•A slow test suite gets in the way of your testing goals because:

•Developers will avoid running tests

•Preparing code for integration becomes painful

•Deployment speed is directly affected


THE FIRST STEP TO FASTER UNIT TESTS:WRITE UNIT TESTS


UNIT TESTS VS INTEGRATION TESTS

•Many Django project test suites are comprised mostly of integration tests

•What is a Unit Test?

•Unit Tests cover a small “unit” of code

• Ideally, these units include as few branches as possible

•What is an Integration Test?

• Integration Tests test the contracts between your “units”



•Django test client is an integration test dead giveaway

• The test client covers way more than you are interested in testing

•URL Routing, Request Middleware, ORM, Template Rendering, Response Middleware, etc



• A good unit tests covers code that is limited in functionality/scope

• Ideally, a single method with limited external calls



• Establish a good ratio of unit tests to integration tests. An example may be:

• for each method that contains business logic, there should exist a unit test

• for each page/view/user path of your project, there should exist an integration test

• YMMV


SET UP CAUTIOUSLY


SET UP CAUTIOUSLY

• Be judicious about how you use setUp/tearDown

• Think like middleware--do I need this for every test in this case?

•One inefficient computation can cripple a large test case



SET UP CAUTIOUSLY

• Take advantage of setUpClass / tearDownClass

• Runs once per test case

• Effective for read-only data that is not altered by tests

• Your data will persist between tests!


THE DATABASE IS HOT LAVA


XKCD.COM/735/Monday, September 2, 13

http://xkcd.com/735/

http://xkcd.com/735/


• If you touch it you’ll die

•Not really, but it’s one of slowest things your application will do in a unit test

•Work with read-only, non persisted data

• use in-memory model instances



• Avoid fixtures. Fixtures add lots of database machinery to tests

• Loaded/purged between each test in a case

• Fixtures don’t adapt with your data model

• Schema changes will often result in test failures

•Not much need for django.test.TestCase

•When you do write: in-memory database (SQLite)



FAKE IT ‘TIL YOU MAKE IT WITH MOCK



•Mock is a library for creating programmable stub objects

•Mock objects can be configured to emulate other objects/structures

• Configure specific behavior for just for testing

•Gets rid of unnecessary overhead



•Use mock to emulate model instances

• Set the attributes you need for testing directly

•Use the spec argument to give guidelines

•No Model/ORM overhead



•Use mock.patch to focus your tests

• Patch in configurable mock objects to sys.modules

• Alter the behavior of code imported elsewhere

• Eliminate branches you are not interested in testing





•Use mock in more complex situations

•mock.patch.multiple decorator lets you patch multiple module references simultaneously

• Track the way objects are used

•Mock.assert_has_calls, Mock.assert_called_with

•many, many more: http://www.voidspace.org.uk/python/mock/


IT’S OKAY TO ENGINEER WHEN TESTING


IT’S OKAY TO ENGINEER WHEN TESTING

•Don’t be afraid to invest engineering effort into your test suite

• Your tests are Python code--take advantage of it!

•Write tools to help you test

• Leverage 3rd party tools (mock, django-nose)

•Decorators, custom test runners

• If you can’t test the code efficiently, refactor the code!


HOW SLOW TESTING YIELDED EFFECTIVE TESTING


HOW SLOW TESTING YIELDED EFFECTIVE TESTING

• Started out working towards speed

• In order to write fast tests, we had to rethink how we tested

•Developed an efficient test philosophy

• Resulted in much more effective tests, and ultimately better code


THANK YOU!

Casey KinseyConsultant, Celerity

Slides available online:

Internetswww.caseykinsey.com

www.celerity.com

Twitter@cordiskinsey

@CelerityITLLC


http://www.caseykinsey.com

http://www.caseykinsey.com

http://www.celerity.com

http://www.celerity.com

DjangoCon 2013 - How to Write Fast and Efficient Unit Tests in Django

Technology

mock mock

mock use mock

test failures

unit test work

integration test ymmv

test case effective

integration tests

large test case