Page 1
magePython Testing Fundamentals
Saturday, July 28, 2012
PyOhio
Ohio State University
Columbus, OH
Chris Calloway
University of North Carolina
Department of Marine Sciences
Python Testing Fundamentals
PyCamp™ Programming For The People
Copyright © 2012
Trizpug
Page 2
mage
http://drunkenpython.org/pytestfund.pdf
http://drunkenpython.org/pyohio.zip
http://drunkenpython.org/pyohio.tgz
Python Testing Fundamentals
PyCamp™ Programming For The People
Copyright © 2012
Trizpug
Page 3
•Fundamental
•Python 3.2.3
•Assertion
•Unittest
•Doctest
Programming For The People
About This Tutorial
Copyright © 2012
TrizpugPyCamp™
Page 4
“Untested Code is Broken Code”
- Phillip von Weitershausen
Programming For The People
Why Test?
Copyright © 2012
TrizpugPyCamp™
Page 5
Programming For The People
Why Test?
•Tests help you design good code
•Test help you find bugs
•Tests help you document code
Copyright © 2012
TrizpugPyCamp™
Page 6
Programming For The People
Assert Statements
•Assert statements use the assert keyword
•Assert statements raise AssertionError
•Based on bool test expressions
Copyright © 2012
TrizpugPyCamp™
Page 7
Programming For The People
Assert Statements
>>> assert 1 == 1
keyword
Copyright © 2012
TrizpugPyCamp™
Page 8
Programming For The People
Assert Statements
>>> assert 1 == 1
expression
Copyright © 2012
TrizpugPyCamp™
Page 9
Programming For The People
Assert Statements
>>> assert expression
is almost the same as:
>>> if not expression:
... raise AssertionError()
>>>
Copyright © 2012
TrizpugPyCamp™
Page 10
Programming For The People
Assert Statements
>>> assert 1 == 1
>>> assert 1 == 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError
>>>
Copyright © 2012
TrizpugPyCamp™
Page 11
Programming For The People
Assert Statements
•A second optional expression on the assert statement provides a message for the AssertionError
•This helps you distinguish one assertion from another
Copyright © 2012
TrizpugPyCamp™
Page 12
Programming For The People
Assert Statements
>>> assert 1 == 2, "Reality check"
second expression
Copyright © 2012
TrizpugPyCamp™
Page 13
Programming For The People
Assert Statements
assert expression1, expression2
is almost the same as:
if not expression1:
raise AssertionError(expression2)
Copyright © 2012
TrizpugPyCamp™
Page 14
Programming For The People
Assert Statements
>>> assert 1 == 2, "Reality check"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError: Reality check
>>>
Copyright © 2012
TrizpugPyCamp™
Page 15
Programming For The People
Assert Statements
•Assertions may be sprinkled liberally throughout your code to check that your code is running as expected
•Think of assertions as Python's reality check
•You decide what "reality" is
Copyright © 2012
TrizpugPyCamp™
Page 16
Programming For The People
Assert Statements
profit = bottom_line(today)
assert profit > 0, "Unacceptable!"
projection = growth(tomorrow)
assert projection > profit, "UR fired!"
Copyright © 2012
TrizpugPyCamp™
bigbiz.py:
Page 17
Programming For The People
Assert Statements
•Assertions usually generate unhandled exceptions which halt your program
Copyright © 2012
TrizpugPyCamp™
Page 18
PyCamp™ Programming For The People
Assert Statements
bigbiz.py:
Text
def bottom_line(timestamp):
"""Compute the profit on date."""
return -1e6
Copyright © 2012
Trizpug
Page 19
Programming For The People
Assert Statements
> python bigbiz.py
Traceback (most recent call last):
File "bigbiz.py", line 20, in <module>
assert profit > 0, "Unacceptable!"
AssertionError: Unacceptable!
>
Copyright © 2012
TrizpugPyCamp™
Page 20
Programming For The People
Assert Statements
•By using Python's -i command line switch, you may interactively inspect what went wrong where the assertion was raised
Copyright © 2012
TrizpugPyCamp™
Page 21
Programming For The People
Assert Statements
> python -i bigbiz.py Traceback (most recent call last): File "bigbiz.py", line 20, in <module> assert profit > 0, "Unacceptable!"AssertionError: Unacceptable!>>> profit-1000000.0>>>
Copyright © 2012
TrizpugPyCamp™
Page 22
Programming For The People
Assert Statements
•By using Python's -i command line switch, you may use also Python's debugger to see what went wrong at the point of assertion
Copyright © 2012
TrizpugPyCamp™
Page 23
Programming For The People
Assert Statements
> python -i bigbiz.pyTraceback (most recent call last): File "bigbiz.py", line 20, in <module> assert profit > 0, "Unacceptable!"AssertionError: Unacceptable!>>> import pdb>>> pdb.pm()> /Users/cbc/pyohio/bigbiz.py(20)<module>()-> assert profit > 0, "Unacceptable!"(Pdb) profit-1000000.0(Pdb)
Copyright © 2012
TrizpugPyCamp™
Page 24
Programming For The People
Assert Statements
•Assertions may be turned off by "optimizing" Python with the -O command line switch
Copyright © 2012
TrizpugPyCamp™
Page 25
Programming For The People
Assert Statements
> python -O
>>> assert 1 == 2
>>>
Copyright © 2012
TrizpugPyCamp™
Page 26
Programming For The People
Assert Statements
> python -O bigbiz.py
Profit is -1000000.00 USD.
Projected profit is -2000000.00 USD.
>
Copyright © 2012
TrizpugPyCamp™
Page 27
Programming For The People
Assert Statements
•It's a fine line whether assert statements are testing or debugging
•Assert statements are slightly more sophisticated than using print
•But assert statements form the basis for testing in Python
•In Python, a test is an assertion of an expected result
Copyright © 2012
TrizpugPyCamp™
Page 28
Programming For The People
Unittest Module
•Unittest is "batteries included" in Python
•Unittest helps you separate test code from the code under test
•Unittest helps you write tests before code
•Unittest helps you organize and discover all your tests
•Unittest hooks into many other Python tools
Copyright © 2012
TrizpugPyCamp™
Page 29
Programming For The People
Unittest Module
>>> import unittest
>>> dir(unittest)
['BaseTestSuite', 'FunctionTestCase', 'SkipTest', 'TestCase', 'TestLoader', 'TestProgram', 'TestResult', 'TestSuite', 'TextTestResult', 'TextTestRunner', '_TextTestResult', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__unittest', 'case', 'defaultTestLoader', 'expectedFailure', 'findTestCases', 'getTestCaseNames', 'installHandler', 'loader', 'main', 'makeSuite', 'registerResult', 'removeHandler', 'removeResult', 'result', 'runner', 'signals', 'skip', 'skipIf', 'skipUnless', 'suite', 'util']
>>>
Copyright © 2012
TrizpugPyCamp™
Page 30
Programming For The People
TestCase Class
•unittest.TestCase is a class
•You create TestCase subclasses
•You add methods whose names start with "test" to your TestCase subclass
Copyright © 2012
TrizpugPyCamp™
Page 31
Programming For The People
TestCase Class
•Each test method you supply in your subclass executes a TestCase supplied method whose name starts with "assert"
•A TestCase provides what is known as a "test fixture" in testing parlance
•The unittest module provides many ways to run the test methods of your text fixture
Copyright © 2012
TrizpugPyCamp™
Page 32
Programming For The People
TestCase Class
test_operator.py:
Text
"""Demonstrate the unittest module."""
import operator
import unittest
Copyright © 2012
TrizpugPyCamp™
Page 33
Programming For The People
TestCase Class
test_operator.py:
Textclass TestOperator(unittest.TestCase):
"""Test the operator module."""
Copyright © 2012
TrizpugPyCamp™
Page 34
Programming For The People
TestCase Class
test_operator.py:
Text
def test_add(self):
"""Test the add function."""
self.assertEqual(operator.add(2, 2),
2 + 2)
Copyright © 2012
TrizpugPyCamp™
Page 35
Programming For The People
TestCase Class
test_operator.py:
Text
def test_sub(self):
"""Test the sub function."""
self.assertEqual(operator.sub(4, 2),
4 - 2)
Copyright © 2012
TrizpugPyCamp™
Page 36
Programming For The People
TestCase Class
test_operator.py:
Text
def test_mul(self):
"""Test the mul function."""
self.assertEqual(operator.mul(2, 2),
2 * 2)
Copyright © 2012
TrizpugPyCamp™
Page 37
Programming For The People
TestCase Class
test_operator.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Page 38
Programming For The People
main Class
•unittest.main is also a class
•unittest.main is normally only used within a script containing test fixtures
•When unittest.main is instantiated, all of the tests in the script's namespace are loaded and run
Copyright © 2012
TrizpugPyCamp™
Page 39
Programming For The People
main Class
> python test_operator.py
...
--------------------------------------Ran 3 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 40
Programming For The People
TestLoader Class
•Notice that nowhere does it appear that your script has instantiated your TestCase subclass or execute any of its methods
•unittest.main instead instantiates a special unittest.TestLoader class which has methods to search your module for TestCase classes and instantiate them
Copyright © 2012
TrizpugPyCamp™
Page 41
Programming For The People
TestRunner Class
•Once the TestLoader instance created by unittest.main discovers and instantiates the TestCase in your script, unittest.main instantiates a special unittest.TestRunner class which has methods to run the methods of your TestCase instances
•unittest.main takes care of handling TestLoaders and TestRunners for you!
Copyright © 2012
TrizpugPyCamp™
Page 42
Programming For The People
TestRunner Class
Copyright © 2012
TrizpugPyCamp™
TestCase(superclass)
TestOperator(subclass)
unittest.main(instance)
TestLoader(instance)
TestRunner(instance)
TestOperator(instance)
createTests()
runTests()
discover()
run()
Page 43
Programming For The People
main Class
test_operator.py:
Textif __name__ == "__main__":
unittest.main(verbosity=2)
Copyright © 2012
TrizpugPyCamp™
Page 44
Programming For The People
main Class
> python test_operator.py test_add (__main__.TestOperator)Test the add function. ... oktest_mul (__main__.TestOperator)Test the mul function. ... oktest_sub (__main__.TestOperator)Test the sub function. ... ok
--------------------------------------Ran 3 tests in 0.001s
OK>
Copyright © 2012
TrizpugPyCamp™
Page 45
Programming For The People
main Class
•Notice that your tests did not run in the same order in which they were defined
•unittest.main loads the test methods from your TestCase instance's __dict__ attribute
•Dictionaries are unordered
•unittest.main() runs the test methods in your Python's built-in order for strings
Copyright © 2012
TrizpugPyCamp™
Page 46
Programming For The People
TestCase Class
test_operator.py:
Text
def test_add(self):
"""Test the add function."""
self.assertEqual(operator.add(2, 2),
2 + 3)
Copyright © 2012
TrizpugPyCamp™
Page 47
Programming For The People
TestCase Class
> python test_operator.py F..======================================FAIL: test_add (__main__.TestOperator)Test the add function.--------------------------------------Traceback (most recent call last): File "test_operator.py", line 13, in test_add self.assertEqual(operator.add(2, 2), 2 + 3)AssertionError: 4 != 5
--------------------------------------Ran 3 tests in 0.082s
FAILED (failures=1)>
Copyright © 2012
TrizpugPyCamp™
Page 48
Programming For The People
TestCase Class
test_operator.py:
Text
def test_add(self):
"""Test the add function."""
self.assertEqual(operator.add(2, 2),
2 + "2")
Copyright © 2012
TrizpugPyCamp™
Page 49
Programming For The People
TestCase Class
> python test_operator.py E..================================================ERROR: test_add (__main__.TestOperator)Test the add function.------------------------------------------------Traceback (most recent call last): File "test_operator.py", line 13, in test_add self.assertEqual(operator.add(2, 2), 2 + "2")TypeError: unsupported operand type(s) for +:int' and 'str'
------------------------------------------------Ran 3 tests in 0.001s
FAILED (errors=1)>
Copyright © 2012
TrizpugPyCamp™
Page 50
Programming For The People
TestResult Class
•Running a test results in one of three outcomes:
★ Success (expected result)
★ Failure (unexpected result)
★ Error (error running the test)
•The outcomes of all tests are accumulated in a unittest.TestResult instance
Copyright © 2012
TrizpugPyCamp™
Page 51
Programming For The People
TestResult Class
•Most of the time you will not need to create your own TestResult instances
•Most of the ways you will run tests will instantiate and report a TestResult for you
•But to run a test always requires a TestResult instance somewhere
Copyright © 2012
TrizpugPyCamp™
Page 52
Programming For The People
TestResult Class
> python -m unittest test_operator
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 53
Programming For The People
TestResult Class
> python -m unittest \
test_operator.TestOperator
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 54
Programming For The People
TestResult Class
> python -m unittest \
test_operator.TestOperator.test_add
.
--------------------------------------
Ran 1 test in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 55
Programming For The People
TestResult Class
> python -m unittest test_operator.py
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 56
Programming For The People
TestResult Class
> python -m unittest -v test_operator test_add (test_operator.TestOperator)Test the add function. ... oktest_mul (test_operator.TestOperator)Test the mul function. ... oktest_sub (test_operator.TestOperator)Test the sub function. ... ok
--------------------------------------Ran 3 tests in 0.001s
OK>
Copyright © 2012
TrizpugPyCamp™
Page 57
Programming For The People
TestResult Class
> python -m unittest -hUsage: python -m unittest [options] [tests]
Options: -h, --help Show this message -v, --verbose Verbose output -q, --quiet Minimal output -f, --failfast Stop on first failure -c, --catch Catch control-C and display results -b, --buffer Buffer stdout and stderr during test runs[tests] can be a list of any number of testmodules, classes and test methods.
Copyright © 2012
TrizpugPyCamp™
Page 58
Programming For The People
load_tests Protocol
> python -m unittest
.......
--------------------------------------
Ran 7 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 59
Programming For The People
load_tests Protocol
Alternative Usage: python -m unittest \ discover [options]
Options: -s directory Directory to start discovery ('.' default) -p pattern Pattern to match test files ('test*.py' default) -t directory Top level directory of project (default to start directory)For test discovery all test modules must beimportable from the top level directory.
Copyright © 2012
TrizpugPyCamp™
Page 60
Programming For The People
TestCase Class
•Notice that TestCase.assertEqual does not appear to raise an unhandled AssertionError
•The TestRunner instance handles the AssertionError for failing tests and updates the TestResult instance
•TestCase.assertEqual is but one of many test methods you may use in your tests
Copyright © 2012
TrizpugPyCamp™
Page 61
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Tests If
assertEqual(a, b) a == b
assertNotEqual(a, b) a != b
assertTrue(x) bool(x) is True
assertFalse(x) bool(x) is False
assertIs(a, b) a is b
assertIsNot(a, b) a is not b
assertIsNone(x) x is None
assertIsNotNone(x) x is not None
assertIn(a, b) a in b
assertNotIn(a, b) a not in b
assertIsInstance(a, b) isinstance(a, b)
assertNotIsInstance(a, b) not isinstance(a, b)
Page 62
Programming For The People
TestCase Class
•You not only need to test if your code produces expected results, you also need to test if your code handles unexpected results in an expected manner!
Copyright © 2012
TrizpugPyCamp™
Page 63
Programming For The People
TestCase Class
test_operatorNG.py:
Text
def test_add_str(self):
"""Test bad args for add."""
with self.assertRaises(TypeError):
operator.add(2, "2")
Copyright © 2012
TrizpugPyCamp™
Page 64
Programming For The People
TestCase Class
> python -m unittest test_operatorNG
....
--------------------------------------
Ran 4 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 65
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Tests If
assertRaises(exception) exception raised
assertRaisesRegex(exception, regex) exception raised and message matches regex
assertWarns(warning) warning raised
assertWarnsRegex(warning, regex) warning raised and message matches regex
Page 66
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Tests If
assertAlmostEqual(a, b) round(a-b, 7) == 0
assertNotAlmostEqual(a, b) round(a-b, 7) != 0
assertGreater(a, b) a > b
assertGreaterEqual(a, b) a >= b
assertLess(a, b) a < b
assertLessEqual(a, b) a <= b
assertRegex(s, re) s matches regex
assertNotRegex(s, re) s does not match regex
assertCountEqual(a, b)a and b have the same elements in the same number, regardless of order
Page 67
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Compares
assertMultiLineEqual(a, b) Strings
assertSequenceEqual(a, b) Sequences
assertListEqual(a, b) Lists
assertTupleEqual(a, b) Tuples
assertSetEqual(a, b) Sets and Frozensets
assertDictEqual(a, b) Dictionaries
Page 68
Programming For The People
TestCase Class
>>> [attr for attr
... in dir(unittest.TestCase)
... if attr.startswith('assert')]
Copyright © 2012
TrizpugPyCamp™
•Plus many more!
Page 69
Programming For The People
TestCase Class
>>> help(unittest.TestCase.
... assertDictContainsSubset)
Help on function assertDictContainsSubset in module unittest.case:
assertDictContainsSubset(self, subset,
dictionary, msg=None)
Checks whether dictionary is a superset of subset.
Copyright © 2012
TrizpugPyCamp™
Page 70
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
"""Demonstrate the unittest module."""
import operator
import unittest
Copyright © 2012
TrizpugPyCamp™
Page 71
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textclass TestAdd(unittest.TestCase):
"""Test the add function."""
Copyright © 2012
TrizpugPyCamp™
Page 72
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_add_int(self):
"""Test with ints."""
self.assertEqual(operator.add(2, 2),
2 + 2)
Copyright © 2012
TrizpugPyCamp™
Page 73
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_add_str(self):
"""Test with strs."""
with self.assertRaises(TypeError):
operator.add(2, "2")
Copyright © 2012
TrizpugPyCamp™
Page 74
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textclass TestSub(unittest.TestCase):
"""Test the sub function."""
Copyright © 2012
TrizpugPyCamp™
Page 75
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_sub_int(self):
"""Test with ints."""
self.assertEqual(operator.sub(4, 2),
4 - 2)
Copyright © 2012
TrizpugPyCamp™
Page 76
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_sub_str(self):
"""Test with strs."""
with self.assertRaises(TypeError):
operator.sub(4, "2")
Copyright © 2012
TrizpugPyCamp™
Page 77
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textclass TestMul(unittest.TestCase):
"""Test the mul function."""
Copyright © 2012
TrizpugPyCamp™
Page 78
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_mul_int(self):
"""Test with ints."""
self.assertEqual(operator.mul(2, 2),
2 * 2)
Copyright © 2012
TrizpugPyCamp™
Page 79
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_mul_str(self):
"""Test with strs."""
self.assertEqual(operator.mul(2, "2"),
"22")
Copyright © 2012
TrizpugPyCamp™
Page 80
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
str_suite = unittest.TestSuite()
str_suite.addTest(TestAdd("test_add_str"))
str_suite.addTest(TestSub("test_sub_str"))
str_suite.addTest(TestMul("test_mul_str"))
Copyright © 2012
TrizpugPyCamp™
Page 81
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Page 82
Programming For The People
TestSuite Class
•Objects you have bound in the test_operatorNG2 namespace:
•TestAdd class
•TestSub class
•TestMul class
•str_suite instance
Copyright © 2012
TrizpugPyCamp™
Page 83
Programming For The People
TestSuite Class
> python test_operatorNG2.py
......
--------------------------------------
Ran 6 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 84
Programming For The People
TestSuite Class
> python -m unittest \
test_operatorNG2.str_suite
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 85
Programming For The People
TestSuite Class
> python -m unittest -v \ test_operatorNG2.str_suitetest_add_str (test_operatorNG2.TestAdd)Test with strs. ... oktest_sub_str (test_operatorNG2.TestSub)Test with strs. ... oktest_mul_str (test_operatorNG2.TestMul)Test with strs. ... ok
--------------------------------------Ran 3 tests in 0.001sOK>
Copyright © 2012
TrizpugPyCamp™
Page 86
Programming For The People
Organizing Tests
Copyright © 2012
TrizpugPyCamp™
pyohio-+ bin/python pycamp-+ __init__.py setup.py fibonacci.py triangle.py tests-+ __init__.py test_fibonacci.py test_triangular.py
Page 87
Programming For The People
Organizing Tests
> python pycamp/fibonacci.py 100
1 2 3 5 8 13 21 34 55 89
> python
>>> from pycamp.fibonacci import fib
>>> fib(100)
[1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>>
Copyright © 2012
TrizpugPyCamp™
Page 88
Programming For The People
Organizing Tests
> python pycamp/triangular.py 100
1 3 6 10 15 21 28 36 45 55 66 78 91
> python
>>> from pycamp.triangular import tri
>>> tri(100)
[1, 3, 6, 10, 15, 21, 28, 36, 45, 55, 66, 78, 91]
>>>
Copyright © 2012
TrizpugPyCamp™
Page 89
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
"""Tests for the fibonacci module."""
from pycamp import fibonacci
import unittest
Copyright © 2012
TrizpugPyCamp™
Page 90
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Textclass TestFibonacci(unittest.TestCase):
"""Test fibonacci's functions."""
Copyright © 2012
TrizpugPyCamp™
Page 91
Programming For The People
TestCase Class
•TestCase also supplies methods you override
•TestCase.setUp() is called before every test method you supply in your subclass
•TestCase.tearDown() is called after every test method you supply in your subclass
Copyright © 2012
TrizpugPyCamp™
Page 92
Programming For The People
TestCase Class
•Your TestCase subclasses, along with all the test methods you supply, and all the TestCase supplied methods you override, possibly all bundled up into a TestSuite, are collectively known as a "test fixture" in testing parlance
Copyright © 2012
TrizpugPyCamp™
Page 93
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
def setUp(self):
"""Test fixture build."""
self.lessThan100 = [1, 2, 3, 5, 8,
13, 21, 34, 55,
89, ]
Copyright © 2012
TrizpugPyCamp™
Page 94
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
def test_fib_100(self):
"""Test fib for numbers < 100."""
self.assertEqual(fibonacci.fib(100),
self.lessThan100)
Copyright © 2012
TrizpugPyCamp™
Page 95
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
def test_fib_10(self):
"""Test fib for numbers < 10."""
self.assertEqual(fibonacci.fib(10),
self.lessThan100[:5])
Copyright © 2012
TrizpugPyCamp™
Page 96
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Page 97
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
"""Tests for the triangular module."""
from pycamp import triangular
import unittest
Copyright © 2012
TrizpugPyCamp™
Page 98
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Textclass TestTriangular(unittest.TestCase):
"""Test triangular's functions."""
Copyright © 2012
TrizpugPyCamp™
Page 99
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
def setUp(self):
"""Test fixture build."""
self.lessThan100 = [1, 3, 6, 10, 15,
21, 28, 36, 45,
55, 66, 78, 91, ]
Copyright © 2012
TrizpugPyCamp™
Page 100
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
def test_tri_100(self):
"""Test tri for numbers < 100."""
self.assertEqual(triangular.tri(100),
self.lessThan100)
Copyright © 2012
TrizpugPyCamp™
Page 101
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
def test_tri_10(self):
"""Test tri for numbers < 10."""
self.assertEqual(triangular.tri(10),
self.lessThan100[:3])
Copyright © 2012
TrizpugPyCamp™
Page 102
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Page 103
Programming For The People
Organizing Tests
> python -m unittest discover \
-s pycamp -t .
....
--------------------------------------Ran 4 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 104
Programming For The People
Organizing Tests
pycamp/setup.py:
Text
"""Setup for pycamp package."""
from setuptools import setup, \
find_packages
Copyright © 2012
TrizpugPyCamp™
Page 105
Programming For The People
Organizing Tests
pycamp/setup.py:
Text
setup(
name="pycamp",
version="1.0",
packages=find_packages(),
author="Chris Calloway",
author_email="[email protected] ",
Copyright © 2012
TrizpugPyCamp™
Page 106
Programming For The People
Organizing Tests
pycamp/setup.py:
Text
description="Testing Fundamentals",
license="PSF",
keywords="testing pycamp",
url="http://pycamp.org",
test_suite="pycamp.tests",
)
Copyright © 2012
TrizpugPyCamp™
Page 107
Programming For The People
Organizing Tests
> python pycamp/setup.py test
running test
..lots of output omitted for brevity..
--------------------------------------
Ran 4 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 108
Programming For The People
Organizing Tests
> python pycamp/setup.py test
Copyright © 2012
TrizpugPyCamp™
Page 109
Programming For The People
Organizing Tests
> python pycamp/setup.py install
Copyright © 2012
TrizpugPyCamp™
Page 110
Programming For The People
Organizing Tests
> python pycamp/setup.py register
Copyright © 2012
TrizpugPyCamp™
Page 111
Programming For The People
Unittest Module
•unittest provides nice separation of tests from code
•One person can write tests, while another writes code to make tests pass
•unittest provides fine grain control over what tests to run when
•unittest conforms to industry standard testing
Copyright © 2012
TrizpugPyCamp™
Page 112
Programming For The People
Unittest Module
•However, it can be difficult to keep tests and code in sync
•Also, writing code to test code can also be more difficult than the code under test
•What about testing and debugging the test code?
•Reading unittest tests is a poor way to figure out how code works
Copyright © 2012
TrizpugPyCamp™
Page 113
Programming For The People
Doctest Module
•Informally, even without written tests, you probably already test your code by simply using it as it was meant to be used
•You've probably imported your code at a Python prompt and inspect how it works manually
•You just don't yet have a way of repeating that informal testing in an automated manner
•What you need is the doctest module
Copyright © 2012
TrizpugPyCamp™
Page 114
Programming For The People
Doctest Module
Copyright © 2012
TrizpugPyCamp™
pyohio-+ bin/python pycampNG-+ __init__.py setup.py fibonacci.py triangle.py tests-+ __init__.py test_pycamp.py
Page 115
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
"""A module of functions about non-zero Fibonacci numbers.
>>> import fibonacci
>>>
"""
Copyright © 2012
TrizpugPyCamp™
Page 116
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
def fib(n):
"""Return the sequence of non-zero
Fibonacci numbers less than n.
fib(n) -> [0 < fibonacci numbers < n]
where n in an int.
Copyright © 2012
TrizpugPyCamp™
Page 117
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
>>> lessThan100 = [1, 2, 3, 5, 8, 13,
... 21, 34, 55, 89]
>>> fib(100) == lessThan100
True
>>> fib(10) == lessThan100[:5]
True
"""
Copyright © 2012
TrizpugPyCamp™
Page 118
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
"""A module of functions about non-zero triangular numbers.
>>> import triangular
>>>
"""
Copyright © 2012
TrizpugPyCamp™
Page 119
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
def tri(n):
"""Return the sequence of non-zero
triangular numbers less than n.
tri(n) -> [0 < triangular numbers < n]
where n in an int.
Copyright © 2012
TrizpugPyCamp™
Page 120
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
>>> lessThan100 = [1, 3, 6, 10, 15,
21, 28, 36, 45,
55, 66, 78, 91]
>>> tri(100) == lessThan100
True
>>> tri(10) == lessThan100[:3]
True
"""
Copyright © 2012
TrizpugPyCamp™
Page 121
Programming For The People
Doctest Module
> python -m doctest pycampNG/fibonacci.py
>
Copyright © 2012
TrizpugPyCamp™
Page 122
Programming For The People
Doctest Module
> python -m doctest -v \
pycampNG/fibonacci.py
Trying:
import fibonacci
Expecting nothing
ok
Copyright © 2012
TrizpugPyCamp™
Page 123
Programming For The People
Doctest Module
Trying:
lessThan100 = [1, 2, 3, 5, 8, 13,
21, 34, 55, 89]
Expecting nothing
ok
Copyright © 2012
TrizpugPyCamp™
Page 124
Programming For The People
Doctest Module
Trying:
fib(100) == lessThan100
Expecting:
True
ok
Copyright © 2012
TrizpugPyCamp™
Page 125
Programming For The People
Doctest Module
Trying:
fib(10) == lessThan100[:5]
Expecting:
True
ok
Copyright © 2012
TrizpugPyCamp™
Page 126
Programming For The People
Doctest Module
2 items passed all tests:
1 tests in fibonacci
3 tests in fibonacci.fib
4 tests in 2 items.
4 passed and 0 failed.
Test passed.
>
Copyright © 2012
TrizpugPyCamp™
Page 127
Programming For The People
Doctest Module
> python -m doctest pycampNG/triangular.py
>
Copyright © 2012
TrizpugPyCamp™
Page 128
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
if __name__ == '__main__':
if sys.argv[1].lower() == 'test':
import doctest
doctest.testmod()
else:
print(" ".join([str(x) for x in
fib(int(sys.argv[1]))]))
Copyright © 2012
TrizpugPyCamp™
Page 129
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
if __name__ == '__main__':
if sys.argv[1].lower() == 'test':
import doctest
doctest.testmod()
else:
print(" ".join([str(x) for x in
tri(int(sys.argv[1]))]))
Copyright © 2012
TrizpugPyCamp™
Page 130
Programming For The People
Doctest Module
> python pycampNG/fibonacci.py test
> python pycampNG/fibonacci.py 100
1 2 3 5 8 13 21 34 55 89
>
Copyright © 2012
TrizpugPyCamp™
Page 131
Programming For The People
Doctest Module
> python pycampNG/triangular.py test
> python pycampNG/triangular.py 100
1 3 6 10 15 21 28 36 45 55 66 78 91
>
Copyright © 2012
TrizpugPyCamp™
Page 132
Programming For The People
Doctest Module
•But what about setuptools.setup?
•The test_suite argument of setup triggers unittest discovery, not doctest discovery
•What you need is a way to turn doctests into unittests
Copyright © 2012
TrizpugPyCamp™
Page 133
Programming For The People
Doctest Module
•The doctest.DocTestSuite() function searches a module for doctests and converts them into a unittest.TestSuite instance
•Now all you need is a way to communicate your TestSuite instance(s) to unittest discovery
Copyright © 2012
TrizpugPyCamp™
Page 134
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
Copyright © 2012
TrizpugPyCamp™
Page 135
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
unittest.TestLoader
Copyright © 2012
TrizpugPyCamp™
Page 136
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
unittest.TestSuite
Copyright © 2012
TrizpugPyCamp™
Page 137
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
"test*.py"
Copyright © 2012
TrizpugPyCamp™
Page 138
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
Copyright © 2012
TrizpugPyCamp™
Page 139
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Textimport doctest
from pycampNG import fibonacci
from pycampNG import triangular
Copyright © 2012
TrizpugPyCamp™
Page 140
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Page 141
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Page 142
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Page 143
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Page 144
Programming For The People
load_tests Protocol
pycamp/setup.py:
Text
setup(
name="pycampNG",
...
test_suite="pycampNG.tests",
)
Copyright © 2012
TrizpugPyCamp™
Page 145
Programming For The People
load_tests Protocol
> python pycampNG/setup.py test
running test
..lots of output omitted for brevity..
--------------------------------------
Ran 4 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Page 146
Programming For The People
Doctest Module
Together, the doctest.DocTestSuite() function and the load_tests protocol from the unittest module enable you to use all the tools available for unittests with doctests
Copyright © 2012
TrizpugPyCamp™
Page 147
Programming For The People
Python Testing Tools Taxonomy
Copyright © 2012
TrizpugPyCamp™
Page 148
Programming For The People
Test Driven Development
•Doctests enable you to do Test Driven Development (TDD)
•TDD is where you write tests for your code before you write code
•Then you write code to make your tests pass
Copyright © 2012
TrizpugPyCamp™
Page 149
Programming For The People
Test Driven Development
•When all your tests pass, then you have finished coding
•You can develop your code incrementally, writing one test at a time, then getting that one test to pass
•That means you can stop coding at any time
Copyright © 2012
TrizpugPyCamp™
Page 150
Programming For The People
Test Driven Development
•If your code needs more features, then what you really need is more tests, and code which makes those tests pass
•Writing tests lets you see what the API for your code is up front, instead of having it designed haphazardly
•Writing tests can provide the documentation for your code
Copyright © 2012
TrizpugPyCamp™
Page 151
Programming For The People
Test Driven Development
Copyright © 2012
TrizpugPyCamp™
Page 152
Programming For The People
Tomorrow 12:15pm in Barbie Tootle
Copyright © 2012
TrizpugPyCamp™
Page 153
Programming For The People
Thank You
Copyright © 2012
TrizpugPyCamp™
[email protected]