The Python unit testing framework, dubbed 'PyUnit' by convention, is a Python language version of JUnit, by smart cookies Kent Beck and Erich Gamma. JUnit is, in turn, a Java version of Kent's Smalltalk testing framework. Each is the de facto standard unit testing framework for its respective language.
This document explains the Python-specific aspects of the design and usage of PyUnit; for background information on the basic design of the framework the reader is referred to Kent's original paper, "Simple Smalltalk Testing: With Patterns".
PyUnit forms a part of the Python Standard Library as of Python version 2.1.
The following information assumes knowledge of Python, a language so easy that even I managed to learn it, and so addictive that I can't stop.
PyUnit is designed to work with any standard Python, version 1.5.2 and higher.
PyUnit has been tested by the author on Linux (Redhat 6.0 and 6.1, Debian Potato) with Python 1.5.2, 2.0 and 2.1. It is also known to work on other Python platforms, including Windows and Mac. If any platform or Python version issues cause you trouble, please let me know.
For details of using PyUnit with JPython and Jython, please refer to the section 'Using PyUnit with JPython and Jython'.
The classes needed to write tests are to be found in the 'unittest' module. This module is part of the standard Python library for Python 2.1 and later. If you are using an older Python version, you should obtain the module from the separate PyUnit distribution.
To be able to use the module from your own code, simply ensure
that the directory containing the file 'unittest.py' is in your Python
search path. You can do this by setting the '$PYTHONPATH' environment
variable, or by placing the file in a directory in your current Python
search path, such as /usr/lib/python1.5/site-packages
on Redhat Linux machines.
Note that you will have to do this before you can run the examples that are provided with PyUnit unless you copy 'unittest.py' into the examples directory.
The basic building blocks of unit testing are 'test cases' -- single
scenarios that must be set up and checked for correctness. In PyUnit,
test cases are represented by the TestCase class in the
unittest module. To make your own test cases you must
write subclasses of TestCase.
An instance of a TestCase class is an object that can
completely run a single test method, together with optional set-up
and tidy-up code.
The testing code of a TestCase instance should be
entirely self contained, such that it can be run either in isolation
or in arbitrary combination with any number of other test cases.
The simplest test case subclass will simply override the
runTest method in order to perform specific testing code:
import unittest
class DefaultWidgetSizeTestCase(unittest.TestCase):
def runTest(self):
widget = Widget("The widget")
assert widget.size() == (50,50), 'incorrect default size'
Note that in order to test something, we just use the built-in 'assert'
statement of Python. If the assertion fails when the test case runs, an
AssertionError will be raised, and the testing framework
will identify the test case as a 'failure'. Other exceptions that do not
arise from explicit 'assert' checks are identified by the testing
framework as 'errors'. (See also the section
'More about test conditions'.)
The way to run a test case will be described later. For now, note that to construct an instance of such a test case, we call its constructor without arguments:
testCase = DefaultWidgetSizeTestCase()
Now, such test cases can be numerous, and their set-up can be repetitive. In the above case, constructing a 'Widget' in each of 100 Widget test case subclasses would mean unsightly duplication.
Luckily, we
can factor out such set-up code by implementing a hook method called
setUp, which the testing framework will automatically
call for us when we run the test:
import unittest
class SimpleWidgetTestCase(unittest.TestCase):
def setUp(self):
self.widget = Widget("The widget")
class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
def runTest(self):
assert self.widget.size() == (50,50), 'incorrect default size'
class WidgetResizeTestCase(SimpleWidgetTestCase):
def runTest(self):
self.widget.resize(100,150)
assert self.widget.size() == (100,150), \
'wrong size after resize'
If the setUp method raises an exception while the test
is running, the framework will consider the test to have suffered an
error, and the runTest method will not be executed.
Similarly, we can provide a tearDown method that tidies
up after the runTest method has been run:
import unittest
class SimpleWidgetTestCase(unittest.TestCase):
def setUp(self):
self.widget = Widget("The widget")
def tearDown(self):
self.widget.dispose()
self.widget = None
If setUp succeeded, the tearDown method
will be run regardless of whether or not runTest
succeeded.
Such a working environment for the testing code is termed a fixture.
Often, many small test cases will use the same fixture. In this case,
we would end up subclassing SimpleWidgetTestCase into
many small one-method classes such as
DefaultWidgetSizeTestCase. This is time-consuming and
discouraging, so in the same vein as JUnit, PyUnit provides a simpler
mechanism:
import unittest
class WidgetTestCase(unittest.TestCase):
def setUp(self):
self.widget = Widget("The widget")
def tearDown(self):
self.widget.dispose()
self.widget = None
def testDefaultSize(self):
assert self.widget.size() == (50,50), 'incorrect default size'
def testResize(self):
self.widget.resize(100,150)
assert self.widget.size() == (100,150), \
'wrong size after resize'
Here we have not provided a runTest method, but have
instead provided two different test methods. Class instances will now
each run one of the test methods, with
self.widget created and destroyed separately for each
instance. When creating an instance we must specify the test method it
is to run. We do this by passing the method name in the constructor:
defaultSizeTestCase = WidgetTestCase("testDefaultSize")
resizeTestCase = WidgetTestCase("testResize")
Test case instances are grouped together according to the features they
test. PyUnit provides a mechanism for this: the 'test suite', represented
by the class TestSuite in the unittest
module:
widgetTestSuite = unittest.TestSuite()
widgetTestSuite.addTest(WidgetTestCase("testDefaultSize"))
widgetTestSuite.addTest(WidgetTestCase("testResize"))
For the ease of running tests, as we will see later, it is a good idea to provide in each test module a 'callable' object that returns a pre-built test suite:
def suite():
suite = unittest.TestSuite()
suite.addTest(WidgetTestCase("testDefaultSize"))
suite.addTest(WidgetTestCase("testResize"))
return suite
or even:
class WidgetTestSuite(unittest.TestSuite):
def __init__(self):
unittest.TestSuite.__init__(self,map(WidgetTestCase,
("testDefaultSize",
"testResize")))
(the latter is admittedly not for the faint-hearted)
Since it is a common pattern to create a TestCase subclass
with many similarly named test functions, there is a convenience
function called makeSuite provided in the
unittest module that constructs a test suite that
comprises all of the test cases in a test case class:-
suite = unittest.makeSuite(WidgetTestCase,'test')
Note that when using the makeSuite function, the order in
which the various test cases will be run by the test suite is the order
determined by sorting the test function names using the cmp
built-in function.
Often it is desirable to group suites of test cases together, so as to
run tests for the whole system at once. This is easy, since
TestSuites can be added to a TestSuite just
as TestCases can be added to a TestSuite:-
suite1 = module1.TheTestSuite()
suite2 = module2.TheTestSuite()
alltests = unittest.TestSuite((suite1, suite2))
An example of nesting test suites can be found in the file
'alltests.py', in the 'examples' subdirectory
of the distribution package.
You can place the definitions of test cases and test suites in the same modules as the code they are to test (e.g. 'widget.py'), but there are several advantages to placing the test code in a separate module, such as 'widgettests.py':
Of course, the whole point of writing these tests is so that we can
run them and find out if our software is working. The test framework
uses 'TestRunner' classes to provide an environment in which your tests
can execute. The most common TestRunner is TextTestRunner,
which can run tests and report the results in textual form:
runner = unittest.TextTestRunner()
runner.run(widgetTestSuite)
By default, TextTestRunner prints its output to
sys.stderr, but this can be changed by passing a different
file-like object to its constructor.
Using TextTestRunner like this is an ideal way to run your
tests interactively from within a Python interpreter session.
The unittest module contains a function called main,
which can be used to easily turn a test module into a script that will
run the tests it contains. The main function uses the
unittest.TestLoader class to automatically find and load
test cases within the current module.
Therefore, if you name your test methods using the test*
convention described earlier, you can place the following code at
the bottom of your test module:
if __name__ == "__main__":
unittest.main()
Then, when you execute your test module from the command line, all of the tests contained therein will be run. Run the module with the '-h' option in order to see the options available.
To run arbitrary tests from the command-line, you can run the
unittest module as a script, giving it the name of an
test case or test suite:
% python unittest.py widgettests.WidgetTestSuite
or
% python unittest.py widgettests.makeWidgetTestSuite
You may also specify particular tests on the command-line. To run the
TestCase subclass 'ListTestCase' in the
module 'listtests' (see the 'examples' subdirectory of the distribution
package) you can execute the command:
% python unittest.py listtests.ListTestCase.testAppend
where 'testAppend' is the name of the test method that is to be run
by the test case instance. To create and run ListTestCase
instances for all the 'test*' methods in that class, you can run:
% python unittest.py listtests.ListTestCase
There is a graphical front end that you can use in order to run your
tests. It is written using Tkinter, the windowing toolkit
shipped with Python on most platforms. It looks similar to the JUnit
GUI.
To use the GUI test runner, simply run:
% python unittestgui.py
or
% python unittestgui.py widgettests.WidgetTestSuite
Note that here, again, the name entered for the test to be run should
be the fully-qualified name of an object which returns a
TestCase or TestSuite instance. It should not
be the name of a pre-created test, since every test must be recreated
each time it is run.
The use of the GUI test runner rather than the text test runner imposes a time overhead due to all those window updates; on my system, it takes an extra seven seconds per thousand tests. Your mileage may vary.
Usually, when a test is run its name is displayed by the
TestRunner. This name is derived from the name of the
test case class, and the name of the test method that the instance has
been initialised to run.
However, if you supply a doc-string for a test method, the first line of that doc-string will be displayed when the test is run. This provides an easy mechanism for documenting your tests:
class WidgetTestCase(unittest.TestCase):
def testDefaultSize(self):
"""Check that widgets are created with correct default size"""
assert self.widget.size() == (50,50), 'incorrect default size'
I have suggested the use of Python's built-in assertion mechanism for
checking conditions in test cases rather than a 'home-brewed'
equivalent; assert is simple, concise and familiar.
Note, however, that if tests are run with Python's
optimisation option turned on (generating '.pyo' bytecode files), the
assert statements will be skipped, rendering the
test cases quite useless.
For those who tend to work with Python's optimisation option enabled,
I have included a method assert_ in the
TestCase class. It is functionally equivalent to the
assert built-in and will not be optimised away, but it is
less convenient and results in less helpful error messages:
def runTest(self):
self.assert_(self.widget.size() == (100,100), "size is wrong")
For good measure I have also provided TestCase with
failIf and failUnless methods:
def runTest(self):
self.failIf(self.widget.size() <> (100,100))
A test case method can also call fail in order to fail
immediately:
def runTest(self):
...
if not hasattr(something, "blah"):
self.fail("blah missing")
# or just 'self.fail()'
The most common type of assertion is an assertion of equality between two values or objects. If the assertion fails, the developer usually wants to see what the incorrect value actually was.
TestCase has a pair of methods called
assertEqual and assertNotEqual for this purpose
(with aliases failUnlessEqual and failIfEqual
for those who prefer):
def testSomething(self):
self.widget.resize(100,100)
self.assertEqual(self.widget.size, (100,100))
Often a test will wish to check that an exception is raised in a certain set of circumstances. If the expected exception is not thrown, the test should fail. This is easy to do:
def runTest(self):
try:
self.widget.resize(-1,-1)
except ValueError:
pass
else:
fail("expected a ValueError")
Usually, the source of the expected exception is a callable object;
for that reason, TestCase has an
assertRaises method. The first two arguments of the method
are the expected exception as it would appear in an 'except' clause,
and the callable object. The remaining arguments are those that should
be passed to the callable object:
def runTest(self):
self.assertRaises(ValueError, self.widget.resize, -1, -1)
Some users will find that they have existing test code that they would
like to run from PyUnit, without converting every old test function to
a TestCase subclass.
For this reason, PyUnit provides a FunctionTestCase class.
This subclass of TestCase can be used to wrap an existing
test function. Set-up and tear-down functions can also optionally be
wrapped.
Given the following test function:
def testSomething():
something = makeSomething()
assert something.name is not None
...
one can create an equivalent test case instance as follows:
testcase = unittest.FunctionTestCase(testSomething)
If there are additional set-up and tear-down methods that should be called as part of the test case's operation, they can also be provided:
testcase = unittest.FunctionTestCase(testSomething,
setUp=makeSomethingDB,
tearDown=deleteSomethingDB)
Although PyUnit was written primarily for 'C' Python, it is possible to write PyUnit tests using Jython for your Java or Jython software. This can be preferable to trying to write JUnit tests using Jython. PyUnit also works correctly with Jython's predecessors, JPython 1.0 and 1.1.
Of course, Java does not have a TK GUI interface, so PyUnit's Tkinter-based GUI will not work with Jython. The text-only interface works just fine, however.
To do so, simply copy the standard C Python library module files
'traceback.py', 'linecache.py',
'stat.py' and 'getopt.py' to a location from
which they can be imported by
JPython. You can get these files from any distribution of
C Python. (These guidelines are based on the standard library of
C Python 1.5.x, and may not be correct for other Python versions.)
Now you can write your PyUnit tests exactly as you would with C Python.
See the caveats in the section "More about test conditions" above.
When exceptions are raised during the running of a test suite, the resulting traceback objects are saved so that failure details can be formatted and printed at the end of the test run. Apart from simplicity, the benefit of this is that a future version of the GUI TestRunner will be able to allow post-mortem inspection of local and global variable values, which are stored with the traceback.
A possible side-effect is that when running test suites with very high failure rates, the memory usage of all these saved traceback objects could become a problem. Of course, if so many tests are failing, this memory overhead is the least of your problems.
You may freely use, alter and redistribute this software under the same liberal terms that apply to Python itself. All I ask is that my name, e-mail address and the project URL be retained in the source code and accompanying documentation, with a credit for me as the original author.
My motive for writing this software was to make a small contribution to the improvement of software quality in the world; I didn't bargain on getting any cash. (That's not to say that sponsorship would be unwelcome.)
One key plan for the future is to integrate the TK GUI with the IDLE IDE. Volunteers are welcome!
Other than that, I have no great plans to extend the functionality of the module. I have kept PyUnit as simple as possible (but no simpler, hopefully!) because I believe that helper modules for such common testing tasks as log file comparison are better written by test writers than by myself.
News, updates and more are available at the project website.
Comments, suggestions and bug reports are welcome; simply e-mail me or, better still, join the very low-volume mailing list and post your comments there. There are a surprisingly large number of people already using PyUnit, and they all have wisdom to share.
Many thanks to Guido and his disciples for the Python language. In tribute, I have written the following haiku (or 'pyku', if you will):
Guido van Rossum
'Gawky Dutchman' gave birth to
Beautiful Python
I gratefully acknowledge the work of Kent Beck and Erich Gamma for their work on JUnit, which made the design of PyUnit a no-brainer.
Thanks also to Tim Voght; I discovered after I had implemented PyUnit that he had also implemented a 'pyunit' module as part of his 'PyWiki' WikiWikiWeb clone. He graciously gave me the go-ahead to submit my version to the community at large.
Many thanks to those who have written to me with suggestions and questions. I've tried to add appropriate credits in the CHANGES file in the download package.
Particular thanks to Jérôme Marant, who packaged PyUnit for Debian.
Steve Purcell is just a programmer at heart, working independently writing, applying and teaching Open Source software.
He recently acted as Technical Director for a Web/WAP start-up, but spends most of his time architecting and coding large Java systems whilst counterproductively urging his Java-skilled colleagues to take up Python instead.