How to move Python doctest examples into another unit test framework?

Question

Years ago AFAIK the mathematitician and software developer Tim Peters discovered that very often the documentation of APIs tends to get out of date over time during the software live cycle, because the comments documenting the API are often overlooked when features are added or the call signatures of functions or methods change.

To solve this problem he invented the doctest module which introduces somewhat "executable documentation" or "literate testing" (see also: the doctest documentation).

As has already been pointed out elsewhere, putting a huge amount of unit tests into the doc strings of all Python modules, functions, classes and methods would clutter up the API documentation too much.

So I'm looking for an elegant and clever way to move tests cases which used to be doctest examples to some other place where they could still be found by some unit test executing driver.

Answer 1

How to move Python doctest examples into another unit test framework?

Doctests are of the form:

def testable_func():
    """
    >>> testable_func()
    'foo'
    """

And unittests, using the builtin unittest module, are of the form:

def test_testable_func(self):
    self.assertEquals(testable_func(), 'foo')

There's a section in the documentation on reusing old test code that says:

In some cases, the existing tests may have been written using the doctest module. If so, doctest provides a DocTestSuite class that can automatically build unittest.TestSuite instances from the existing doctest-based tests.

But this only runs the doctests as unittests. It does not create a scaffold of pre-written unittest code. If you want to leave your doctests in place, you could use this to run them in conjunction with other unittests.

Keeping the doctests

You can use the preexisting doctests, if you would like to keep them as they are, with the unittest module, (see the documentation), you just need to plop this function where you run your unittests. Of course you'll need to import the module the doctests are in:

import my_module_with_doctests

def load_tests(loader, tests, ignore):
    tests.addTests(doctest.DocTestSuite(my_module_with_doctests))
    return tests

You can omit the my_module_with_doctests argument if the objects or functions with doctests are imported into the test module (this should be cut-and-paste working code, just paste it in the test module):

def load_tests(loader, tests, ignore):
    tests.addTests(doctest.DocTestSuite())
    return tests

Moving the doctests to unittests

But I think, because you're asking about "moving" them, you want to migrate your doctests to be used as unittests.

I would strongly recommend that you do the conversion manually to ensure that your assumptions about the doctests are correct, taking notes about tests you want to add in the future if you think of any, so that you have a one-to-one conversion and can iron out any bugs with minimal changes. You may choose to leave the doctests as helpful documentation on use.

After you are certain you have a successful one-to-one conversion of the tests, then go back to your notes/comments and add further tests.

If you must scaffold:

If you must create a scaffolding (perhaps the time allocated is too small for manual conversion) you can iterate over the objects and/or functions, look at their __doc__ attributes, give them to

example_list = doctest.DocTestParser().get_examples(a__doc__)

And for each example in the list, insert into the assertEquals call each objects' source and want attribute. Such a template might look like this:

'self.assertEquals({0}, {1})'.format(x.source, x.want)

Conclusion:

Going forward, be aware of doctests as a way of providing excellent documentation, but they are no substitute for the scalability of unittests.