Auto generate doctest output with Sphinx extension

Question

I think I am missing something about the sphinx extension for doctest.

The typical example in the documentation is:

.. doctest::

   >>> print 1
   1

Isn't there a way to let sphinx generate the output (here: 1) automatically?

As far as I understood, it is possible to run:

$ make doctest

which has the effect to test the code snippets, and compare the real output with the expected output. For example, if you have

.. doctest::

   >>> print 1
   3

doctest will warn you that it got 1 while it was expecting 3.

Instead, I would like sphinx to insert the real output alone in my docstring or in my .rst file. For example, if we have something like:

.. doctest::

    >>> print 1
    >>> print [2*x for x in range(3)]

I would like that when we run make doctest with an option, it changes the docstring to:

.. doctest::

   >>> print 1
   1
   >>> print [2*x for x in range(3)]
   [0,2,4]

I'm sure it's possible, and would be very convenient!

Answer 1

I have to strongly (but kindly) advise against what you're trying to do.

What you're asking is against the "test part" of the doctest module:

The doctest module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.

These tests have a reasons to be if you write the input and the expected output and let Python check if the expected output match the actual output.

If you let Python produce the expected output, well.. it will no longer be expected (by the user/author), so the doctests will never fail, hence those tests will be useless.

Note: If inside a function there's no logic (if/else, while-loops, appends, etc..) there's no need to test them. And tests must not reproduce the testing logic, otherwise they're not testing the function anymore.

I found this video about test driven development very interesting, maybe it could be of interest to you if you want to know more about this argument.

Answer 2

Here is a suggestion on how you could achieve what I suspect you might be looking for:

Doug Hellmann has written an interesting article called Writing Technical Documentation with Sphinx, Paver, and Cog. It has a section describing how the Cog tool can be used to automatically run code examples and capture the output for inclusion in Sphinx-built documentation.

---

There is also a contributed Sphinx extension called autorun that can execute code in a special runblock directive and attach the output to the documentation.