Automated Python Documentation with Sphinx

Code documentation is simultaneously one of the most important and one of the most neglected parts of any good coding. Reliable and well formatted code documentation is necessary to allow others to use one’s code easily and with minimal user errors. It is also a great way to provide other developers the insights needed to help modify or enhance an existing project. However, good documentation is often left as a side task to receive minimal attention or disregarded entirely due to the tediousness of it’s creation. Today, we are going to review a guide on how to use the Sphinx package to automatically create beautiful HTML and PDF code documentation for Python automatically and without the knowledge of coding HTML pages. This guide will provide the following guides:

• How to install Sphinx and automatically create Sphinx docs structure with Sphinx Quickstart
• Create HTML hompage
• Use Sphinx API Doc and Sphinx Extensions to automatically create API-Style documentation
• Change Sphinx Theme and use ReStructuredText to convert in-code docstrings to HTML documentation

Installing Sphinx and Using Sphinx Quickstart

First, to install Sphinx, ensure you have Python’s package install pip installed and upgraded with the following commands:

apt install python3-pip
pip3 install --upgrade pip

Next, install the Sphinx package. For later use, we will also install the RTD Theme:

pip3 install sphinx sphinx_rtd_theme

Now it is time to create the proper directory structure for Sphinx to operate in. Let’s say our project root folder is calls sphinx_learning/ and it contains the standard main.py along with a python package called calculator/ where we have created a simple module, basic_functions.py . We will now add the docs/ directory to our project root. This will be the main directory where Sphinx will operate and generate all of it’s files. We now have a project structure as follows:

├── sphinx_learning/
    ├── calculator/
        ├── __init__.py
        ├── basic_functions.py
    ├── docs/
    ├── main.py

If you are using and IDE such as PyCharm, you would see something in your project tab similar to:

Next, from the command line, we will head into the docs/ directory and use Sphinx Quickstart to generate the initial required files. To do this, headc to the docs/ directory and run sphinx-quickstart

cd docs/
sphinx-quickstart

You will be prompted with several messages to determine how you would like to set up Sphinx. The first is a yes/no questions regarding how to set up the source/ and build/ directories that Sphinx will use. Enter y for this prompt. Next, it will ask for the project name. This is the name that will appear in the Documentation, but it does not have to be the root project folder name. Here we will enter Calculator . It will then ask for the authors, we will put in the author names (multiple authors are separated by a comma). Next it will ask for a project release, we will put 1.0 for this release. Finally, Sphinx will wish to know what language to generate the Documentation in. We will use the default en for English. Once that is complete, Sphinx Quickstart will populate the docs/ directory with several important subdirectories and files. The new project structure should look similar to this:

├── sphinx_learning/
    ├── calculator/
        ├── __init__.py
        ├── basic_functions.py
    ├── docs/
        ├── build/
        ├── source/
            ├── _static
            ├── _templates
            ├── conf.py
            ├── index.rst
        ├── make.bat
        ├── Makefile
    ├── main.py

The files and directories Sphinx Quickstart just created are as follows

• build/ : The directory where the HTML and PDF documentation files will be stored once generated. For now this should be empty
• source/ : The directory containing the majority of the source code and files used to generate the documentation
• _static/ : A folder for static HTML files if they are to be used. We will not use them in this project and this directory will remain empty.
• _templates/ : A folder to store any HTML templates for custom HTML pages. We will not use them in this project and this directory will remain empty.
• conf.py : The Python configuration file. This file controls the extensions and templates that Sphinx will use for generating documentation. We will be modifying this later. It also is where we will declare the root folder for Sphinx to look to when generating documentation.
• index.rst : A ReStructuredText file used to generated HTML files. ReStructuredText is an enhanced Markdown format that is used by Sphinx to generate HTML and PDF documentation. The index.rst will be used to generate the index.html later on, which will act as the homepage for our documentation.
• make.bat : Compilation file for Windows
• Makefile : Compilation file for non-Windows

Creating an Initial HTML Home Page

We have now used Sphinx Quickstart to generate the basic structure of our Sphinx Documentation along with several key files. However, before we generate our HTML code, we must first point Sphinx to where our code is. In order to do this, we must add our root project path to the conf.py file. We will use Python’s built-in os and sys packages to add the root project path to the working Path. To do this, import the two aforementioned packages and use the sys.path.insert() to insert them into the working path. Use the os package to define the absolute path to insert.

import os
import sys
sys.path.insert(0, os.path.abspath('../..'))

The conf.py file should now look like this

We are now ready to generate our HTML website files! To do this, we will use the Makefile in the docs/ directory. Head to ~learning_sphinx/docs/ and run make html

cd ~learning_sphinx/docs/
make html

Sphinx should provide a relatively verbose output providing updates to the user

We can see at the bottom that Sphinx dumped the generated HTML files in build/html . If we head there we should see some new files

As mentioned before, index.rst is the ReStructuredText file that is used to create our home page. So it would follow that index.html is the HTML file for our home page. If we open this up in a browser, we can see the beginnings of an HTML website for our code. If you have never opened a local HTML before, simply navigate to the file, right click and select the open with option, then select your default browser. Here we will see our new documentation HTML page

Notice that it has the project name as well as the authors and creation date. And now we have a homepage, but there is still no real documentation present.

Using Sphinx API Doc and Sphinx Extensions to Generate Documentation from Source Code

So we now have a homepage. However, all we have are a few hyperlinks to empty pages. This is because we have not yet provided Sphinx with the .rst files to create the HTML files for our actual code. Fortunately, Sphinx provides a simple tool to do this. The sphinx-apidoc CLI command takes in source code and generates corresponding .rst files to create HTML documentation in API style for the input code using Sphinx. The command to specify the input source code and to specify where to output the .rst files is sphinx-apidoc -o <output-dir> <input-dir> . Since we are trying to document our calculator, the input code will be our calculator/ module. The output directory should be the docs/sources directory so that when we later generate the HTML code, the .rst files are in the right location. Head back to the /docs directory and execute the following

sphinx-apidoc -o source/ ../calculator/

Sphinx will inform us that the proper .rst docs where created

We should also be able to find the new .rstfiles in the source/ directory. The modules.rst is a ReStructuredText file that is automatically generated by Sphinx to encompass all of the modules we wish to create. An .rst file is also created for each individual module. Since we only had 1 module, we will see the calculator.rst file also present here.

Unfortunately, Sphinx does not natively support generating code documentation simply from source code. However, there is a very nice Sphinx extension that allows us to do this; the autodoc extension. In order to add extensions to our Sphinx compilation, we must head back to conf.py . Below all of our project variables, there is an empty list called extensions . It is here that we will define what extensions to use while compiling. As mentioned before, we will be adding the autodoc extension that will automatically generate documentation from source code. We will also be adding the viewcode extension, which will give a link in the documentation to view the associated source code. There are plenty of other extensions that Sphinx provides and it may be worth your time to check those out and add any useful ones, but for now we will stick with these 2. Head to conf.py and add the following to the extensions list

The final step to get Sphinx to generate our HTML documentation is to ensure that the modules page is included in our homepage, index.html . In order to do this, we need to modify the index.rst file to ensure it points to the modules.rst file when recreating the HTML. Simply head to the index.rst file and below the first chunk of code, add modules

NOTE: In ReStructuredText, indentation is enforced, so ensure that modules lines up with the rest of the code block.

In order to fully view the capabilities of Sphinx documentation, we will need some functions to document. Let’s go ahead and add a simple function to our basic_functions.py for addition of 2 integers.

Now we can regenerate our HTML files from the docs/ directory by running the following make command

make clean html

And presto! Sphinx will automatically generate the documentation for our module from the source code.

If we head back to our docs/build/html directory, we should see HTML files for both modules and calculator created.

Now if we reopen or refresh our index.html page, we can see that the links lead to code documentation for our code!

If we click on these links, we will see basic documentation of our code

We can even go deeper into our addition function and view the basic function definition

Finally, because we have the viewcode extension, we can also see the source code by clicking the [source] hyperlink

We now have working documentation for our simple calculator!

In-Code Docstring Documentation and Sphinx Themes

For the final section, we will do some beautification of our documentation by adding in In-Code docstrings that Sphinx is able to interpret and by changing the theme of the documentation to one used by many well known packages such as matplotlib . First, we will add a few more functions and provide some in-code ReStructuredText style docstrings to better define our functions

def addition(a: int, b: int):
    """
    A function to add two integers

    :param a: The first integer to add
    :param b: The second integer to add
    :return: The addition of a and b
    """
    return a + b

def subraction(a: int, b: int):
    """
    A function to subtract two integers

    :param a: The first integer being subtracted from
    :param b: The integer to subtract from a
    :return: The subtraction of b from a
    """
    return a - b

def multiplication(a: int, b: int):
    """
    The multiplication of two integers

    :param a: The first integer to multiply
    :param b: The second integer to multiply
    :return: The multiplication of a and b
    """
    return a * b

Notice the structure of the docstrings. This is the ReStructuredText format and it is required to follow the documentation to be generated properly. Many IDEs, such as PyCharm, support auto-creation of these docstrings in ReStructuredText format when creating functions. Additionally, there are many other fields that can be defined in these docstrings in ReStructuredText format, such as parameter and return types. Feel free to research and add in whatever docstrings you wish to use. Now we will change the theme of the documentation. Head back to conf.py and towards the bottom we will see the html_theme variable. This is defaulted to alibaster , but we will be changing to the theme we installed at the beginning of this tutorial, sphinx_rtd_theme

Now, for the final time, head back to docs/ and recreate your HTML code

make clean html

Finally, open index.html in your browser and you may see a more familiar HTML theme for your documentation. Navigate to our calculator module and you will see much better code definitions. Congratulations! You have created your first Sphinx Documentation!