IDEs & Debugging

Running tests

Testing your code is a very important part of the development process. If you want to read more into the background of testing and learn more about the feautes available in Python, take a look at the testing chapter of the Best Practices in Software Engineering course.

For now, we'll assume we know how to write tests and will instead see how the IDE can make running them easier.

The first thing we need to do is enable the integration feature in the IDE.

1. Open File → Settings.../Preferences.... Go to Tools → Python Integrated Tools in the area on the left.
2. In the Default test runner field select pytest.
3. Click Apply to save the settings.

If "No pytest runner found in selected interpreter" shows up in that window near the bottom then choose Fix. This will install pytest in the background.

Press Ok to close the settings window.

You can see more information on this in the official documentation.

1. Go to View → Command Palette...
2. Search for Python: Configure Tests and press enter
3. Select pytest
4. Select . Root directory when asked to select the directory containing the tests
5. A popup will ask Test framework pytest is not installed. Install? Choose Yes

You can see more information on this in the official documentation.

Running a single test

Now that testing is enabled we can go ahead and write our first test. A test setup is is made of two parts: the code to be tested and the code to do the testing. We'll have to create both.

First the code to be tested in utils.py:

def add_arrays(x, y):
    """
    This function adds together each element of the two passed lists.

    Args:
        x (list): The first list to add
        y (list): The second list to add

    Returns:
        list: the pairwise sums of ``x`` and ``y``.

    Examples:
        >>> add_arrays([1, 4, 5], [4, 3, 5])
        [5, 7, 10]
    """
    z = []
    for x_, y_ in zip(x, y):
        z.append(x_ + y_)

    return z

Then make another file called test_utils.py which will contain our testing code:

from utils import add_arrays


def test_add_arrays():
    a = [1, 2, 3]
    b = [4, 5, 6]
    expect = [5, 7, 9]

    output = add_arrays(a, b)

    assert output == expect

Now that our code is all in place, we can go ahead and run the test.

As soon as PyCharm recognises that this is a test, there should be a green "⏵" button next to the function name.

Click it and select Run 'pytest for test_util....'

This will open up a new tab at the bottom of the screen with two panes. The left-hand one containing the list of tests and the pane on the right containing the output of pytest. You can read more about this tab in the official documentation.

Above the name of the function , you should see Run Test.

If you don't see the Run Test link above the test name then you may see ⚡ Test discovery failed in the bottom status bar. If so click it and it try to find the tests again. If you don't see ⚡ Test discovery failed either then open the command palette (View→Command Palette...) and find and run Python: Discover Tests.

Click Run Test and it should change to ✔ Run Test which means that the test has passed successfully.

Running all tests

Running a single test is useful while you are developing it or the code which it is testing but you will often want to run all the tests in the whole project to make sure that something else hasn't been broken.

Add a second test to test_utils.py:

from utils import add_arrays


def test_add_arrays():
    a = [1, 2, 3]
    b = [4, 5, 6]
    expect = [5, 7, 9]

    output = add_arrays(a, b)

    assert output == expect


def test_add_empty_arrays():
    assert add_arrays([], []) == []

In the project browser pane on the left-hand side of the screen, right-click on the name of the folder utils and select Run 'pytest in utils'. This will run both of the tests and show the output in the Run tab at the bottom of the screen.

On the far left-hand side of the screen there is a column of icons. Click on the conical flask icon (when hovered it says Test). This opens the testing tab.

At the top of that pane there is a green play button.

Press that button which will run all tests in the project. You should see the list of items below the button change briefly to a ↻ icon and then to a green tick:

Failing tests

In the situation where all the tests are passing (hopefully most of the time) the IDE will simplify what it shows you since there's not much more to say. However, if a test fails you will be given more information.

Let's deliberately break one of the tests to simulate this:

from utils import add_arrays


def test_add_arrays():
    a = [1, 2, 3]
    b = [4, 5, 6]
    expect = [5, 7, 999]  # ← This was changed from 9 to 999

    output = add_arrays(a, b)

    assert output == expect


def test_add_empty_arrays():
    assert add_arrays([], []) == []

Run all the tests again.

PyCharm allows you to easily rerun whatever you ran previously. This applies to running scripts, debugging scripts and running tests.

In the top-right corner of the screen there are the run widgets:

The first thing is the drop-down which lets you reselect previous runs as well as create new configurations to run whatever script or tool you wish. Next to that is the green "play" button which reruns the last configuration. Next is the "insect" debug button which runs the same configuration but through the debugger. Finally there is a red stop button (greyed-out be default) which allows you to forcibly stop any running task.

This drop-down is part of a very powerful run configuration system in PyCharm. We don't need most of its power, but it's good to know where to find it.

We see the fact that the test failed in the test summary pane in the IDE.

The information about how the test failed is in the right-hand pane. If you expand out test_utils in the summary list by clicking the arrow next to it, you can then click each entry to see the results for just that test.

By default, tests that passed are hidden in this list. To show them, press the tick button shown in the top-left of the above screenshot.

In the code editor pane the test name will now have a red underline beneath it and ✔ Run Test will have changed to ✘ Run Test. Hover over the name of the test and you will see a popup with the details of how the test failed.

Before continuing, fix that test by changing it back to 9.

Previous | Next