Unit Testing

Unit Testing#

As our code continues to grow, how can we be sure it is working as expected? If we make minor changes to the code, what tests can we run to make sure we didn’t break anything? Are our functions written well enough to capture and correctly handle all of the edge cases we throw at them? In this module, we will use the Python pytest library to write unit tests: small tests that are designed to test specific individual components of code. After working through this module, students should be able to:

  • Find the documentation for the Python pytest library

  • Identify parts of code that should be tested

  • Identify appropriate assertions and exceptions to test for

  • Write and run reasonable unit tests

Getting Started#

Unit tests are designed to test small components (e.g. individual functions) of your code. They should demonstrate that things that are expected to work actually do work, and things that are expected to break raise appropriate errors. The Python pytest unit testing framework supports test automation, set up and shut down code for tests, and aggregation of tests into collections. It is not part of the Python Standard Library, so we must install it. Make sure to load a virtual environment if you are using one (which we highly recommend!).

[terminal]$ python3 -m venv groceries

[terminal]$ source groceries/bin/activate # macOS/Linux
[terminal]$ groceries\Scripts\activate # Windows

[terminal]$ pip install pytest

Find the documentation here.

Pull a copy of the groceries script we have been working on, and a copy of the groceries json file, if you don’t have copies already.

Devise a Reasonable Test#

The functions in this Python3 script are relatively simple, but how can we be sure they are working as intended? Let’s begin with the compute_average_quantity() function. We might choose to test it manually using the Python3 interactive interpreter:

>>> from groceries import compute_average_quantity
>>>
>>> data = [{'thing': 1}, {'thing': 2}]
>>>
>>> print(compute_average_quantity(data, 'thing'))
1.5

So simple! We import our code, hand-craft a simple data structure, and send the data plus the key we are interested in to our function. We know off the top of our heads that the average of 1 and 2 is 1.5, and that is in fact the number we get back.

Instead of writing that out each time we want to test, let’s instead put this into another Python3 script. When writing test scripts, it is a common convention to name them the same name as the script you are testing, but with the test_ prefix added at the beginning.

[terminal]$ ls
groceries.json  groceries.py
[terminal]$ touch test_groceries.py
[terminal]$ ls
groceries.json  groceries.py  test_groceries.py

Open up the script and put in our testing code from before:

1from groceries import compute_average_quantity
2
3data = [{'thing': 1}, {'thing': 2}]
4print(compute_average_quantity(data, 'thing'))

Next try to execute the test script on the command line:

[terminal]$ python test_groceries.py
1.5

Great! We assume the test is working. But we still have to look at the output (1.5) and remember back to our hand-crafted data and make sure that is the correct result. It would be more efficient if we had a way to check that the correct answer is returned in our test script itself. To do this, we can use the assert statement.

1from groceries import compute_average_quantity
2
3data = [{'thing': 1}, {'thing': 2}]
4
5assert( compute_average_quantity( data, 'thing' ) == 1.5 )

Now instead of printing the result, we use assert to make sure it is equal to our expected outcome. If the conditional is true, nothing will be printed. If the conditional is false, we will see an AssertionError.

EXERCISE#

  • Write a few more tests to convince yourself that the function is in fact returning the average of the input values.

  • Modify one of the tests so that it should fail, and execute the tests to confirm that it does fail.

  • If you have multiple tests that pass and multiple tests that fail, how would you know?

Automate Testing with Pytest#

Pytest is an excellent framework for small unit tests and for large functional tests (as we will see later in the semester). If you previously installed pytest with pip3, now would be a good time to double check that the installation worked and there is an executable called pytest in your PATH:

[terminal]$ pytest --version
pytest 8.0.0

Next, we just need to make a minor organizational change to our test code. We group all of our tests for a given function (e.g. all the tests for compute_average_quantity) into their own function. By convention, we typically name that function as “test_” plus the name of the function we are testing. Pytest will automatically look in our working tree for files that start with the test_ prefix, and execute the test functions within.

1from groceries import compute_average_quantity
2
3def test_compute_average_quantity():
4   assert compute_average_quantity([{'a': 1}, {'a': 2}], 'a') == 1.5
5   assert compute_average_quantity([{'a': 1}, {'a': 2}, {'a': 3}], 'a') == 2
6   assert compute_average_quantity([{'a': 10}, {'a': 1}, {'a': 1}], 'a') == 4

Call the pytest executable in your top directory, it will find your test function in your test script, run that function, and finally print some informative output:

==================== test session starts ====================
platform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/ajs2987/projects/cs401/docs/unit05/scripts
plugins: anyio-4.6.2.post1
collected 1 item

test_groceries.py .                                                                                                           [100%]

===================== 1 passed in 0.01s =====================

What Else Should We Test?#

The simple tests we wrote above seem almost trivial, but they are actually great sanity tests to tell us that our code is working. What other behaviors of our compute_average_quantity() function should we test? In no particular order, we could test the following non-exhaustive list:

  • If the list only contains one dictionary object, the function still behaves as expected

  • The return value should be type float

  • If we send it an empty list, that should raise some sort of exception

  • If we send it a list of non-uniform dictionaries (e.g. the dictionaries don’t all have the expected key), we should get a KeyError

  • If we send it bad values (e.g. a value is a string instead of an expected float), we should get a ValueError

  • If we send it a string that doesn’t appear in the dictionaries, we should get a KeyError

Tip

A list of all of the built-in Python3 exceptions can be found in the Python docs.

To test some of these behaviors, let’s create some additional assertions and organize them into their own functions.

 1from groceries import compute_average_quantity
 2import pytest
 3
 4def test_compute_average_quantity():
 5   assert compute_average_quantity([{'a': 1}], 'a') == 1
 6   assert compute_average_quantity([{'a': 1}, {'a': 2}], 'a') == 1.5
 7   assert compute_average_quantity([{'a': 1}, {'a': 2}, {'a': 3}], 'a') == 2
 8   assert compute_average_quantity([{'a': 10}, {'a': 1}, {'a': 1}], 'a') == 4
 9   assert isinstance(compute_average_quantity([{'a': 1}, {'a': 2}], 'a'), float) == True
10
11def test_compute_average_quantity_exceptions():
12   with pytest.raises(ZeroDivisionError):
13      compute_average_quantity([], 'a')                               # send an empty list
14   with pytest.raises(KeyError):
15      compute_average_quantity([{'a': 1}, {'b': 1}], 'a')             # dictionaries not uniform
16   with pytest.raises(TypeError):
17      compute_average_quantity([{'a': 1}, {'a': 'x'}], 'a')           # value not a float
18   with pytest.raises(KeyError):
19      compute_average_quantity([{'a': 1}, {'a': 2}], 'b')             # key not in dicts

After adding the above tests, run pytest again:

==================== test session starts ====================
platform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/ajs2987/projects/cs401/docs/unit05/scripts
plugins: anyio-4.6.2.post1
collected 2 items

test_groceries.py ..                                                                                                          [100%]

===================== 2 passed in 0.02s =====================

Success! The tests for our first function are passing. Our test suite essentially documents our intent for the behavior of the compute_average_quantity() function. And, if ever we change the code in that function, we can see if the behavior we intend still passes the test.

EXERCISE#

In the same test script, but under new test function definitions:

  • Write tests for the calc_total_price() function

  • Write tests for the count_categories() function

Capturing Standard Out#

If you have a function that prints to standard out (stdout), we can write a unit test for that using the capsys utility. Imagine a function that takes an argument and prints something to screen:

1def print_func(num):
2    print(f'hello {num}')
3
4def main():
5    print_func(5)
6
7if __name__ == '__main__':
8    main()

Executing this code prints hello 5 to screen. To write a unit test for this, we import the function into our test script, call the function normally, then capture the response using the capsys.readouterr() method. Then we assert that the response matches our expectations. Assume the above Python code is in a script called print_hello.py.

1from print_hello import print_func
2
3def test_print_func(capsys):
4    print_func(1)
5    captured = capsys.readouterr()
6    assert captured.out == 'hello 1\n'

Notice that we put a newline character (\n) at the end of the expected output. This character is automatically added by the print function. See the additional resources below for more information on using capsys.

Additional Resources#

Contents