PyTest

Introduction

Pytest supports tests written with unittest .

The main advantage of Pytest lies in the way a TestCase is written.

A TestCase in pytest is a series of functions in a Python file whose names start with test_.

Pytest has other distinctive features too:

• support for the built-in assert statement (no need for the special self.assert methods);
• filtering support;
• the ability to rerun starting from the last failed test;
• an ecosystem of hundreds of plugins that extend the functionality.

A TestSum test in pytest looks like this:

def test_sum(): assert sum([1, 2, 3]) == 6, "Should be 6" def test_sum_tuple(): assert sum((1, 2, 2)) == 6, "Should be 6"

Here the base TestCase class and any use of classes at all are gone, as is the command-line entry point. As usual, more information is available on the Pytest site.

Example without an IDE

The simplest example in the console of Linux

You need to prepare the environment and create two files - one with the code and one with a test for that code.

The file with the code can be named anything - preferably in lowercase and not matching a reserved name.

The test file is usually named the same but with the test_ prefix, or less often with the _test postfix

mkdir /home/$(whoami)/python/pytest/qa
cd /home/$(whoami)/python/pytest/qa
python -m venv venv
source venv/bin/activate
python -m pip install --upgrade pip
python -m pip install pytest
touch psum.py test_psum.py

In the file psum.py write a simple function, for example addition

def add(x, y): return x + y

And in the file test_psum.py there will be a test for this function

from psum import add def test_psum(): assert add(2, 3) == 5, "Should be 5"

To run the test, execute

pytest test_psum.py

=================== test session starts =================== platform linux -- Python 3.9.5, pytest-7.0.1, pluggy-1.0.0 rootdir: /home/andrei/python/pytest/qa collected 1 item test_psum.py . [100%] ========================= 1 passed in 0.01s =========================

If the function returns a result different from the expected one, for example when the test itself has an error

assert add(2, 3) == 999, "Should be 999"

pytest test_psum.py

======================== test session starts ========================= platform linux -- Python 3.9.5, pytest-7.0.1, pluggy-1.0.0 rootdir: /home/andrei/python/pytest/qa collected 1 item test_psum.py F [100%] ============================== FAILURES ============================== _____________________________ test_psum ______________________________ def test_psum(): > assert add(2, 3) == 999, "Should be 999" E AssertionError: Should be 999 E assert 5 == 999 E + where 5 = add(2, 3) test_psum.py:4: AssertionError ======================= short test summary info ====================== FAILED test_psum.py::test_psum - AssertionError: Should be 999 ========================= 1 failed in 0.02s ==========================

Project structure with PyTest tests

As soon as a project outgrows the microscopic size, keeping the tests in the same directory as the code becomes inconvenient.

Suppose the project code is in the app directory.

A typical solution is to put the test file test_psum.py into the tests subdirectory

app ├── psum.py ├── tests │ └── test_psum.py └── venv ├── bin ├── include ├── lib ├── lib64 -> lib └── pyvenv.cfg

If you now run the test from the app directory, pytest will understand how to import psum

python -m pytest tests/test_psum.py

If you run the same test right from the tests directory, an error appears: ImportError while importing test module

Testing argument type validation

Demonstrations of PyTest often start with addition or multiplication functions.

Let us make sure we are in the app directory and create two more files prod.py , tests/test_prod.py

touch prod.py tests/test_prod.py

Now the project structure looks like this:

app ├── prod.py ├── psum.py ├── tests │ ├── test_prod.py │ └── test_psum.py └── venv

Let us add the following code to these files:

# Code def prod(a, b): return a * b

# Test from prod import prod def test_prod(): res = prod(3, 4) assert res == 12

python -m pytest tests/test_prod.py

This function expects numbers. But there is no argument validation.

Let us add validation and write a test.

# prod.py def prod(a, b): if not all( map(lambda p: isinstance(p, (int, float)), (a, b)) ): raise TypeError("Not valid argument data type") print("prod.py: Valid arguments") return a * b

# test_prod.py import pytest from prod import prod def test_prod(): res = prod(3, 4) assert res == 12 def test_arguments(): try: # A deliberately wrong data type must be caught # "" is a string, not a number res = prod("", 4) # With valid arguments the exception is not raised # and the test fails # res = prod(1, 2) except TypeError as err: # if TypeError is caught pass else: print("test_prod.py: Invalid argument is not caught") pytest.fail() # assert False

Instead of pytest.fail() you can use assert False, and then importing pytest is optional

python -m pytest -v tests/test_prod.py

================================ test session starts =============================== platform linux -- Python 3.9.5, pytest-7.0.1, pluggy-1.0.0 -- /home/andrei/python/pytest/otus/venv/bin/python cachedir: .pytest_cache rootdir: /home/andrei/python/pytest/app collected 2 items tests/test_prod.py::test_prod PASSED [ 50%] tests/test_prod.py::test_arguments PASSED [100%] ================================= 2 passed in 0.01s ================================

If the test breaks, PyTest produces the following result

================================ test session starts ================================ platform linux -- Python 3.9.5, pytest-7.0.1, pluggy-1.0.0 -- /home/andrei/python/pytest/otus/venv/bin/python cachedir: .pytest_cache rootdir: /home/andrei/python/pytest/app collected 2 items tests/test_prod.py::test_prod PASSED [ 50%] tests/test_prod.py::test_arguments FAILED [100%] ===================================== FAILURES ====================================== __________________________________ test_arguments ___________________________________ def test_arguments(): try: # "" is a string and should not be accepted res = prod(3, 4) except TypeError as err: # if TypeError is caught pass else: print("test_prod.py: Invalid argument is not caught") > pytest.fail() E Failed tests/test_prod.py:19: Failed ------------------------------- Captured stdout call -------------------------------- prod.py: Valid arguments test_prod.py: Invalid argument is not caught ============================== short test summary info ============================== FAILED tests/test_prod.py::test_arguments - Failed ============================ 1 failed, 1 passed in 0.02s ============================

unittest from PyTest

If the project already has tests written with unittest you do not have to rewrite them - PyTest understands the unittest syntax

Consider the unittest check of the quadratic equation solution

If you run these same tests with PyTest

python -m pytest -v tests/test_quadratic.py

==================== test session starts ===================== platform linux -- Python 3.9.5, pytest-7.0.1, pluggy-1.0.0 -- /home/andrei/python/unittest/app/venv/bin/python cachedir: .pytest_cache rootdir: /home/andrei/python/unittest/app collected 5 items tests/test_quadratic.py::TestQuadratic::test_raises_type_error PASSED [ 20%] tests/test_quadratic.py::TestQuadratic::test_result_is_tuple PASSED [ 40%] tests/test_quadratic.py::TestQuadratic::test_single_root PASSED [ 60%] tests/test_quadratic.py::TestQuadratic::test_two_roots PASSED [ 80%] tests/test_quadratic.py::TestQuadratic::test_zero_a_and_b PASSED [100%] ======================= 5 passed in 0.01s ====================

--last-failed: rerun only failed tests

If a test has failed because of an error in the test itself, you fix it and rerun it.

To avoid rerunning all the tests, use the --last-failed option

Suppose there is an error in test_prod.py

def test_prod(): res = prod(3, 4) assert res == 120

python -m pytest test_prod.py

==================== test session starts ==================== platform linux -- Python 3.9.5, pytest-7.1.1, pluggy-1.0.0 rootdir: /home/andrei/github/pytest1/app collected 2 items tests/test_prod.py F. [100%] ========================== FAILURES ========================= _________________________ test_prod _________________________ def test_prod(): res = prod(3, 4) > assert res == 120 E assert 12 == 120 tests/test_prod.py:8: AssertionError ------------------- Captured stdout call -------------------- prod.py: Valid arguments ================== short test summary info ================== FAILED tests/test_prod.py::test_prod - assert 12 == 120 ================ 1 failed, 1 passed in 0.02s ================ (venv) andrei@LL-andrei2 {12:04}~/github/pytest1/app:master ✗

After fixing the error you can rerun only the failed test

python -m pytest --last-failed tests/test_prod.py

=============================== test session starts ============================== platform linux -- Python 3.9.5, pytest-7.1.1, pluggy-1.0.0 rootdir: /home/andrei/github/pytest1/app collected 2 items / 1 deselected / 1 selected run-last-failure: rerun previous 1 failure tests/test_prod.py . [100%] ======================== 1 passed, 1 deselected in 0.01s ========================= (venv) andrei@LL-andrei2 {12:06}~/github/pytest1/app:master ✗

In PyCharm there is a special Rerun Failed Tests button - a green triangle and a red circle with an exclamation mark inside.

--capture=no: do not hide the output

If your tests call print(), a normal pytest run does not show the output

def test_prod(): print("testing prod") res = prod(3, 4) assert res == 120

To see the print() output, use the option

--capture=no

python -m pytest -v --capture=no test_prod.py

======================== test session starts ========================== platform linux -- Python 3.9.5, pytest-7.1.1, pluggy-1.0.0 -- /home/andrei/github/pytest1/venv/bin/python cachedir: .pytest_cache rootdir: /home/andrei/github/pytest1/app collected 2 items tests/test_prod.py::test_prod testing prod prod.py: Valid arguments PASSED tests/test_prod.py::test_arguments PASSED ========================= 2 passed in 0.01s ==========================

Testing the quadratic equation solution

Suppose we solve a quadratic equation with the following script.

from math import sqrt TYPE_ERROR_TEXT = "Not valid argument type" def quadratic_solve(a, b, c): if not all( map( lambda p: isinstance(p, (int, float)), (a, b, c) ) ): raise TypeError(TYPE_ERROR_TEXT) print("Types are OK") if a == 0: if b == 0: # a and b are 0: no solution return None, None return -c / b, None d = b ** 2 - 4 * a * c if d < 0: return None, None d_root = sqrt(d) divider = 2 * a x1 = (-b + d_root) / divider x2 = (-b - d_root) / divider if d == 0: x2 = None elif x2 > x1: x1, x2 = x2, x1 return x1, x2 if __name__ == "__main__": print(quadratic_solve(1, -1, -2)) print(quadratic_solve("", 2, 3))

The equation can be solved in other ways too; what the test needs is validation of the arguments and the roots returned as a tuple, where a missing root is passed as None.

Let us write a test that checks that the arguments are numbers and the result is a tuple

# test_quadratic.py import pytest from quadratic import quadratic_solve, TYPE_ERROR_TEXT class TestQuadratic: def test_raises_type_error(self): with pytest.raises(TypeError) as exc_info: quadratic_solve(1, "", 2) assert str(exc_info.value) == TYPE_ERROR_TEXT def test_result_is_tuple(self): res = quadratic_solve(0, 0, 0) assert isinstance(res, tuple)

Now at least three variants have to be checked: both roots exist, there is one root, there are no roots.

You can write three separate tests as was done with unittest but it is more convenient to use the test parametrization built into PyTest.

-k: running a specific test

If a test file holds more than one test, you may need to run only some of them

To run a test by its name, use the -k option

python -m pytest -v -k "test_basic_param_prod" tests/test_prod.py

================================================= test session starts ================================================== platform linux -- Python 3.9.5, pytest-7.1.1, pluggy-1.0.0 -- /home/andrei/github/pytest1/venv/bin/python cachedir: .pytest_cache rootdir: /home/andrei/github/pytest1/app collected 18 items / 15 deselected / 3 selected tests/test_prod.py::test_basic_param_prod[args0-0] PASSED [ 33%] tests/test_prod.py::test_basic_param_prod[args1-0] PASSED [ 66%] tests/test_prod.py::test_basic_param_prod[args2-132] PASSED [100%] =========================================== 3 passed, 15 deselected in 0.01s ===========================================

Add PyTest to PyCharm

To add PyTest to PyCharm follow this instruction

Settings (Ctrl + Alt + S) → Tools → Python Integrated Tools → Default test runner:

Select pytest. If it has not been added yet, a warning appears together with a Fix button.

Press the Fix button

A message appears at the bottom of the main settings screen

Installing package 'pytest'

Wait until it changes to a successful installation message and the warning disappears from the screen.

Once PyCharm is configured, a green triangle appears to the left of every test, and clicking it runs that test.

If the tests are in a separate directory, you may have to point at the project directory manually as the source for the tests.

It is done as follows

Right click on the directory → Mark Directory as → Test Sources Root

Article author: Arthur Isaev