Testing with pytest - part 1

Why to write tests?

  • Who wants to perform manual testing?
  • When you fix a bug or add a new feature, tests are a way to verify that you did not break anything on the way
  • If you have clear requirements, you can have matching test(s) for each requirement
  • You don't have to be afraid of refactoring
  • Tests document your implementation - they show other people use cases of your implementation
  • This list is endless...

Test-driven development aka TDD

In short, the basic idea of TDD is to write tests before writing the actual implementation. Maybe the most significant benefit of the approach is that the developer focuses on writing tests which match with what the program should do. Whereas if the tests are written after the actual implementation, there is a high risk for rushing tests which just show green light for the already written logic.

Tests are first class citizens in modern, agile software development, which is why it's important to start thinking TDD early during your Python learning path.

The workflow of TDD can be summarized as follows:

  1. Add a test case(s) for the change / feature / bug fix you are going to implement
  2. Run all tests and check that the new one fails
  3. Implement required changes
  4. Run tests and verify that all pass
  5. Refactor

Running pytest inside notebooks

These are the steps required to run pytest inside Jupyter cells.

In [1]:
# Let's make sure pytest and ipython_pytest plugin are installed
import sys
!{sys.executable} -m pip install pytest
!{sys.executable} -m pip install ../../utils/ipython_pytest/
Requirement already satisfied: pytest in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (3.5.0)
Requirement already satisfied: six>=1.10.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (1.11.0)
Requirement already satisfied: setuptools in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (39.0.1)
Requirement already satisfied: attrs>=17.4.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (17.4.0)
Requirement already satisfied: more-itertools>=4.0.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (4.1.0)
Requirement already satisfied: pluggy<0.7,>=0.5 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (0.6.0)
Requirement already satisfied: py>=1.5.0 in /Users/jerry/.virtualenvs/learn-python3/lib/python3.5/site-packages (from pytest) (1.5.3)
You are using pip version 10.0.0, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Processing /Users/jerry/github/jerry-git/learn-python3/utils/ipython_pytest
Building wheels for collected packages: ipython-pytest
  Running setup.py bdist_wheel for ipython-pytest ... - done
  Stored in directory: /private/var/folders/nk/wgt71p5s1_d1wssm881h7ykc0000gn/T/pip-ephem-wheel-cache-tfei96ba/wheels/cd/57/26/ee80676d18b225631db5c78e9ddda568e7d2abf6f03c97047a
Successfully built ipython-pytest
Installing collected packages: ipython-pytest
  Found existing installation: ipython-pytest 0.0.1.dev0
    Uninstalling ipython-pytest-0.0.1.dev0:
      Successfully uninstalled ipython-pytest-0.0.1.dev0
Successfully installed ipython-pytest-0.0.1.dev0
You are using pip version 10.0.0, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
In [2]:
# Let's load ipython_pytest extension which makes it possible to run
# pytest inside notebooks
%load_ext ipython_pytest

Due to technical limitations of Jupyter notebooks and pytest extension, all code that runs inside tests has to be in the same cell. In other words, you can not call e.g. functions that are defined in other cells.

In real applications you have separate directory and files for your tests (see TODO LINK ). The above mentioned limititation is only present for writing tests in Jupyter environment.

pytest test cases

Pytest test cases are actually quite similar as you have already seen in the exercises. Most of the exercises are structured like pytest test cases by dividing each exercise into three cells:

  1. Setup the variables used in the test
  2. Your implementation
  3. Verify that your implementation does what is wanted by using assertions

See the example test case below to see the similarities between the exercises and common structure of test cases.

In [3]:
%%pytest
# Mention this at the top of cells which contain test(s)

# This would be in your e.g. implementation.py
def sum_of_three_numbers(num1, num2, num3):
    return num1 + num2 + num3


# This would be in your test_implementation.py
def test_sum_of_three_numbers():
    # 1. Setup the variables used in the test
    num1 = 2
    num2 = 3
    num3 = 5
    
    # 2. Call the functionality you want to test
    result = sum_of_three_numbers(num1, num2, num3)
    
    # 3. Verify that the outcome is expected
    assert result == 10
=========================================================== test session starts ===========================================================
platform darwin -- Python 3.5.4, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /private/var/folders/nk/wgt71p5s1_d1wssm881h7ykc0000gn/T/tmpgm5fi86g, inifile:
plugins: nbval-0.9.0
collected 1 item

_ipytesttmp.py .                                                                                                                    [100%]

======================================================== 1 passed in 0.02 seconds =========================================================

Now go ahead and change the line assert result == 10 such that the assertion fails to see the output of a failed test.