A complete python testing tutorial to write better code

You start to write a program, and it turns out great. Then, you add features over time and slightly change its behavior. You went way beyond the original planning, and this feels great. However, at some point, your program stops working. You desperately search for the bug in the code with no luck. Eventually, you will drop the project and start a new one. We have all been there, that ugly moment when your code collapse. Today we see how we can avoid it with tests. This python testing tutorial will present everything you need to know about tests. Can’t wait to start!

Introducing this Python Testing Tutorial

What is python testing?

Generally, you write a program and run it to see if it works as expected. This is the simplest way to test the program’s functionalities. Testing, and specifically python testing, means that another program does that for you. In other words, you have a script that launches your program and sees if it runs as you wish.

A script launches your program and see if it runs as you wish.

What does this mean? You can have several testing scripts, and each can test a specific portion of your code. Ideally, you want to test every part of your code. To do that, we can classify python tests into three categories.

• Unit Tests are your first step. They verify an individual part of your code, like a function or a class, verifying if it works as expected on its own.
• Integration Tests are an evolution of unit tests, as they verify how different parts of your code interact. Thus, they may use multiple modules to test an entire functionality.
• End to End Tests are the most complete type of test. They verify the whole stack of a part of your application, including a connection to external resources like a database. In other words, they emulate the real-world scenario and test it. Since they often rely on external resources, they are harder to implement.

Each of this category serves a purpose. You should write tests of all the three categories above.

Why python testing? The technical debt

People often say “My code works, I don’t need to test it”. Or, even better “I will never ever change that part, I will skip tests”. Of course, both are utterly wrong.

You write tests because you can’t know how things will evolve. In fact, you may end up changing the script, or some of its dependencies may change with updates. An OS call you relied on may be unavailable in the future, and so on. Programs are a maze of entangled lines of code, if something changes something else will break. Period.

Furthermore, if you write tests, expanding your code will be much easier. You can modify a piece of existing code without fear of disrupting something else. If the tests succeed, you are good to go. Thus, you speed up your release cycle: you know immediately if your application works, and you can ship it to the customer faster. At first, writing tests seems like additional work to do. In the end, it will make you save a ton of time. Go with the motto “If it is not tested, it doesn’t work”.

Instead, if you don’t write tests you will increase your technical debt. What is this? If you have a debt, you owe someone money, money that you will need to return at some point. Not writing tests means you save time now, but this won’t continue forever. At some point, you will have to “return” the time you borrowed. This is the moment when your script breaks, and you spend days finding out how to fix it. You will spend much more time doing this rather than writing tests as you progress. You are now repaying the technical debt, and with high interest.

Python Testing Tutorial

A DIY Approach

We can start our Python Testing Tutorial in a Do-It-Yourself fashion. Our program that we want to test is simply a function that performs a division.

def division(a, b):
  if b == 0:
    raise ZeroDivisionError
  return a/b

To test if this function works, we can create another set of functions that call it. Such testing functions will compare it to the expected results, and print an error in case something is out-of-order.

def test_division_1():
  if division(10, 2) == 5:
    return True
  return False
  
def test_division_2():
  if division(100, 5) == 20:
    return True
  return False
  
def test_division_3():
  if division(100, 1) == 100:
    return True
  return False
  
def test_division_by_zero():
  try:
    division(10, 0)
  except ZeroDivisionError:
    return True
  return False
  
def test_all():
  errors = 0
  if not (test_division_1() and test_division_2() and test_division_3()):
    print('Error while dividing integers..')
    errors += 1
  if not test_division_by_zero():
    print('Error while dividing by zero...')
    errors += 1
  if errors == 0:
    print('No error!')
  else:
    print('{0} errors found!'.format(errors))

Now we can call test_all() to see if our division() function works under all circumstances. At first, it will say there is no error. Imagine at some points you want to change the division by zero behavior. You change the raise ZeroDivisionError into a plain return None, because another part of your code needs that. If you do that, tests will fail.

C:\Users\aless\Desktop>python temp.py
Error while dividing by zero...
1 errors found!

You know it is time to correct your code.

The standard approach to python testing

Writing DIY tests was cool, but it is not something we would do in a production program. Like in everything, best practices exist for testing as well. Since this python testing tutorials aim to get you up and running with tests, we need to use the unittest module. This module provides everything you need for testing, and it is already included in python.

First, we need to structure a project in a scalable manner. The size of the code we write for tests will grow as our application grows. Generally, the bigger the application, the more tests it needs. Furthermore, more tests are always better. Unlike any other part of the code, multiple tests doing the same thing is an acceptable practice. So, we need to prepare a dedicated part of our projects to tests. We do that by creating the test folder in the project root.

Here you can find more information about structuring your project. Now, our test folder is literally a module, so we create __init__.py. Now, we can add files containing tests.

Our Testing File

Whenever you create a file for testing, let the name start with test_, as this is a good practice. Now, we can start writing our test class.

Each file may contain one or more Test Cases. A Test Case is simply a set of tests that verify a specific functionality. For example, all the tests we wrote in the DIY section can be part of a “Test Division” Test Case. In code, you need to create a class extending the TestCase class, available from unittest. So here is what our test_division.py file will look like.

import sys
sys.path.append('..')
from myapp.utils import division
import unittest

class TestDivision(unittest.TestCase):
  def setUp(self):
    pass
    
  def tearDown(self):
    pass
    
  def test_divisions(self):
    self.assertEqual(division(10, 2), 5, 'Failed to divide 10 by 2')
    self.assertEqual(division(100, 5), 20, 'Failed to divide 100 by 5')
    self.assertEqual(division(100, 1), 100, 'Failed to divide 100 by 1')

  def test_division_by_zero(self):
    with self.assertRaises(ZeroDivisionError):
      division(10, 0)

Breaking it down

The first to lines are important in case you are not using an advanced ide like PyCharm. They allow you to import files that are not part of this module, but from the parent directory instead (..). Then, we import our division function that is in utils.py, and unittest.

At this point, we create our TestDivision class, and we add two coll methods: setUp and tearDown. These methods contain instructions to run when setting up the test case, and when tearing it down. In the case of a plain unit test, we generally don’t need them. However, they are important in case of integration or E2E tests, as they prepare the environment.

Then, each method of the class starting with test_ will be considered a test. We created two, one that tests some divisions, and another that tests division by zero. We grouped all division attempts into one function because they test the same exact thing.

Python Testing Tutorial: about assertions

Our TestCase base class offers several methods of self we must use to create our tests. Below the most important ones

The third parameter you saw in the code from the previous paragraph is an optional message. That message will appear in the output in case the test fails. If you need more information, here is a link to the official documentation.

Running our test

If you attempt to run the test we wrote with python test_division.py, nothing will happen. This is because this file has no configuration to handle a direct call, and it shouldn’t. Instead, python has a built-in tool to run all the tests in a directory. Just run the following command.

python -m unittest discover

This will automatically discover and run all the tests in the specified directory. If we are in the test folder, we can use . as directory, as in the example below. Since we haven’t changed back to raise ZeroDivisionError, the division by zero will still fail, and this will be the output.

C:\Users\aless\Desktop\src\test>python -m unittest discover .
F.
======================================================================
FAIL: test_division_by_zero (test_division.TestDivision)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\aless\Desktop\src\test\test_division.py", line 20, in test_division_by_zero
    division(10, 0)
AssertionError: ZeroDivisionError not raised

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

About writing tests

The Test Coverage

In this Python Testing Tutorial, we don’t want just to explain how to write tests. We want to explain how to write meaningful tests. To do that, we need to talk about test coverage. This is exactly what the name says: how much of your application is verified by your test cases.

Forget about a 100% coverage, this requires a huge load of work that is often not justifiable. Instead, aim to cover with tests all the core components. Specifically, write tests that are generic enough to cover several circumstances. If, for example, you need to test a function containing if x < 0: there is no point in testing with 100 values of x, but only with one greater than zero, and with another less than zero.

You can have functional testing cases covering 40-60% of your application and consider it tested. Of course, the more tests you can afford, the better. Anything below 40% is not acceptable.

Test-Driven Development

Now that you know how to write tests, we can conclude our Python Testing Tutorial talking about test-driven development. We can consider this a buzzword of modern programming, and for good reasons.

Test-Driven Development is an approach to write code all around the tests. When you create a program, you should have a clear idea of its specifications. Instead of having a document listing all of them, or in addition to it, you start writing tests. After all, a test simply says “I expect that this works this way”. Then, after the tests are ready, you start to create the code and continue until you see no error.

A conclusion for our Python Testing Tutorial

With this python testing tutorial, we explained everything you need to start writing your own tests. With them, you will be able to scale your code and maintain it over time. Hopefully, you will avoid those frustrating moments where you don’t know where the code is broken. You understand the concepts of test coverage and test-driven development and are ready to do take your code to the next level.

In case you were looking for a TL;DR, here we have a few key points.

• Create a test folder in the project root and write tests into files starting with test_.
• Inside each file, extend the unittest.TestCase class with one or more classes: each represents a set of tests on the same component.
• For integration and end-to-end tests, you can use the setUp and tearDown methods of TestCase to prepare and destroy the environment.
• Any method of your TestCase class starting with test_ will be executed as a test.
• Inside each test method, use assertions like assertEqual to verify values.
• Run your tests with python -m unittest discover .

What do you think about tests? Are you ready to embrace them in your development style? Let me know what you think in the comments!