In this article: I recently spoke at DjangoMaine’s December meetup about writing spec-like tests with Python’s unittest. I am adapting the slides into an article, but in the meantime I want to apply the approach to some actual tests encountered in the wild.

Edit 2014-09-17: Despite my prior claims to the contrary, these tests have too much boilerplate. I no longer do it this way. I now use a much flatter testing style powered by py.test and mock. Ignore the text below.

We will start with example tests from the Python and Django documentation. Although these are not real tests I insist we treat them as real. Programmers copy & paste these examples when creating their own tests. Because of their tremendous influence, I consider examples like these at least as important as real tests.

Python - unittest - 25.3.1 Basic example

The following snippet comes from Python’s unittest documentation.


    import random
    import unittest

    class TestSequenceFunctions(unittest.TestCase):

        def setUp(self):
            self.seq = range(10)

        def test_shuffle(self):
            # make sure the shuffled sequence does not lose any elements
            random.shuffle(self.seq)
            self.seq.sort()
            self.assertEqual(self.seq, range(10))

            # should raise an exception for an immutable sequence
            self.assertRaises(TypeError, random.shuffle, (1,2,3))

        def test_choice(self):
            element = random.choice(self.seq)
            self.assertTrue(element in self.seq)

        def test_sample(self):
            with self.assertRaises(ValueError):
                random.sample(self.seq, 20)
            for element in random.sample(self.seq, 5):
                self.assertTrue(element in self.seq)

After rewriting these in spec-like style, I end up with the following:


    import random
    from unittest import TestCase


    class WhenShuffling(TestCase):
        def setUp(self):
            self.seq = range(10)
            self.shuffled = list(seq)

            random.shuffle(self.shuffled)

        def test_it_does_not_lose_any_elements(self):
            self.assertEqual(sorted(self.shuffled), self.seq)


    class WhenShufflingAnImmutableSequence(TestCase):
        def setUp(self):
            self.seq = (1, 2, 3)

            # This assertion does not belong here, but we are forced to put it
            # here if we want to keep our Function Under Test inside `setUp`.
            # If this raises, all of our `test_` methods fail; otherwise, they
            # may succeed. We should define them with this behavior in mind.
            with self.assertRaises(TypeError):
                random.shuffle(self.seq)

        def test_it_refuses_to_shuffle(self):
            # If `setUp` fails, this test fails; otherwise, it succeeds.
            pass


    class WhenChoosing(TestCase):
        def setUp(self):
            self.seq = range(10)
            self.element = random.choice(self.seq)

        def test_the_chosen_element_exists_in_the_sequence(self):
            self.assertIn(self.element, self.seq)


    class WhenSampling(TestCase):
        def setUp(self):
            self.seq = range(10)
            self.sample = random.sample(self.seq, 5)

        def test_each_sampled_element_is_in_original_sequence(self):
            for element in self.sample:
                self.assertIn(element, self.seq)


    class WhenSamplingMoreUniqueElementsThanWhatExist(TestCase):
        def setUp(self):
            # This assertion does not belong here, but we are forced to put it
            # here if we want to keep our Function Under Test inside `setUp`.
            # If this raises, all of our `test_` methods fail; otherwise, they
            # may succeed. We should define them with this behavior in mind.
            with self.assertRaises(TypeError):
                random.sample(range(10), 20)

        def test_it_refuses_to_sample(self):
            # If `setUp` fails, this test fails; otherwise, it succeeds.
            pass

Is this any better than the original? Yes, but it is not obvious from our current perspective.

You could argue that it is worse. It feels over-engineered; it has significantly more lines and longer names. The strength of these arguments fades over time though, and the tests will still exist long after these arguments have weakened.

Suppose a bug report comes in about “random.sample” repeating elements from the sequence. Adding a regression test is easy.


    class WhenSampling(TestCase):
        # ... previous definitions elided

        def test_it_contains_no_duplicate_elements(self):
            self.assertEqual(len(set(self.sample)), len(self.sample))

After a year of not looking at this code a bug report comes in about the above regression test failing on an obscure platform. The test runner output in the bug report will show things like “WhenSampling” and “test_it_contains_no_duplicate_elements,” which is more informative for our hazy memory than the original names, “TestSequenceFunctions” and “test_sample.”

Django - Testing Django applications - Example

The following snippet comes from Django’s unittest documentation.


    from django.utils import unittest
    from django.test.client import Client

    class SimpleTest(unittest.TestCase):
        def setUp(self):
            # Every test needs a client.
            self.client = Client()

        def test_details(self):
            # Issue a GET request.
            response = self.client.get('/customer/details/')

            # Check that the response is 200 OK.
            self.assertEqual(response.status_code, 200)

            # Check that the rendered context contains 5 customers.
            self.assertEqual(len(response.context['customers']), 5)

After rewriting these in spec-like style, I end up with the following:


    import httplib as http

    from django.test import TestCase


    class WhenRequestingCustomerDetails(TestCase):
        def setUp(self):
            self.response = self.client.get('/customer/details/')

        def test_it_returns_OK(self):
            self.assertEqual(response.status_code, http.OK)

        def test_the_response_contains_5_customers(self):
            self.assertEqual(len(self.response.context['customers']), 5)

We replaced tedious comments and meaningless boilerplate like “SimpleTest” and “test_details” with meaningful names. We also made our setup code more explicit and reduced the risk of different assertions interfering with each other. Again we have significant improvement, though we may have to imagine ourselves in maintenance mode to see it.

Further Reading

These tests are inspired by test_dingus.py

Published: 2012-01-01
Edited: 2012-01-01 · 2014-09-17