So I want to write this function that downloads a file and unpacks it in python using TDD.

The function will look like this approximately

import urllib.request
import tarfile

def download_and_unpack(tarball_url, destination)
    datastream = urllib.request.urlopen(tarball_url)
    thetarfile = tarfile.open(fileobj=datastream, mode="r|gz")

Now, I don't want to test the implementation but the behaviour, and I don't need to test the python standard library, so I don't know where to start.

I was thinking of creating a tarball in /tmp, pass it to the function and see that the files are unpacked, but that feels like testing too much at once. Any pointers or advice on how to start here?

As someone suggested, I could wrap each line in a custom interface and test the function by mocking these instead, but then I would have to test the interfaces themselves, and I feel I would end up in the same dead end...


1 Answer 1


In reality, I would probably not test this function in more than one TDD step - chaining three ready-made system functions together isn't particular error-prone, and if it fails, finding the root cause should be done pretty quick with one type of test alone.

However, lets do this for the sake of an exercise. This function consists mainly of two steps (as it's name says), so it should be possible to develop it in two TDD cycles. One possible approach here could be introducing a testmode parameter, which is set to "False" as default, but when set to True, it will write the downloaded data stream directly into a file.

This gives you the opportunity to first write a test to verify the download works, and make the first implementation of the function look like this:

def download_and_unpack(tarball_url, destination, testmode=False)
    datastream = urllib.request.urlopen(tarball_url)
    if testmode:
         # add code here to write datastream to a file

You need to call download_and_unpack with testmode=True in this test, of course.

Afterwards, you write a test asserting the function does both steps (download and unpack), then complete it. Finally, you add a test verifying when testmode=True, the result will be the same with no intermediate file.

Of course, I don't think in this contrived example it would be really worth the hassle, bot for larger functions with more complex code sections, allowing access to some intermediate steps' data, and logging it or writing it to some temporary file is an old, simple and practice-proven technique which can play well together with TDD.

  • You are right that the function does two things, so I should probably split this in two functions and test each one independently. I'm not really found of adding a extra argument to the function to change its behaviour during tests, doesn't this defeat the purpose of testing? But maybe this can be viewed as an alternative to mocking! Jun 10, 2021 at 7:00
  • @MartinRaspaud: no, the function does not do two things, I intentionally avoided this misleading wording. It does one thing: coordination of the calls to 3 other functions, where I would consider two of those calls as one step. Concerning the behaviour change: yes, one has to be careful not to introduce a "Heisenbug" tihs way, but this is not "black box" testing, it is TDD, which means you design your functions for better testability. Injecting a mock into a class changes the behaviour of a class as well.
    – Doc Brown
    Jun 10, 2021 at 10:35
  • 1
    ... of course, my suggestion is not the only solution: one could also make the download and unpack functions parameter of this function (or parameters of an outer class), use TDD to develop this generelized function, and finally wire it together with the real download and unpack functions. However, wire-up code is not really suitable for "TDD".
    – Doc Brown
    Jun 10, 2021 at 10:40
  • Got it. Yes, I was thinking of heisenbug possibilities, so maybe that's just something I have to learn to live with in TDD. Jun 10, 2021 at 11:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.