I am building a binary file importer using Python. The specific file structure is defined in a specification. This looks like:


  • Map Block

String, Short Integer, Long Integer, String, Short Integer, Long Integer...

  • Block1

String, String, Long Integer, String, Short Integer, String, Long Integer, Long Integer....

  • Block2

  • Block3


I have proved the concept of collecting and decoding the binary data and I can access each block of data individually by collecting the Map Block first.

However I can't figure out the best way to Test it. The current tests I have are: test_get_string test_get_short test_get_long

Can you suggest what suite of tests I might build next? Would I be best testing with an actual file or create long binary strings to test with?


If you work with a stream to import (as suggested by Killian) then you can test each of the functions to get individual values by mocking that stream. Mocking the stream means that you setup the next few bytes before calling the to be tested function and let the test fail when it requests more than the needed number of bytes. After the function all those bytes should have been read.

Part of the correctness assertion would be that it doesn't read more or less than the amount of bytes it needs to decode and the correctness of the value.

You can also test getting each block individually again with ensuring it only reads exactly as many bytes it needs and correctness of the read in values.

For testing the map block decoding you can construct one and then query the offsets of some blocks using that map block.

| improve this answer | |
  • I am using the stream (docs.python.org/2/library/io.html) when I open the file correct? This function I am not looking to test. – fiberman May 12 '15 at 9:02
  • To construct the Map Block would you simulate the stream? as in put the binary into the tests? Or create mock return values from the individual values. – fiberman May 12 '15 at 9:08
  • You can use BytesIO to provide the mocked stream (or implement your own) – ratchet freak May 12 '15 at 9:10
  • So would you recommend this sort of test.. (see next comment) – fiberman May 12 '15 at 9:12
  • mock._file = io.BytesIO() mock._file.write(b'\x64\x00\x80\x00\x00\x00\x09\x00\x47\x65\x6E\x50\x61\x72\x61\x6D\x73\x00\x6E\x00\x18\x00\x00\x00\x53\x75\x70\x50\x61\x72\x61\x6D\x73\x00\x64\x00\x3B\x00\x00\x00\x46\x78\x64\x50\x61\x72\x61\x6D\x73\x00\x6E\x00\x36\x00\x00\x00\x4B\x65\x79\x45\x76\x65\x6E\x74\x73\x00\x6E\x00\x90\x00\x00\x00\x44\x61\x74\x61\x50\x74\x73\x00\x64\x00\xAE\x86\x01\x00\x41\x52\x53\x70\x65\x63\x69\x61\x6C\x00\x6E\x00\x16\x02\x00\x00\x41\x52\x45\x76\x65\x6E\x74\x00\x64\x00\xCE\x00\x00\x00\x43\x6B\x73\x75\x6D\x00\x64\x00\x02\x00\x00\x00') file_importer.Map(file=mock._file) – fiberman May 12 '15 at 9:14

The purpose of your implementation is to get comething decoded. It's not to move data from disk to memory, that is just an incidental auxiliary function. Therefore, your code should be decoupled so that the importer works with a stream, and in practice it should be coupled to a standard file-reading implementation (preferably from the standard library).

Similarly, your unit test should test the decoding logic, not the file I/O - I should hope that that has been tested with the standard library, and anyway it's not your concern. Therefore, testing with handcrafted input data, without touching the file system, is both faster and better according to sound software engineering principles.

| improve this answer | |
  • I am using the standard library to do the I/O and the decoding. (struct.unpack). Should I mock the response from this decoding for the block tests? – fiberman May 12 '15 at 8:46
  • Generally, a unit test shouldn't touch the file system, but only exercise the business data flow. Python's struct.unpack is a bit of a grey area; it isn't exactly system I/O, but also not strictly business logic - so you should do what lets you create a reliable test most easily. – Kilian Foth May 12 '15 at 8:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.