Should I write integration test that test my code with real-network servers, before I start writing my code?

Question

I am trying to write integration tests for a client-server app. The client app runs on a user's machine and connects to the server app running on a specific Windows server, network-serverA. The server app also accesses files on a different network server, network-serverB.

In this post, the author mentions that integration tests are more important than writing unit tests. Further, in the book Code Complete 2, Steve McConnel mentions that it's better to write tests beforehand rather than afterwards.

In my case, to get true integration tests, I would have to either start up two network servers in VMs or just somehow simulate that. Towards that I did some research to figure out how to implement that, and these are the few options I came up with to simulate the network servers:

1. Use the default Windows file share (https://msdn.microsoft.com/en-us/library/windows/desktop/aa365233(v=vs.85).aspx). The advantage of this method is that it's very easy to set up for me. The disadvantage of this method is that I would have to do these steps in an automated script.

   Steps:

   a. change the name of the user's computer to "FileServerName", so the program can access it via \\FileServerName

   b. share a folder in the current directory

   c. change the name of the user's computer back to the previous state.

2. Use FreeNAS installed as a VM on VirtualBox and map it to a certain drive name to match actual production file-server: https://www.windowscentral.com/how-to-set-up-freenas-home-file-server

   Steps:

   a. set up FreeNAS as a file server under virtualBox

   b. change name of FreeNAS such that files are accessible via \\FilServerName

   c. start up FreeNAS as the file server and distribute files from \\FileServerName

   d. shut down FreeNAS VM

3. Run a Docker container for both network-serverA and network-serverB. In network-serverA run a file-server which serves files as \FileNameServer and in network-serverB run a ipython instance. The challenge in this setup is that it's versatile, but also requires more knowledge than have at this point.

4. Using Python, create a virtual network interface, create a name for that virtual network interface so that it can be accessed by \\FileServerName and then access that as normal.

All of these options require some upfront cost and tradeoffs. Starting up a FreeNAS server or Docker containers probably simulate true integration tests better, but also has higher setup costs, in that I would have to spend a week or so to understand. On the other hand, just using Windows File sharing is cheaper to set up, but if somebody else comes along tomorrow and wants to test this, it's not easily transferrable to that developer.

When you are starting a project, and are using the TDD, PseudoCoding (from Code Complete 2) frameworks, do you typically write out integration tests first (setting up database/file server access) before writing a single piece of code?

Or should I forget about integration tests for now, write out the high level classes, functions to implement the requirements and write unit-tests for these function, create functions that can be tested? Once I have the unit tests, can I come back and write the integration tests?

Answer 1

You need a development or test environment which has these things setup the same as you live environment.

Sure it takes time and maybe for your particular case cutting corners is fine. But generally speaking you need something which matches your live setup if you are going to have faith in your integration tests.

By the time we get to integration testing, we should already have compiled, manually tested on our local dev machine, run unit tests etc.

Our integration tests are to catch more subtle bugs with environment setup and deployment. "Does it work in the field?"

You dont want these to pass on your windows file share only to find they fail against the live unix one.

Answer 2

Start with whatever is the lowest hanging fruit and work from there.

That said, I have the suspicion that integration tests are more useful, so spend some time and prototype how you will do them. Just remember, what works for one grpup may not exacty work for another.