Evaluating whether to write unit test or integration tests first on blue-sky/prototype projects

Question

Something I've noticed recently is when I am doing the following types of projects:

• When beginning a project
• Working on an MVP/prototype
• Adding features which aren't entirely defined
• Working on a smaller scale project

For reference, I am working on a Python project now which currently has ~1k lines of code, including some comments and all whitespace.

I find it immensely easier to first write integration tests, work on the code, and then once it the API is somewhat hardened actually work on adding unit tests. The types of tests that I can run on my main function, so to speak, and are more "end to end" than anything else.

This is because unit tests are really annoying when an API is changing fairly rapidly, which often is the case when working on a project matching any or most of the criteria above.

Is this approach a good approach and what criteria should be considered when making the decision whether to start with unit or integration tests first for these types of projects? Am I missing the value of unit testing these sorts of projects before the APIs are more solidified?

Answer 1

Am I missing the value of unit testing these sorts of projects before the APIs are more solidified?

No. You're doing fine.

The two big aims of TDD are:

• Defining interfaces by actual usage, rather than by internal implementation¹
• Maximizing test coverage

Test coverage can be pretty well maximized either way you go. That is, regardless of whether you first test small, isolated units or large, "integrated" units, you have the option to write your tests before your implementations.

What you gain in writing higher level ("integration") tests first, as you're doing, is the assurance that your higher level interfaces and interactions are also defined primarily according to their usage, rather than by their internal implementations.

The same effect can be largely achieved with some good "architecting" and diagramming. But, those high level tests can often reveal things you missed in your diagrams — or that you just got wrong in your "architecture" work.

---

If you're not actually doing TDD (or anything like it), the order you write the tests doesn't matter much. The interfaces already exist by the time you test, so it's far less likely that your tests will change anything — they will only serve to protect against particular breaking changes.

But, if you're concerned with building the implementation top-down versus buttom-up, the first bullet point still applies to a large degree. The high level code helps define low level interfaces. Whereas, if the low level interfaces are written first (or otherwise already exist), the high level code is at their mercy ...

---

^{1. This one also applies even if you're not doing full-on TDD. Even if you're just writing 1 or 2 tests before your implementation, those 1 or 2 tests can help you define or refine your interfaces before it's too late!}

Answer 2

I've worked the way you're working. And I'm not going to tell you you can't. I will warn you about something you can run into.

When every unit test is simply a retrofit it's hard to learn to make them flexible. They tend to be nothing more than regression tests. Since you've never used them to help you refactor it's very easy to write the kinds of tests that actually make refactoring more difficult. This tends to spiral out of control until you lose all faith in TDD.

However, you're already working on something. I'm not going to tell you to stop. I will say it might be worth it to start something else that you have time to explore and follow the red green refactor cycle from the beginning. Make sure you actually do use the tests to help you refactor. Until you've mastered this way of working use it sparingly on something that matters. This is a very different way to code and takes getting used to. Doing it half way will do no one any good.

That said

I find it immensely easier to first write integration tests, work on the code, and then once it the API is somewhat hardened actually work on adding unit tests. The types of tests that I can run on my main function, so to speak, and are more "end to end" than anything else.

Understand that a unit test is NOT simply a test that acts on one class. So long as the API you're working on can be tested without doing any of the following you're doing unit testing just fine:

• It talks to the database
• It communicates across the network
• It touches the file system
• It can't run at the same time as any of your other unit tests
• You have to do special things to your environment (such as editing config files) to run it.

Michael Feathers: A Set of Unit Testing Rules

So if your end-to-end test involves more than one object that's fine. This is unit testing not object testing.

Just like private methods do not need to be tested any more then they get tested through testing the public methods that use them, objects do not need to be initially developed under their own testing harness. Only when objects are being considered for use independent of the end-to-end story do they really need to be treated like they have their own interface and behavior to confirm. If you're being careful about it this is when you make those objects public. This way you make no promises you haven't tested.