How to do you judge the value of using TDD or plain unit testing in an Open Source project? Is there a good rule of thumb? [duplicate]

Question

In my current OSS project, I knew literally nothing about what I was doing when I began.

I was integrating with a larger project as a plugin, and that in and of itself had a steep learning curve.

If I had begun with TDD it would have never been written as my code was changing as I was learning.

This has worked for quite a long time. Now I am past the point where unit tests provide negative returns, and I need to write them. So as I fix things, for items I know how to write a test for I will.

My question, does anyone have a good rule of thumb as to what point you should say to yourself - stop - unit tests need to be used from here on out.

Believe me - it is not day 1. I am looking for a pragmatic answer. Remember, I don't make any money off of my work, I enjoy writing new features not new tests for old ones. That is the root of the dilemma.

Answer 1

No, there is no rule of thumb.

It really depends on how dependable and maintainable you want your code base to be. If your project is a small class or one-shot utility, you may not need any unit tests at all. If, on the other hand, it is a framework or core program that will be used by many people, you may need extensive unit tests, and you'll probably need them from the beginning.

Unit testing, when used correctly, guides your overall software design. Testable code is easier to maintain, easier to reason about, and has fewer bugs. This is as true from day one as it is at the end of your project. Delaying this process is based on the false idea that unit testing takes more time than just writing an application and testing it manually. While unit testing takes a little longer to set up, it pays for itself the first time you have to re-run the tests.

Unless you are writing in a dynamic language with a REPL, I honestly don't see how delaying your unit tests is going to help you. I use unit tests as a shim for trying out experimental code. When I have code that works, I graduate both the unit test and the working code to "first-class" status in my project, and the unit test serves as a design prototype for the actual code, as well as verifying that the code still works after a refactor.

If you are working in a dynamic language with a REPL (as opposed to a statically-typed language like C#), unit tests become even more important, because you no longer have the safety net of static type verification during compilation.

Nowhere is this philosophy more evident than in the SQLite code base. SQLite has roughly 84 thousand lines of production code, covered by more than 90 million lines of test code. I'm pretty sure the author wasn't ever thinking "so when should I start writing unit tests for this thing?"

Further Reading
http://www.sqlite.org/testing.html

Answer 2

Testing and test coverage are difficult things to quantify, I don't believe 100% code coverage is essential, or even a good idea, but a good set of tests developed using TDD or not is going to save you time in the long run and improve the quality of your code.

As a rule of thumb : if you really know what the code should be doing and can define it clearly and exactly I'd use TDD, it helps you get the requirements down and keeps you focused on the thing you are developing. Otherwise if the exact nature of the code is still forming or changing as you go, write a test as soon as you have a working version you are happy with and you think the signatures etc are not likely to change.

I find it much harder to retro fit tests to a project, it's easier to code tests when the thing you are working on is still fresh in your memory.

If its trivial code I'd probably not write a test at all.