I think I may be writing -too- many unit tests.

For example, say I was doing a coursework management system, and I coded a feature for submitting coursework on behalf of a student. If invalid credentials are specified in the API call, a 401 forbidden is returned. If a student tries to submit a different student's coursework, a different error is returned. If a student sends in his own coursework, its accepted but in a 'pending' status, awaiting approval from staff. If a faculty member sends in coursework on behalf of a student, its automatically accepted and also put in 'approved' status automatically.

As you can see, there are about 4-5 different scenarios here, each with a different outcome.

I tend to write tests for each scenario. That means, it might take me 10-15 mins to code the feature, but then another 15 mins - an hour, to write and pass all the unit tests. May be even 2 hours.

Obviously this is making me slow, and I'm expected to deliver results much faster. But unless I write all those tests, it feels 'wrong', and that the code may not work as expected.

What's the solution here? What's the right amount of unit tests to write? For every 1 hour of coding, how much time, max, should be spent writing tests for that feature?

  • 1
    Obviously, there's no universal answer to this except "write the right amount of tests for your context". Start with an arbitrary amount, collect metrics about what matters in your environment (time to deliver, bug count, etc.), analyze them and use them to adjust amount of tests as you progress. – guillaume31 Apr 5 '16 at 9:03

Every one of those outcomes that you described are valid testing scenarios. The way you know that is that each behavior is tied to a different outcome. That makes each one a prime candidate for testing.

From a Cyclomatic Complexity point of view, the fact that there are different outcomes for each test corresponding to different program states almost certainly means that your testing different paths through the code, which improves your testing code coverage.

If you were just repeatedly testing the same behavior over and over and asserting the same result, then I might be worried that you're writing too many tests. But not if each test is testing a specific behavior and result.

  • Good point. I do re-use parts of code which are repeated in tests. there's very little duplicated code. But, the question remains about how to balance the time in coding actual features vs writing tests. It seems counter productive to spend 4x the time in writing tests than it does to write the code and deliver results 4x faster. – Click Upvote Apr 4 '16 at 20:05
  • 2
    Until your code makes it out to production with an error that could have been caught in unit testing, and now the cost to fix it has just gone up by an order of magnitude. Anyway, part of the reason you write unit tests is that you save time later by making your code easier to refactor and retest. You need to start thinking of unit tests as an investment, not a cost. – Robert Harvey Apr 4 '16 at 20:08
  • 1
    @ClickUpvote - There are times when writing the test gives you some insight into how to write the code. Even if you're not doing TDD, the lines get blurred if you're going back and changing code as a result of something you learned from the test. – JeffO Apr 4 '16 at 20:26
  • 1
    As a rule of thumb: cyclomatic complexity is a lower bound on the number of tests (assertions, to be precise) you need to achieve branch coverage. NPath complexity is a lower bound on the number of tests you need to achieve path coverage. – Jörg W Mittag Apr 5 '16 at 0:16
  • @JörgWMittag Unless I misunderstand you, I don't think you're right there. Consider the function if (a) { b(); } else { c(); } ; if (d) { e(); } else { f(); }. The cyclomatic complexity of this function is 3, but you can achieve branch coverage with only 2 tests (a=1,d=1; a=0,d=0). I would say, however, that cyclomatic complexity is a good rule of thumb for how many tests a function actually needs -- that extra test in this case may expose some dependency between the other functions called, for example, and is generally a good idea. – Jules Apr 5 '16 at 8:07

There's always a trade-off between creation and testing, I could create a perfect product, just come back in 10 years and it'll be ready.. and no manager will ever accept that estimate (or the cost :) )

So you have to be pragmatic, unit tests do not catch all bugs, so you should not try to create unit testing environment that is 100% perfect with 100% code coverage. You can spend 20% of the time writing 80% of the tests, and spend the rest of the time writing the integration tests that you need to do as well.

However, there's one part of your question that stands out:

to write and pass all the unit tests.

Now, the time spent making the code work so it passes the tests should be considered time spent in the initial coding stage. You can crack out any old rubbish code and say "finished!" but it isn't really finished - you just deferred some coding time to later (when the bug is discovered) or on someone else (which is worse, fix your own mess). So never think that writing code is the end of it, the time spent fixing code that is highlighted by your tests counts as coding time, not testing time.

So, to answer your question. I prefer integration testing in the main, so I don;t unit test everything. I tend to unit test only those parts of the code that is complex or awkward. So I would not try to limit the time spent testing, but to prioritise what needs testing. A simple method can be visually inspected to be correct if its simple enough, its pointless writing a unit test just for a getter for example. Then you can spend more time giving the complex methods the testing they deserve, saving the time you spent testing the trivial and your overall quality and productivity should rise.


"Hammer in a nail - if the wood splits you should have used a screw"

How do you know if you're not writing enough tests? If bugs appear further upstream that could have been caught at the unit test phase then you haven't written enough tests.

But this aside, I sense you're equally interested in the expended effort as heading off possible bugs. If you find yourself writing code to cover off ranges of parameters or combinations of the same, then frameworks such as NUnit can take a lot of the donkey work away. If however they're bona fide business cases then you just have to bite the bullet and code as many as you need. There is no rule of thumb for this really - some code can be extremely quick to write but difficult to test and vice versa.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.