Is it normal to spend as much, if not more, time writing tests than actual code?

Question

I find tests a lot trickier and harder to write than the actual code they are testing. It's not unusual for me to spend more time writing the test than the code it is testing.

Is that normal or am I doing something wrong?

The questions “Is unit testing or test-driven development worthwhile?”, “We are spending more time implementing functional test than implementing the system itself, is this normal?” and their answers are more about whether testing is worth it (as in "should we skip writing tests altogether?"). While I'm convinced tests are important, I'm wondering if my spending more time on tests than actual code is normal or if it's only me.

Answer 1

I remember from a software engineering course, that one spends ~10% of development time writing new code, and the other 90% is debugging, testing, and documentation.

Since unit-tests capture the debugging, and testing effort into (potentially automate-able) code, it would make sense that more effort goes into them; the actual time taken shouldn't be much more than the debugging and testing one would do without writing the tests.

Finally tests should also double as documentation! One should write unit-tests in the way the code is intended to be used; i.e. the tests (and usage) should be simple, put the complicated stuff in the implementation.

If your tests are hard to write then, the code they test is probably hard to use!

Answer 2

It is.

Even if you take just unit testing, it is not unusual to have more code within the tests than the actually tested code. There is nothing wrong with it.

Consider a simple code:

public void SayHello(string personName)
{
    if (personName == null) throw new NullArgumentException("personName");

    Console.WriteLine("Hello, {0}!", personName);
}

What would be the tests? There are at least four simple cases to test here:

1. The person name is null. Is exception actually thrown? That's at least three lines of test code to write.

2. The person name is "Jeff". Do we get "Hello, Jeff!" in response? That's four lines of test code.

3. The person name is an empty string. What output do we expect? What is the actual output? Side question: does it match the functional requirements? That means another four lines of code for the unit test.

4. The person name is short enough for a string, but too long to be combined with "Hello, " and the exclamation point. What happens?¹

This requires a lot of testing code. Moreover, the most elementary pieces of code often require setup code which initializes the objects needed for the code under test, which also often leads to writing stubs and mocks, etc.

If the ratio is very big, in which case you may check a few things:

• Is there code duplication across the tests? The fact that it's test code doesn't mean that the code should be duplicated (copy-pasted) between similar tests: such duplication will make the maintenance of those tests difficult.

• Are there redundant tests? As a rule of thumb, if you remove a unit test, the branch coverage should decrease. If it doesn't, it may indicate that the test is not needed, since the paths are already covered by other tests.

• Are you testing only the code you should test? You are not expected to test the underlying framework of the third-party libraries, but exclusively the code of the project itself.

With smoke tests, system and integration tests, functional and acceptance tests and stress and load tests, you add even more test code, so having four or five LOC of tests for every LOC of actual code is not something you should be worried about.

A note about TDD

If you're concerned about the time it takes to test your code, it might be that you are doing it wrong, that is code first, tests later. In this case, TDD may help by encouraging you to work in iterations of 15-45 seconds, switching between code and tests. According to the proponents of TDD, it speeds up the development process by reducing both the number of tests you need to do and, more importantly, the quantity of business code to write and especially rewrite for testing.

---

^{¹ Let n be the maximum length of a string. We can call SayHello and pass by reference a string of length n - 1 which should work just fine. Now, at Console.WriteLine step, the formatting should end up with a string of length n + 8, which will result in an exception. Possibly, due to the memory limits, even a string containing n / 2 characters will lead to an exception. The question one should ask is whether this fourth test is a unit test (it looks like one, but may have a much higher impact in terms of resources compared to average unit tests) and if it tests the actual code or the underlying framework.}

Answer 3

I think it is important to distinguish between two types of test strategies: unit testing and integration/acceptance testing.

Although unit testing is necessary in some cases, it is frequently hopelessly over-done. This is exacerbated by meaningless metrics forced on developers, like "100% coverage". http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf provides a compelling argument for this. Consider the following issues with aggressive unit testing:

• An abundance of pointless tests that does not document a business value, but simply exist to get closer to that 100% coverage. Where I work, we have to write unit tests for factories that does nothing else but create new instances of classes. It adds no value. Or those lengthy .equals() methods generated by Eclipse - there is no need to test these.
• In order to make testing easier, developers would sub-divide complex algorithms into smaller testable units. Sounds like a win, right? Not if you need to have 12 classes open to follow a common code path. In these cases, unit testing can actually decrease code readability. Another problem with this is that if you chop your code into too small pieces, you end up with a magnitude of classes (or pieces of code) that seem to have no justification beyond being a sub-set of another piece of code.
• Refactoring of highly coveraged code can be difficult, as you also need to maintain the abundance of unit tests that depend on it working just so. This is exacerbated by behavioural unit tests, where part of your test also verifies the interaction of a class's collaborators (usually mocked out).

Integration/acceptance testing, on the other hand, is a hugely important part of software quality, and in my experience you should spend a significant amount of time on getting them right.

A lot of shops have drunk the TDD kool-aid, but as the link above shows, a number of studies shows that its benefit is inconclusive.

Answer 4

Can not be generalised.

If I need to implement a formula or algorithm from physically based rendering, it can very well be that I spend 10 hours on paranoid unit tests, as I know the slightest bug or inprecision can lead to bugs that are almost impossible to diagnose, months later.

If I just want to logically group together some lines of code and give it a name, only used in file scope, I may not test it at all (if you insist on writing tests for every single function, without exception, programmers might fall back to writing as few functions as possible).

Answer 5

I find it to be the most important part.

Unit testing is not always about "seeing if it works right", it's about learning. Once you test something enough it becomes "hardcoded" into your brain and you eventually lower your unit testing time and can find yourself writing entire classes and methods without testing anything until your done.

This is why one of the other answers on this page mention that in a "course" they did 90% testing, because everyone needed to learn the caveats of their objective.

Unit testing is not only a very valuable use of your time as it literally enhances your skills, it is a good way to go over your own code again and find a logical mistake along the way.

Answer 6

Is it normal to spend as much, if not more, time writing tests than actual code?

Yes, it is. With some caveats.

First of all, it is "normal" in the sense that most big shops work this way, so even if this way was completely misguided and dumb, still, the fact that most big shops work this way makes it "normal".

By this I don't mean to imply that testing it is wrong. I have worked in environments with no testing, and in environments with obsessive-compulsive testing, and I can still tell you that even the obsessive-compulsive testing was better than no testing.

And I do not do TDD yet, (who knows, I might in the future,) but I do the vast majority of my edit-run-debug cycles by running the tests, not the actual application, so naturally, I work a lot on my tests, so as to avoid as much as possible having to run the actual application.

However, be aware that there are dangers in excessive testing, and specifically in the amount of time being spent on maintaining the tests. (I am primarily writing this answer in order to specifically point this out.)

In the preface of Roy Osherove's The Art of Unit Testing (Manning, 2009) the author admits to having participated in a project which failed to a large part due to the tremendous development burden imposed by badly designed unit tests which had to be maintained throughout the duration of the development effort. So, if you find yourself spending too much time doing nothing but maintaining your tests, this does not necessarily mean that you are on the right path, because it is "normal". Your development effort might have entered an unhealthy mode, where a radical rethinking of your testing methodology might be necessary in order to save the project.

Answer 7

Yes, it is normal if you're talking about TDDing. When you have automated tests, you secure the desired behaviour of your code. When you write your tests first, you determine whether existing code already has the behaviour you desire.

This means that:

• If you write a test that fails, then correcting the code with the simplest thing that works is shorter than writing the test.
• If you write a test that passes, then you have no extra code to write, effectively spending more time to write test than code.

(This does not account for code refactoring, which aims at spending less time writing subsequent code. It is balanced by test refactoring, which aims at spending less time writing subsequent tests.)

Yes too if you're talking about writing tests after the fact, then you will be spending more time:

• Determining desired behaviour.
• Determining ways of testing desired behaviour.
• Fulfilling code dependencies to be able to write the tests.
• Correcting code for tests that fail.

Than you'll spend actually writing code.

So yes, it is an expected measure.

Answer 8

It can be for many people, but it depends.

If you're writing tests first (TDD), you may be experiencing some overlap in the time spent writing the test is actually helpful to writing the code. Consider:

• Determining results and inputs (parameters)
• Naming Conventions
• Structure - where to put things.
• Plain old thinking

When writing tests after writing the code, you may discover your code is not easily testable, so writing tests is harder/takes longer.

Most programmers have been writing code a lot longer than tests, so I would expect most of them to not be as fluent. Plus you get to add the time it takes to understand and utilize your testing framework.

I think we need to change our mindset about how long it takes to code and how unit testing is involved. Never look at it in the short-term and never-ever just compare the total time to deliver a particular feature because you have to consider not only are you writing better/less buggy code, but code that is easier to change and still make it better/less buggy.

At some point, we're all only capable of writing code that is so good, so some of the tools and techniques can only offer so much in improving our skills. It's not like I could build a house if I only had a laser guided saw.

Answer 9

Is it normal to spend as much, if not more, time writing tests than actual code?

• Yes for writing unit-tests (Test a module in isolation) if the code is highly coupled (legacy, not enough separation of concerns, missing dependency injection, not tdd developed)
• Yes for writing integration/acceptance-tests if the logic to be tested is only reachable via gui-code
• No for writing integration/acceptance-tests as long as the gui-code and the business-logic is separated (The test does not need to interact with the gui)
• No for writing unit-tests if there is seperaton of concerns, dependency injection, code was testdriven developped (tdd)