143

I'm a solo developer with a pretty time-constrained work environment where development time ranges usually from 1-4 weeks per project, depending on either requirements, urgency, or both. At any given time I handle around 3-4 projects, some having timelines that overlap with each other.

Expectedly, code quality suffers. I also do not have formal testing; it usually goes down to walking through the system until it somewhat breaks. As a result, a considerable amount of bugs escape to production, which I have to fix and in turn sets back my other projects.

This is where unit testing comes in. When done right, it should keep bugs, let alone those that escape to production, to a minimum. On the other hand, writing tests can take a considerable amount of time, which doesn't sound good with time-constrained projects such as mine.

Question is, how much of a time difference would writing unit-tested code over untested code, and how does that time difference scale as project scope widens?

8
  • Comments are not for extended discussion; this conversation has been moved to chat.
    – maple_shaft
    Commented Jun 16, 2016 at 12:50
  • 9
    You are solving the wrong problem. You are too busy and seem to have no project management support. Are you estimating project effort? Are you reserving 20% of your time for bug fixes, meetings, and other non-coding tasks? How much overtime are you working?
    – Tony Ennis
    Commented Jun 16, 2016 at 22:06
  • 24
    Do you realize that you're essentially saying "I have time to do it twice, but not time to do it once the right way."?
    – RubberDuck
    Commented Jun 16, 2016 at 23:07
  • 6
    @RubberDuck there is actually a point in the curve of project complexity measured as Time to Write vs Time to Test, where "Wring it twice", takes less time than "Write it and it's tests". I think it may be somewhere in the region of a bash oneliner. Commented Jun 17, 2016 at 13:48
  • 1
    One time the developers got presents and thanks when a project was cancelled. I pointed out that we could have been even more productive had we known that the product wouldn't ship. So, this is a case where developing without testing would be advantageous.
    – JDługosz
    Commented Jun 19, 2016 at 5:03

14 Answers 14

161

The later you test, the more it costs to write tests.

The longer a bug lives, the more expensive it is to fix.

The law of diminishing returns ensures you can test yourself into oblivion trying to ensure there are no bugs.

Buddha taught the wisdom of the middle path. Tests are good. There is such a thing as too much of a good thing. The key is being able to tell when you are out of balance.

Every line of code you write without tests will have significantly greater costs to adding tests later than if you had written the tests before writing the code.

Every line of code without tests will be significantly more difficult to debug or rewrite.

Every test you write will take time.

Every bug will take time to fix.

The faithful will tell you not to write a single line of code without first writing a failing test. The test ensures you're getting the behavior you expect. It allows you to change the code quickly without worrying about affecting the rest of the system since the test proves the behavior is the same.

You must weigh all that against the fact that tests don't add features. Production code adds features. And features are what pay the bills.

Pragmatically speaking, I add all the tests I can get away with. I ignore comments in favor of watching tests. I don't even trust code to do what I think it does. I trust tests. But I've been known to throw the occasional hail mary and get lucky.

However, many successful coders don't do TDD. That doesn't mean they don't test. They just don't obsessively insist that every line of code have an automated test against it. Even Uncle Bob admits he doesn't test his UI. He also insists you move all logic out of the UI.

As a football metaphor (that's American football) TDD is a good ground game. Manual only testing where you write a pile of code and hope it works is a passing game. You can be good at either. Your career isn't going to make the playoffs unless you can do both. It won't make the superbowl until you learn when to pick each one. But if you need a nudge in a particular direction: the officials calls go against me more often when I'm passing.

If you want to give TDD a try I highly recommend you practice before trying to do it at work. TDD done half way, half hearted, and half assed is a big reason some don't respect it. It's like pouring one glass of water into another. If you don't commit and do it quickly and completely you end up dribbling water all over the table.

9
  • 75
    There is such a thing as too much of a good thing Neither you or Buddha have tested my grandmother's cookies :-) Commented Jun 15, 2016 at 13:05
  • 3
    @NickAlexeev I love that chart there. One thing it doesn't point out is that unit tests (which are typically automated) are really good at finding bugs when the code is modified. I'd love to see that split out into "bugs found before release" and "bugs found after release". Unit tests are the best line of defense against regression.
    – corsiKa
    Commented Jun 15, 2016 at 15:44
  • 3
    I think this is a very balanced answer: testing everything, even trivial things, can be a waste of time. Having good tests for the complex parts that can easily break can really help. I have just finished porting a small but non-trivial project from Java to C++. I first ported the tests and this guided me setting up the whole C++ implementation. Once all the tests were green, only a few easier classes needed to be ported and it went pretty smoothly. On the other hand, I do not have tests for all the code: this would have prolonged the implementation by at least 3, 4 days with little gain.
    – Giorgio
    Commented Jun 15, 2016 at 17:03
  • 5
    A slight disagreement with this : 'You must weigh all that against the fact that tests don't add features. Code adds features. And features are what pays the bills.' I would strongly suggest that it isn't features which pay the bills - it is working features. (or do people get paid for non-working deliverables ?). The rest of the answer I completely agree with. Commented Jun 16, 2016 at 20:06
  • 7
    @TonySuffolk66 you're correct, it's working features that pay the bills (barring flimflam salesmanship) However, people were creating working features long before TDD was a thing. They will long after it's gone. Remember, TDD is a disciplined way of testing. It is not the only disciplined way of testing. Commented Jun 17, 2016 at 3:38
118

I agree with the rest of the answers but to answer the what is the time difference question directly.

Roy Osherove in his book The Art of Unit Testing, Second Edition page 200 did a case study of implementing similarly sized projects with similar teams (skill wise) for two different clients where one team did testing while the other one did not.

His results were like so:

Team progress and output measured with and without tests

So in the end of a project you get both less time and fewer bugs. This of course depends on how big a project is.

36
  • 33
    The sample size is way too small to consider this to be scientific but I think it's representative of what lots of people experience. I find that when I do TDD, most of the extra time is spent fixing the bugs that cause my unit tests to fail, not writing the tests themselves. That's not really adding extra time, just shifting when you find and fix those issues. Any real extra time is in fixing problems that you wouldn't have found, at least not in the first go-round.
    – JimmyJames
    Commented Jun 15, 2016 at 15:38
  • 7
    @JimmyJames It's a case study, which is used extensively in business, and a lot in science when it's not (yet) possible to run a large scale reproducible experiment. There's psychology journals full of them. "Unscientific" is not the right word.
    – djechlin
    Commented Jun 16, 2016 at 2:12
  • 26
    Why do I think if the outcome of that case study had been showing the opposite, it would not have made it into the book ;-) ?
    – Doc Brown
    Commented Jun 16, 2016 at 10:40
  • 11
    @DocBrown I wonder how many case studies were made and discarded before they found one with the right answers :-)
    – gbjbaanb
    Commented Jun 16, 2016 at 12:13
  • 7
    @djechlin I don't think conflating a statistically valid study and something that you can't even apply statistics to is helpful to humanity. We've got enough crappy unrepeatable research without further muddying the waters by calling the comparison of two uncontrolled situations 'scientific'. The term for this is 'anecdotal evidence'. It can pretty much guarantee I could find two projects that showed very different results simply because of the development teams involved. It wouldn't 'prove' anything.
    – JimmyJames
    Commented Jun 16, 2016 at 16:48
32

There is only one study I know of which studied this in a "real-world setting": Realizing quality improvement through test driven development: results and experiences of four industrial teams. It is expensive to do this in a sensible way, since it basically means you need to develop the same software twice (or ideally even more often) with similar teams, and then throw all but one away.

The results of the study were an increase in development time between 15%–35% (which is nowhere near the 2x figure that often gets quoted by TDD critics) and a decrease in pre-release defect density from 40%–90%(!). Note that all teams had no prior experience with TDD, so one could assume that the increase in time can at least partially attributed to learning, and thus would go down even further over time, but this was not assessed by the study.

Note that this study is about TDD, and your question is about unit testing, which are very different things, but it is the closest I could find.

1
  • 1
    It would be more interesting to impose additional constraints: no mutable state, following SOLID, static typing, no relying on null, functional over imperative, code contracts, static analysis, automated refactoring, no IoC containers (but DI) etc. I bet that value of unit tests would decrease (but not disappear).
    – Den
    Commented Jun 21, 2016 at 13:12
24

Done well, developing with unit tests can be faster even without considering the benefits of extras bugs being caught.

The fact is, I'm not a good enough coder to simply have my code work as soon as it compiles. When I write/modify code, I have to run the code to make sure it does what I thought it does. At one project, this tended to end up looking like:

  1. Modify code
  2. Compile application
  3. Run application
  4. Log into application
  5. Open a window
  6. Select an item from that window to open another window
  7. Set some controls in that window and click a button

And of course, after all that, it usually took a few round trips to actually get it right.

Now, what if I'm using unit tests? Then the process looks more like:

  1. Write a test
  2. Run tests, make sure it fails in the expected way
  3. Write code
  4. Run tests again, see that it passes

This is easier and faster then manually testing the application. I still have to manually run the application (so I don't look silly when I turn in work that doesn't actually work at all), but for the most part I've already worked out the kinks, and I'm just verifying at that point. I actually typically make this loop even tighter by using a program that automatically reruns my tests when I save.

However, this depends on working in a test-friendly code base. Many projects, even those with many tests, make writing tests difficult. But if you work at it, you can have a code base that's easier to test via automated tests than with manual testing. As a bonus, you can keep the automated tests around, and keep running them to prevent regressions.

2
  • 1
    And using something like nCrunch can cut steps 2 and 4, making the feedback loop even tighter.
    – Euphoric
    Commented Jun 15, 2016 at 7:22
  • "I still have to manually run the application" is an important observation, IMHO. No silver bullets.
    – Den
    Commented Jun 21, 2016 at 13:18
23

Despite there being a lot of answers already, they are somewhat repetitive and I would like to take a different tack. Unit tests are valuable, if and only if, they increase business value. Testing for testing's sake (trivial or tautological tests), or to hit some arbitrary metric (like code coverage), is cargo-cult programming.

Tests are costly, not only in the time it takes to write them, but also maintenance. They have to be kept in sync with the code they test or they're worthless. Not to mention the time cost of running them on every change. That's not a deal-breaker (or an excuse for not doing the truly necessary ones), but needs to be factored in to cost-benefit analysis.

So the question to ask when deciding whether or not (or of what kinds) to test a function/method, ask yourself 'what end-user value am I creating/safeguarding with this test?'. If you can't answer that question, off the top of your head, then that test is likely not worth the cost of writing/maintaining. (or you don't understand the problem domain, which is a waaaay bigger problem than a lack of tests).

http://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

16
  • 1
    I'm not super familiar with BDD but will guess that it operates at a slightly coarser granularity than the method/function level and probably has a less tenuous connection to user-value. Commented Jun 15, 2016 at 15:32
  • 1
    Related
    – Robbie Dee
    Commented Jun 15, 2016 at 16:09
  • 10
    "Testing for testing's sake (trivial or tautological tests), or to hit some arbitrary metric (like code coverage), is cargo-cult programming." So true and so well-said. Test in such a way that you feel like a cool badass - think about yourself as a ... spy, elite athlete ... DON'T test like a "government department". You know?
    – Fattie
    Commented Jun 16, 2016 at 12:06
  • 2
    @SteveJessop disagree, code coverage (in the sense of being a metric) is inherently arbitrary: the number of paths through a non-trivial program at the machine instruction level (i.e. the one that counts) will be greater than the number of atoms on Earth or possibly even the visible universe. This is not testable. So any claim of 'code coverage' will be to some heuristically chosen arbitrary threshold. Programmers are good at gaming such metrics at the expense of the things that actually matter. Commented Jun 17, 2016 at 12:35
  • 2
    I would also say that tests provide business value approximately (albeit not precisely) whenever they fail, and the resolution is to improve the code under test. So if you're using TDD that pretty much automatically happens at least one per test. Tautological tests by definition cannot fail and so are useless. For "trivial" tests -- having worked with the Java TCK early in my career, I'm no longer surprised by how trivial of a test it is possible to fail when re-implementing an API from scratch ;-) But business value is almost always predicted heuristically, so is "arbitrary" too. Commented Jun 20, 2016 at 8:32
10

It depends on the person, as well as the complexity and shape of the code you're working with.

For me, on most projects, writing unit tests means I get the work done about 25% faster. Yes, even including the time to write the tests.

Because the fact of the matter is that software isn't done when you write the code. It is done when you ship it to the customer and they're happy with it. Unit tests are by far the most efficient way I know of to catch most bugs, isolate most bugs for debugging, and to gain confidence that the code is good. You have to do those things anyways, so do them well.

6
  • 9
    I think it's worth noting though, it's an acquired skill. I see so many people hear the claim that TDD really isn't even a time-sink upfront that pays itself off in the long run, it's just faster, period. then they try it for a day and it's painful because they have 0 experience, have read 0 books, no practice, they just expect it to magically work. there's no secret to TDD that makes you a better developer, you still need to practice, still need to think, still need to make good educated decisions.
    – sara
    Commented Jun 15, 2016 at 5:48
  • 1
    @kai - +1. I spent weeks reading about TDD before I tried it. I read everything I could find. I read books. I read through all the well-known agile blogs for examples. I read xUnit Test Patterns cover-to-cover. For the first few weeks, it still took me twice as long.
    – Jules
    Commented Jun 15, 2016 at 8:33
  • 2
    I agree. TDD is hard. The mindset is difficult. Anyone who says "Just write the tests first" and claims that it's free doesn't know how to do it. It takes practice.
    – duffymo
    Commented Jun 15, 2016 at 14:03
  • @kai: for similar reasons a lot of people can't touch-type. They tried it once and after a whole hour still weren't typing any faster than before ;-) Commented Jun 17, 2016 at 1:05
  • @SteveJessop I guess that's a pretty neat comparison. Or being really unfit and going out for a 10 minute jog, getting exhausted and wondering why you can't run 10 miles in an hour. it really illustrates how you need to work before you get the benefits.
    – sara
    Commented Jun 17, 2016 at 8:36
4

Some aspects to consider, not mentioned in the other answers.

  • Extra Benefit/Extra Cost depend on experience with writing unittests
    • with my first unit-test project the extra costs trippled because i had to learn a lot and i made a lot of mistakes.
    • after 10 years of experience with tdd i need 25% more coding time to write the tests in advance.
  • with more tdd-moduls there is still need for manual-gui-test and integration-testing
  • tdd only works when done from the beginning.
    • applying tdd to an existing, grown project is expensive/difficuilt. But you can implement regression-tests instead.
  • automated tests (unittests and other kind of tests) require maintanace const to keep them working.
    • having created test through copy&paste can make testcode-maintanace expensive.
    • with growing experience testcode becomes more modular and more easy to maintain.
  • with growing experience you will get the feeling when it is worth to create automated tests and when not.
    • example there is no big benefit to unittest simple getters/setters/wrappers
    • i donot write automated tests via the gui
    • i take care that the businesslayer can be tested

Summary

When starting with tdd it is difficuilt to reach the "more benefit than cost" state as long as you are under "time-constrained work environment" especially if there are "clever managers" that tell you to "get rid of the expensive, useless testing stuff"

Note: with "unit testing" i mean "testing moduls in isolation".

Note: with "regression testing" i mean

  • write some code that produces some output-text.
  • write some "regression testing" code that verifies that the result of the generation ist still the same.
  • the regression test let you know whenever the result changes (which might be ok or an indicator for a new bug)
  • the idea of "regression testing" is similar to approvaltests
    • ... taking a snapshot of the results, and confirming that they have not changed.
1
  • Needs proofreading (the literary equivilent of testing?)
    – JDługosz
    Commented Jun 19, 2016 at 5:10
4

Programmers, like people dealing with most tasks, underestimate how long it actually takes to complete it. With that in mind, spending 10 minutes to write a test can be looked at as time one could have spent writing tons of code when in reality, you would have spent that time coming up with the same function name and parameters you did during the test. This is a TDD scenario.

Not writing tests, is a lot like having a credit card; we tend to spend more or write more code. More code has more bugs.

Instead of deciding to have total code coverage or none at all, I suggest focusing on the critical and complicated part of your application and have tests there. In a banking app, that might be the interest calculation. An engine diagnostic tool may have complex calibration protocols. If you've been working on a project, you probably know what it is and where the bugs are.

Start slowly. Build some fluency before you judge. You can always stop.

4

Question is, how much of a time difference would writing unit-tested code over untested code, and how does that time difference scale as project scope widens?

The problem gets worse as the age of the project increases: because whenever you add new functionality and/or whenever you refactor existing implementation, you ought to retest what's previously be tested to ensure that it still works. So, for a long-lived (multi-year) project, you might need to not only test functionality but re-test it 100 times and more. For this reason you might benefit from having automated tests. However, IMO it's good enough (or even, better) if these are automated system tests, rather than automated unit tests.

A second problem is that bugs can be harder to find and fix if they're not caught early. For example if there's a bug in the system and I know it was working perfectly before you made your latest change, then I'll concentrate my attention on your latest change to see how it might have introduced the bug. But if I don't know that the system was working before you made your latest change (because the system wasn't properly tested before your latest change), then the bug could be anywhere.

The above applies especially to deep code, and less to shallow code e.g. adding new web pages where new pages are unlikely to affect existing pages.

As a result, a considerable amount of bugs escape to production, which I have to fix and in turn sets back my other projects.

In my experience that would be unacceptable, and so you're asking the wrong question. Instead of asking whether tests would make development faster you ought to ask what would make development more bug-free.

A better question might be:

  • Is unit-testing the right kind of testing, which you need to avoid the "considerable amount of bugs" you've been producing?
  • Are there other quality control/improvement mechanisms (apart from unit-testing) to recommend as well or instead?

Learning is a two-stage process: learn to do it well enough, then learn to do that more quickly.

3

There have been a long history of Programmers board promoting TDD and other test methodologies, I won't recall their arguments and agree with them, but here is additional things to consider that should nuance a bit:

  • Testing isn't equally convenient and efficient depending of context. I develop web software, tell me if you have a program to test the whole UI... right now I'm programming excel macros, should I really develop a test module in VBA ?
  • Writing and maintaing the test software is real work that counts in the short run (it pays off in a longer run). Writing relevant tests is also an expertise to get
  • Working as a team and working alone, haven't the same tests requirements because in team you need to validate, understand and communicate code you did not write.

I'd say testing is good, but make sure you test early and test where the gain is.

4
  • 1
    "Should I really develop a test module for VBA?" Damn right you should. rubberduckvba.com/Features#unitTesting
    – RubberDuck
    Commented Jun 16, 2016 at 23:23
  • There is a few reasons I won't use this because it doen't fit my needs (I'm on few days tasks at most, locked environment, successor won't bother 3rd parties). Good comment though, language is not an excuse in itself :)
    – Diane M
    Commented Jun 17, 2016 at 6:25
  • All fair points @ArthurHavlicek.
    – RubberDuck
    Commented Jun 17, 2016 at 10:02
  • 2
    Writing tests is still trivial in VBA. Having all the fancy features that some unittest frameworks have? That's harder, but running a program called mainTest() that calls all your test modules isn't really that hard.
    – enderland
    Commented Jun 17, 2016 at 17:20
2

An oft overlooked benefit of TDD is that the tests act as a safeguard to make sure you aren't introducing new bugs when you make a change.

The TDD approach is undoubtedly more time consuming initially but the takeaway point is you'll write less code which means less things to go wrong. All those bells and whistles you often include as a matter of course won't make it into the code base.

There's a scene in the film Swordfish where if memory serves, a hacker is having to work with a gun to his head and being erm... otherwise distracted. The point is it is a lot easier to work when your headspace is in the code and you have time on your side rather than months down the line with a customer screaming at you and other priorities getting squeezed.

Developers understand that fixing bugs later is more costly, but flip that on it's head. If you could be paid $500 a day to code how you code now or $1000 if you wrote in a TDD way, you'd bite the hand off the person making you the 2nd offer. The sooner you stop seeing testing as a chore and see it as a money saver, the better off you'll be.

1
  • That thing in your first sentence is called Regression testing
    – cat
    Commented Jun 15, 2016 at 11:06
2

I can relate to your expierience - our code base had almost no tests and was mostly untestable. It took literally ages to develop something and fixing productions bugs took precious time from new features.

For a partial rewrite, I vowed to write tests for all core functionality. At the beginning, it took considerably longer and my productivity suffered noticeably, but afterwards my productivity was better than ever before.

Part of that improvement was that I had fewer production bugs which in turn led to fewer interruptions -> I had a better focus at any given time.

Furthermore, the abillity to test AND debug code in isolation really pays of - a suite of tests is vastly superior to a system which cannot be debugged except with manual setup, e. g. launching your app and navigate to the screen and do something...perhaps a few dozen times

But notice that there is a drop in productivity at the beginning, so start to learn testing on some project where the time pressure is not already insane. Also, try to start it on a greenfield project, unit testing legacy code is very hard, and it helps when you know how a good test suite looks like.

1

Just to complement previous answers: remember that testing is not a purpose itself. The purpose of making tests is for your application to behave as expected through evolution, within unexpected contexts, etc.

Therefore, writing tests does not mean to prove all behaviors of all endpoints of an entity. This is a common error. A lot of developers think they need to test all functions/objects/methods/properties/etc. This leads to a high workload and a bunch of irrelevant code and tests. This approach is common in big projects, where most developers are not aware of the holistic behavior, but can only see their domain of interaction.

The right approach when dealing with sparse resources and testing is pretty obvious and of common sense, but not commonly formalized: invest testing development resources first on high-level functionalities, and gradually descend into specificities. This means that at some point, as a lonely developer, you would not only focus on unit-testing, but on functional/integration/etc. testing and depending on your time resources, gradually into the main unitary functions, as you would plan and consider. High-level testing will provide the necessary information to address low-level/unitary testing and to plan your testing development strategy according to the resources you have.

For example, you would like to test a processing chain first as a black-box. If you find that some member of the chain fails due to the behavior didn't considered some extreme condition, you write the tests that guarantee the functionality not only on this member but also on others. Then you deliver. For the next cycle, you detect that sometimes the network fails. So you write tests that address such issue on the modules that could be vulnerable. And so on.

0

The answer to the question is: Unit testing means less development time.

That's the wrong question of course.

Also, an answer to this question really requires a "how do you get to a point where unit testing is giving me benefits" - I cover this in the "my story" portion of my answer.

Here's the real answer:

Unit testing creates a shorter development cycle, it eases refactoring allowing better reuse of existing software, it allows unweildy legacy codebases to be brought under control, and it enables the fast development of high quality software.

It should be a requirement of all investors in public companies that all software in which their company invests time and money that the software include unit tests because it improves the quality of those investments which translates to greater profitability.

Stack exchange wanted to know if I am contributing something new to the conversation.

yes I am.

Early testing - which can only mean unit testing - solves many of the problems that plague managers of software, far beyond the tests themselves.

The excuse that "we have a final test where we check the whole system" is inexcusable and I would fire anyone who worked for me who said that. It's mathematically provable (not here though) that that statement is objectively negligent.

There are studies of software development showing that a bug that makes it to production costs something like 20 times what a bug caught in design costs.


So my story

I worked on a code base that was millions of LOC.

It was well organized, thankfully.

But it had...zero automated tests but lots of manual ones.

Problem: The knowledge of how to do the manual tests was in the heads of engineers, and engineers leave and retire and die. You can't have your IP in the head of an engineer, but SO MANY organizations do.

Problem: There were features of this system that were not commonly used, so it took a while to remember how they were supposed to behave.

Problem: The path for data from acquisition to presentation to end user was convoluted and there were multiple binaries making unpredictable changes to the data, based on other external inputs controlled by other entities.

Problem: The entire system consisting of some 10s of binaries - maybe 40 - had to be run in order to exercise any portion of the system, due to the required interactions.

Problem: Every change to the system required a day of testing. Our dev cycle made us afraid to change anything that wasn't absolutely necessary. Consequently we didn't remove dead code, we didn't improve and refactor, and we built up technical debt to unsustainable levels.

Problem: It was considered impossible to build automated tests. Those who believed this will be proven absolutely wrong by the facts.


Time to problem solve.

Problem: The entire system consisting of some 10s of binaries - maybe 40 - had to be run in order to exercise any portion of the system, due to the required interactions. This sort of test took up to a day due to the nature of the processing we were doing.

Problem: We have so much to do, how can we take the time to set up the testing environment?

I spent maybe 16 hours researching google test, I downloaded the google test gtest library, it took about 2 hours to figure out how to install it and write my first test and have it included in our build system. Now I was able to test functions without running the whole system.

2 Problems solved. 18 hours.

So the belief that it was "impossible" was wrong. It was possible, if you didn't incorrectly constrain the question to "is it possible to get every single function under a unit test regime".

I just said, "of course it's possible" and I did it.

Then I found places where not only could I write tests, I could use the library to debug problems and speed up my productivity immediately. I had 2 hours back within the day, and in fact by the end of that week I had recovered all 18 hours invested and was up about an hour in productivity, and every week I added 18 hours to that.

Multiply that by my hourly rate and that's how much money I saved the company.

Now I was able to debug more quickly - instead of putting in logging and running the whole system and looking at log files, I just had to write a 10-line test and let the framework handle the details. The test ran in 2 minutes.

Problem: I have existing functions, this isn't greenfield development - if I have to change an existing function how do I make sure I didn't break it?

Problem: Every change to the system requires a day of testing. Our dev cycle makes us afraid to change anything that's not absolutely necessary. Consequently we don't remove dead code, we don't improve and refactor, and we are building up technical debt to unsustainable levels.

With tests I was able to refactor more confidently - I just wrote tests for the existing function, refactored it, and made sure the refactored function passed the tests. The test runs at compile time, I remove all dead code, i improve and refactor at will, and technical debt never has a chance to build up.

2 Problems solved. Hundreds of thousands of dollars saved.

Problem: How do I test complex functions?

Once I had most of my "leaf-node" functions under test (the ones that didn't require much refactoring to test them and that did not use other functions) I looked at the more complex ones. I picked a library that mostly depended on my leaf-node functions, wrote as many tests as I could for it as is, and then refactored and manually tested functions that were not easily automated. I used mocking and stubbing wherever it helped, and soon I had a complex library refactored, simplified, and testable.

Problem solved.

Problem: How do you get most of the codebase under test?

I continued working through the codebase like that, making sure each investment in test effort returned a benefit immediately.

Problem solved.

Problem:

The knowledge of how to do the manual tests was in the heads of engineers.

Problem: There were features that were not commonly used, so it took a while to remember how they were supposed to behave.

Now I was able to reproduce tests accurately, and for years to come those tests will be available for others to use. The knowledge of the expected behavior is out of the engineers' heads and is now in written code in a repository. The tests themselves were a few lines of code, so I could put in lots of documentation about how the tests worked.

2 Problems solved.

My conclusion:

Testing (with a good library like gtest) Does not cost anything or take more time. It is not only free, it has a huge roi on that non-investment.
It costs more and is slower to not unit test.

This is a solved problem folks, let's get to the hard ones please.

2
  • 1
    I understand you find some things important, but using CAPITALS is the same as SCREAMING. Please use boldface font instead.
    – Dominique
    Commented Feb 5 at 10:12
  • Thank you for your comment, i have changed the post. Though to be honest, most of the time when I talk about this topic i feel like screaming in the face of a dev community that just doesn't seem to get it. Commented Jun 6 at 20:33

Not the answer you're looking for? Browse other questions tagged or ask your own question.