Looking for Case Studies of How TDD Improved Quality and/or Speed of Development [closed]

Question

At my company I am trying to make a case for why we should be doing TDD. Currently most developers just do whatever they can to get the project done, then go add unit tests after the fact in order to meet manager metrics. Any examples from reputable companies doing TDD and seeing the benefits would be greatly appreciated.

Answer 1

A study of 4 projects in IBM and Microsoft. Published in Emperical Software Engineering journal.

Empirical Studies Show Test Driven Development Improves Quality

A paper first published in the Empirical Software Engineering journal reports: "TDD seems to be applicable in various domains and can significantly reduce the defect density of developed software without significant productivity reduction of the development team." The study compared 4 projects, at Microsoft and IBM that used TDD with similar projects that did not use TDD...

The paper includes 1 case study at IBM and 3 from Microsoft. Each of the case studies compare two teams working on the same product, using the same development languages and technologies, under the same higher-level manager, only one of which was using test-driven development (TDD). None of the teams knew that they would be part of the study during their development cycles. The IBM case study followed teams doing device driver development. The Microsoft cases followed teams working on Windows, MSN, and Visual Studio.

The paper describes the TDD practices used by the teams as minute-to-minute workflows, as well as task-level workflows...

Answer 2

There is a chapter about TDD with a case study in the recent book, "Making Software: What works and why we believe it". But you may be disappointed, since if I recall correctly the study did not uncover any real benefits to TDD. The case study was interesting anyway, and the book in general is one of the best software books I'v read recently. It contains many case studies of things like pair programming, code review, etc.

Answer 3

Definitely check out this: TDD Proven Effective! Or is it?

...when Phil Haack announced that Research Supports the Effectiveness of TDD I was more than a little interested in seeing what the linked report actually contained. Phil quotes from the abstract.

We found that test-first students on average wrote more tests and, in turn, students who wrote more tests tended to be more productive. We also observed that the minimum quality increased linearly with the number of programmer tests, independent of the development strategy employed.

Phil has obviously read the rest of the report and provides his favorite pieces that seem to do as his title suggests. One of the things I worry about when I see things supporting the latest and greatest software development practices, however, is a strong tendency towards confirmation bias — of looking for confirmation of current theories and overlooking counter-indicators.

So, being the curious type and since TDD is something I'm keeping an eye on to see if its something I might want to adopt myself some day, I went into the report...

...without question, testing first leads to having more tests per functional unit. The question is if this is valuable. This study would seem to indicate that this is probably not the case, at least if quality is your intended gain. But then, I'm not that surprised that number of tests doesn't correspond to quality just as I'm not surprised that the number of lines of code doesn't correspond to productivity.

The author has a lot of good points about TDD not being all that effective (imo despite being hyped to death)

Answer 4

look at how much time you and the client spent manually testing the software; compare that to an estimate of how long TDD-style automated tests would have taken. Pocket the difference

in my experience, TDD's automated tests are gold because they provide insurance and eliminate enormous amounts of manual testing

as Andres F pointed out, you can get these benefits merely from automated testing, not necessarily TDD - however, TDD requires automated tests instead of it being an afterthought or nice-to-have

Being forced to think about testing first also forces you to think about quality-related issues - such as modularity, interface design, and so on - before you start writing code.

Personally, I believe one of the biggest benefits of TDD is that writing the test first keeps the specification of what the code actually has to do fresh in your mind while you are writing the code, rather than sort-of-figuring-it-out-as-you-code.

Answer 5

you want to make the case for it: suggest you do it for the next project, and then learn from it. If it turns out it works great for you, then I hope you'll continue to use it and if it took longer to do the project and/or spend all your time writing tests instead of coding, then you'll surely dump it as a failure.

I think the real-world solution is (like most things) a mid-way, you want tests but you don't want the tests to be more important than the project.

(personally I think TDD is a fad, sounds good in theory, but in practice... not so good. I find integration testing is far more important, but that could be just the kind of complex projects I work on).

Answer 6

I have been working using TDD for 2 years and where I worked at the time ,we were all reluctant to use including managers.However it soon turned to be the right thing todo.The benefits that we soon noticed were

• Discovering bugs at an early stage.
• Writing Better code without even realising.
• Your code is now more maintainable as due to your testing is all in small chunks (we hadfunctions that were 300-400lines)silly. Now max 30 and all indenpendently tested.

The managers would not know as they are all interested in one thing "Have you finished". But then they complain when the software keeps breaking without realising. With a good coverage and sensible tests.It's not the quantity but quality you can really see when somebody breaks a functionality. Also unfortunately it's difficult if you are on your own.I had the same problem,as you might need to change code eg base classes etc so that you can make parts of the software testable.

I give you an example.I wanted to mock the repository but there was no interface and i need to inject the repository into my service layer and therefore add/modify a constructor all over the shop,this turned out to be a big deal but in the end i have more than 200 tests just testing one area of the system and they were impressed.

I usually do the following:

• I keep my unittests very short
• Only 1 assert .No russian roulette.
• I test a positive -negative and exception scenario

Regarding case studies I am afraid ,I am not sure I have seen any. You need to build your project and become your case studies.They might be impressed too.

I hope it helps