How can I justify software testing to management?

Question

I work for a small company (less than 200 employees) whose software group only makes up a small part of our staff (4 employees, occasionally with a few contractors). The four of us have been making strides in transitioning to better practices, and one of the next logical steps is to improve our testing.

As anyone who has done any meaningful tests knows, testing takes a lot of time - and at my company, it takes too much time to justify to management, so we generally do what little we do on the sly. I don't think this is serving us well, as we keep coming up against otherwise avoidable problems when we ship under-tested software.

I would like to be able to come to management with a justification for hiring a dedicated software test engineer (someone who can both write automated tests and perform manual ones). Are there any good published studies that show the benefits of adding such a position to a small company? Where can I find information about costs associated with the position? I plan on doing a little number crunching on our own history, but having some external sources to point to would help bolster my case.

Answer 1

I believe that you should convince your management not by using numbers, but by explaining the rational.

Joel Spolsky makes some very smart points on the benefits of hiring testers in his article Top Five (Wrong) Reasons You Don't Have Testers, that may help convincing management.

Also here:

If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products, or you're wasting money by having $100/hour programmers do work that can be done by $30/hour testers. Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't recognize it.

Some more pearls of wisdom here:

There is no better way to improve a programmer’s morale, happiness, and subjective sense of well-being than to have dedicated testers who get frequent releases from the developers, try them out, and give negative and positive feedback. Otherwise it’s depressing to be a programmer. Here I am, typing away, writing all this awesome code, and nobody cares. Boo hoo.

Answer 2

Don't waste time on number crunching. It's not compelling. Everyone disputes the details.

Instead, focus on specific risk scenarios.

Poor (or no) testing == failure in production == loss of revenue (or worse). Follow this line of reasoning with specific losses that can accrue from software failures.

Be detailed and focused so that the logic of "loss comes from bad software comes from bad testing" is inescapable.

What happens is -- of course -- focus on the specific risk scenarios. But that's better than no focus at all.

Very, very specific stories are easier to use than general studies and summarized numbers. It's even better when you can tie historical events, specific customers or products, or other factual details to your risk scenario.

"Remember when we spent 18 hours trying to fix the [X] product software? How much revenue did we lose from cancelled orders? And help desk calls?"

Answer 3

If you don't test enough, you lose profit to pay to fix buggy software in the field. If you test a lot and deliver late, you lose revenues because software was delivered late.

From a business leader's point of view, the former is usually preferable because delivered software is actually getting used, was probably paid for, and it is therefore easier to justify bug-fixing expenses after it was deployed and paid for.

If you want better testing, you need to show how testing will help the business leader get the software to market faster, within the same budget. In a large-ish software project, time-to market is enhanced when test coverage is good at the lowest layers of the system (e.g. OS, data layer) because that means that the folks developing on top of that don't have to fix their bugs and the lower layer bugs. It also helps to ensure good testing of success paths of key use cases.

You may want to consider outsourcing some testing to a shop in India or Vietnam. They do good work, and their loaded labour rates are much less than those in N. America. And they'll test while you sleep.

Answer 4

It sounds like you have a critical first step:

As anyone who has done any meaningful tests knows, testing takes a lot of time - and at my company, it takes too much time to justify to management, so we generally do what little we do on the sly. I don't think this is serving us well, as we keep coming up against otherwise avoidable problems when we ship under-tested software.

It sounds like you need to make the business case for testing at all and embedding into the schedule before you do anything else.

Here's how I did something similar:

1. Projects were using classic Big Bang testing. There are plenty of resources showing that this method is the least reliable, most costly and (therefore) most likely to be cut for the purposes of "savings."

2. In order to "hide" testing inside the project (rather than at the end), I dedicated every fourth week to testing and planning. This means that no developer was ever more than three weeks from something that they'd been working on. It also meant that every task was tested more than zero.

3. Then I made the business case to the project managers. My justification was that (a) people have to test just to know if they've actually finished their tasking and (b) with early testing, I could easily claim a net cost savings of at least 50% savings.

That last point was a bit dramatic for some people. Considering that post-release bug fixes are considered to be at least ten times more expensive to find and fix, it was a fairly easy argument to win.

The four of us have been making strides in transitioning to better practices, and one of the next logical steps is to improve our testing.

I agree but I don't agree that the first step is to hire. You're talking about an increase in your group's salary budget of 20% without first demonstrating the return on investment. Invest in the testing capabilities and resources of the people that you have now. Demonstrate the value of testing as a core development practice. Then start working on the business case for a new hire when you have the history to back you up.