5

I work in a small contracting firm doing small custom projects. We have this continual problem where the customer wants to be responsible for testing the final build to save some money. Most of the projects are reading from a file from one system and creating orders or whatever in another system. So the customer figures they can run a few files through and make sure the orders come out ok.

However, as you might already predict, they never do a very good job. Inevitably, the order they see created ends up not looking quite right on the report, or is missing some flag that keeps it being converted to a job, or whatever. Then I get blamed because "the software doesn't work". Of course, it does work. It works really great. It catches errors and writes logs, and does everything it was asked to do. It is just that they missed a requirement, or they missed it missing, because they didn't do a thorough test.

Help me out here. I would like to take over testing in order to be thorough.

What is this called, integration testing? And how do I work with the customer to make sure every scenario is being covered? I need to find a way to draw out every possible scenario from them so that I can hand them the final build and say, (1) this software works and (2) this software works according to YOUR specification.

  • 1
    "problem where the customer wants to be responsible for testing" - honestly, IMHO one has a much bigger problem when the customer wants not to be responsible for any kind of testing. – Doc Brown Jun 9 '17 at 6:27
  • 1
    There are a whole harem of conflicts of interest here that would make this a bad idea. You might as well offer a 5% discount for every bug they find. – Neil Jun 9 '17 at 14:24
6

We have this continual problem where the customer wants to be responsible for testing the final build to save some money

I would suggest for future clients that you don't allow them to be the exclusive tester of the software. Typically clients don't have dedicated QA personel, and they don't have the qualifications or experience to adequately test software.

For small projects, it's often useful to create a specification that describes the features of the system. Along with the specification you should create a suite of "acceptance tests". These are end-to-end tests of the system that show it's working correctly. The client should sign off on both the specification and the acceptance tests.

The only tests the client should be responsible for running are the acceptance tests so that you can show the software is working as agreed. You should also run them yourself prior to delivering the system.

Help me out here. I would like to take over testing in order to be thorough.

It's a bit tough with your current project, because it sounds like you've stipulated in the contract somehow that you won't be charging the client for testing. It would still be valuable to write acceptance test cases together with your client so that you can agree about what a working system looks like. It also gives you a documented way to charge the client for changes of requirements.

Regardless of whether you can bill the client for testing, I would still suggest you write unit tests and automated integration tests and acceptance tests. Perhaps you cannot bill this client for that work, but it's still worthwhile to do because it'll increase your chances of the project being a success, and will give your client reason to ask for more of your business. After this project you can make the policy change I suggested above.

The other option is you try to negotiate a change to the contract so that you can be paid for testing your software. It sounds like you have evidence to justify this change, and the client may be amenable to it.

3

You wrote your software works fine on the technical level, but you have the problem of missed requirements, and you try to solve this by taking the responsibility for tests away from the customer? Really? To me this sounds like you currently do not get enough customer feedback and try to solve this by reducing the customer feedback even more, right?

Sorry, but this can obviously not work.

Here is a better approach: give the customer more responsibility, focussing on acceptance testing. Write some things about the amount of testing they need to do into the contract, demand a protocol for each test, and make it clear once they accepted a feature by not complaining about it in the protocol, they will have to pay you an additional fee when they want a change afterwards. The contract can also contain something about the customer providing one or more persons who do the testing regularly and on time, in defined cycles.

You can be sure, if doing the tests sloppy will cost your customer money, they will take it more seriously.

1

I take it you do not have the setup to do the final test, which would be called an integration test. If it is not easy to get a copy of that system at your site for whatever reason, you may want to talk about having a contact you can toss versions to on a daily basis and have them do tests more frequently, early in the development cycle. They will feel more involved and responsible and whatever bugs are found will not have such an impact because you are still working on it, it is not supposed to work yet. Once it is time to deliver for real, everyone should be confident it is OK and there will not be any surprises.

1

Just because the customer wants to perform their own testing, does not mean it alleviates you from doing your own acceptance testing before issuing a release.

Your company needs a process and part of that process is that you don't release software to the customer unless it has gone through internal acceptance testing. That acceptance testing verifies that every customer requirement is met.

There is no way for the customer to take over the acceptance testing responsibility without creating significant impacts to the development team and not requiring several iterations of releases as bugs are discovered and fixed during the acceptance testing.

Thus, the customer can't simply exclude it from the contract in order to avoid the costs. The costs are still going to be there (one way or the other) for the developers. At best, there's probably some book-keeping and testing environment costs the customer might be able to save on.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.