Should programmers help testers in designing tests?

Question

How much should programmers help testers in designing tests?

I don't think they should help at all. My worry is that if they help testers in designing tests for their own code, they'll 'infect' the testers with their own prejudices and blind spots about that code.

I feel that the requirements should be sufficient for giving the information needed for testers to create their tests. If there is some part of the implementation that the programmers find worrisome, then I think it's their duty to implement unit tests to test that part or even run their own informal system tests to test that part.

Not everyone I know agrees with this though (and I understand some of their points to a certain extent). What do others think about this? Is this discussed in the literature anywhere?

Answer 1

I agree. Programmers can help the testers to understand the functional specs, to find resources for research but should not pollute the testers' minds with their own ideas about how to approach testing.

Answer 2

I think there's room for developers and testers to coexist peacefully in the realm of QA. :)

More specifically, I think developers should be responsible for the first level of testing -- unit tests and basic integration tests to make sure that their stuff works in most cases before they hand it off to the testers.

It's up to the testers to create acceptance tests based on requirements that are completely agnostic of any implementation details (this is typically referred to as 'black box testing'). If there's a discrepancy in how testers and developers understand the requirements, it's a problem that should be addressed either by the project manager (if any) or by making sure everyone is on the same page back in the design phase of the feature.

Answer 3

I think both testing and development are collaborative efforts, so of course (IMO), devs should give test ideas to testers. I don't think it infects them or taints testers at all. The tester, of course, should enhance those tests with many other testing approaches.

I'm also a big fan of testers helping developers - I've brainstormed design ideas with developers and paired with them to fix bugs (and pointed out the bugs and suggested fixes) many times in my career, and don't think I've ever tainted a developer with tester cooties.

If you don't see it as a collaborative effort, you'll just have code "thrown over the wall" from dev to test...and you'll end up with lower quality. That's the truth in my world, but (of course), ymmv.

Answer 4

The way I see it is that it's not QA's job to test my code. The tester's job is to make sure my code fulfills all the requirements for that task.

When I pass something to QA, I make sure they know the task I was doing, not the specifics of my code. I never pass anything to QA that has 'bone head' bugs in it. That wastes my time, their time...and pretty much everyone's time.

At my last job, we had QA involved from the beginning. That sat in the requirements gathering sessions, the project meetings, and the design meetings as well. They listened and asked questions and while the developers were writing code, they were writing their test plans. It worked out great and we caught a lot of issues that probably would have slipped through.

Answer 5

I think you're quite wrongheaded here. I've been a tester and a developer, and have greatly benefited as a tester from guidance by developers on areas that they considered high-risk or fragile; as a developer, I want testers to find the problems I haven't deeply probed.

There was no "pollution" unless your code is raw sewage, and that would be for a completely different reason.

Requirements do a terrible job of communicating the technical issues that a QA professional would care about, because they elaborate at best only what business analysts have managed to capture. Good developers will be aware that their code is optimized around the "happy path", and will want to know what they've left unconsidered. They'll at least have an intuition of what could go wrong, and what areas they'd like QA to probe. They also know what the big picture is for risk around a particular feature based on their design.

As a tester absent guidance from the development team, I've sometimes gone off on a wrongheaded approach that generated good bug reports, but didn't completely exercise the high-risk code paths and bigger problems, which could have been avoided through better collaboration with the development team, shipped to customers.

While a tester certainly shouldn't limit themselves to testing just what the developer says is important, they won't be damaged by learning what the developers own concerns about the code are. Sometimes, they can fine tune their approach based on their knowledge of the implementation. Only if a tester is particularly short-sighted will they consider the developer's opinion about what the risks are as the final word; they won't completely shut out things that the developer identifies as low risk, but they'll invest more effort in things that could have a higher customer impact.

The QA team is likely to see areas that have big combinatorial test scope than the requirements gatherers or developers of a system, but they may not be aware of components of the system that have a more subtle kind of fragility that benefits from awareness of the design or implementation of the system.

In my experience, collaboration between QA and Development produces better quality products. I would never recommend doing only a black box handoff.

Answer 6

As a tester, I have no objection at all to programmers suggesting test cases (though that doesn't mean I'll stick to only those test cases), or describing implementation details. Sometimes it can be really helpful to have someone say "I think this bit might be risky, I'd really like it if you tested this bit pretty thoroughly". Knowing some of the implementation details helps me to apply years of experience to choose the tests I think most likely to fail. Sometimes just a brief mention means a few tests suddenly zoom right up my priority list.

Does it taint me? I'm kinda tickled by the idea of programmers chivalrously striving to preserve my tester purity, but really - no, this is a myth. More information usually triggers even more questions for me, not fewer. I guess it's a mindset thing - I don't find bugs because I'm ignorant, I find issues because I'm a skeptical, untrusting type who just is too darn stubborn to wholly take anything on trust. On every system I've tested, I've found that I find more issues, and more "interesting" ones, the more deeply I come to understand it.

Answer 7

I like to review the test plans and suggest additional test cases that QA might not have thought of. I'm not sure how that would "infect the testers with my own prejudices".

Answer 8

I found myself in this weird position that I need to implement and write test cases in Selenium afterward since we're short on QA staff. I believe test-driven development would be extremely helpful but it's not adapted in my shop yet.

One thing I find writing tests helpful is that I find bugs when I write tests. I think in different perspective to help me write more robust code. But it's true that the test coverage could be limited. In this case, QAs can always let us know what they would like to be covered. Or, we can passively add more tests when we see errors.

Answer 9

I'm doing QA, and unlike most domains knowing how to use our code is far more difficult than learning any programming lanquage. So we count on developers to give us test cases for their brand new whizzbang features, because we wouldn't know how to. In any case the QA problems are more to catch regressions and things that get broken than original testing of new features. In any case when the result is a complex computation, its hard to know what is a correct answer and what is a wrong answer, or even if an abnormal termination is a good or bad thing.

In any case, a developer, if he's honest probably knows something of his babies vulnerabilities. He probably knows at what parameter values, he has to select different algorithms, or domains in a table lookup or whatever. Knowing that, if he is sincere about rigorous testing, he should be able to generate a reasonable sized suite of tests that covers a lot of the code.