Do testers have to look at source code?

Question

How necessary is it for testers to do white-box testing in addition to black-box testing? As a developer, I see value in having a QA-minded person go through my code and look for weak points, but it also seems like if a non-technical tester is thorough and methodical enough, they can cover the application well.

White box testing may reveal important test cases that wouldn't be obvious while looking only at the application from a user's perspective, but exclusively black-box testing might take less time, and significantly increases the number of people who are capable of doing the job. Also, a nontrivial amount of white box testing should be performed by the developer before it ever gets to the tester, am I right?

Answer 1

White box and black box testing are two different skills - someone skilled at one isn't necessarily skilled at the other.

A lot of what QA does will be black box testing - making sure that the system works "as advertised" and in this regard they don't need to know how the system works in order to do a good job. In fact knowledge of the code may lead them into the same traps that catch the unwary developer doing their own testing. It's far better that they have no preconceptions of how the developer thinks the system is supposed to work.

That's not to say that there shouldn't be white box testing - which is (partly) what developer unit testing is. A good suite of unit tests should be testing valid inputs, invalid inputs and edge case inputs. However, I don't think that it's always necessary for QA to do this - if for no other reason than it is really the developer's responsibility.

However, as SnOfus points out in his comments good white box testing can be extremely useful.

Answer 2

I've had good luck on my team with tester code reviews. Testers tend to look at code the same way they look at applications - they ask themselves, "in what ways would this code not work", and often find issues that programmers don't find during code review.

For a longer answer, I wrote this paper.

Answer 3

Absolutely

Like I said in response to @ChrisF: QA is there to make sure that production code works always. Without being exhaustive, that means knowing what platforms are expected, what the state of the deployed machine will be, what performance requirements are like and being certain that every line is going to succeed.

IME, a good QA team will write, read and review more code than the dev team does.

If your feature teams and (major) bugfix teams don't have a QA developer in them working to design the API, workflow, security and all of the other aspects of the feature/fix, then you're really missing out on a strength that your QA department can bring to your product while it's being developed.

Answer 4

It isn't necessary at all. Rather then having a QA-minded person go through your code, I recommend having a developer go through your code. In addition, if you are writing unit tests as you develop your code, having a QA-minded person go through you code isn't the best use of the QA's time.

The QA person shouldn't be focused on HOW the code was written, but WHAT the code does.

The QA person should be focusing no functional testing and leverage the requirements to do those tests. In the end, unit tests and a QA functional tester doing black box testing will be the best use of everyones time.