Is context-driven testing suitable for all projects?

Question

One thing I cannot find anywhere is: is this approach limited to non-critical software? Because in critical (or just when customer wants us to follow his rules) we have no freedom in testing and just follow the procedures. Am I right?

Answer 1

It depends on what you mean by "freedom."

Testing isn't necessarily about "freedom." Testing is about finding out if the software does what it is supposed to do. In most "contexts," that means determining if the software meets the customer's requirements. So any testing that occurs will be "driven" by that goal, and any testing that doesn't advance that goal is a waste of time. This is true regardless of the testing methodology used.

In some testing methodologies like TDD, the tests themselves are part of the requirements. Each test identifies a requirement, and the passing of that test verifies that the requirement is fulfilled.

So let's look specifically at some of the things that Context-Driven Testing advocates:

1. Testing is done on behalf of stakeholders in the service of developing, qualifying, debugging, investigating, or selling a product.

2. Test artifacts are worthwhile [only] to the degree that they satisfy their stakeholders’ relevant requirements.

3. The value of any [testing] practice depends on its context.

Consider two projects:

• One is developing the control software for an airplane. What “correct behavior” means is a highly technical and mathematical subject. FAA regulations must be followed. Anything you do — or don’t do — would be evidence in a lawsuit 20 years from now. The development staff share an engineering culture that values caution, precision, repeatability, and double-checking everyone’s work.

• Another project is developing a word processor that is to be used over the web. “Correct behavior” is whatever woos a vast and inarticulate audience of Microsoft Word users over to your software. There are no regulatory requirements that matter (other than those governing public stock offerings). Time to market matters — 20 months from now, it will all be over, for good or ill. The development staff decidedly do not come from an engineering culture, and attempts to talk in a way normal for the first culture will cause them to refer to you as “damage to be routed around”.

Testing practices appropriate to the first project will fail in the second.

Practices appropriate to the second project would be criminally negligent in the first.

As you can see, Context-Driven Testing adapts itself to the project, not the other way around. But this should really come as no surprise, since all software testing is (or should be) about proving that customer expectations are met, not about conforming oneself to some arbitrary methodology just because.

Context-Driven Testing is less about a methodology, and more about some guidelines and principles that drive all of testing. It basically says that your testing methodologies, your rigor, your testing culture, are all based on the nature, character and culture of the software project and its participants.

In that sense, consider it the "freedom" philosophy. You are free to adapt your testing methods to best fit the needs of the software project under test.