Do test-driven and behavior-driven development belong to the "verification" category?

Question

In university, we were introduced to the two terms "verification" and "validation". The definitions can be summarized as follows:

• Validation checks that the specifications and requirements that were defined fulfill the customer's actual needs. (Do we build the right product?)
• Verification checks that the software that is being built matches the specifications and requirements that were defined. (Do we build the product right?)

I stumbled across the terms test-driven development and behavior-driven development. While reading into them, I tried to connect these two to the knowledge that I mentioned above.

Is it a valid generalization to say that TDD and BDD are classified as "verification"?

Because my understanding is that these two approaches focus on checking the software rather than the requirements. Or are there any aspects I missed that speak against such a strict categorization?

Answer 1

Is it a valid generalization to say that TDD and BDD are classified as "verification"?

No. TDD and BDD incorporate verification procedures into the development process in a specific way, but they are much more than just that, so it is not correct to classify TDD and BDD themselves as verification.

Nor either are TDD and BDD validation procedures, but one of the things that differentiates BDD from TDD is a focus on pushing validation-like things up in the development timeline, and treating them at least to some extent with the same approaches that serve verification-like purposes.

Verification and validation are different facets of quality control. They are not categories of development methodologies. A software development effort will generally involve both, regardless of the particular development methodology employed. Verification is usually more tightly integrated with day-to-day development efforts, but if a flaw is discovered during validation then there is every reason to think that that will be fed back into the development process too, probably provoking additions or updates to the verifications being performed.

Answer 2

Is it a valid generalization to say that TDD and BDD are classified as "verification"?

Yes. All testing techniques that take the customers specification as input (and therefor as the truth) fall in the category of Verification.

Validation can only be done in a User Acceptance Test or something similar, where the actual customer (and preferably, the end-user) can see the product in action and/or interact with it. Only the customer can tell you if you built the right product.

Answer 3

Tests are a verification activity: they demonstrate that the software system exhibits certain behaviour.

However, the act of creating test cases can sometimes have validation-like aspects. TDD and BDD are not primarily about testing the software, but also about finding out what to build. They are a technique for discovering requirements, and the test suite becomes an executable description of the requirements.

For example, the TDD red-green-refactor cycle starts by us thinking about a new atomic requirement for a piece of software, then writing a test case for that requirement, and then implementing it. Strictly speaking, this doesn't allow us to make any statements about the requirements themselves. However, the act of writing the test case forces us to imagine how we would be able to use the software system if it satisfied that requirement. This can sometimes show that some functionality is unsuitable, e.g. that it is redundant, or that it conflicts with other aspects of the design.

The connection to requirements is more apparent with BDD practices, since BDD is targeted at capturing business-level requirements. For instance, describing example scenarios not in code but in semi-formal Gherkin syntax can facilitate a conversation between developers and subject matter experts, so that the SMEs can point out when the scenarios are missing relevant details.

Answer 4

Bart van Ingen Schenau's answer is correct as far as I'm concerned, but I wanted to add this as maybe an easier way to remember the distinction.

In the beginning of the process of having an application built, there is a person/team/company with a very specific need that needs to be filled (who I will call the customer). They are the best source of stating the requirements. They know what they need. However, they don't know how to find a solution for their need.

At the other end of the line, there is the software developer. They are the best source of solutions to problems. However, they don't know which specific problem currently is in need of a solution.

In some way, the customer needs to convey the specific problem to the developer, so that the developer can then find the right solution and report it back to the customer. There may be several layers of analysts between customer and developer, or the customer could be talking to the developer directly. In either case, there is room for misinterpretation of the requirements between customer and developer, and that's the main thing to remember here.

Using an example of miscommunication, let's say that the customer needs a tool to send marketing emails to many people at once, and the developer misunderstood and thought that they needed a tool to send many emails to one person.

TDD and BDD are something that developers do. They write the tests. Therefore; the developer is going to write tests to confirm their own understanding of what the problem is: does the application send many emails to the same person?

If they wrote their software correctly, the tests will confirm that the application indeed sends many emails to the same person. The verification has succeeded.
The developer will demo the application to the customer, who immediately sees that this tool does not send emails to many people at once, which is what they had asked for, and therefore the validation has failed.

The only way in which the tests written in BDD/TDD could be part of the validation, not verification, is if the customer is the one writing the test suite.

I've seen this happen once, where a software company hired another company to create a machine-learned algorithm. The customer company had gone through the effort of quantifying their requirements in a set of easy to run tests, therefore suggesting the developer company to use some kind of test-oriented development while also pretty much removing any room for misinterpretation of the requirements (barring silly things like building software that is hardcoded to only handle the provided test cases).

Answer 5

TL;DR: Who cares? I would call it "testing" and not care about the distinction.

I worked on a project for multiple years that is now CMMI Level 3 (aka PITA level). I helped set that up. It is now (mostly) open source. Early on, we tried very hard to follow an early version of NASA's NPR 7150.2. Software Engineering Requirements exactly as written. We read the requirements quite literally, resulting in repeated wrangling over whether we should classify some procedure or test as verification or validation. (Note: Those words (verification and validation) have a slightly different meaning in the physics-based simulation world compared to their meaning in other software domains.) I suggested we simply call it all "testing" so as to avoid the bickering. We pushed that concept up the chain, as a test of our compliance. We were informed that NPR 7150.2 was not meant to be read literally. It was intended to be tailorable. (Newer versions of NPR 7150.2 specifically address tailoring.)

We deemed that anything we did to verify or validate the performance, accuracy, or correctness of our models was "testing", with "traceability" from requirements to tests as a separate section of our many documents. We passed that metatest, thereby saving hours of wrangling over which was which (verification or validation). The project has passed both CMMI Level 3 and NASA reviews for many years.

Tailoring NPR 7150.2 to an Agile organization is not easy, but it can be and has been done. Someone writes a story, others review it, someone with sufficient privileges approves it, someone writes and runs tests (which typically fail; the implementation doesn't yet exist), someone implements the story and writes and runs more tests (which should now pass). Another group might write and run even more tests. Someone finally accepts the story, the software, and all myriad tests as passing muster. All of this is in a database from which reports can be generated. There is no need to distinguish between verification vs validation. What is important is that tests of multiple types must exist, so call them tests, and don't worry about the distinction.