What advantages are there to using a BDD test tool like SpecFlow over simple unit test tools like MSTest?

Question

I'm looking at BDD test tools, like SpecFlow, MSpec or NSpec, and I can't see from the examples what advantages they offer over simple unit test tools like MSTest.

As far as I can see, when creating a BDD test I would write a test method similar to a normal unit test that I would write in MSTest. The difference I can see is that the BDD test tools seem to require me to write several test methods that can then be combined to create the actual tests. I do something similar when writing normal unit tests to avoid repeating myself: I have several arrange methods I can call from the test methods. So combining several methods to create a test doesn't seem like it's very different from what I normally do.

Another difference is that, at least in SpecFlow, I would have to map plain English statements to the test methods via attributes. Then plain English statements would be used to specify the exact tests. Given that I would still have to write the test methods themselves, it seems like coming up with the plain English text and mapping it to the test methods is extra work compared to just writing a normal unit test.

Given the general enthusiasm for BDD tools I feel like I missing something and I'm not seeing the advantages of using these sort of tools. What am I missing?

Answer 1

The term "BDD" was coined by some TDD practitioners to solve a specific problem: when teaching TDD, the most important hurdle to overcome, is to understand that TDD is not about testing.

Dan North realized that one problem with understanding that TDD is not about testing, is that everything screams "TEST" at you: you are using a test framework to write test cases by inheriting from a class named TestCase and writing test methods whose name starts with test. You use testing terminology everywhere: "assertion", for example. So he thought: what if we simply remove all the words that are about testing and replace them with words related to behavior specification via examples? It's still exactly the same thing, except without the misleading words!

And that's exactly what a BDD tool is: it's the same as a TDD tool, but without the testing terminology. That's how Dan North wrote JBehave, for example. If you already understand TDD, and understand that TDD is not about testing, then it doesn't matter what tool you use. But I, personally, find it easier to get into the mindset that I'm not writing tests, when I'm, well, not writing tests but rather examples. So, for me, BDD tools are easier to use, when I use TDD tools, I am constantly battling with myself "I know this is called a test, but it isn't one, it's an example, you stupid brain!"

This is even more pronounced if you show your tests to someone else, e.g. a non-technical stakeholder. You could either explain to them that while this looks like a test, it isn't really a test but rather an example of the behavior … or you could use a framework that makes it look like an example of the behavior. The Great Dream™ that non-technical stakeholders would write executable specifications has not come to pass, but I know of several organizations where the non-technical stakeholders routinely read, and sometimes even edit Gherkin examples, even if they are not writing them from scratch. Obie Fernandez once wrote a web-based structural editor for Cucumber (which probably doesn't work anymore) that would let non-technical users edit Cucumber Scenarios in a structured manner, and immediately execute them against the codebase, and his non-technical customers actually did use this editor to write new scenarios, and communicate their wishes in the form of executable examples.

tl;dr: TDD is not about testing, BDD is TDD without the testing terminology, BDD tools are testing tools without the testing terminology. If you can manage the mental divorce from testing while using testing terminology, it doesn't matter what you use. However, be aware that you may not be the only client of those "tests", even if you are the only one writing them, you may not be the only one reading them.

Answer 2

The first thing to know is the the person that coined the BDD term (Dan North) doesn't use such tools. He uses standard unit test tools.

The supposed advantage is the easiness to transform structured English (or other language) into code. Structured English means that the stakeholders can use plain English to give their requirements, but on a way that is code relatable for developers.

I say supposed advantage because in the past I have found that: a) most stakeholders will not learn nor try to learn to express their requirements on a structured manner (we try to impose over them a system for which they usually don't see the benefits); and b) those tools tend to actually make more difficult the creation of tests.

I used Specflow for two years before I completely discarded the idea and went back to plain NUnit.

Answer 3

It will be hard to get an answer that will satisfy you, because everyone who is answering is biased in some way (including myself of course).

I think (or at least I hope) that the hype is not about the BDD tools that can be used "instead of" unit test tools, but about the BDD process.

The BDD process asks you to think about the problem you would like to solve first, find some good examples and only jump to the coding afterwards, using these examples as tests.

The external DSL BDD tools, like Cucumber or SpecFlow use the plain text file to help you to structure your thoughts and discuss it with the business representatives without the distraction of the coding.

My math teacher was able to draw a perfect strait line by hand. I use a ruler if I need draw one. This is the same with BDD tools. If you can focus on the examples and discuss it with the business, you can capture them to well-structured and named unit tests. That's fine.

But - regarding my experience - for many teams having this separation is beneficial. This is their ruler.

Once you get practice, the overhead of mapping English to the automation code pieces and maintaining it is not as huge as it seems first. Nowadays I even capture tests with SpecFlow for some my internal, one-person projects, because it helps me to know whether I have my developer or my product owner hat on.