9

I'm currently exploring BDD test frameworks like cucumber and I find it curious when people say

since the feature files are in simple natural language it improves clarity and gives a clear vision

but, isn't natural language the cause for most of the trouble we have in software Engineering?

Natural language is ambiguous and that's the reason many software projects fail because of the misinterpretation of clients' requirements and the developer's understanding. I don't get the niche here.

Yes, breaking down the tests into tiny simple doable actions makes sense and gives a certain level of clarity but does that improve productivity on a whole?

PS: I'm not an expert and I'm not making an opinion here. I'm just curious to understand the concept.

2
  • 1
    Very good question. I must say that I've never seen things like what cucumber proposes work in practice. Natural language is unsuitable for precise technical tasks, such as specifying tests. – Andres F. Jul 17 '17 at 21:59
  • BDD's use of language is intended to reflect the existing language of the business domain, which should already be unambious to the business. The Wikipedia article does state that early in the text. – Martin Spamer Oct 9 '18 at 16:52
8

You are correct. BDD does not eliminate problems with language ambiguity - not at all. As others pointed out, the snippets that get translated need to be matched by properly defining them, but this also does not address the underlying ambiguity problem.

Now why is BDD actually worthwhile despite not solving this problem? There are some reasons and this list certainly isn't complete.

Ambiguity has not been solved

This is neither a reason in favor of BDD nor against it. But when you contrast it to other approaches like user stories or requirements, then all SW development approaches suffer from language ambiguity as they all start in one way or another with a natural language formulation.

Technically, the problem of language ambiguity has been solved with artifical languages like lojban, but then again, your customer and developers will most likely not know that language.

Ubiquitous language

BDD goes hand in hand with the idea of an ubiquitous language. Being able to specify scenarios together with all customers, testers and developers, just gives BDD an edge over other approaches.

Consider a traditional requirements engineer writing down all the requirements. Once you as a tester or customer get that 300 page document full of requirements to review you will have a lot more pressing problems than the terminology that was used there.

User stories do a little better on that front, since they also include all stakeholders in their creation. In terms of the ubiquitous language, I wouldn't say that either BDD or user stories are better - though they do differ in the next point significantly.

Testability

A major aspect of BDD is that your specifications are actually executable (via Cucumber or the like). Neither requirements nor user stories offer this feature. For me personally, that's the main selling point for BDD.

Contrast that with traditional requirements - we have been telling requirements engineers for ages that their requirements should be testable. Yet, every project sees a case where somewhere down the line testers realize that they have no idea how to test a certain requirement.

User stories, if done right, include testers in their early creation stage to make sure of that. Unfortunately, this is a case of theory clashing with the real world, where I have seen numerous stories that no tester has seen before.

BDD on the other hand automatically gives you an executable test scenario. There are no excuses and no ways around that (well unless you completely ignore the automation layers and just write down scenarios for the fancy poetry).

More generally, Test First is a principle that has been very rewarding across all stages of software development and BDD is its application to the outermost layer of the development (as compared to f.ex. TDD on the unit level).

Summary

In summary, BDD does not eleviate you from the problems of natural language ambiguity. It does, however, help you to tackle that problem via two important points: Focusing on an ubiquitous language in order to reduce the ambiguities (it won't eliminate them entirely, but it helps a ton!) and by forcing you to write executable specifications. The latter point is helping to address ambiguity problems mostly because that is the point where ambiguities start to show up as problems otherwise.

1
  • this is an awesome answer. I did a bit of research on this after asking this question and I should agree with most of your points.One major problem with using tools like cucumber or any BDD tool is that the developer don't understand the idea of BDD on a zen level.Here's an interesting article on this by Elizabeth Keogh. – Raghuram8892 Jul 19 '17 at 4:34
4

When something is written in “natural language” this can mean a number of things:

  • This is literally English. As English is a very ambiguous and imprecise language, this input mode is unsatisfying in the context of software development.

  • This is English, but relevant terms are precisely defined. Such language is used in legal documents, or mathematical texts.

  • This is a formal language, but the language is modelled very closely after conventions of natural language. This describes all programming languages, to some degree. The more English-like the formal language is, the easier it is to understand for untrained readers. Notable examples of programming languages with this goal include COBOL and SQL: select id, name from persons where age > 18 is immediately obvious. The disadvantage of these languages is that you do need to understand the formal language to write working code. Also, these languages are often very verbose.

DDD suggests that the project uses an ubiquitous language to describe the domain. This is essentially case 2: define relevant terms so that you can communicate precisely within natural language.

Cucumber itself is case 3: a formal language that has the intention to read very close to normal English. More precisely: Cucumber is a framework that allows you to define a simple formal language that can be used to express requirements/tests. The point here is that the same document represents the natural-language requirements and automatically executable tests, so the two will always be in sync. You can read a cucumber scenario and verify that it expresses your requirement correctly without having to understand how all of this works.

Cucumber works by matching the document against known snippets of natural language. These snippets have to be defined first. To write a scenario in Cucumber, you need to be aware of the available snippets – the software won't read your mind. These snippets are also a source of possible problems: When you implement the behaviour of a text snippet, your code might do something slightly different than the snippet text suggests. This is unlikely to be a problem if the same snippet is used many times. Cucumber is therefore well-suited to describe business rules that consist of a smaller set of conditions, actions, and outcomes. Other testing frameworks might be better if each snippet were to be used only once or twice, e.g. because the setup for each test case is unique.

2
  • thanks for the detailed info. I feel that cucumber is somewhat in the grey area between case 2 and case 3. its unlike SQL where you can't actually have some sort of "Free Will" and stick with strict formal syntax. To my limited knowledge, don't cucumber allow any form of text after the "Given" ,"When" keywords for its scenario? It may be as simple as that but I'm from a non English native country and its most likely hard to make people give precise snippets. – Raghuram8892 Jul 17 '17 at 15:32
  • 1
    Yes, you can put anything you want after the Given / When / Then, but a) you and your team define precisely what that means, and b) you define the meaning in the matchers in code, i.e. a formal language. – Jörg W Mittag Jul 18 '17 at 7:04
0

@Raghuram8892, the text after the Given/When/Then/And keywords has to match the "snippet", otherwise the step fails as undefined or "pending". As such, it falls squarely into case 3.

Regarding "English", Cucumber and its language, Gherkin are designed for international use. You can invoke the command, cucumber --i18n help to see the list of languages currently supported, and cucumber --i18n $CODE to see the keywords for a particular language code. For example, cucumber --i18n eo gives the keywords for Esperanto.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.