Is BDD actually writable by non-programmers?

Question

Behavior-Driven Development with its emblematic “Given-When-Then” scenarios syntax has lately been quite hyped for its possible uses as a boundary object for software functionality assessment.

I definitely agree that Gherkin, or whichever feature definition script you prefer, is a business-readable DSL, and already provides value as such.

However, I disagree that it is writable by non-programmers (as does Martin Fowler).

Does anyone have accounts of scenarios being written by non-programmers, then instrumented by developers?

If there is indeed a consensus on the lack of writability, then would you see a problem with a tool that, instead of starting with the scenarios and instrumenting them, would generate business-readable scenarios from the actual tests?

Update: regarding the “scenario-generator” tool, it would of course not guess business language magically  ;)  But, just like we currently use regexp matchers to create tests in a top-down approach (on the abstraction dimension), we could use string builders to create scenarios in a bottom-up approach.

A “to give an idea only” example:

Given I am on page ${test.currentPage.name}
And I click on element ${test.currentAction.element}
…

Answer 1

I have seen it. Didn't end up well.

I think that cucumber is cumbersome (<--lol :D) abstraction for this exact reason. Too hard for non-technical people to write by themselves; too verbose for technical people.

Non technical people just haven't learned to think like programmers. It is our privilege to understand abstract knowledge, break it down, build up anew and still be able to run away from ambiguousness successfully. That's what we are paid for.

If there is indeed a consensus on the lack of writability, then would you see a problem with a tool that, instead of starting with the scenarios and instrumenting them, would generate business-readable scenarios from the actual tests?

Tool itself won't be able to generate them. Computer knows nothing about business domain. In the end - programmer would be responsible for drawing what needs to be generated anyway and even then - likely those scenarios would be too verbose/cryptic for their end-users.

Answer 2

Part of the difficulty in terms of the customer writing a specifications document is that the customer often doesn't know how to translate the things the customer wants into a language which actually describes what the customer needs. While the customer may say that they want a certain behaviour to exist in a system, they are generally not so concerned with the minutiae until they have seen and used and experienced the software working in a manner which the customer feels doesn't quite match their needs.

When customers describe a business process, they often leave a lot of relevant information out. Often this information relates to things about a process that are commonly understood within the customer's particular domain and which is therefore taken for granted and often not relayed to the programmer. At other times, the customer doesn't actually know how to deal with all of the boundary conditions within a system, and is looking to the programmer for guidance. Sometimes it's all a simple case of usability, with the customer thinking they want something to work in one way, but later changing mind when it becomes clearer that things should work differently.

Ok, so enough of "customer-relations 101 for programmers". The question is whether there is still value in having the customer use a business readable DSL to define how to define a specification. I believe that with guidance, the answer is a tentative 'yes', and I say tentative because the next question that comes to mind is, why would you have a customer craft a DSL when you can have a programmer more easily define one that will provide a customer with a simple yet rich language to define how a system needs to work?

When you have provided a customer with a language for describing how they would like a system to work, you are going to end up with statements that say something along the lines of:

"for a given 'subsystem', as a 'business entity' I want 'some feature' so that I might achieve 'some result'".

This type of statement ends up describing a requirement in a very clear way, providing the the overall shape that the customer basically wishes the system to assume, or another way of looking at it is that the customer is describing what the system is. If you wish to have your customer think about things a little further, you can then ask them to describe the rules which the feature must obey using a number of statement similar perhaps to:

"Given 'some system state', When 'some action occurs', Then expect 'some result'

Again very clear descriptions, this time about how the system should behave. The thing is, this will not replace the need for a software developer to fill in all of the blanks, and to tease out further details that the customer may be only peripherally aware of. While the customer may be able to be 'trained' by the programmer to describe features and behaviours in a nice programmer-friendly format, the customer won't really have the skills or the knowledge to generate meaningful test cases, nor to provide the implementation code. This was I think the point of the Martin Fowler article the OP has referred to. So yes, the software itself isn't writable by the customer, but the description of the software most certainly can - and IMHO should - be written by the customer. For what it's worth, I didn't read Fowler's article as saying that the customer shouldn't or can't write the DSL, but rather that the cost in creating a tool that would ensure the customer can understand how to successfully implement a DSL is effectively prohibitive, which is why I feel it's easier and perhaps more cost effective to show the customer a DSL, and help the customer tune it if needed in order to provide a common language which both business people and programmers can use to collectively stay on the same page.

I feel that we programmers can sometimes forget that our customers are generally very smart in terms of their understanding of their businesses and business processes, certainly much better than we do. When they don't have a programmer to tell them how to build a software system, the customers generally resort to other - perhaps less efficient - means to solve their particular business management problems. By this I mean simple databases (think Access) or spreadsheets, or even in hand written ledgers, and with well defined rules and procedures to manage those processes. What many customers lack isn't a means to determine how a system needs to work, but rather how it should be built, and more importantly how efficiently describe the behavioural rules of a system to the people who do have the skills to actually build the system.

If there is indeed a consensus on the lack of writability, then would you see a problem with a tool that, instead of starting with the scenarios and instrumenting them, would generate business-readable scenarios from the actual tests?

I think that this is looking at the the issue the wrong way around. I would see a large problem with a tool that generates documentation from tests if that documentation was intended to represent a specification in any way. In order to test a scenario, you need to understand it, therefore the scenario needs to already exist for you to both define a test for it. If you describe the scenario in a BDD-syntax, then you have already specified it, and thus you can only instrument the scenarios after the fact. If on the other hand you had a tool that would allow the customer to describe a system in a nice programming-friendly DSL, and if that tool could be used to generate the code templates that would be used as a test suite, then I'd say there would be great value in such a tool. It would not see the customer taking programmers out of the equation, and it would help reduce the effort required to take the customer's wishes and generate test-encoded requirements in a BDD fashion, and would perhaps make the customer's wishes more easily understood. It would not however be a substitute for having an experienced software developer on hand to help the customer to separate the customer's needs from the customer's wants.

Answer 3

I've seen developers write scenarios; testers write scenarios and even a product owner write scenarios. I've also had conversations explicitly designed to bring out scenarios - "Given <this other context>, when what should happen then?" - and written down the words the business use.

The best results I've had were from having a conversation with the product owner while he was in the office, capturing them on a wiki then sending them to him so he could correct and add more. He found a couple we'd missed in our conversations. That rocked.

The worst scenarios I've seen are the ones developers wrote themselves without any conversation with the business. I can tell because they use terms like "When I perform a search", or "When I submit the form with a date of today + 3". They're usually not very interesting scenarios, far too many of them, too low a level of detail and therefore completely unmaintainable. The business also don't read these.

Much better to focus on the conversations IMO. I've seen a few teams do this now for a couple of months with huge improvements to quality, irrespective of the automation. One team managed to get automation working in a very difficult environment some weeks later, much to the joy of the tester! But really, as long as the team are having the conversations and using scenarios to draw out other scenarios, I don't think it matters who writes them down.

Answer 4

My experience is that it is best left to the BA (Business Analyst) to write the GWT(Given-When-Then)s(experience using SpecFlow). The BA can translate the Customer requirements into the GWT and the business can read it. Customers understand the systems but they do not have the tech think to write the requirements in a way that we can use.

Ideally the BA would write some GWT, I would make some revisions, the BA would review/revise repeat until the BA and business was confident in the coverage. Practically the BA would give me a rough draft that I would clean up and make work. The Business shrugs it off saying sure then wonders why it didn't cover some areas no one thought of.