automated acceptance testing / BDD & workflow for designing a system

Question

Recently, I started reading the book Specification by Example, which relates to automated functional testing and BDD (from what I've understood till now).

I've tried using Concordion (.Net), and seems very interesting. I've been having issues with keeping any form of useful documentation for any designed system, and this might help

My issue is, how would one suggest that the workflow in designing a complete system is? Some questions that arise are:

• Is it ideal to try to design the entire whole system in the beginning?
• Should you design a very high level overview of the system, and then create specifications one main feature at a time, i.e create detailed specifications -> develop -> test -> move to next main feature?
• Should you create BDD-style specifications for each and every method in the system, even trivial ones like some GetProductByReferenceCode?

The issue is that most of the times when you actually start developing, you start to realise something's that need to be done differently than originally thought, or omissions not noticed during the initial design. I find that sometimes the initial design stage takes a lot of time, only for the actual design of the system to be very different once the product is launched.

My current workflow for designing a system is:

1. First start with the user-interface, creating mockups of each and every screen the users will be dealing with. I find this to be the most visual method that the business users can understand.
2. Define logic that is directly related to the user-interface
3. Define any logic that happens in the background, for example notifications, etc.

Does this make sense? Any ways this could be improved?

Answer 1

BDD at its essence is a software development methodology with the aim of driving (or guiding) code from examples of how the system is used. It has nothing to say about the actual look and feel of the user-interface (which can be regarded as a simple "controller" for your application and could in theory be easily swapped with an a REST API, a browser-ui or anything through which the user can interact with the application). Therefore starting from the UI could be quite misleading if attempting to practice BDD.

Think of your UI as a thin layer that contains no business logic at all and focus instead on driving the "guts" of the application. This is the logic you describe in step (3) of your workflow.

For example, imagine this were a shopping application and the end behaviour you would want to drive is buying a product. This requires being able to give the system command to add a product to a "basket", being able to pay for etc. "Add Product X to Basket" and "Pay for my Basket" are commands you can give to the system regardless of the UI.

Even if you have already spec'd the UI - you can / should identify the commands given to the system. It is essential to conceptualize the system in terms of business logic so that you don't get distracted by details of how the UI will work (you can think about this later).

Each of these commands you identify becomes the When in your Given/When/Then scenario. The action that is ultimately tested when the suite is run.

Identify business rules - these will help you understand the preconditions that need to be met for each command to have the expected outcome. Say you have a business rule about customers getting a free Toaster on every Microwave when a sale is on:

---

Background:
  Given the price of a microwave oven is £100
  And the price of a toaster is £10

Scenario: Customers get free toaster if they buy a microwave 
  Given the winter sale starts on Sun 31st Dec at 9am
  When I buy a microwave oven on Sun 31st Dec at 9am
  Then my basket should contain the following line items
    | Product        | Price |
    | Microwave oven | £100  |
    | Toaster        | £0    |

Scenario: Customers do not get free toaster if they buy a microwave before sale
  Given the winter sale starts on Sun 31st Dec at 9am
  When I buy a microwave oven on Sun 31st Dec at 8am
  Then my basket should contain the following line items
    | Product        | Price |
    | Microwave oven | £100  |
    | Toaster        | £10   |

From these scenarios you can start writing code - in the Given steps you set up the products with their prices and the sales rule. In your When step you take the action - either by calling the code directly (fastest development & feedback), calling an API or automating the actual browser (the most time consuming and slowest feedback) In your Then step you assert that the outcome happened as expected - this might involve inspecting the contents of the data persistence and probably the response from the system or UI.

Building the application like this forces you to do quite a lot of thinking up-front (the business logic) but the coding is easier because you simply need to make the scenarios (which become tests at this point) pass. Your code is literally guided by the scenarios/examples./ When they pass you can be guaranteed that the system works correctly - go and build a UI that will allow customers to do what the scenarios say.

Answer 2

This is a broad question with no specific answer--you've probably come up with one since posting the question.

You might think of acceptance testing plus BDD as red-green development for each level of the project. In a pure world, you could write upper level acceptance tests first using BDD knowing that the functionality doesn't exist and all of the tests are going to fail. Then you could write general BDD style tests for each of the major components or subsystems in the application. As each subsystem is developed you flesh out the the general BDD tests to be more specific. Then you would use test driven development to create the tests for the behind the scenes logic and private functions as you implement them.

Personally, I've found that all of the levels of BDD are helpful to some degree, but it takes a disciplined team to do it from beginning to end. BDD also may not be appropriate for all projects, depending on business requirements, time constraints, staffing, budget, ad infinitum.

Answer 3

I tend to write BDD tests as acceptance for the endpoints of my system WCF/MVC/WEBAPI.... since those are stories that the business cares about. I'm using SpecFlow to create Feature files and acceptance criteria scenarios. So with regards to my MVC testing, I'll create a Feature file that looks like

Feature: High School Manamgement 
    In order to manage a list of high schools
    As a coach 
    I want to be able to create, edit or delete a high school

@web
Scenario: Navigate to the high school home page
    Given I am logged in as ''
    When I navigate to the page '/HighSchool'
    Then I should see a table named 'HighSchoolGrid'
        And I should see the 'Create High School' button

@web
Scenario: Navigate to high school create page
    Given I am logged in as ''
    When I navigate to the page '/HighSchool/Create'
    Then I should see the fields
    | Field               |
    | Name                |
    | Type                |
    | PhoneNumber         |
    | Mascot              |
    | AddressLine1        |
    | AddressLine2        |
    | AddressCity         |
    | AddressState        |
    | AddressZipCode      |
    | AddressZipExtension |
        And the required fields should be marked required  
        | Required Field |
        | Name           |
        | AddressState   |
        And I should see the form submit button 'Create'
        And I should see the 'Cancel' button

@web
Scenario: Submit Empty Create Form
    Given I am logged in as ''
    And I have navigated to the page '/HighSchool/Create'
    When I submit the form
    Then I should see the required fields highlighted
        | Required Field |
        | Name           |
        | AddressState   |
        And I should see the error message ''

@web
Scenario: Submit Form Successful
    Given I am logged in as ''
        And I have navigated to the page '/HighSchool/Create'
        And I have filled out the form as follows
            | Name      | Type | PhoneNumber    | Mascot  | AddressLine1  | AddressLine2 | AddressCity | AddressState | AddressZipCode | AddressZipExtension |
            | xxxxxxxxx |      | (215) 555-1212 | XXXXXXX| 1 Main Street |              | XXXXXXXXX| PA           | 18964          |                     |
    When I submit the form
    Then I should be redirected to '/HighSchool'
        And  I should see the success message 'xxxxxxxxx has been added'

So these scenarios are my acceptance tests which are integration tests that exercise the whole system. I've created the @web tag to generate a Selenium webdriver instance to run through the website. I've also created an @api tag that spins up a HttpClient that then will post to WebApi endpoint.

So like your current workflow, I use ui mock(s), ui-logic, ... to capture a test that a manual qa tester or business partner might perform. The other nice thing is that I now I have automated regression tests to that if my requirements change when I run my test suite I now should see a failure for the changed requirement. I can create a new BDD test that captures that change so any failed regression can be evaluated and removed if it is no longer a valid test. This also can help you find where various requirements are in competition with each other.

That's really the extent of the true BDD/acceptance testing I do.

I also have unit tests that mirror a somewhat BDD style in that they are expressed in a somewhat Gherkin syntax. I start with this Specification base class for testing that I found from Jimmy Bogard on Los Techies.

public abstract class SpecificationBase
{
    [SetUp]
    public void SetUp()
    {
        Given();
        When();
    }

    protected virtual void Given() {}
    protected virtual void When() {}
}

public class ThenAttribute : TestAttribute {}

Then my unit tests all follow a standard pattern. See my answer here for an example implementation of the this abstract class. I find this gives me good coverage on both integration and unit testing.