4

I am unsure where/how to specify the behavior of a mock in a test scenario when using the Given-When-Then syntax. It seems to me both Given and When could be correct.

Consider the following example:

  • I want to test the control panel for a car.
  • This panel has a start button for the engine and a LED that shows the engine status.
  • When the user presses the start button, the engine should be started and the LED should be green or red, depending on whether the engine was started successfully.
  • The control panel is the SUT, the engine is mocked.
  • In the specific test scenario, we test what happens when the engine fails to start.

Which of the following Given-When-Thens is the more correct one:

This one:

  • Given a control panel in its initial state
  • And an engine that will fail to start
  • When the start button is pressed
  • Then the LED will become red

Or this one:

  • Given a control panel in its initial state
  • When the start button is pressed
  • And the engine fails to start
  • Then the LED will become red

Personally I feel that the second way reads more natural, but I also feel that the mock behavior should not be part of the action on the SUT.

The example is made up, but the key element here is the action towards the engine that is taken as a consequence of the test action given in the When. I.e. the engine response cannot be phrased as a state (unless one would say "given a malfunctioning engine".)

6

The scenario should be set up in the Given step. This involves mocking parts of the system. However, a mock might be used in the When step when that's part of the action.

When reading your two scenarios, they both look reasonable as natural language:

  • Given a control panel in its initial state, And an engine that will fail to start, When the start button is pressed, Then the LED will become red.
  • Given a control panel in its initial state, When the start button is pressed, And the engine fails to start, Then the LED will become red.

However, the second variant does not conform to the Given-When-Then structure.

  • Given describes the initial context of the scenario.
  • When describes actions or interaction with the system.
  • Then describes the outcome of the actions.

Both of your scenarios obfuscate the actions by using passive voice: “the button is pressed”. In my experience, it is better to name an actor who interacts with the system (third-person perspective) or to write the scenario from a first-person perspective: “When the driver presses the button” or “When I press then button”.

From this, it is clear that “the engine fails to start” is not an action – it is an outcome. But asserting that as an outcome makes no sense for the scenario that you're trying to test. That the engine will fail to start is an essential part of the context of your scenario, so it should be specified in the Given step.

I would write your scenario as:

Scenario: the engine status LED reports a failing engine.

Given a control panel
And and a broken engine
When the driver starts the car
Then the engine fails to start
And the engine status LED lights up red.

Thanks to Greg Burghardt for pointing out in the comments that the dashboard – not the entire car – might be the system under test, so that the engine might be the actor. This would lead to the following, equally valid scenario:

Scenario: the engine status LED reports a failing engine.

Given a control panel
And that the driver tried to start the engine
When the engine reports a failure
Then the engine status LED lights up red.

10
  • Could you explain why you added "Then the engine fails to start"? Since this is mocked behavior, it doesn't seem to be something that needs to be validated by the test.
    – slingeraap
    Sep 14 at 13:40
  • 1
    @slingeraap Yes, that's not an essential part of the test. I think it's clearer to document that as part of the outcome of the “start the car” action, especially if the setup for the mock is more complex. I would leave it out if Given-When-Then is just used to structure a test case, but keep it if I'm using Gherkin/Cucumber.
    – amon
    Sep 14 at 13:45
  • 2
    It almost feels like it should be "When the engine fails to start" and then using "Given the start button was pressed" as part of the setup. Sep 14 at 18:33
  • 1
    @GregBurghardt That is an incredibly good point! Yes, the engine should probably be the main actor here. I edited the answer, does this look reasonable?
    – amon
    Sep 14 at 18:58
  • 1
    @slingeraap No, from this perspective the test would take the engine's view and the driver/button might be mocked.
    – amon
    Sep 15 at 7:25
2

I understand starting a vehicle is a representative example, so I will focus more on the Behavior-Driven Development and mocking aspects of the question.


I am unsure where/how to specify the behavior of a mock in a test scenario...

You don't. The behavior of a mock is transparent to the scenario. Remember that BDD focuses on the business process, not the technical details that support that business process.

In unit testing, creating mocks is typically done in the setup. This leads me to believe that a Given step would wire the engine and control panel together, but I don't think a step phrased as Given an engine followed by And a control panel is very descriptive of the business process. Again, the business process is the focus, which may include a larger system than you might realize at first.

An engine connected to a control panel is the minimal requirement, but not a useful business process. Think bigger. What would a driver need? An engine wired to a control panel sitting on a table isn't so useful. A driver needs a vehicle. The entire vehicle is the System Under Test.

Now we have a conundrum. We have specific components that need to interact. One of them is the real component. The other is a mock. Behavior-Driven Development is il-equipped to handle this level of detail. BDD scenarios are typically used in end-to-end tests against a real system. If any component is mocked, stubbed or faked, it is not a detail captured in the scenario.

My honest advice is to either not use Gherkin, or limit the scope of your BDD scenarios to the control panel. Limit them to the point that every scenario has a real control panel and everything else is a mock. A step like Given a vehicle would mock everything except the control panel. If you need to test the interaction between two other components, then you will need a whole new suite of features in a separate project so you can mock different objects using different step definitions.

I honestly just don't think this is a good use case for Gherkin. This fine level of control can only be achieved by straight code. Any solution involving Gherkin will compromise BDD best practices. While no laws will get broken for violating a best practice, it does make routine testing techniques like mocks and fakes very difficult. There is no good answer. Gherkin may be the wrong tool for this job.

2
  • If we're talking about what the mock does (observable behavior), and not about how it does it, then the statement "the behavior of a mock is transparent to the scenario" is not quite right. Yes the abstract behavior of the mock is test-independent and is determined by what is being mocked, but the actual setup (the state of the mock) is test-dependent. Otherwise, the test and the mock are next to useless. What the mock is specifically set up to do is scenario-dependent, and can be different for every test (and is part of the "the givens"). I think the OP's question is more about that. Sep 16 at 0:34
  • @FilipMilovanović: "... but the actual setup (the state of the mock) is test-dependent. Otherwise, the test and the mock are next to useless." --- this gets right to the heart of my answer. With Gherkin, it is difficult to phrase steps in a manner that allows such fine grained control of mocks and stubs, which leads me to believe that Gherkin (along with the philosophical overhead of behavior-driven development) is not a good fit for this kind of test. Sep 16 at 11:10
0

Which of the following Given-When-Thens is the more correct one:

At the end of the day, that is a really uninteresting question. The primary audience of the Gherkin (Given-When-Then) syntax are humans and humans are usually far less bothered with correctness than with usefulness.

The two interesting questions are which version is seen as more understandable by your team and stakeholders and is your tooling capable of handling that order of clauses.

And to make it explicit that the panel gets to handle a sequence of two events, my team might write the testcase even as

Given a control panel in its initial state
When the start button is pressed
Then the engine is requested to start
When the engine fails to start
Then the LED will become red

Such multi-event, multi-response scenarios are quite common in the testing we do, but it is hard to argue this is correct Gherkin syntax, with the two When-Then blocks. But the usefulness of this construct for us far outweighs the fact that it cannot be called correct.

2
  • To me Given/When/Then or Arrange/Act/Assert is not fundamentally about syntax or blocks, or even readability, it's about how to approach thinking about the test case. You have one specific thing that's the subject of the test case (some object (possibly composite) + the interfaces associated with the behavior of interest), and a specific behavior you want to test, observable by the client (role-played by the test here) either directly through the client-facing API, or through dependencies that the client has either itself provided or otherwise has access to. 1/2 Sep 16 at 0:58
  • Making the call that initiates that specific behavior is the When/Act. Anything you do to prepare for that is the Given/Arrange. The Then/Assert part then involves the checks that give you information about the specific thing the test case is for, insofar as any "normal" (non-test) client code would be able to determine it (in other words, it can be written to rely on these behaviors). Any Then/Assert beyond that is testing the internals. 2/2 Sep 16 at 0:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.