1

I'm using BDD and I'm writing Gherkin feature files. Is it good practice to add a step in a Scenario to wait for a while?

Something like:

Waits for 2 seconds after all the operations complete

or should this be implemented in the code without having an explicit step? A stakeholder would not know about the necessity to wait for a bit before checking for something so in some way, this is not part of the business logic but it is convenient to implement this step to allow to easily enforce that the checks run after a bit.

  • I changed the title of the question to make it clear that this is not about whether it is savvy to wait but about whether, in case you decide that you need to wait, you should say it in the feature file or not – 65536 Mar 12 at 10:54
5

Given that the stakeholder "would not know about the necessity to wait", then no, there should not be an explicit wait step. There are times when it's appropriate to have one, but they are times when the wait is something that the user would care about (for example, automated logouts on financial software).

In fact, you probably shouldn't have any explicit waits in your tests. You don't care that the time has passed, you care that the operations have completed. It's still not ideal to have a Given the operations have completed step (since, as you say, the user wouldn't phrase it that way) and to instead have that in the code behind the When step.

Even besides the question of whether the user would phrase the step that way, there's a direct technical reason not to use explicit waits: they are slow, and they stay slow. If your code takes 2 seconds to process things, then waiting 2 seconds isn't actually enough - you need to wait long enough that you can be pretty sure that it will have finished (even if the system is under additional load). So you might need to build in a little extra time, say 3 seconds. Which isn't too bad for one test, but if you end up with a suite of a few hundred tests, then executing the suite gets slower and slower (especially since, when doing BDD, you often reuse steps.) Furthermore - if later on, you come up with a faster way to do things (or get a faster machine, or a faster execution environment for automated tests), then you don't get any of that speed benefit, since the test is still explicitly waiting an amount of time.

  • 2
    I think the main reason against waits is not that they violate BDD policy, or that they are slow, but that they are incorrect. A test is supposed to demonstrate reliably that a certain task succeeds, but a constant amount of time pulled out of your hat like "2 seconds" cannot possibly be reliable - there will always be false positives or false negatives. If at all possible, it is always preferable to depend on things having actually happened rather than to wait a set time and hope. – Kilian Foth Mar 12 at 7:24
  • I know that this is not ideal but if I want to know whether something that is not supposed to happen actually happens there isn't a way to avoid it. If I just wait for the operation to complete how would I know whether waiting a bit more I would have seen the issue? Of course, it is not ideal to have a fixed wait but how would you handle that case? – 65536 Mar 12 at 10:36
3

Such a step is probably not a good idea, mostly because this will make your tests terribly slow. But why do you need such a step?

  • You have a nonfunctional requirement that some operation must finish within some deadline.

    Then let the operation run and measure the elapsed time, for example

    Given that I have a timer
    When I perform some action
    Then I get some result
     And no more than 2 seconds have elapsed on the timer
    
  • You need to synchronize some operations, e.g. some external system might need those two seconds to respond.

    Then using a timer is a completely backwards approach. Instead, use some proper synchronization or at least wait with some timeout.

  • You need to test time-dependent behaviour, e.g. a token should be expired after 24 hours.

    Then don't wait until that time passes, but use some kind of mock time source that you can advance in your tests. For example:

    When I perform some action
     And 24 hours pass
    Then I get some result
    

    With huge timeframes like 24 hours this approach is obvious, but for small timeframes like 2 seconds the same concepts hold.

  • generally you need these "wait a second" steps in UI testing because the speed of the UI is unpredictable and it offers no easily read 'finished' event – Ewan Mar 12 at 7:22
  • In my case the operation I'm waiting for is not supposed to happen so I do not have a way to synchronise. That's why I have waited for the operation to complete and then I wait a bit longer. Of course, no one would add a step to wait 24 hours the waiting is a way to be pragmatic, so I don't thing that "but for small timeframes like 2 seconds the same concepts hold" is correct. – 65536 Mar 12 at 10:41
  • I'm not sure that to mock the time is a good idea in integration tests. I'm doing it all the times in unit tests with RxJava and TestScheduler but should I do it in integration tests? – 65536 Mar 12 at 10:58
  • @Ewan event loop UIs often provide a method to check if the event queue is empty. – Caleth Mar 12 at 11:14
1

With BDD your test should match the requirements for the feature. Include the wait step when its a requirement

Good:

When the "Win a Prize" button is clicked
and the 2 seconds mandatory wait between clicks has elapsed
the "You have won!" message should be displayed

And not when it is a simple technical restriction on the system

Bad:

When the "Win a Prize" button is clicked
and the 2 seconds round time trip to the prize api has elapsed
the "You have won!" message should be displayed
0

The way I answer this question is by asking a different question: "how would you tell a human to test this feature?". Would you explicitly tell them to wait, or is waiting implied?

  • I did ask the tester and she said: of course I would wait! But does this mean that I do not need to say it explicitly? What about how long I should wait? And what about if in the step definition I forget to wait and then the test succeeds even if something is wrong? Isn't it better to enforce the fact that I need to wait? – 65536 Mar 12 at 10:52
  • @65536: there is no general purpose answer that we can give to cover all that. It really depends the specifics of what you're actually testing and what the requirements are. Unfortunately, the question is too vague and generic to give better advice. – Bryan Oakley Mar 12 at 13:24
0

What is your system under test doing "after all the operations complete"?

Either nothing is happening, in which case the wait is superfluous, or something is happening, so your definition of "all the operations" is incomplete, and should be changed to include the extra things going on.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.