Today I read about how to deal with pauses and timeouts (links at the bottom of this post).

My testers keep adding pauses into Given-When-Then. They claim it’s necessary because of how the tests work. I don’t like it because the numbers look random and do not reflect our business rules. Also, sometimes pauses aren’t long enough. The tests sometimes fail although there are no bugs in the system version we’re testing, because the pauses aren’t long enough. How do we avoid that?

They rightly say that this problem is symptomatic of working with an asynchronous process and here is a Scenario Example:

Given a user checks out with a shopping cart worth $200
When the user submits an order
And the system waits 3 seconds
And the user authorises the payment
Then ...

Specifying a given time frame does not guarantee consistent quality test results. There are factors that could lead to tests failing erroneously, the main one being system resources during test runtime.

Focus on the what rather than the how

Instead of specifying a specific time constraint to make the test work, this detail can be dealt with in the step implementations rather than the feature file. This shifts the focus to discuss the business requirements which is one of the goals of Gherkin.

If it is important to specify that step then an alternative way is to document the event rather than an arbitrary ‘3 seconds’. The alternative way to write it would be like this, refer to the third step:

Given a user checks out with a shopping cart worth $200
When the user submits an order
And the order submission completes successfully
And the user authorises the payment
Then ...

This is helpful because it also branches out and gives space to ask more questions to do with the system. For example:

  • What happens if the order submission doesn’t complete due to a dodgy connection?
  • How would this impact on user experience and how will we deal with this scenario?
  • What if the order item suddenly goes out of stock? (last item was purchased a second before this particular user submitted)

Modelling business time

What do you do when you have to automate a test where you have to wait months or years? An interesting concept I didn’t think about.

…If time is critical to an aspect of your system, model it in the domain and represent it as business time. Technically, this often involves creating a wrapper around the system clock, with methods to schedule and wait for events. Anything else that depends on time should not use the system clock, but connect to the business clock instead.

So far I have tested such cases manually so this point is for me to consider going forward when building on test automation skills. In one example for GDPR regulations, we need to autodelete user accounts after a given amount of time. We usually change the database value for the user accounts’ last login time.

Automating this, I can see how this technique of modelling business time would come in handy. Not relying on the system clock will give us more freedom to control time virtually and build more faster and reliable tests.

Links here: