Where does Behat/Mink fit into the CI/CD pipeline

Question

So I have a project which has BDD tests with Behat/Mink. The scenarios which I have use mink, and so require the testable code to be deployed so that Mink can actually test the pages.

I'm wondering how these tests would fit into a CI/CD pipeline/workflow. Currently our workflow is as simple as push, deploy, manually test on staging, manually release, manually test on production.

In my head I have the following workflow in mind for a regular pipeline:

1. Push to repo
2. Post receive hook to CI service or our own CI server (Jenkins?)
3. CI service/server runs composer install and runs behat tests
4. If they pass it deploys

I don't see how step 3 can happen before deployment though, given that mink needs a url to test the pages, and the new code isn't on the server yet as it hasn't passed it's tests.

Can anyone recommend a sequence of events which is best for behat, where the tests are run before deployment to production.

Answer 1

WebDriver tests (such as Mink tests in your case) are essential for any web application serving an interactable frontend/UI, because they give you confidence about the behaviour of your application in response to user interactions. Without them, in order to gain the same confidence about your applications functionality, you would have to test your application manually in an excessively time consuming fashion.

In general, you want to already run your tests locally, before pushing your build to CI. You can do this by running the server on localhost, and running the tests against the locally served pages.

This allows you to catch errors as soon as possible, keeping the feedback loop short, and it reduces the frequency of having broken code in your repo which you and other devs rely on as a baseline for development.

Then, run them again on your staging environment(s), which will give you the confidence of them not only running on your machine, but also on an environment which is much closer in terms of configuration to your production environment.

A work flow for a pipeline with 2 environments (staging & production) might look something like this:

1. Write some really neat code :D
2. Run tests locally (including WebDriver tests)
3. If all tests are green, push to repo
4. Deploy new build to staging environment
5. Run tests against the build on your staging environment (including WebDriver tests)
6. If all are green, deploy the new build to production

Hope this helps :)