2

We're using (and loving) the jest test framework. Most of our tests are nice compartmentalized good-code-hygiene style unit tests.

However we've also written some service level tests (setting up servers and issuing requests) and also some longer running deployment tests for testing the automation around deploying our stack. These are segregated so that most of the time you just run the first unit tests, but when necessary you can invoke (or CI can) the other longer tests.

Setting up these kinds of integration tests up can be tricky as you sometimes have to work-around (or hack around) limitations of the tools you are using which need testing -- however once you get a good model for it, its not too bad.

Now that many of these things are set up -- I'm having this crazy idea that maybe I should just be using our test framework for actually executing the production deployment process itself ... This way the real deployment logic more exactly matches the testing of that logic. To the extent that actual deployment involves chaining together various components and tools -- why not express that in a manner which allows easily leveraging code from the rest of the project. And the logic that's needed to correctly test variations of the deployment process can just be directly re-used to actually DO the real life deployment -- and keep all the required configuration knobs centralized/verified in a really nice way. Furthermore snapshots can be used to assert all kinds of expectations about the deployment environment and prevent all manner of unexpected sources of variation ...

Does this idea sound crazy? I'm really tempted to try it ... Anybody done this before?

6

See it this way: what you actually did is, you automated the steps of your deployment process in a test-driven manner, just like you can implement any other program or component in such a manner. Since you did this in a "bottom up" way, as a "side effect" you can run the building blocks of the deployment not only for real deployment, but also for the purpose of testing.

By simply changing your point of view, it does not sound so crazy any more, I guess?

One thing to consider here might be, how entangled is your automated deployment with your testing framework currently? If those two are strongly coupled, it might be a good idea to decouple them, so for running the production deployment one does not require necessarily installation or special knowledge of the testing framework. On the other hand, maybe the two are strongly coupled but it is not an actual problem for you, then you might leave it as it is.

  • Great answer! My deployment process is not currently coupled to the testing framework. I'm pondering adding that coupling in order to explicitly leverage features of the testing framework within the deployment process. – Ben Aug 18 '17 at 23:28
5

If your "test code" is being used to deploy your production code, it's no longer test code - it's production code.

That being said, your idea is sound in that mirroring your production environment as best possible in your test environment is a good thing. If your test code contains better and more robust ways of setting up your production environment (?) it makes sense to look into moving that code into production as well.

However, do not treat this as "deploying with test code" but rather treat it as "integrating our test code into production code." If you do not have any sort of configuration management or deployment teams I suggest you start discussing who owns this piece of your codebase, too.

  • I'd just like to add that it's not uncommon, for me at least, to discover I've written some bit of code in my test suite that ends up being useful in the production code. Spot on about migrating the code from the test suite to the production code. – RubberDuck Aug 16 '17 at 10:42
  • The weird conceptual thing about what I'm proposing is that I'm pondering actually kicking off the production deployment process inside the test runner. – Ben Aug 18 '17 at 20:56
  • More specifically -- jenkins would be setup to run a particular jest invocation when code lands on the production branch to kick-off production deployment (rather than running some 'more normal feeling' shell command). Benetifs being: leveraging the test-runner's multi-process execution, snapshots like crazy to validate expectations, easier matching of testing and deployment logic ... – Ben Aug 18 '17 at 21:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.