Is it crazy to move deployment logic into test suite …?

Question

We're using (and loving) the jest test framework. Most of our tests are nice compartmentalized good-code-hygiene style unit tests.

However we've also written some service level tests (setting up servers and issuing requests) and also some longer running deployment tests for testing the automation around deploying our stack. These are segregated so that most of the time you just run the first unit tests, but when necessary you can invoke (or CI can) the other longer tests.

Setting up these kinds of integration tests up can be tricky as you sometimes have to work-around (or hack around) limitations of the tools you are using which need testing -- however once you get a good model for it, its not too bad.

Now that many of these things are set up -- I'm having this crazy idea that maybe I should just be using our test framework for actually executing the production deployment process itself ... This way the real deployment logic more exactly matches the testing of that logic. To the extent that actual deployment involves chaining together various components and tools -- why not express that in a manner which allows easily leveraging code from the rest of the project. And the logic that's needed to correctly test variations of the deployment process can just be directly re-used to actually DO the real life deployment -- and keep all the required configuration knobs centralized/verified in a really nice way. Furthermore snapshots can be used to assert all kinds of expectations about the deployment environment and prevent all manner of unexpected sources of variation ...

Does this idea sound crazy? I'm really tempted to try it ... Anybody done this before?

Answer 1

See it this way: what you actually did is, you automated the steps of your deployment process in a test-driven manner, just like you can implement any other program or component in such a manner. Since you did this in a "bottom up" way, as a "side effect" you can run the building blocks of the deployment not only for real deployment, but also for the purpose of testing.

By simply changing your point of view, it does not sound so crazy any more, I guess?

One thing to consider here might be, how entangled is your automated deployment with your testing framework currently? If those two are strongly coupled, it might be a good idea to decouple them, so for running the production deployment one does not require necessarily installation or special knowledge of the testing framework. On the other hand, maybe the two are strongly coupled but it is not an actual problem for you, then you might leave it as it is.

Answer 2

If your "test code" is being used to deploy your production code, it's no longer test code - it's production code.

That being said, your idea is sound in that mirroring your production environment as best possible in your test environment is a good thing. If your test code contains better and more robust ways of setting up your production environment (?) it makes sense to look into moving that code into production as well.

However, do not treat this as "deploying with test code" but rather treat it as "integrating our test code into production code." If you do not have any sort of configuration management or deployment teams I suggest you start discussing who owns this piece of your codebase, too.