0

Is it common to write test for CI/CD scripts?

Background: Some CI/CD jobs/scripts have become rather complex over the years (new features, edge cases) and I believe it would help rewriting them with e.g. python instead of bash. However, since these scripts have no tests, rewriting would definitely be error-prone. Therefore, the question above. Bonus points for any references.

4
  • 1
    This question is a variation on "who polices the police?", which when addressed raises the question of "who polices the police's police?", and from there on it's turtles all the way down. Secondly, I also get the feeling that what you call CI/CD scripts are significantly bulkier than what they should be (what edge cases? How is this a thing?), which may render the question unanswerable as to what your specific script may or may not be.
    – Flater
    Commented Jul 13, 2022 at 10:51
  • 1
    Indeed, if the police is the new python CI/CD, the police's police is the old bash one. Let both work in parallel until you gain trust on the new one.
    – mouviciel
    Commented Jul 13, 2022 at 11:15
  • 1
    Do you even care about the scripts or do you simply care about the resulting published artefacts and the working software built, deployed and running correctly in its target environment? Presumably if all the artefacts exist and everything is working as expected in that environment after those scripts have finished running, then it must logically follow that everything is fine in the CI/CD scripts too. Perhaps a better question to consider would be whether you have sufficient monitoring, observability and reporting in your system to know when something is wrong if CI/CD fails? Commented Jul 13, 2022 at 11:51
  • @mouviciel: That seems like a good idea, thanks!
    – gebbissimo
    Commented Jul 13, 2022 at 13:09

1 Answer 1

1

Yes, we test our CI/CD scripts, but those tests are primarily done manually.

Our CI/CD scripts are split into two parts. The first part contains the (sometimes complex) logic of how to build the software, documentation, etc. This part is designed to be able to run it locally on a developer's machine. When making changes here, they are tested locally.

The second part consists of wrapper scripts that allow our CI/CD engine (Jenkins) to call the first part and to have it execute on the correct slave node. This second part is not really tested, but just tried out on the CI/CD system. We don't consider that to be a problem, because the complexity is limited.

1
  • Thanks, I appreciate the answer! The phrase "designed to be able to run it locally" was very helpful!
    – gebbissimo
    Commented Jul 13, 2022 at 13:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.