I am part of a small team which develops several internal apps for our company. We are in a process of becoming more Agile, this especially includes lots of automated testing. Now we are in a situation where, for the one or two apps which we have worked on most recently, we generally make small, reversible changes, run lots of tests, and deploy quite fast with not much human intervention.
I would consider us to be still a long way off from doing 'real' CI. For the one or two apps mentioned above we could probably get very close to it pretty soon. However, I am having a hard time imagining what our setup is going to look like, sometime in the future where we have started using CI for most of our legacy code.
Suppose we have several standalone apps, which all can be deployed independently to different servers. We also have some shared code which is used by many of them, and which we want to use consistently between them. This includes utility functions, code which enforces things like a consistent interface over some parts of all apps, and the ORM definitions for our database (shared across all apps).
I see two alternatives, neither of which looks very easy or elegant:
- All code gets merged into one huge repo. Deployment means running all the tests for all the code, integration tests for everything and acceptance tests for all the apps, before rolling out everything in one go. This seems to make deployment a much Bigger Deal than it was before, contrary to the philosophy of CI, which suggests that it should be fast and easy. It also means we don't have any separation between different parts of our codebase, with things which never work together in the same repo forever, just because they both depend on some third thing.
- We keep each app and each component of the shared code separate. Deployment means testing the latest version of one component extensively, before 'dropping it in' to a working system consisting of all other components. This seems like a cleaner design. However, it seems to imply that we have to manage dependencies and versioning for all these things. Every integration/acceptance test has to have some foreknowledge of what versions of the other components it is going to be used with and can tolerate. In other words, although every component becomes a lot more reliable, we have to worry loads more about how the pieces fit together, and integration bugs. When the pieces which everything else depends on change, there is potential for breakage eveywhere.
The way out of this dilemma is probably to do a limp version of CI, where the individual components get tested extensively and deployed quickly, but we stick to having big 'flag day' releases of the utilities and database schema. But it really seems like there are lots of benefits of proper CI/CD which we would be missing out on. It isn't an aesthetically pleasing solution, and we aren't looking to satisfy our bosses that we have ticked a box, but we want to improve our working practices.
How should we organize the code for proper CI, and what are the key lessons to learn about planning and designing the architecture, both for legacy and future newly written code?