I am in the process of migrating a pretty much monolithic PHP application to microservices. With the development pipeline, docker containerisation, CI/CD and scalability in mind, what would be the best practices for designing and deploying the following, which I think is a pretty common scenario:
Our service has a public facing REST API with a well defined data model backed by an RDBMS that has been rewritten using Spring Boot. There are, however, a set of helper processes like for polling external systems and importing data, publishing time-based events to redis etc. Of course, these also rely on the same RDBMS and, in Spring terms, would share the same model and repository classes.
While the REST API itself should be horizontally scalable, those helper processes are pretty much singletons hence I don't want to run them within each of the REST API containers. So it seems to make sense splitting the REST API and each helper process into individual Spring Boot applications, each running in its own container.
Now that I have separate applications, I need a way to share the model/repositories between them. I see two major approaches to this:
- either make one big Git repository holding all applications as a multiproject build where common application parts are in their own module and shared among each application. This repository's build would have to result in a handful of Docker container images (one for the REST API and one for each helper process)
- The problem with this approach is CI/CD: each time I make any change in that repository, the build pipeline will kick off, rebuilding each container as it cannot know where the change was made. It seems this will result in too much unnecessary rebuilding / redeployment.
- or split the codebase and put each helper process into its very own git repository. Each repository's build would have to result in only a single Docker container image, except for the common code that would just result in a jar that should be uploaded to a maven repository. That way, CI/CD deployments can be restricted to only the parts that really changed.
- The problem with this approach is that it will be a pain to maintain model/repository code in a separate git repository than the application code and have each consuming project sync with it if there are any changes.
My question is: what are some best practices for solving this type of problem? At what level does it make the most sense to split things and where does it make the most sense to reuse/couple things that actually really belong together?