Microservices: how far should coupling / decoupling of application parts relying on the same data backends go?

Question

I am in the process of migrating a pretty much monolithic PHP application to microservices. With the development pipeline, docker containerisation, CI/CD and scalability in mind, what would be the best practices for designing and deploying the following, which I think is a pretty common scenario:

Our service has a public facing REST API with a well defined data model backed by an RDBMS that has been rewritten using Spring Boot. There are, however, a set of helper processes like for polling external systems and importing data, publishing time-based events to redis etc. Of course, these also rely on the same RDBMS and, in Spring terms, would share the same model and repository classes.

My thoughts

While the REST API itself should be horizontally scalable, those helper processes are pretty much singletons hence I don't want to run them within each of the REST API containers. So it seems to make sense splitting the REST API and each helper process into individual Spring Boot applications, each running in its own container.

Now that I have separate applications, I need a way to share the model/repositories between them. I see two major approaches to this:

1. either make one big Git repository holding all applications as a multiproject build where common application parts are in their own module and shared among each application. This repository's build would have to result in a handful of Docker container images (one for the REST API and one for each helper process)
   • The problem with this approach is CI/CD: each time I make any change in that repository, the build pipeline will kick off, rebuilding each container as it cannot know where the change was made. It seems this will result in too much unnecessary rebuilding / redeployment.
2. or split the codebase and put each helper process into its very own git repository. Each repository's build would have to result in only a single Docker container image, except for the common code that would just result in a jar that should be uploaded to a maven repository. That way, CI/CD deployments can be restricted to only the parts that really changed.
   • The problem with this approach is that it will be a pain to maintain model/repository code in a separate git repository than the application code and have each consuming project sync with it if there are any changes.

My question is: what are some best practices for solving this type of problem? At what level does it make the most sense to split things and where does it make the most sense to reuse/couple things that actually really belong together?

Answer 1

I'm personally more in favor of an approach closer to option 2.

Consider how many existing frameworks organize themselves: they often have a core library and optional libraries for additional functionality where these optional libraries share the core as a common dependency. Also consider how some some depend on just an API at compile time, allowing any implementation to be provided at runtime (such as slf4j).

In your case, your model is the core API.

I would create a model API, then the model implementation(s), then your REST API and helper utilities, each in their own repository and with a dependency on the model API.

Your framework tree might look like this:

---> = depends on

[Model Impl.] (supplied to other libraries at runtime)
   |
   |           ┌─ [REST] 
   V           │
[Model API] <──┼─ [Utility 1]
               │
               └─ [Utility 2]

With this you can have different model implementations and your REST API or Utilities may depend on specific versions of the Model API.

If you have any questions or concerns with this design then ask away and I'll try to get back to you.

Answer 2

I think it is a mistake to try to reuse the models for different microservices. All you can do is copy the relevant parts. Ofc. this goes against DRY from one perspective, but from another perspective a single thing in one context means something else in a different context. For example a person can be user, customer, recipient, etc. depending on the context. So there is a good chance that you will change tomorrow what you copy today and you cannot do these per context changes if you try to keep everything in a single place and reuse it. Google domain driven design and bounded contexts, there are many articles and books in the topic, e.g. https://martinfowler.com/bliki/BoundedContext.html