How to develop independent logical blocks without having too many microservices?

Question

I have a service which is composed of various independent small "blocks" of logical code. A block stands for a class which implements an interface "IBlock" which includes an "Execute" function that gets a BlockRequest parameter and returns a BlockResponse parameter. Each logical block usually does the following: Get some data from a DB, Make an API call, Any specific simple logic over the data, Return a normalized response.

The purpose of this service is to serve an external orchestrator (different service) by executing blocks dynamically according to request. The service is unaware of any business flow, and only knows to execute a specific block on demand. Basically the service is some kind of repository of independent logical blocks.

The service receives request messages via queue, runs some logic according to the request (a specific block) and return an answer by publishing a response message to a different queue.

Outside of executing blocks, the service itself is quite lean and holds little logic - mostly validating the request, logging, writing to DB and publishing back a response. However, there are many "logical blocks" and the number will only increase as we keep using this service.

I will eventually have too many logical blocks which will be hard to maintain and every time a small piece of logic in the form of a new block will be added - Meaning adding another class to the solution -
I will need to test the entire service and deploy all of it again to support the new block or a minor change.

Separating the blocks into different services could help as it would allow us to deploy and maintain each block separately, but eventually this would mean a lot of services which would also be tough to maintain and keep track of - making this a lacking solution.

Is there any middle ground pattern\technology\design that could help me build this service properly?

Answer 1

Let’s kill a myth: the fact that a block is independent does not justify by itself a microservice.

There is a balance to be found:

• between the advantage of having independently deployable microservices and the inconvenience of the increased complexity;
• between the maintenance benefit and the performance overhead of inter-service communication.

The second point is not of your concern, because your orchestration architecture requires the inter-service communication anyway.

The first point remains open for discussion. In view of the many misunderstandings about the desirable granularity of microservices, Chris Richardson, the microservice guru in chief just released a new rule of thumb:

A team should ideally own just one service since that’s sufficient to ensure team autonomy and loose coupling and each additional service adds complexity and overhead. A team should only deploy its code as multiple services if it solves a tangible problem (...)

I do not have sufficient elements to advise, but ask yourself what your goal is, and if releasing each “block” independently would really be a benefit.

For example would your independent block rely on some shared code ? How often does that code change and how will you cope with such changes in the future ? Or could there be some hidden coupling because of expectations about the data exchanged in the requests ?

As an alternative, have you considered a service that offers all the blocks ? You could still achieve the horizontal scalability by having several running occurrences of your coarser grained service.

Now, if after having challenged your own ideas you conclude that microservices would be the right solution, go for it :-)