1

We're building a new web-based industrial application and one of the questions that are hammering our heads for the last few days is about the integration between different "microservices" on this architecture.

I'm using microservices with just a pinch of salt because we're not totally embracing the concepts to define real microservices. One (and I think the biggest) difference relies on the fact that we're using the same shared database along the different modules (that I'm calling "microservices"). A sort-of logical view of our system could be drawn as:

                  ╔══════════════╗
                  ║    Client    ║ ══╗
                  ╚══════════════╝   ║ (2)
                                     ║
                                     ▼        
        ╔══════════════╗  (1) ╔══════════════╗
        ║  Serv. Reg.  ║ <==> ║  API Gatew.  ║
        ╚══════════════╝      ╚══════════════╝
            █       █   █████████████     (4)
           █         █              ████
╔══════════════╗  ╔══════════════╗  ╔══════════════╗
║   Module A   ║  ║   Module B   ║  ║   Module C   ║  <===== "Microservices"
╚══════════════╝  ╚══════════════╝  ╚══════════════╝
        ║║ (3)           ║║ (3)            ║║ (3)
        ║║               ║║                ║║
╔══════════════════════════════════════════════════╗
║                Database Server                   ║
╚══════════════════════════════════════════════════╝

Some things that we've already figured out:

  • The Clients (External Systems, Frontend Applications) will access the different Backend Modules using the Discovery/Routing pattern. We're considering the mix of Netflix OSS Eureka and Zuul to provide this. Services (Modules A,B,C) registers themselves (4) on the Service Registration Module and the API Gateway coordinates (1) with the Register to find Service Instances to fullfill the requests (2).
  • All the different Modules use the same Database. (3) This is more of a client's request than a architecture decision.

The point that we (or me, personally) are stuck is about how to do the communication between the different modules. I've read a ton of different patterns and anti-patterns to do this, and almost every single one will tell that API Integration via RestTemplate or some specialized client like Feign or Ribbon.

I tend to dislike this approach for some reasons, mainly the synchronous and stateless nature of HTTP requests. The stateless-ly nature of HTTP is my biggest issue, as the service layer of different modules can have some strong bindings. For example, a action that is fired up on Module A can have ramifications on Modules B and C and everything needs to be coordinated from a "Transaction" standpoint. I really don't think HTTP would be the best way to control this!

The Java EE part inside of me screams to use some kind of Service Integration like EJB or RMI or anything that does not use HTTP in the end. For me, it would be much more "natural" to wire a certain Service from Module B inside Module A and be sure that they participate together on a transaction.

Another thing that needs to be emphasized is that paradigms like eventual inconsistencies on the database are not enough for our client, as they're dealing with some serious kind of data. SO, the "I promise to do my best with the data" does not fit very well here.

Time for the question:

Is this "Service Integration" really a thing when dealing with "Microservices"? Or the "Resource Integration" wins over it?

It seems that Spring, for example, provides Spring Integration to enable messaging between services as would a tech like EJB do. Is this the best way to integrate those services? Am I missing something?

PS: You may call my "Microservices" as "Microliths", how we usually name them around here. :)

  • It doesn't seem like there's any reason to put a network boundary between your modules, and quite a few not to. The whole system would be far simpler (no need for registration, discovery, gateways) and more performant (no network overhead when A needs to call B and C) if you built a monolithic app. – jonrsharpe Jan 17 at 9:29
  • This is the classical example of a distributed monolith. I suggest, that you rethink your architecture. – Thomas Junk Jan 18 at 9:51
4

I think you correctly diagnose the problem, Micro-service architecture is designed for scalability and distributed systems. You don't want them to talk to each other, or use the same database.

Either, go for the distributed stateless approach. Or, merge the services to make one macro-service. You can still separate the code out into service classes etc, but you can manage transactions and inter service communication better.

Alternatively you could implement a manual transaction with some database tables. But it's going to be a hacky solution.

  • Since it is one DB, you could implement a state model; and only if data is in the mode accepted or so, it is considered the current truth. But yes »it's going to be a hacky solution.« Fully agree. – Thomas Junk Jan 18 at 10:05
1

My idea is that the pattern being used here with "microservices" is not applicable for what you're trying to accomplish. However that said, the next best thing is to make each step as stateless and independent as possible from one another.

They all share the same database perhaps, but I would make it in such a way that each of these microservices has its own database configuration and connection pool. The line to process should be identifiable using a numeric id, and this id gets passed to each service in the request to process it for each successive step.

To avoid issues where you don't know where the last successful step was performed when errors occur, you should be updating the status of the record associated with this id every step of the way. Therefore if a request to perform a change to record #347 doesn't have the state you expect it to have at that stage, the service should return a bad request status and should go no further.

To emulate a transaction, you would need to create the record associated with an id, and call each microservice with that id in order to ensure that that record is processed properly. If like a transaction, you need to "roll back" an operation when an error occurs, each microservice should have a rollback option as well, which given the state of last successful step, you perform a call to each microservice in reverse order with this rollback option to completely go back to its previous state. Even if this rollback operation does nothing else, it should restore the previous state.

So to sum up:

  1. Each microtransaction should have its own dedicated connection settings.
  2. Each microtransaction should know the state of the record to process beforehand.
  3. Each microtransaction should know what final state to put it in.
  4. In case of error, each microtransaction should reverse its operation and restore the previous state.

In this way, your microtransactions remain as stateless as they can be while providing you a means of performing transactional multi-step processing.

  • Assuming microservices share a database is a pretty big assumtion – Esben Skov Pedersen Jan 18 at 7:17
  • @EsbenSkovPedersen "One (and I think the biggest) difference relies on the fact that we're using the same shared database along the different modules" It's a safe assumption to make, seeing how the OP explicitly said so. – Neil Jan 18 at 7:21
  • Fair point. I was reading something else into "perhaps" – Esben Skov Pedersen Jan 18 at 14:06
0

I do agree with Neil. Although we have taken the approach Neil has propsed above (as we wanted full event choreography), you can consider Saga or 2PC aprroach, as eventual consistency is ruled out, and you need atomicity of your transaction. One more point for consideration would be, your workflow as how many microservice/module, you are touching for each workflow. If it is fairly low(less than 5 is a safe assumption), then you can ditch saga pattern, else it is recommended to consider saga.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.