6

I've created about a dozen services for a intranet. Now I've gotten to the point that these services are more coupled to each other than I'm comfortable with, and I've been having problems where once service can cause a few other service to degrade.

I've been working on a EDA design so that I can make these services decoupled from each other but as I'm the only programmer here I need some feedback.

The design aims to solve these scenarios:

Scenario 1: Client requests data from Service A, to fulfill that request Service A needs data from Service B but Service B has failed and is unresponsive

Scenario 2: Client updates an entity in Service B, the reference to this entity needs to be updated in Service A and C

I've split the design into two parts, Operations and Events

An operation is sent by a service or client to change an entity in a service. This always results in an event being fired. Each operation contains a user id, operation id and a few timestamps.

An event is a reaction to a operation, it notifies anyone who's listening that a change has occurred. Each event contains the id of the operation that started it, the id of the user that caused it and some timestamps. Each event also has a hash of the complete entity so that any listeners can compare against their versions.

I'm using RabbitMQ as a message bus and each service has persistent durable queues to store any pending events or operations.

In order to decouple service from each other, each service caches any entities it depends on that belong to another services.

When a service requires a entity from another service it fetches it and stores it in a table in its database. It then listens for any changes made to any entities in its cache and updates them when an event related to it occurs.

When storing or fetching an entity from the cache fails, it will re-fetch it from the appropriate service.

For auditing, I've got a service that listens to events from all services and stores them. It can then recreate any entity at any given point in time.

I'd love comments from anyone more experienced than I am and whether there are any glaring issues with the design.

2
+50

I got a couple of things I'd like to point out:

  1. I'd refer to commands instead of operations. That fits the lingo around the system you're modeling better I think.
  2. It seems like your using the events as a trigger to retrieve data from other services. Will you also use the events to create those entities in the first place? If that's the case, I'd strongly suggest to introduce some form of EventStore for each application to store the events in which you're publishing. You can then reuse those events to recreate your entities if necessary. Most people refer to this as Event Sourcing, although I prefer to use that term for the write side (the 'operation handling' in your example) of the application. Nonetheless, I'd create your entities on those events as well. If you'd take that approach, you could even let the other services listen to all those events and create the entity they'd need themselves, which will omit the need to query the entities.
  3. You point out you'll add another service for auditing. If you follow my suggestion from point 2, your event store will automatically also be your audit log. This will of course only hold if you model your events correctly, but nonetheless I think making your events first class citizens and store them immediately after publishing omits the need of a dedicated audit-log service.
  4. Without knowing what programming language you are in, I'd still suggest taking a look at some CQRS/EDA/DDD frameworks out there, like the Axon Framework for example. It's Java based, but even if you're not on Java, the concepts the framework tries to help you with resemble what you're trying to do.

All over, I think that remodeling your application set to something like you're suggestion is a good thing. Making your communication message based and your services location transparent ensure that you lose the coupling you're worried about at this moment.

  • Thanks for the response. #2 & #3 I was planning on implementing event sourcing in the auditing service and use that as a centralized event store but rather than using the events as the single source of truth, I'd store the current state of each entity in the service that owns it. The audit service could then recreate entities at any point in time.This is mostly because some services are used to read and insert data into databases that belong to 3rd party services and I can't modify those to fit the event sourcing model. I'd prefer that all the services were modeled in the same way. Cont. – grimurd Feb 19 '18 at 16:46
  • #2 I'm planning on using the events to create entries in each services entity cache when It's possible. #1 Good point, better get the naming right. #4 I'm using Node but I'll definitely look into those. – grimurd Feb 19 '18 at 16:54
  • 1
    For Node I believe a nice framework to use is Wolkenkit. Haven't worked with it myself, but I believe it's somewhat similar as to what the Axon Framework does. Thus, any lessons learned from how people approach an Axon Framework project would probably apply to your situation. – Steven Feb 21 '18 at 8:46
2

First of all, if service A needs data from service B, then your service boundaries are wrong. It seems that the criteria you used for splitting your system is by entity. It correlates to layered services concept, which was already considered evil back in 2007.

The approach I'm advocating is for is based on the concept of business capability. You identify some areas in your domain that are really very cohesive, that comprise of some encapsulated data used only within that area, business-processes implemented there, applications, and people. Then you delve deeper in those areas and repeat that step. I find it convenient to use a value-chain analysis for that. Consider you prime business-process. How do you make money, what's the main reason of your business' existence? Take a look at this process as a sequence of steps. If some step contains some business-processes that are not connected with any other steps, than probably these steps can be a standalone service.

Your technical services map to those business-services as 1:1.

There is a series of posts on SOA that contain some examples as well, check it out.

  • You definitely have a valid point there about these services being entity based. Unfortunately there isn't much I can do to fix these boundaries. Many of these services are management for 3rd party systems which have their own databases that I'm connecting to. These 3rd party services form an physically interconnected/dependent system of their own and my services are used to provide customer support and the support agents need to be able to do some cross referencing between these systems. To do that I had to reference data in other services directly (which I didn't want to do) Cont... – grimurd Feb 20 '18 at 20:03
  • The only way I can see is to create another entity to contain these relationships and another service for managing those entities that could fetch the required data from all the other services. But that still feels like I'm only moving the problem around. On your other point, value-chain-analysis. The business is very small and mostly based around a single source of revenue which all these services are for. Would probably work better in a bigger more diverse business. But these SOA posts are very interesting, I'll have to read those. Thanks for your response. – grimurd Feb 20 '18 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.