1

Lets say there are two services. One generates event A and the other event B. We need to build a new service that implements the function C = func(A, B) , which produces the result C.

But as A and B are asynchronous events, they may reach (and invoke) the new service separately. Depending on what is required, it may be apt for the new service to:

  1. either wait for both events,
  2. or implement different variations of the function: C = func(A,C') or C = func(B,C') where C' is the previous state of C (assuming that it is even possible to build that function)
  3. or query back to retrieve the complimentary event, e.g. if A invoked the service, then query for B.

Question:

Lets say the requirement is that we update the state of C ASAP, and A and B have a significant time gap in occurrence - that rules out option #1.

Additionally assume it is hard/impossible to build different variations of the function (#2).

That leaves us with option #3, which probably has drawbacks because it may be breaking the event driven paradigm (or maybe not, I am not sure).

How should we solve this problem? I expected this to be a common problem with a recommended pattern, but I could not find it. Any suggestions or pointers on where I could find the answer, or if the question is wrong, how should I look at it?

3

Whether it breaks the event-driven paradigm should not be your primary concern. The question is if this solves your business problem.

There are a number of questions that need to be answered:

  • If A and B are events, it doesn't really make sense to query them. But I believe you mean they represent some state, and an event is raised when the state changes.

  • Will querying service B yield a different result than simply taking the latest received event? In other words, could you simply call func(A, B') where B' comes from a cache? What do you do if the cache is empty? And do you need to invalidate it at some point?

  • A query will cause the client of service C to block until service B responds. Will that be a problem? What if service B is unreachable?

  • What happens if you receive a new A or B event while you are querying?

These are general problems of asynchronous systems and finding the right trade-offs is very specific to the requirements of your particular application.

Fundamentally, you just can't guarantee to get the most up-to-date state if your state is distributed. That means you either deal with temporary discrepancies (read up on 'eventual consistency') or you find a way to make the relevant parts synchronous (giving up the advantages of the asynchronous approach).

  • thanks, yes i did assume 'they represent some state, and an event is raised when the state changes', but again that assumption is not really inherent to this problem. I also think this problem is related to event-driven ETL, confluent.io/blog/changing-face-etl but a lot of blogs do not talk about this scenario, where data enrichment or data transformation is from other events. like you said, i have to weight in trade-offs for my specific problem, but I like the idea service C caches only latest A or B, and deal with empty cache. – Avinav Apr 30 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.