0

I have a distributed platform which allows customers to make purchases, and the items which are purchased are stored in an inventory:

Sales app -> PurchaseEvent -> Inventory app

The Sales app raises the PurchaseEvent onto a message bus, which is asynchronously consumed by the Inventory app. This all works great.

There's one piece of functionality which makes it possible for two customers to be merged into one. When this happens, a CustomerMergedEvent is raised, and the Inventory app consumes this to update its data (so that all inventory for those two customers is now under one merged customer).

All is smooth when everything works fine. The challenge arrives when there is a performance backlog in PurchaseEvents being consumed. Any purchase consumed by Inventory after the CustomerMergedEvent has been consumed, will not know the customer merge has taken place. We'll also not even be alerted to the fact this has happened.

We could make it so every customer merge results in a new customer, and have the Inventory app alert us if it receives information about a customer which no longer exists. But are there solutions which solve this time-related issue with events on a higher level?

2
  • Does your Inventory app know about the backlog in PurchaseEvents? Does your message bus allow you to explicitly pull the PurchaseEvents relating to a particular customer to expedite the processing of those events? Apr 6, 2023 at 11:57
  • It has no knowledge of the backlog, and there's no sense of event priorities in terms of consumption or anything like that. I worry that might become very complex as things scale
    – FBryant87
    Apr 6, 2023 at 12:00

2 Answers 2

3

When merging customers, you just keep both of them and have a merged_into field in one of them which points at the other (or you create a new customer and have such merged_into fields in both original customers).

When you process a PurchaseEvent for a customer who has been merged into another one, you just handle it for the target customer instead, possibly following merged_into fields recursively if needed.

1

I guess what you are looking for is to process the events in the order they come in. with the additional wrinkle of the events being on different queues?

This is a common built in functionality to many MQ style products, but.. I'm not sure they will work across queues out of the box.

The standard way of dealing with this type of aggregated processing, ordering, batching etc is to have an intermediate router processor which divides up the events as they come in and orders and batches them before passing them on to the real workers

so for example you could have

Sales App -> purchase queue

Admin App -> merge queue

merge queue -> delay router
{
    are there pending purchases for affected customers?
       wait/skip/put to back of queue
    else
       send to real merge queue
}

purchase queue -> pending count router
{
    check customer id
    increment count of pending purchases for customer
    send to real purchase queue
}

real purchase queue ->
{
    process
    decrement pending count for customer
}

Obviously even if you can do this you can still have lock issues while the merge is processing. But i'm guessing you have thought of several possible solutions to that already and are only interested in the queuign problem.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.