Mircroservices - Ordering of Integration Events

Question

I'm currently learning about event driven architecture and have the following 2 microservices.

• Product Service - Responsible for managing products
• Ordering Service - Responsible for processing orders

The Product Service publishes the following events whenever a product is created

• ProductedCreatedEvent
• ProductedUpdatedEvent

The Ordering Service consumes these messages and updates its own local Product data store.

Questions

The ordering of the events seems quite important as I don't want the ordering service to consume a ProductUpdatedEvent before the product has actually been created.

I've been looking at Azure Service Bus as this can guarantee ordering however their latest product, which is designed for event driven architecture, Azure Event Grid, doesn't support guaranteed ordering.

Should you only use a messaging system which supports FIFO in a microservice architecture?

If the ordering cannot be guaranteed through the message system, what other ways can we address this issue, if it is indeed an issue.

Answer 1

Deciding if order is important is up to each microservice. In your case, it definitely is. Your example above is actually easy to solve since if the OrderService gets an UpdateProduct event when the product doesn’t exist, it can put the UpdateProduct event back on the queue assuming there’s a Create event coming soon. However, two UpdateProduct events arriving out of order could cause problems so we still have to solve the issue.

There are ways of ensuring order in the publisher and subscriber but they are clumsy. If your Update events contain the entire new state of the product, you can add a timestamp and refuse to update your cache if the incoming timestamp is older than the persisted one. If your Update events only send updated data, you can put an incrementing version number on your aggregate and if an incoming event is more than 1 greater than your existing version, requeue the event assuming the intervening event is coming soon. As you can see, these are clumsy. You’re much better off ensuring order in the queue.

You may consider Azure Event Hub instead of Azure Event Grid. The Event Hub will guarantee order in a single partition, so wisely partitioning your app can give you the order guarantee you need. The Event Hub acts more like Kafka than a traditional Queue in that it provides a queue plus persistence of events for a small number of days. This can be advantageous if a system goes down and needs to recover.

Answer 2

Timestamp based solution:

The incoming messages are:

{
 id: 1,
 timestamp: T2,
 name: Samuel
}
{
 id: 1,
 timestamp: T1,
 name: Sam,
 age: 26
}
{
 id: 1,
 timestamp: T3,
 name: Marlon Samuels,
 contact: 123
}

And what we expect to see irrespective of the ORDER in the database is:

{
 id: 1,
 timestamp: T3,
 name: Marlon Samuels,
 age: 26,
 contact: 123
}

For every incoming message, do the following:

1. Get the persisted record and evaluate the timestamp.
2. Whichever's timestamp is greater that's the target.

Now let's go through the messages:

1. T2 arrives first: Store it in the database as it's the first one.
2. T1 arrives next: Persisted one (T2) & incoming (T1), so T2 is the target.
3. T3 arrives: Persistent one (T2) & incoming (T3), so T3 is target.

The following deepMerge(src, target) should be able to give us the resultant:

public static JsonObject deepMerge(JsonObject source, JsonObject target) {
    for (String key: source.keySet()) {
        JsonElement srcValue = source.get(key);
        if (!target.has(key)) { // add only when target doesn't have it already
            target.add(key, srcValue);
        } else {
            // handle recursively according to the requirement

        }
    }
    return target;
}

Let me know in the comment if you need full version of deepMerge()