I am using Kafka. I am developing a simple e-commerce solution. I have a non-scalable catalog admin portal where products, categories, attributes, variants of products, channels, etc are updated. For each update, an event is fired which is sent to Kafka.
There can be multiple consumers deployed on different machines and they can scale up or down as per load. The consumers consume and process the events and save changes in a scalable and efficient database.
Order of events is important for me. For example, I get a product-create event. A product P is created and lies in category C. It is important that event for the creation of category C is processed before the product-create event for product P. Now if there are two consumers, and one consumer picks up product-create event for product P and the other consumer picks up event for creation of category C, it may happen product-create event is processed first, which will lead to data inconsistency.
There can be multiple such dependencies. How do I ensure the ordered processing or some alternative to ensure data consistency?
Two solutions that are right now in my mind:
- We can re-queue an event until its dependent event is successfully processed.
- We can wait for the dependent event to get processed and try processing the event at some intervals say 1 second with some maximum retries.
Requeuing has issues that event is now stale and no longer required. Example:
- Initial Order = Create-Event(Dependent on event X), Event X, Delete-Event .
- After Requeuing, Order = Event X, Delete-Event, Create-Event(Dependent on event X).
Create event is processed after delete event again leading to inconsistent data.
The same issue is applicable to the second solution (waiting and retrying).
Above issues can be solved by maintaining versions for events and ignoring an event if the targeted object(which is going to be modified by the event) has a higher version than that of the event.
But I am very unsure of the pitfalls and the challenges of the above solutions that might not be very obvious right now.
PS: Stale data works for me but there should be no inconsistencies.