I'm interested in integrating with an external system which uses webhooks to notify clients of events. The system is very similar to Stripe - the REST endpoints have rate limits to avoid undue polling load and instead relies on clients/users receiving webhooks to keep track of state. In the rest of this post, I'll write as if I'm integrating with Stripe since it's similar and familiar.
I'm interested in what it would take to build this webhook integration as some sort of event stream. I'm imagining that I would be able to save each event I receive into my database (Postgres) or stream (Kafka) and then build out some form of projection to aggregate these events into views I care about. For example, having a projection of
all products purchased by each user.
Normally, I feel like I would be able to handle this quite easily. However, one issue I run into is that Stripe (and the system I'm integrating with) doesn't:
- guarantee the order of delivery of webhook events
- guarantee high fidelity timestamping of each event - events delivered have a
created_atattribute which is a measure of seconds since epoch (not milli or nanoseconds)
For example, the Stripe docs mention
Stripe doesn’t guarantee delivery of events in the order in which they’re generated. For example, creating a subscription might generate the following events:
- charge.created (if there’s a charge)
Your endpoint shouldn’t expect delivery of these events in this order, and needs to handle delivery accordingly.
These events may all be delivered with the same value of
created_at which means I can't rely on that field to order the events.
Is there a well known design pattern for handling aggregations of events which are not guaranteed to be in order?
One idea I have is to provide my projections with a list of events that occurred with, say, 3 seconds, and let the projection do various permutations of the order to figure out which is "correct". That being said, I know this idea isn't scalable (what happens if 100 events are generated with 1 second) and is quite brittle (what happens if there are issues in with the upstream service?).