Let's say I have a system that parses network traffic, and generates event of things that happened. example:
- ip1 was talking to ip2
- ip2 was talking to ip3
Now let's say that these events are published into event sourcing thing (like kafka for example).
I have another service (service B) that listens to these events, and keeps track of all IPs in the system. so that the work of service B is like:
ip = read_ip_from_event() if not ip_exists(ip): add_new_ip(ip)
add_new_ip also publishes event (
IpAddedEvent) in kafka.
Now here is my question: Let's say that I have a lot of traffic and I want to have multiple instances of service B. how can I make sure that eventually I get only one
An issue can happen when ip doesn't exist, and two instances of serviceB reads from kafka (two different events with same ip), and both check
ip_exists at the same time, and then both publish
How can I avoid that, publishing only a single event?