-2

I have X number of servers in NLB listening to an event that gets published by a backend system.

I am planning to develop a windows service that listens to the event and then calls an internal API. Here, each server in the NLB is subscribing to the event.

Is there a way that I can make sure that my API is called only once for a published message instead of X times (once per server) ?

The API I am calling updates the data and I don't want the data to be updated more than once.

2
  • You could redesign your system to use a FIFO queue which would allow a single instance to retrieve the message. Or allocate a lock to the first instance and allow it to make the API call while blocking all other instances. – Dan Wilson Apr 30 '20 at 17:21
  • 2
    Sharing your research helps everyone. Tell us what you've tried and why it didn't meet your needs. This demonstrates that you've taken the time to try to help yourself, it saves us from reiterating obvious answers, and most of all it helps you get a more specific and relevant answer. Also see How to Ask – gnat Apr 30 '20 at 17:31
0

Processing an event only once is an extremely hard problem to solve in a distributed context like you describe.

Instead of reinventing a (very) complex wheel, you should consider:

  • either to rely on middleware that can handle this challenge for the event queue,
  • or think of a more robust design, in relation to multiple processings of the same event.

For the first option, Kafka for example has this feature since 2017. Here a blog post from Neha Narkhede (Confluent's co-founder and Chief Technical Officer) who explains how this works. Note the the "exact once" delivery just ensures that that the event is delivered exatly once: you'd still have to add some logic to ensure that the processing takes place.

For the second option, you could check if you really need "exact once". If it is confirmed, you could try to design your processing in an idempotent manner. If this is not possible, it becomes more complex: you'll need to combine transactional processing (e.g. a shared database, but this might become a bottleneck if you have very high volumes) with some trick to avoid repetition (e.g. keeping a hash of the messages already processed, keep the id or the timestamp of the last message processed if your middlewaren ensures delivery in-order).

0

You can't stop multiple calls. But you can stop multiple updates.

When the event happens you'll get X calls from X servers observing the event. Have each server sync an event count though and they'll all be saying:

Ooh oh its happed 3 times now!

Update the first time you see the 3 and ignore the rest. Now wait for someone to mention a 4.

One of the X servers goes down and misses a few events? Have it ask which number you're waiting for.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.