async notifications delivery

Question

i have some back end service that is in charge of sending notifications to my end users via mapping between some identifier to their real web sockets.

currently the service works in a sync way, meaning every loop iteration it checks if i got some new messages in my queue and if so i am polling the message from the queue and sending it to the relevant users based on their identifiers in a sync way(until the message is not sent the loop wont continue).

i came up with an idea to send these messages in async way. meaning that after polling a message from the queue i will delegate it to some thread pool to deal with it and send it when possible.

i have some concerns about this solution.

first, is there any scenario that messages will be received by the user out of order?.

second, will i see an improvement in the time the user will receive his messages? the sending involves I/O operations so i thought it will be a good idea to make it asyc.

are there any other concerns you think i should consider?

Answer 1

I see this question is unanswered, so, I will attempt on answering it but this will be my first answer on stackexchange so please go easy on me.

A Lot of things depend on whether the order of the messages matter to you, if yes, maybe async way is not the best way to go.

Now, there are multiple async patterns which you can follow while sending async requests and you can read about them here : https://blogs.mulesoft.com/dev/design-dev/asynchronous-messaging-patterns/

1. "is there any scenario that messages will be received by the user out of order?. " : Definitely, this is a good possibility. Say, you are sending the message to a user asynchronously and it fails but then you have also sent another message (since you are not waiting for the failure ack) and it succeeds,then in that scenario the 2nd message will go first.

   Also, in the era of distributed system, this is always a possibility where there two messages are sent async to two different consumers or machines and the second message's IO processing gets done faster so the order of messages will change.

   This is also explained here : https://cloud.google.com/pubsub/docs/ordering

2. Good performance improvement will be observed. Since you are not waiting for your message requests to be completed and the message includes I/O operations, performance is bound to get improved as I/O operations are getting handled in the background and your app is not waiting for it. Add some horizontal scaling and you will see tremendous improvement in latency and end-to-end time.

Pitfalls :

1. Order of messages can not be guaranteed and there are very few use cases where order matters
2. Now, you are not waiting for an ack or a response, you have to handle failures and retries in a different way.
3. It is always better to profile out both approaches and notice the performance yourself. No two systems are same and there are different requirements for different systems. It is better to experiment it out for 5% requests and see if the performance improves.

Hope this helps and all the best.