.NET Core Microservice messaging with Amazon SNS and SQS

Question

So I've done some searching but can't seem to find a whole lot of suggestions on this topic. My question is what are some opinions on the best way to receive messages from an Amazon SQS queue on a .NET Core based WebAPI microservice ? The whole project consists of around 5 microservices and is utilizing AWS for infrastructure. I've implemented a basic pub/sub style event bus in which services will publish events to an Amazon SNS topic and then those messages will be delivered to whatever SQS queues are subscribed to it. Since SQS queues must be polled to receive messages, I've been able to come up with a couple solutions.

• Use long polling to continuously poll the queue for messages.
• Trigger a lambda function when a message is delivered to SQS, which will then send an HTTP request to the subscribing microservice and notify it that a message has arrived.

The first option seems inefficient to me as it could cause unnecessary network traffic and tie up resources. The latter option seems like a decent solution to me but may require some extra effort to deal with the competing consumer scenario since the services themselves will be hosted on ECS and may have multiple instances running and consuming messages.

So, I guess my question is does it seem like I'm on the right track here ? Or maybe there are some other solutions that I haven't thought of that someone could propose ? Appreciate any feedback. This is my first post here, so I apologize if I didn't give enough background info. Thanks.

Answer 1

Just use the built in long polling, or even just poll on a fixed interval.

There is no problem of using too much network bandwidth as you only waste bandwidth when there are no messages to receive. This limits the wasted bandwidth to a small controllable amount, say 1 message every 10 seconds per queue.

If you move to a push model then when you have high volumes of traffic you will be pushing thousands of messages, which won't be used as a single Receive call will retrieve multiple messages.

Here you waste far more bandwidth and at an uncontrollable amount.