0

I have read in a DDD book that using message queues between communicating services can make the whole architecture more scalable, amazon's documentation mentions that queues provide granular scalability. I don't see how it is 'more' scalable than services communicating synchronously using http. I agree that it scales more gracefully, for eg., if we use the queue size as an indicator to grow and shrink the number of consumers if required.

But that doesn't mean that it is impossible to scale without queues right ? And for granular scalability, any microservice based architecture can scale individual services independently to achieve the same right ? Am I missing something here ?

Note: I'm only curious about the scalability side of this, I'm aware of other benefits of queues like better decoupling, availability. I agree that queues scale more gracefully but don't see why its impossible without them.

Improve this question
1
  • 2
    Your use of "impossible" could mean that you're thinking in a way that's too black & white. Even assuming that there's no hard limit in terms of what's theoretically possible, in practice, if something scales, as you've said, more gracefully, then you can scale it to a larger extent before you hit so much complexity that it becomes hard to control all of it (too resistant to change, too much knowledge for various teams to all share, too many interactions to keep track of, too many unpredictable circumstances to be robust against, too many factors that can degrade the design).
    – Filip Milovanović
    Apr 7, 2021 at 20:37

1 Answer 1

Reset to default
5

It's not that queues are more scalable, its the fact that two services communicating through queue means the communication is asynchronous. Asynchronous communication is far more scalable than synchronous communication, as it allows the writer (who enqueues messages into the queue) and the consumers (who dequeues the messages) to progress in their own rhythm.

Let's say you have a REST API service which is being called from your client side and perform some long running operation task.

If the communication was synchronous (the API service doesnt return a HTTP status code before it finishes the task), on high volumes of traffic your API service may crash as it will be too loaded or it will not finishes his task on time, which will eventually result in HTTP timeouts.

If the communication is asynchronous, your API service will only queue messages that will be processed by some background worker, and it will be able to return HTTP response message relatively quickly.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.