why and when is queues used in backend architectures?

Question

I am implementing a backend with a typescript framework called nestjs and I was reading some documentation about their usage of queues. https://docs.nestjs.com/techniques/queues I have never used queues between services and I don't understand why or when I should either. (hence my question)

So I had some thoughts regarding some of the statements regarding why queues are useful in the documentation. This is just to provide context to my question.

Smooth out processing peaks. For example, if users can initiate resource-intensive tasks at arbitrary times, you can add these tasks to a queue instead of performing them synchronously.

1. If a user has to initiate a cpu intensive task on the server, and the queue should handle performing the code asynchronously (waiting until the server isn't under load), then I don't see it being directly beneficial, since instead of a lot of users should wait a bit for the user asking for the backend to do the syncrhonous task to be performed then the user asking for it would potentially wait a long time. If neither of the outcomes were relevant, then the queue wouldn't matter in the first place?

Break up monolithic tasks that may otherwise block the Node.js event loop. For example, if a user request requires CPU intensive work like audio transcoding, you can delegate this task to other processes, freeing up user-facing processes to remain responsive.

2. So if a user would maybe upload some music to a external service out of the scope of the nestjs application, then the queue api would assign the task to the queue and respond a 201 to the user? Isn't most webservers implemented with queues in the first place? Since a server can respond to multiple requests asynchronously? Would this only be relevant if the external service uses another protocol than http then?

So why and when is queues used in backend architectures?

Answer 1

since instead of a lot of users should wait a bit for the user asking for the backend to do the syncrhonous task to be performed then the user asking for it would potentially wait a long time

No one said the work on the queue was being done synchronously.

• Many such implementations will have a thread pool performing several pieces of work in parallel.
• Other implementations would use a Priority Queue to ensure that quicker jobs were done first
• Even better implementations would use both with a high priority pool and low priority pool for obtaining work on.

The relevance of a queue is to ensure that requests once received are remembered till they can be processed.

Isn't most webservers implemented with queues in the first place?

Yes but those queues are on the front end, and once your program receives the request the timer starts. You have exactly X seconds to complete processing before the web server terminates the connection. A Queue permits those requests to keep being processed beyond this limit.

Since a server can respond to multiple requests asynchronously?

Also those requests are processed very quickly (assuming a multi-threaded server) which could easily overwhelm your actual resource budget. Delaying work till later is something a queue is good for.

Queues are also useful for transferring longer running work to other back-end servers tooled for intensive processing.

Would this only be relevant if the external service uses another protocol than http then?

No queues are supremely useful data structures. They are also known as:

• Futures
• Rendezvous
• Channels
• Streams
• Pipes

They are used everywhere.

But more specifically the network protocol has zero to do with how a server should be implemented.

Answer 2

The previous answer by Kain0_0 is very good. In summary, I would add the main need for queuing systems is because of background processing, parallel execution, and recovery from failure.

And operating systems and databases could hardly exist without them.

If you want to explore queueing further, as a starting point, learn what you can about Reddis and RabbitMQ.

Answer 3

Here's another example, lets say you have an app where a lot of users will simulataniously be clicking a button that does some calculation and updates a single field in a database. Maybe the button increments/decrements a single record's count/balance/whatever field. If you have each button click set to to synchronously do the calculation, then you can end up with weird race conditions.

One solution for this using queues would be to have the button click submit an event to a queue first, and then have a subscriber to the queue process the queue messages one by one. This way, all the messages are pushed into the queue damn near simultaneously, but each queue item is processed one-by-one. If the button-click calculation was an expensive database operation, for example, this could be extremely beneficial.

Now of course you now have the UI/UX complexity that you'll need to figure out a strategy for informing the user about the calculation result, so make sure the whole thing is worth it to you.