Whenever I am asked how I would scale out an application I inevitably tend toward a queueing model: split the application responsibilities into individual services and add queues between those services. This means you can spin up more or less instances of a given service as required and they simply pull from the queue to get work.

Get enough of these services with different roles and almost inevitably there is the creation of orchestration layer - something that understands the necessary flow of a work item through the queues and manages that end to end.

What are some of the alternative approaches for scaling out applications that don't use queues and end up with an orchestration layer?

Update based on comments

As @tofro pointed out, I'm probably talking more about elasticity instead of scalability https://stackoverflow.com/questions/9587919/what-is-the-difference-between-scalability-and-elasticity.

Here is an example. Let say I have a service that does video encoding. A user uploads a file, selects one or more different encoding formats (quicktime, divx), the file is encoded into the formats and the user can download the resulting output files.

One way to make this elastic using queues would be to have different services, QuickTimeEncoder and DivXEncoder, with queues QTEQueue and DivXQueue and put jobs on queues as required. More instances of the encoders could be added over time as demand changes.

  • Queues are just one way to do middleware. Depending on your requirements, what about fire-n-forget events with sagas, priority queues, etc.. Also, the concept of a "queue" comes with a bunch of parameters: are the messages persisted and guarenteed-to-be-delivered, optimised for speed, what happens in case of a queue overflow, etc. etc. etc.. So, to me, it sounds like you need to rethink your question, because it sounds like "a queue" = "interprocess communication" for you. And it's not really possible to build anything scalable without interprocess communication.
    – cwap
    Nov 2, 2016 at 13:28
  • MapReduce might be what you are looking for - it is probably very similar to what you have in mind by queues, but existing MapReduce frameworks provide you - at least partially - with the necessary orchestration infrastructure in a generic manner.
    – Doc Brown
    Nov 2, 2016 at 13:32
  • 3
    Queues don't in my opinion handle scalability and thus are no fundamental requirement for it. What they give you is elasticity, which is a different thing.
    – tofro
    Nov 2, 2016 at 14:03
  • What is your motivation for eliminating queues? The reason that scalable architectures have queues is to maximize the efficiency of your "agents;" without queues, your agents would all have to wait to pass control to the next agent. Nov 2, 2016 at 14:44
  • I don't want to eliminate queues, i'm just curious to know if there are viable alternatives
    – user783836
    Nov 2, 2016 at 15:17

2 Answers 2


In the absence of queues, you must spin up a new agent immediately for each new task to be executed; the new agent holds the task instance instead of a queue.

The Erlang programming language is capable of doing this, because it has the capability of spinning up millions of lightweight agents.

Note that Erlang also has a queue module, so it still gives you that option.

  • is it possible to spin up agents on remote machines?
    – user783836
    Nov 2, 2016 at 15:12
  • 1
    Erlang doesn't have any notion of shared state, so any time you need to preserve state, you have to spawn a process (Erlang's term for an agent) to hold it. The only way to communicate with the process is via its mailbox which, amusingly, is a queue.
    – Blrfl
    Nov 2, 2016 at 15:14
  • @user783836: Yes, it is. In fact, if you have a handle that describes a process, where it lives is completely opaque. Give Learn You Some Erlang for Great Good! a read. Even if you don't use the language, it will change how you think about concurrent systems.
    – Blrfl
    Nov 2, 2016 at 15:15

The easiest way to scale an application in my experience is to make it stateless and add instances. This is basic horizontal scaling. I an approach that is a little more sophisticated is to decompose the service across a your resources and then use orchestration to compose these various pieces into the needed functionality. There is no need for queues to do this. HTTP works just fine.

Queuing used to play a more prominent role than it does in contemporary design. When memory was highly constrained, it was important to be able to have a place to store a mass of transactions in order to push them through the small pipe of memory. Now we have 64 bit memory addressing and it's cheap and common to have more memory than you need. Systems with 1TB of RAM are nothing special.

The other aspect of queuing is that it could be used to manage IO waits. That is, you would have threads doing little pieces of work and moving the transaction on to the next thing instead of making blocking calls and waiting. With non-blocking IO libraries and other faster approaches to concurrency distributed queuing has become more of a niche approach. I think it still has a place in asynchronous processing but as @tofro mentions in the comments, the need for scalability doesn't necessarily indicate distributed queuing. There are lots of complications and problems that come with queueing as well as a lot of baggage about 'guaranteed delivery' which doesn't guaratee much in reality. I would avoid it unless you have specific problems that it solves better than the other techniques at your disposal.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.