Currently my architecture is very much event driven. Every API request is served as quickly as possible sending events that run after the response.

However this is a monolith and the event handlers run under the same process that manage the API. Responding a single API request is efficient, but the process CPU will quickly choke managing API + background work.

I have two ways to scale this:

  1. Offset background tasks to workers

  2. Make a cluster and load balance the same process

Although both approaches should be implemented on the long term: Which one should be the first one to scale?

My assumption is that a non healthy monolith would be easier to fix with option (1), while option (2) covers earlier important infrastructure needs (so it will be optimal for well structured monoliths)

  • It would be helpful to describe a little bit more how the architecture of your system currently looks like. Just "web" and "event driven" and "monolith" is, lets say, "a little bit" vague. – Doc Brown Dec 31 '17 at 8:34
  • vagueness 1 is always better than vagueness 2 given hand wavey assumptions 3 – Ewan Dec 31 '17 at 8:45

I have too little information to give a definite answer, but I have a packet processing application where the approach (2) is preferred over approach (1). I got over million packets per second more performance with approach (2) when compared with approach (1). The reason for this improved performance is reduced inter-CPU communication costs.

The approach (1) requires you to dispatch tasks to workers from a central thread. This means the central thread needs to allocate memory that the worker threads then free. This means there is unbalanced allocation flow (one thread allocates, another thread frees) which makes practically any reasonable allocator slow, meaning only few million blocks per second of performance.

Also, you cannot use blocking queues to transfer work blocks from one thread to another thread at a performance of more than few million operations per second unless doing some weird performance-enhancing tricks.

Now, I don't know how big is you task rate. Thousands of tasks per second? Millions of tasks per second? I can see benefit in the worker approach if your task rate is merely thousands of tasks per second. At millions of tasks per second, you'll start to find that the worker approach is inefficient.

  • I was assuming that 1 meant sending the tasks off over the network, so much slower than your case! – Ewan Dec 31 '17 at 10:43

I would always go with the worker approach.

Given that you have come to the point where you have hit a problem with the existing monolith design, simply duplicating and having 2 is unlikely to solve the underlying problem.

Additionally, having being built as a monolith rather than a worker process; its probably not designed to be run as multiple instances. You may find that changes are needed before you can simply spin up another instance.

Further more, as you have just hit the problem, there is probably a single event which is hogging the cpu and causing the slowdown.

You can make a small change to move just this event to a worker and leave the rest of the monolith as is.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.