Does creating one thread per event subscriber scale well?

Question

I'm working on an application implemented in different microservices, which uses a message broker (event bus) to consume events (simple pub/sub).

As for now, in a given microservice, we are spawning 1 thread per event subscriber, which will be running in background checking for new published events (specifically) to process. I was told that the idea behind this approach is to avoid one event consumer to shutdown other subscribers in the case of an unhandled exception in the processing of this specific event.

The problem as I see is that this approach doesn't scale well. As our application grows and different events are created, this list of active threads will grow as well.

I think we should stick with 1 background thread responsible for handling any new event needed by a given microservice and handling the processing of an event in the same thread via async/await or use a queue to dispatch the processing of each event; and implement strategies to ensure that this thread will keep alive for the duration of the application.

Which approach is better in this scenario? One thread per event subscriber or 1 thread for processing all events?

Answer 1

One thread per subscriber may not be ideal if there are many subscribers. Each thread has some memory overhead, and if each needs to check something they will also use some CPU time.

If you are concerned about exceptions a simple try/catch should be sufficient. There are uncatchable exceptions, but if you get one, your process will probably be killed anyway. If you want isolation between subscribers you probably need to use separate processes.

If events can be processed concurrently it might be useful to use a limited number of threads to raise events, for example by having a shared queue of events to process, and multiple consumers of said queue. This could help by avoiding delays if some events can take a long time to process.

The easiest way to do it might be to just use tasks, like Task.Run(MyEventProcessing). This will catch any exception, causing the task to be put in a failure state and not crash the application. This will use threadpool threads for processing, and the pool has a soft upper limit. Just make sure to add appropriate logging etc. so errors do not escape undetected.

Answer 2

Sounds like a ThreadPool is exactly what you want:

The System.Threading.ThreadPool class provides your application with a pool of worker threads that are managed by the system, allowing you to concentrate on application tasks rather than thread management.

Source: https://docs.microsoft.com/en-us/dotnet/standard/threading/the-managed-thread-pool

The whole point here is to keep a number of threads available and ready to perform the work required when required without having to create and destroy them between each operation. This saves resources, and should be a lot more effective (and simpler!) than trying to manage it yourself.