How to design a dynamic thread allocator

Question

Forgive the title if it isn't correct, I am just not sure what I am trying to design is called.

Say I have a number of tasks I want to run, N.

Additionally, I have a maximum number of threads I can allocate at once, T.

I want to run as many of these tasks in async as I have threads available (up to T), and once a thread frees up I want to immediately start a new task on the thread that just finished until all tasks have been run.

EXAMPLE

say I have 10 tasks to run and 4 threads available to run these tasks. As the app starts, 4 threads begin running 4 of the 10 tasks, leaving 6 remaining tasks, when one of the running tasks finishes, immediately another task starts running on a new thread and there are 5 remaining tasks. This goes on until there are no more remaining tasks to execute.

QUESTION

Is there a design pattern or common application architecture I can study to build a task runner like this?

Answer 1

The design pattern that most closely matches your description would be a Producer Consumer pattern with a queue between each side.

For your exact specs you would have 4 threads consuming the queue which will limit to 4 items being worked on at one time.

Depending on the language there may be built in constructs for multi threaded queue concurrency (C# System.Collections.Concurrent for an example) or you could use an external queue server to manage locking for you (RabbitMQ, MSMQ etc).

Additionally there may be constructs like the background Thread Pool in .net which will manage some of the complexity of actually running the threads but that will depend highly on the language this will be developed in.

I use the producer consumer pattern extensively in a C# application that I developed using both in memory and external queues. As long as you take care of locking so that only a single consumer grabs each queue item it provides a very simple way to distribute work among threads.

(For my specific implementation I use System.Collections.Concurrent.BlockingCollection for some of my queues and System.Threading.Tasks (the Task Parallel Library) for managing my consumer threading.)