We have a system that publishes requests for processing into a database table. We also have a scheduled task that runs every minute and looks for pending requests in that table. If a request is found, the task processes it, marks it as complete, then picks up the next pending request, if any, and so on. If there are no more requests, the task terminates. A next task instance is started at the top of the next minute.

That all works fine, but we now have a requirement to process all of the requests as soon they are added to the database table, i.e., with a minimum delay. Which means we need to be able to process them in parallel, not one by one.

I am seeking ideas on how to implement this. One approach would be to have a monitoring thread in the task that would query the table for new requests periodically, and when a new request is found, span a processing thread for that request. Or, taking a similar approach, split the task in two executables - one for monitoring and another for processing. The former will start instances of the latter on demand.

Any thoughts?

  • In your current system, how long does it take to process one task? Sep 1, 2016 at 20:02
  • From a few seconds to a minute or so. That's why processing more than one request at a time is highly desirable.
    – uncoder
    Sep 1, 2016 at 20:04
  • Next question - in your current system, how many requests are typically pending? Sep 1, 2016 at 20:10
  • And a third question - does all of this run on one box? If so, does that box have spare capacity? (currently low CPU, IO and memory usage?) Sep 1, 2016 at 20:10
  • It varies, but at certain times hundreds of requests can be added over a short period of time. Obviously, processing them one by one causes huge delays in such cases. Most of the processing occurs on a remote server (via web service calls), and that portion is heavy on CPU usage and some I/O.
    – uncoder
    Sep 1, 2016 at 20:19

3 Answers 3


I think you have the right idea - have a monitoring thread/process/application that does nothing but monitor. If it sees tasks that need processing, it hands them off as quickly as possible so it can get back to its job - monitoring.

A very long time ago a lot of Unix daemons (ftp servers, etc) were written that way. One thread would listen on the port for incoming messages. As soon as a message arrived, that thread would spawn a new thread to process the message and the original thread would go back to listening.

Since this is now the 21st century, can you do this the easy way? Convert each task request into a web call then run that through a load balancer to a bunch of worker machines? It sounds counter-intuitive (since processing a single task might take a little longer) but you'll get some pretty easy to maintain parallelization.

Spawning threads on the local box is certainly also an option. If you're using Windows there are some tools that make it relatively easy to manage the thread pool for you so you don't have to keep track of how many threads you've spawned. If possible, try to create "fire & forget" (i.e., non locking) threads.

  • Thank you! "...there are some tools that make it relatively easy to manage the thread pool for you so you don't have to keep track of how many threads you've spawned." Could you elaborate, please?
    – uncoder
    Sep 1, 2016 at 21:13
  • I was thinking of BackgroundWorker, but see also this answer to the question BackgroundWorker vs background thread. Note that the question is ancient - 2009 Sep 1, 2016 at 21:31

This has less to do with parallel then it has to do with database polling vs messaging.

You could make your current solution parallel by just adding a step where the process looks at the number of tasks and decides if it wants help. If it does it starts another task. Oh, you will want to mark a task as pending when something has started work but hasn't finished.

Bang, zowie, you're parallel. Big woop.

Everything I just said has to be done in a transactional way. That means it's got overhead. Sure it's parallel but that doesn't mean there isn't a faster way.

A messaging system would let clients start the processes that handle their requests directly and immediately. When resources are overwhelmed this would still queue. The problem is you have to rewrite the clients that used to talk to the database to now talk using the messaging system.

If rewriting the clients to talk to a messaging system is a no go, you can cheat. Put a packet sniffer on the database and parse out the requests. Soon as you spot them you can start work on them. Hurry because apparently 60 seconds is to long to wait.

  • I agree with the necessity of flagging the requests records with "in process" data, preferably a time stamp.
    – uncoder
    Sep 1, 2016 at 20:17
  • Whatever, just mark it before some other task sees it. Probably want to mark something as started before starting an extra task just to be sure it can't wake up and see the same state you saw. Cause, that could be bad. :) Sep 1, 2016 at 20:20
  • Yep, I have been considering all that, too.
    – uncoder
    Sep 1, 2016 at 20:22

Spawning new threads has considerable OS overhead o set up the environment, I would also look into having each task look for more work on the queue and only terminate when none found.

  • That's true, but it still cheaper than spawning new processes. :) In our case the overhead can be ignored, however, as each task takes a considerable time to finish - it is a sum of data transfer during web calls, database lookups and transactions, etc.
    – uncoder
    Sep 7, 2016 at 21:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.