If I am building a webservice/Web API for servicing requests, there are 2 threading strategies I can think of.

I will explain this in terms of Java (though the question may be relevant for any other language also)

  1. In the main thread, I accept the request and pass it to a thread. The thread does the processing.

  2. In the main thread, I accept the request and pass it to a thread. This thread just puts the request into a job queue and exits. You have a separate thread pool which processes requests in the queue and processes them.

How do I pick of these methods?

In the first method, I can run as many run as many threads as my OS/memory allows me and each request starts getting processed as soon as it is received. However, things may slow down if there are too many threads. In the 2nd method, if I have a fixed number of worker threads, then if all threads are occupied, then new requests may have to wait before the processing starts.

There is an SLA that there will be an average of so many requests per hour and each webservice call will have be processed within x seconds.

Is there any theory about how to figure out how best to do this. A lot of testing will be done to compare but is there already some theory for this? What factors need to be considered?

This is a Java program running on Tomcat.

  • 3
    Why can't the main thread just put the request to the queue? What happens if 1000 long requests arrive to the main thread, and you don't want to start 1000 threads? How does a thread pool fit into this picture? I think your options need a bit more of further contemplation. – 9000 Aug 5 '15 at 19:17
  • 1
    Doing your own multi-threading in a container is typically something that is to be avoided. Java EE specification and multi threading, Why is spawning threads in Java EE container discouraged? – user40980 Aug 5 '15 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.