4

I'm writing a chat server and client. For the chat server I have 3 "services" all running on their own threads:

  • ConnectionListener that loops over accept, and adds any new clients to a ConcurrentLinkedQueue
  • MessageListener that waits for incoming messages from clients; also adding them to another ConcurrentLinkedQueue
  • MessageBroadcaster that sits on the other end of MessageListener's queue, and sends out any received messages to the clients in ConnectionListener's queue.

I think that's all the background tasks that I'll need, but say in the future, I have another task that I think should operate concurrently in the background. At that point, the server will be using 4 threads for 4 separate background tasks.

Is there a point where I should reconsider my design? What is a better way of arranging this?

My question stems from 2 concerns:

  • I hear threads are expensive. While that isn't an issue yet, I don't want to get into the habit of spawning a new thread for everything if there's a better way.
  • I'm not sure if I should sacrifice "separation of concerns" to save using threads. In theory, the message receive/broadcast jobs could be combined; although they're separate duties.
5
  • So your problem with this is "too many threads"? Web applications sometimes spawn tens of thousands of threads (that's not an exaggeration). If you have doubts about your design it should be whether those background tasks are actually distinct (and are actually tasks, not reactive methods) and if you can properly separate them and make them communicate properly, rather than how many Thread objects you have.
    – Ordous
    Jun 11, 2015 at 14:45
  • @Ordous My question stems from 2 things (which admitting I should have added to the question): I heard threads are expensive. While that isn't an issue yet, I don't want to get into the habit of spawning a new thread for everything if there's a better way. And I'm not sure if I should sacrifice "separation of concerns" to save using threads. In theory, the message receive/broadcast jobs could be combined; although they're separate duties. Jun 11, 2015 at 14:49
  • Threads are expensive when you've got a limited number of cores vs threads. I.e. you're running at least tens of threads doing small stuff (like adding two integers...), you're running on decade old hardware or you're limited in resources because the machine is virtual/shared. For comparison - a simple Swing GUI application will by default spawn 4 threads (Main, EDT, 2x SwingWorker pool). Honestly though, what are you aiming for? If it's a small company-wide/friends-only app, server performance should never even come into consideration unless you screwed up something badly.
    – Ordous
    Jun 11, 2015 at 14:56
  • 1
    Ok, thank you. I guess I'm prematurely sweating the small things. Jun 11, 2015 at 14:59
  • 5
    It would seem so. The real cost of multi-threading is development cost. It's easier to mess up, and the more threads and interactions you have, the trickier it becomes.
    – Ordous
    Jun 11, 2015 at 15:00

2 Answers 2

6

I hear threads are expensive. While that isn't an issue yet, I don't want to get into the habit of spawning a new thread for everything if there's a better way.

Threads have overhead. The longer a thread lives, the less of a concern the overhead is. For example, let us assume a thread has a ton of overhead and requires 100ms to spin up. However, the thread lives for an hour. That is a tiny fraction of its life, so who cares? If you can bury the thread initialization in application startup when users expect delays anyway, all the better. If the thread lived for less than a second, then the overhead is a bit much and maybe you should consider alternatives.

That being said, do not spawn threads to do nothing. Each thread should have a purpose.

I'm not sure if I should sacrifice "separation of concerns" to save using threads. In theory, the message receive/broadcast jobs could be combined; although they're separate duties.

In my experience, multithreaded applications need multiple threads. Combining concerns with e.g. a listener and a broadcaster in this case will increase complexity to the point of it being unmanageable. It will create bugs. The key here is where do the threads block? If you have a listener that blocks on accepting a socket, it makes no sense for it also to handle requests. One of its concerns will suffer: requests will be delayed, or new connections might fail. Just create more threads.


Note 1: you should be using thread primitives, not raw threads. Since you tagged this question I will point to the specific class: ExecutorService. The Executors class provides several factory methods to make it easy to create them for various tasks. The basic idea is you submit tasks (Future or Runnable) and let the framework manage the threads.

This has two primary advantages:

  1. Managing threads can be hard. While it appears easy, there is a lot of boilerplate logic that is easy to screw up, resulting in weird, difficult to replicate (and fix) bugs. Push this responsibility into a robust framework that is well-tested and used in millions of other programs. If there were bugs then Oracle (or Microsoft for .NET, or whatever other vendor) would have found them by now.

  2. Your program's purpose is not "to manage threads" it is "to run a chat server." Let the framework handle the threading concerns, and focus on the chat server aspect. By focusing on the tasks and not the threads, your code is more concise, expressive, and clear.

I wrote up an answer to this old Stack Overflow question that shows a use case for this. I do not want to derail the focus on this answer too much, so you can read more here as well as search both Programmers and SO: Creating an unknown amount of threads in Java?


Note 2: given that this appears to be a desktop/server application as opposed to embedded, there is not much need to be concerned with spawning threads. Modern CPUs (amd64, Intel Core) have multiple cores on-die, typically between two and six. Some models are hyperthreaded or superscalar, adding more logical cores that can execute code concurrently. Add in time slicing with today's high clock speeds, and there is no need to concern oneself with spawning too many threads as long as the threads are reasonable. In the case of your question, the threads appear to be performing small bits of work and possibly blocking on I/O. You are not spawning a million threads for digital video encoding, for example.

Even on embedded architectures, speed and concurrency are not as strong as desktop and server CPUs but should have no problem handling a few threads. My smart phone and tablet both have many applications running in the background and have zero performance problems even with many applications and threads running.


Note 3: if you are interested in learning more about the tradeoffs of a single-threaded algorithm vs. splitting it into pieces and using concurrency, there are a few topics worth exploring. Given the problem described in your question this is not an immediate need, but could make for some interesting and educational reading.

  • MapReduce is an algorithm for splitting a task into pieces that do not rely on each other, performing those tasks, and joining the results together.

  • Parallelism in databases involves multiple threads querying data and joining the results. There has been a lot of research into the tradeoffs of using multiple threads for a single database query and a lot of information is out there. Outside of schema/query optimization this is very useful information from a more theoretical perspective.

3
  • Thank you for the detailed response. What does "Note 1" gain me though? Is it basically a thread-pool? Jun 11, 2015 at 15:22
  • Please see my most recent edit.
    – user22815
    Jun 11, 2015 at 15:28
  • Note: handling lots of connections on a single thread is difficult in Java. For C programs, you can handle lots of connections and accept new ones, at the same time, on a single thread. However, it gets harder the more different types of connections you have open. In node.js, everything is single-threaded and it's not a problem because nothing blocks ever.
    – user253751
    Feb 24, 2020 at 16:34
2

4 threads are nothing, especially since these background tasks that you described are mostly doing nothing but waiting for something to happen.

A modern server can easily have hundreds of such threads simultaneously running, precisely because the word "running" here is mostly a figure of speech: they are not really running, they are just waiting.

I open up Sysinternals Process Explorer on my desktop computer right now, select "System Information", and it says that I have about 957 threads running. Does that answer your question?

enter image description here

The problems start when your threads actually have processing to do, but this is actually quite rare. You might be writing lots of code and you might be thinking that this thread of yours is going to have lots of stuff to do, but in most cases this is just an illusion.

The vast majority of threads written by the vast majority of programmers spend the vast majority of their time waiting for something external to the thread to happen. When we say that a thread is waiting, we mean that it is sitting in an efficient wait state, and consuming zero CPU.

Examples:

  • When a thread opens a file to read or write data from/into it, it is mostly waiting for the data to be transferred from/to the disk. While the data are being transferred, the thread is sitting around doing nothing.

  • When a thread issues a SELECT statement to a database, it is mostly waiting for the database to execute the command, and then for data to arrive via the network. During that time, the thread is sitting around doing nothing.

  • When a thread performs a network operation like reading from a socket, --you guessed it, it is just sitting around doing nothing.

So, unless you do something silly, like spin-looping, (also known as busy-waiting,) all of the code that you have written which constitutes your thread represents but a minuscule percentage of the total running time of that thread. Over the course of an hour it will usually wait for almost entirely an hour, and it will do actual work for a grand total of a few milliseconds.

In other words, the thread sleeps for aeons of computer time, and once every few centuries it wakes up to eat a quick breakfast and immediately go back to sleep.

So, I would not worry about these 4 threads of yours at all.

2
  • It might be worth adding a note about interrupts which allow a thread to go idle without consuming resources until some condition is met. You hint at it in your answer but I think explaining that or adding a link would make it better.
    – user22815
    Jun 12, 2015 at 17:05
  • Interrupts are something the operating system does. You don't handle interrupts in your program.
    – user253751
    Feb 24, 2020 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.