-4

I wanna know what is the importance of multithreading in network applications? I have researched for a while but I didn't find references related to this specific topic.

  • 1
    Do you know what the term multithreading means? – JacquesB Jun 20 '17 at 8:19
  • 3
    The question is basic, but I find it important. There are too many books out there that simply assume that if you want to learn about multi-threading, you must also be interested in HTTP and Transport Layer Encryption, which is simply not true. I find spreading awareness that the two topics aren't the same rather valuable. – Kilian Foth Jun 20 '17 at 8:29
  • 3
    recommended reading: Open letter to students with homework problems "If your question... is just a copy paste of homework problem, expect it to be downvoted, closed, and deleted - potentially in quite short order." – gnat Jun 20 '17 at 8:41
4

Multithreading means (apparently) doing many things simultaneously. Technically, there is no particular connection of threading with networking; they are two unrelated methods which are sometimes appropriate and sometimes not.

But when you are running a server that can take requests from any source on the network, particularly from any source on the entire internet, it becomes very likely that several requests will come in simultaneously - or at least faster than you can serve the first one. If you don't want service to become slow very quick, a common solution is to maintain a thread pool and serve up to N requests pseudo-simultaneously, so that the response time doesn't suffer from waiting overhead (at least not until all the threads are used).

So the importance of multi-threading for network applications is simply that a common pattern of writing network software (servers) has an obvious problem for which threads are an obvious solution.

  • Another solution would be to use a asynchronous server, which is single threaded. – Oskar Skog Jun 21 '17 at 20:03
  • 1
    @OskarSkog true, but (a) in modern multicore systems single-threaded asynchronous server may be slower than a multithreaded server and (b) it is extraordinarily hard to actually implement an asynchronous server correctly, because non-blocking I/O is just very hard, which means that most servers available are multithreaded and not asynchronous non-blocking. But I know there is at least nginx available for those that like an asynchronous event-driven non-blocking approach. – juhist Jul 17 '17 at 12:54
  • "Parallelism" means doing many things simultaneously. "Concurrency" means starting new tasks before other, previously started tasks have finished. Multithreading is just one particular technique that can be used to achieve concurrency (on pretty much any computer system), and parallelism (on multiprocessing computer systems). – Solomon Slow Jul 17 '17 at 16:52
2

In general people expect a server to be able to handle multiple simultaneous clients, anywhere from a handful to tens of thousands.

There are several approaches to handling this. Each with pros and cons.

Fork on connect

The main server process only monitors the listen sockets, when it accepts a connection it forks off a new process to handle it. The forked instance of the server process may handle the connection itself or it may use exec to start a new program to handle the client connection. When the client disconnects the child process ends.

This provides a high level of isolation between clients. This may or may not be desirable. It has a relatively high overhead when a new client connects and also a relatively high overhead per simultaneously connected client.

A further complication with fork based approaches is portability. Not all operating systems have the concept of "forking".

Prefork

This also uses multiple processes but the child processes are kept around reducing the overhead of forking new ones. All of the idle client processes monitor the listen sockets for new connections. When a connection is accepted the child process dedicates itself to handling that connection until the connection completes, then returns itself to the pool of idle children.

Client isolation is good but not as good as with fork on connect since the state of a child process may have been affected by previous clients but it's better than with approaches that handle multiple clients in one process.

Create thread on connect

Similar to fork on connect but instead of forking a thread is created. This has a lower overhead and makes it easier to share data between clients. On the downside there is a much lower level of isolation, a bug on one client thread can easily crash the whole server process and great care must be taken over locking of data shared between threads.

Thread pool

Similar to prefork but using threads instead of child processes. Saves the overhead of creating a new thread when each client connects but adds additional complexity.

Event driven

The program contains an "event engine", this monitors all of the sockets, both listen sockets and connected sockets. It then calls event handlers when the rest of the application needs to do something, for example when there is data available to read or when it is time to write more data.

The upside of this approach is that the overhead of threads is avoided. Thread safety concerns are also avoided.

However it means that the application code must be structured as a series of event handlers. Explicit state machines must be created to track the state of a connection rather than it being handled naturally by the program flow.

Another downside is that it's difficult to avoid inserting blocking stuff into the event handlers. For example on most operating systems file IO is blocking. So is the standard API for translating between hostnames and IP addresses. This can easily kill performance.

And finally a purely event driven process can't take account of multiple processors.

Hybrid approaches

It is possible to combine threaded and event-driven approaches. One approach is that when a connection is in an "interesting" phase it gets a dedicated thread, once only relatively simple tasks such as sending out bulk data remain it is handed off to a thread that handles multiple connections.

Another approach is to have multiple independent event-engine threads each handling a subset of connections.

This can improve performance over a one-thread per socket approach but at the cost of additional complexity.

Similarly it is possible to combine multiple processes with multiple threads. This can reduce the damage when a thread crashes and reduce issues with running out of address space while avoiding much of the overhead of having lots of processes around.



That's enough about servers, what about clients?

A simple command line client that connects to a single server at a time has no need for multi-threading or event driven approaches. However not all clients are that simple.

A client that wants to talk to multiple servers at once has many of the same considerations a server has and either a threaded or an event driven approach is likely to be needed to handle that.

Similarly for a GUI client people expect the GUI to continue working while the client talks to the server. So either an event engine that fits with the GUIs event system is needed or a seperate thread is needed for networking.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.