Should multi-threading be used for tasks which does not involve IO operation?

Question

I was reading a book on OS and got confused between different terminologies. I hope i will get my doubt cleared here. These questions might be very basic for some experts, but i find this platform best suitable for such clarifications.

Processes are run by CPU and this whole operation is managed by OS. For Single core CPU, at any instant only one process will be running. CPU saves the state of a process to Process Control Block and starts running another process which was waiting and this switching is so fast that it feels like all processes are running simultaneously.

Need of different threads in a process was felt to get more fine-grained control over process , that means if process is waiting because of some I/O bound operation then even if this process is selected by CPU, it will not do anything.

So, if multiple threads are running under one process, then if one thread takes care of I/O activity , other threads can do some real computation, provided this process is selected by CPU.

Here are my questions: Is Multi-threading effective only if task involved I/O activity? Why is Multi-threading preferred over Multi-processing ? (is it because threads will share same memory space? )

Why Multi-core processors are better for multi-threading? (I think they will be better even if you don't use multi-threading, why so excitement in name of multi-threading?). And anyway threads in different processes will not share same memory, so two threads running in different processes on different cores are like running different processes. It is just that they will be parallel, and the real parallel, not the pseudo parallel.

Answer 1

One of the main reasons for using threads even in a completely CPU-bound process is to allow interaction or update progress output via a UI of some sort. While running the calculation in the foreground without allowing interaction would be more efficient, the slight cost of task switching and handling UI events usually outweighs the penalty of forcing a user to wait indefinitely for a process.

Another case where running multiple threads on a single core would be more efficient is when a multi-stage process involves one stage that produces a lot of temporary results that are processed by a later stage. Running these stages in series might exhaust available memory. Running them in parallel allows the second stage to free up memory as it processes results.

Finally, as we add more caches to our architectures, threads can often become idle when a cache miss occurs while accessing memory. This gives some time for another thread to activate and do some work.

The main benefits of multi-threading over multi-processing include

• shared memory
• less overhead for context-switching
• easier abstraction

Most modern operating systems provide some method of sharing memory across processes, too. Even still, allowing the OS or virtual machine to shift threads across multiple processors when available is very appealing. For example, when you move a multi-threaded Java program from a single-core machine to one with multiple cores, the JVM will automatically make use of those other cores.

Answer 2

There seem to be a few misconceptions underneath your question.

• Although it is possible to have an architecture where each CPU has its own, dedicated, memory, this is not the case for multi-core processors. You might encounter such an architecture when working with super-computers or computing clusters.
  In a multi-core processor, each core has its own cache, but the bulk of the memory (all the RAM) is shared between the cores. This means that it is possible for two threads of the same process (application) to be executing on different cores at the same time.
• It is not the CPU that selects a task to execute, but it is (the scheduler in) the OS that selects the tasks to execute on each CPU. In this selection, only tasks will be considered where the OS knows that the task can make some progress. Tasks that wait for something (an IO operation to complete, a lock to be released, etc.) will not be considered for scheduling until the OS knows that the condition has been satisfied.

To answer your questions:

1. It is usually recommended to have a separate thread for user-interaction that does no long-running computations or potentially blocking I/O, because that improves the perceived responsiveness of the application.
   It might also be beneficial to have multiple threads that perform computations, because they might get executed on different cores (if there are multiple cores available) and it can make the design of the application simpler, which is an advantage even if you only have a single CPU.
2. Multi-threading is often preferred over multi-process, because inter-thread communication is often easier to understand/specify than inter-process communication, because threads share the same memory. Also, depending on the thread implementation in the OS, task-switching between threads can be more efficient than task-switching between processes.

Answer 3

There are several reasons why people started using threads besides doing I/O. One of the reasons is that it is supposedly easier to program if you are used to a single-threaded world. Working with lots of I/O in a single process in the old times would mean creating very elaborate state machines that were difficult to debug and understand. Also, if you do such a state machine you have to break-up your computations into small pieces since a long computation can make your application unresponsive. The old Netscape Navigator was done like that.

Different threads can usually use different cores, so if you have CPU-bound operations without much I/O those operations can run in a different core without blocking the rest of your application. It's the opposite of what you said, so I think what you are thinking of threads is what in some places is called green threads, i.e., threads that are not scheduled by the operating system. A piece of code runs until it gives up controle, i.e., performs I/O or yields.

Now, there are several reasons why people would sometimes prefer multithreading to multi-processing. Processes usually don't share memory, so lots of programming styles are not possible. Instead of updating a data structure in memory you have to send a message to another process and the other process has to interpret it. That's how many languages work today and they are great for many things, but it's a different programming style. Another reason is that processes are usually heavier and more resource-intensive than threads. You can have your application have hundreds of threads, but you usually can't have hundreds of processes.

Answer 4

When it comes to understanding parallel processing (I'm going to use this term as a generic term), it's best to start out with the basics. Take it slow and simple at first.

It's my understanding that threads and processes are much the same, but threads are a light-weight work system whereas processes are heavy-weight. These are the terms used to indicate how much system work must be generated in order to support these work arrangements. This is an important matter because parallel processing causes the computer some extra work. If a task spawns a great many threads, this is more efficient than if it has to spawn an equal number of processes to do the same work.

Try to understand I/O in the broadest possible sense. It's not only about user input, or I/O to mass storage devices (disk). From the CPU's perspective a read from main memory is I/O. A cache read is I/O too, even the fastest Level 1 cache. Parallel processing allows productive work to continue even if one thread (or process) is waiting on I/O, because other non-blocked threads can continue to execute.

And never get stuck thinking that you know or control the order of execution of threads you spawn! While you can force synchronization of parallel tasks, only do so if the problem logic requires it. Synchronization slows down the pace of program execution and is bad unless strictly necessary.

Most modern CPU's are complex beasts internally and will break down even Assembly language statements into micro-instructions.

All talk of multiple CPU's versus core utilizing CPU's is just hardware implementation stuff. Even a single CPU, non-core based, can often make very effective use of parallel software. It just requires OS and application support.

The crude hierarchy of processor power (in increasing order of instruction-executing power) is:

1. Single CPU, no multi-core
2. Multiple CPU, no multi-core
3. Single CPU, multi-core
4. Multiple CPU, multi-core

Why? Multi-core CPU's can route tasks around to internal cores much faster than it takes to send a task to a different CPU. Also, multi-core CPU's routinely share cache levels, more so than single-core CPU's. That permits faster data sharing too.

The crude metrics I've seen say that sending a task from one core to another, within a single CPU is approximately 10X faster than having to send that task off-chip, to another CPU.

Note too that shared-nothing architectures, like supercomputers routinely use, abstract this up even higher. Then you have multiple whole computers all working together as a coordinated unit. The instruction processing power goes up but so does the inter-process communications (IPC) overhead. This is directly analogous to the increase in power with attendant IPC overhead found between multiple single-core CPU's and one multi-core CPU. All found within a single computer with a single operating system.

Here are some general guidelines for programming parallel processing:

• implement as many parallel tasks as you can, as the logic will permit;
• stay away from trying to coordinate the number of subtasks you spawn to correlate with your current CPU design;
• stay away from second-guessing the CPU's instruction processing abilities;
• stay away from process synchronization wherever possible;

In fact it's a good design pattern to not design to your current hardware, as much as possible. Software can have a very long life while hardware has a relatively short and fixed lifespan. Therefore bet on (by spending time on) the software more so than the hardware. The hardware will change, you can count on it.