# How is it possible to have limitless availability of threads while only having a finite number of physical processing units?

So I'm having trouble understanding the relationship between threads (software) and processor cores (hardware).

Basically my understanding of a processor core is a unit that deals with commands and is essentially what executes them. A processor with 2 cores operating at 1 kHz can execute 2000 commands per second, right? This must mean that the 2 cores are simultaneously executing a thousand commands per second.

Then there are software threads, which allow a programmer to execute code simultaneously, however they are not limited (theoretically).

I can't understand how is it possible to have limitless availability of threads that execute commands simultaneously while only having a finite number of physical units that do so?

• Computers reckon time in micro- and nanoseconds. People reckon it in milliseconds at most. This means that what is actually staggered or interlaced to the computer seems very much simultaneous to us people. Aug 16, 2017 at 15:31
• As you allude, the number of threads is not really unlimited. There is a non-zero cost associated with a CPU switching thread contexts. Having too many threads is one cause of poor performance in applications. Aug 16, 2017 at 15:49
• Please note that there is a practical limit to the number of threads you can have. It's very high, but it is not infinite. If you have too many threads, your application spends all it's time context switching and almost no time doing work. The practical limit is much lower than the operating system limits in many cases. Aug 16, 2017 at 16:28

A processor with 2 cores operating at 1 kHz can execute 2000 commands per second, right?

Not quite. For the simplest processors, that might be true. But many processors can can execute multiple instructions simultaneously (in a single core, without hyperthreading), and most processors have instructions require more than one clock cycle to complete.

how is it possible to have limitless availability of threads that execute commands simultaneously while only having a finite number of physical units that do so?

Time slicing. Basically, dividing the CPU's time into short durations. Keep a queue of all the threads, run one thread for a while (e.g a few hundred microseconds), then stop it, stick on the back of the queue, and run the next thread.

This does mean that only one thread per core is running at any given moment. But, even on a single core computer, we can maintain the illusion of multiple threads running simultaneously because human consciously process things in the tens to hundreds of milliseconds range, a duration during which there will have been many different threads running.

Most operating systems today use preemptive multitasking, in which the OS will forcibly pause a thread after its allotted time slice expires. Some use cooperative multitasking, in which a thread will continue executing until the thread itself decides to yield and let someone else run.

You may wonder how the operating system gets to run in order manage all these threads.

In cooperative multitasking, it should be fairly obvious that the executing thread can execute an instruction that hands control to the OS (comment if it's not clear). Yes this does mean that it's easy for a poorly behaved application to bring the whole system down, which is why most mainstream operating systems aren't cooperative these days.

In preemptive multitasking, the hardware generates an interrupt after a given length of time (which the OS can usually configure). Interrupts cause the CPU to jump to an interrupt service routine, to handle that particular interrupt. ISRs are set up by the kernel while booting, so when the timer interrupt fires, the OS gets control and can swap out which thread to run next.

You can have many threads because a processor core can execute instructions on one thread for awhile, and then switch to another thread, executing some instructions there. This process occurs rapidly and continuously, making it appear that all threads are executing simultaneously. But the only time threads actually execute simultaneously is when you dedicate a single thread to each processor core.

Threads require some overhead to work. If you allocate too many threads, you can actually decrease performance, because you spend more time on the overhead than doing actual work. So the number of threads you can practically spin up is not limitless.