3

I need to ask a question that have been bugging me for some time now:

If I have a single core and one OS thread, this thread will get 100% of the CPU time and all is good.

If I have a single core and two or more OS threads, they will share the CPU time using time slices.

So, is the time slices always the same amout of time no matter how many threads?

What I'm trying to get at, is the amout of work the CPU can do the same when I have two threads or 10000 threads? I'm well aware that each individual thread will progress slower since they share a resource, but the actual amount of work the CPU can do, will it be the same?

e.g.

[T1 ] [T2 ] [T1 ] [T2 ] [T1 ] [T2 ] [T1 ] [T2 ] [T1 ] [T2 ]

[T1 ] [T2 ] [T3 ] [T4 ] [T5 ] [T6 ] [T1 ] [T2 ] [T3 ] [T4 ]

----time-------------------------------------------------->
img. 1

In the illustration above, there are 2 vs 6 threads, but the total amount of work would be the same. Is this true? Or are there something else that affects this when there are more threads, that cause each slice to be smaller or the context switch between the threads to be longer?

e.g.

[T1 ] [T2 ] [T1 ] [T2 ] [T1 ] [T2 ] [T1 ] [T2 ] [T1 ] [T2 ]

[T1 ]   [T2 ]    [T3 ]    [T4 ]    [T5 ]    [T6 ]    [T1 ] 

----time-------------------------------------------------->
img. 2

I'm not asking if it is a good idea to use 1000 threads...

[Edit]

Trying to clarify what I'm trying to understand:

Given x amount of time, e.g. 1 minute. And given that the code does not use locks or any other thread interrupting code.

If I have two threads, there will be y% time spent on context switching. If I have 1000 threads, will y be a greater number?, or will it be the same as in the previous case?

  • You could fill a book with information about thread scheduling and context switching. There are many ways it can work with many variations. Modern OSes have very complex schedulers and modern multi-core CPUs are very complex. This adds a lot of complexity to any particular implementation. – user22815 Oct 1 '14 at 18:36
2

If your quantum (the time each thread gets to run) is the same, then you'll have the same number of context switches regardless of the number of threads (assuming no threads are blocked). The amount of time each thread gets will be affected, but the context switching overhead will be constant. This is in an idea world; obviously, in the real world, you'll have locks and resource contention, so many processes will be waiting on I/O or otherwise not ready, so there will be more context switching overhead, and it's possible that the amount of time spent switching can exceed the amount of time spent actually running user code. The amount of extra time is entirely dependent on your workload, which is why it's not possible to generally predict the amount of overhead.

3

Context switching itself imposes an overhead on the system, because the OS has to perform book-keeping on which threads exist, which are running, which are ready, which have already received their fair share, etc. In a well-designed OS this overhead is not great, and usually it is not dependent on the number of threads (up to some hard maximum), but it is definitely there.

Everything else is system-dependent. For instance, a scheduler may decrease the nominal length of each slice depending on how many many candidates there are, or it may not. As always, a reasonable answer to decide what to do in your case can only come from running careful performance tests on your system.

  • So, from a Windows perspective, is the scheduler capable of changing nominal length of slices? my impression was that the time slice length was fixed for windows (depending on priority) – Roger Johansson Oct 1 '14 at 10:56
1

Even with a single threaded app running on a single core machine you're going to get context switches as the OS will want to run multiple applications, similar with 2 apps running on a dual core machine, you'll still get switches. Its just that having many threads running mean you get more context switches.

On Windows the scheduled time slices (or quanta) is a fixed 10ms (IIRC). A thread can give up part of its slice by sleeping, which is why you occasionally see sleep(0) calls in code. Its a way to say "I'm only going to do nothing for a bit, someone else can have a go before they're scheduled".

Another thing that can cause a context switch is locking. If you lock on a waitable object, you're giving up the rest of your slice - no point waiting for the object to be signaled by a different thread if that thread hasn't had a chance to run after all!

This is the main cause of excessive switching - too much locking.

When a context switch happens, the CPU has to store away the state of the registers and load the registers that were stored for the other thread, possibly it will have to flush a cache to get different memory in that the other thread wants to work with. This takes time, and although its "CPU time" its wasted housekeeping, ie its not spent making your program do what its supposed to do.

For example, I once worked on an enterprise program that spent more time switching than doing useful work. Once we solved the locking bug, performance improved dramatically - suddenly the CPU coudl perform useful work rather than simply spin between threads.

  • Yes, I totally get those parts, but the main question is, over a given time, eg. 1 minute, if you have two threads or 1000 threads, and no locking code, will there be more time spent doing context switching in the latter case, if so, why? – Roger Johansson Oct 1 '14 at 13:18
  • 1
    Every time the CPU changes from running one thread to a different one, there will be a context switch. If you have 1000 threads all trying to run at the same time, there will be more switches as the scheduler tries to satisfy each one of their requirement for CPU time. With 2 threads, the same will happen. But in the real world, one of the 2 threads will be in a blocked state and so the thread 1 will never be interrupted and generate no context switches. – gbjbaanb Oct 1 '14 at 13:52
  • But given that in Windows, a time slice defaults to 10ms, how can there be more context switching, shouldn't each thread just progress less during the same time period. (e.g. 1 minute) as shown in my first illustration (img1) in the question, where thread 1 gets to run 5 times in the first case and only 2 times in the second case.. but still the amount of context switching time, is the same in both cases.. or is that illustration incorrect? – Roger Johansson Oct 1 '14 at 14:08
  • 1
    my point was that thread 1 will run for 10ms, and then be interrupted and thread 2 runs for 10ms. However, in reality, thread 2 will be blocked and so thread 1 will not need to be switched at all. The difference is in the time taken to switch between threads, if you only run t1, there will be no delays due to switching. – gbjbaanb Oct 1 '14 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.