In general, there is very little you can do anything with the information you gather from a task. The reason is this: once you start a task, you have to let it finish, no matter what.
- You can't suspend it forever. Some other parts of code might be depending on the task result; suspending it means the dependent code will be waiting forever.
- You can't kill it. The task might have been making changes in the file system or database; it might leave the system or datastore in an inconsistent state if you kill the task in the middle of execution.
- For the same reason you can't kill it, you can't even move an executing task to another computer, or even to another thread.
Basically, once a task is executing, you have to let it finish, no matter much much resource it takes.
The only other remedial action you can take is that if you see a significant number of worker threads are:
- Executing on some tasks;
- Are not consuming much CPU time due to them being in the blocked state (by blocking I/O)
Then you can perhaps spin up a few more worker threads and give them new tasks, in the hope that some of them are CPU intensive and therefore recoup the "wasted CPU cycles".
Some thread pool designs simply try to opportunistically increase the number of worker threads (and have them run concurrently on tasks) until the point where they see diminishing marginal returns.
http://msdn.microsoft.com/en-us/magazine/ff960958.aspx
I can only give a rough overview of ideas. Feedback are welcome.
First, some definitions.
"CPU intensive" - tasks which, once started executing on a thread, will keep on executing on the CPU until finish, making continuous progress along the way, and rarely volunteer up for a "pause" (blocking operation).
"IO intensive" - tasks which, during its execution on a thread, tend to frequently request for a "pause" (entering sleep or a blocking operation) voluntarily. This "pause" typically occurs when the task is waiting for an external event to arrive, such as waiting for the data from reading a file, or from a database query.
"CPU wasting" - tasks which superficially look like the "CPU intensive" type, but is in fact "busy-spinning" - running a tight empty loop without making progress. It could be further divided down to "programming errors (software defect)" or "malicious workload".
Notice I use the word "volunteer" several times. This is because an operating system may preemptively suspend a thread as it context-switch to other threads so that the OS can support a total number of threads in far excess of the number of CPU cores. (Typical CPU cores are in single-digit or low ten's, but typical total number of threads in OS is in the hundreds to thousands.)
It is possible for a watchdog thread (with suitable administrative privileges) to query the OS for the CPU time consumed by each worker thread. This can be used to decide how "CPU intensive" a particular running task is.
Also, it might be possible for the watchdog thread to wake up with a regular interval to check what each worker thread is up to. If one wants to find out whether a thread spends most of its time being blocked (waiting for I/O), one will need to drill down to the reason of the thread's reason of suspension. As described before, one has to distinguish between threads that are preempted by the OS, versus threads that are blocked by its own IO operations. This is doable on some OSes, but may be impossible on some other OSes.
I have never used any of these techniques so I don't know how feasible or effective these techniques are.
