Is there a way to calculate whether to spawn a new thread or use current thread

Question

I am building an interpret DSL in which parallelism is part of the language - the user simply states that a calculation can be done in parallel and the interpreter may decide whether to make the calculation in parallel or not. Since it is not always helpful to do parallel calculations, specially for very small jobs (context switch, memory copying between cores, etc may actually make the program slower).

I want to know if there are any "best practices" available that can calculate (with high degree of certainty) whether to do a new calculation in the current thread or spawn a new thread for the calculation. Note that since my language is interpreted, I can do all kinds of monitoring on the executed program and make this decision depending on how the program executes (kind of a JIT compiler).

Update By "Spawn" I didn't mean that I will create a new thread, but transfer the processing to a different thread, maybe from a thread pool. Note that even when using a thread pool there is overhead of possible memory transfer between processor cores and stuff like that, so even in this case the amount of work should be relatively high.

Answer 1

Not to my knowledge, it seems extremely hard to get right. Even if you could do it, you still have to consider system load, number of cores, other os activities etc.

Some alternative approaches I can think of:

• Have you considered a pool of threads or a dispatch queue? At least that allows you avoid some issues with context switching etc and you can balance the system as a whole dynamically based on the total amount of usage and jobs.

• Can you model the calculations as map-reduce problems (or otherwise separate input data from code)? Then you could calculate how hard a problem is based on input data size and throughput for each item. After a bunch of samples you should be able to determine how much time it should take in total for the calculation to complete, and thus decide to run it locally or distributed.

• Would it be possible to start the calculation and simply wait until a certain time has passed and then move it to a separate thread, once you realize this calculation is not trivial?

Answer 2

My initial thought is that everytime you find a parallel construct, you first run it serially measuring the time it takes to complete. If it takes a long enough time, switch over to parallel the next time the operation executes. The theory is that the operation probably takes a similar amount of time each time it is executed.

Answer 3

From a high-level perspective, a thread probably shouldn't know enough about the system well enough to resolve this problem. The usual solution to this problem is to use a thread pool (many languages and operating systems offer automated support for this). You hand the thread pool tasks and it allocates threads as needed, reusing existing threads when reasonable. This provides the performance benefit of avoiding wasted thread allocation without adding inappropriate responsibilities to your thread workers.