0

First of all, let me clarify the terms in order to avoid any possible misunderstandings.

Language is considered to be compiled when a program written in it's source code cannot be run directly without additional steps under normal conditions. That means it's usually and normally done that way. Technically we have online REPLs for C++ which let you run C++ code line by line. That is not considered normal conditions, because normally (i.e. in production usage) it is preprocessed, assembled, compiled and linked. Same goes for Java, technically you can write bytecode by hand and then have it directly executed by the virtual machine. Is it normally done that way? The answer is no.

A few examples of what is considered a compiled language:

g++ -Wall -pedantic thread.cpp

java c Thread.java
jar cfm Thread.jar manifest.txt

cargo build --release    

By contrast, language is considered to be interpreted when a program written in it's source code is usually run directly, without any additional steps under normal conditions. I don't care if it's JIT-compiled by some state-of-the-art technology that makes it look like it's interpreted. If it looks like a duck, quacks like a duck, you know the saying.

A few examples of what is considered an interpreted language:

python3 thread.py
node thread.js --trace-opt

Multithreading is considered as a process of executing multiple threads simultaneously i.e. concurrent execution of two or more parts of the program at the same time. So, there must at least be:

  • true parallelism (execution in two different threads happens really in parallel, at the same time, not some kind of sneaky asynchronicity, when one is actually happen after another but it's so optimized that it looks like it runs simultaneously)
  • an ability to spawn multiple threads in one source code file that have direct access to shared variables
  • threads spawn from the main thread must have exactly the same functionality, without any limitations
  • possibility of race condition as a consequence might exist
  • some kind of locks to prevent it
  • an ability to observe said threads by OS means

A language must have native multi-threading. That is, it's directly built-in in the language itself. What is not considered a native multithreading:

  • some kind of third party library you need to install via package manager
  • some non-standard implementation of the language that runs in some kind of environment
  • not coming from the language itself, but from it's environment, like Browser API for example, which provides workers for JavaScript in the Browsers

All interpreted languages that I know of are single threaded. Some of them are limited by GIL (like Python and Ruby), others by the runtime environment (like JavaScript). By contrast all compiled languages like C++, Java, Rust support multi-threading.

Question is: can interpreted language be multithreaded? If it can, please name an example. If it can't is there any computational reason for that?

24
  • 4
    @tnsaturday: The CPython docs that you quoted already prove that Python is multi-threaded. Because if Python weren't multi-threaded, then only being able to run one thread at a time wouldn't make sense: there is only one thread. The fact CPython explicitly documents the restriction that only one of multiple threads can run at a time obviously means that there can be multiple threads, ergo, Python is multi-threaded. Oct 2, 2023 at 23:32
  • 6
    @tnsaturday: that's not true. Hop in your Delorean, time travel back to 2001 or earlier, and you'll find mutlithreading existed, but most computers only had a single CPU, which meant that only a single thread was executing at any given time. What threading provides, even on a single CPU, is the means to abstract away seemingly concurrent tasks without having to melt your brain with select() and other management to keep track of the two lines of computation, and sharing resources instead of having to jump through a bunch of IPC hoops with multiprocessing. Oct 3, 2023 at 6:10
  • 4
    Your definition of "interpreted" is at best shaky. No language, except machine code, is run "without additional steps". There is always some sort of translation going on. Jitting is at least considered as a form of interpretation by wikipedia. While javascript used to be interpreted, by now jitting should be more common, so what is "normal" can, and has, changed.
    – JonasH
    Oct 3, 2023 at 6:59
  • 2
    There is also "GIL-free Python" github.com/colesbury/nogil which really does execute on multiple threads on the same time.
    – pjc50
    Oct 3, 2023 at 8:30
  • 3
    When you said "A language must have native multi-threading. That is, it's directly built-in in the language itself." your question became a farce - by that definition C is not multithreaded since it relies on OS system calls to provide threading - ego - no language is multithreaded.
    – DavidT
    Oct 4, 2023 at 6:34

5 Answers 5

5

Question is: can interpreted language be multithreaded?

Yes.

If it can, please name an example.

Python.

Ever since 2003 the Twisted community has been leaning pretty hard on python's ability to interleave threads of execution within a single address space. Given GIL constraints, the focus has been on I/O-bound processing, such as seen in a web server.

With modern asyncio we see threads awaiting futures all the time.

Compute-bound data science workloads that rely on BLAS-based libraries like {numpy, pandas, scikit-learn} will often release the GIL. That way sibling threads can do work while a solver written in a compiled language is keeping other cores busy.


Your line of questioning seems directed at compute-bound {C++, java, rust} programs like this:

Init common counter to zero.

Spawn thread 1, with private i variable:

    for (i = 0; i < 900; i++) { counter += 1 }

Spawn thread 2, with private j variable:

    for (j = 0; j < 900; j++) { counter += 1000 }

And then we wonder if, once the dust settles, we will see a value of 900900. If not, we worry about using semaphores or other IPC to protect the "increment" critical region. We might also add sleep(0) or similar voluntary "yield" statements.

We can write such programs in python, as well.

The difference is, the language-level GIL prevents conflicting counter updates. It is either a blessing or a curse, take your pick. Because a mutex, outside of app developer control, is held while each app thread runs, a whole class of potential bugs has been avoided. That is the tradeoff python made long ago, and which we still live with.

Is this the threading behavior OP desires? Probably not.

Is the bytecode interpreter running multiple threads? Absolutely.

3
  • What your "counter" example points at it the language's memory model. Some languages have an explicit memory model, which explains how concurrent access to objects is handled. For example, Java guarantees that variable assignments are atomic, C guarantees you'll be accidentally invoking UB. "Scripting" languages have a more complex data model, so specifying a useful memory model is difficult without breaking old code. Python's memory effectively ensures atomicity for most opcodes. But increments involve multiple opcodes (a += ba = a.__iadd__(b)), making data races possible here…
    – amon
    Oct 3, 2023 at 9:20
  • @amon I think "guaranteeing UB" is an oxymoron - the whole point of UB is that it doesn't guarantee anything. Famously, it was once remarked that undefined behaviour cloud mean the compiler will "make demons fly out of your nose" - but it might also document and implement a much more sane result for that code. So what you're really saying is "the C standard doesn't define a memory model covering this situation, but a particular compiler might do so".
    – IMSoP
    Oct 3, 2023 at 16:10
  • @IMSoP I'm being a bit facetious. While C (since C11) is one of the few languages with a thoroughly defined memory model for atomics/multithreading (to the point that other language's memory model often just says "whatever C does"), the language famously lacks guardrails. Shared memory will exhibit defined behaviour when explicitly using the atomics library or guarding accesses with mutexes, but nothing will warn you if you forget that. Some langs like Erlang solve this via a shared-nothing architecture, others like Rust with stronger type systems. Java/Python only ensure memory safety.
    – amon
    Oct 3, 2023 at 17:24
4

TL;DR - Java byte Code - i.e. you are hand coding the byte code and running it on the JVM - It's interpreted by the JVM and provides full support for native threads and locking primitives.

--

Operating systems typically provide the ability to share memory between processes and also share certain types of locks as a result it is reasonable to say that two processes that are sharing state and are additionally controlling access to the shared state via synchronization primitives are a form of multithreading - you have multiple threads (that happen to be in different processes) that are communicating.

"Interpreted Language" is more difficult to define - personally I am happy with the definition that: if it isn't directly executed on the CPU it's interpreted. That does leave some gaps with respect to JIT compilers, however for this argument we can just assume you can disable the JIT so the code remains purely interpreted.

With these definitions, any language that is not compiled to native code and can use OS level shared memory + synchronization primitives can be considered to be both interpreted and multithreaded (including the example I started this post with).

--

If you want a different argument, please provide two requirements:

  • One that is a defining attribute of all multithreaded applications.
  • One that is a defining attribute of all interpreted languages.

Such that you believe that both could not be implemented in the same "toy" language invented just to prove this point. I cannot think of any such pair of requirements.

4

I think this is a good question. At a high-level your observation seems correct, i.e.: that what people tend to think of as 'compiled' languages tend to have more or better support for 'true concurrency' where two or more threads can be executing at the same time. However, there are a few minor misconceptions here that are leading you to an incorrect conclusion.

I don't want to get too deep into the details of this but first off, the whole concept of interpreted versus compiled is very fuzzy. I don't think there's a precise way to define the difference. You point to Java as compiled but you should understand that the first thing that happens when you run a newly written CPython script is that it compiles the program into a folder called __pycache__. If you look in that folder, you will find .pyc files. Those contain the compiled python bytecode that is actually executed by the runtime. This essentially no different than the way Java works, other than that it's automatic instead of requiring a separate compile step before running.

You asked for an example of an interpreted language that allows for true multithreading and the immediate thought that I had was: Jython. I consider the project to be defunct, but I used in extenively in the past. Jython is basically a java library that implements a Python interpreter/runtime on top of the Java interpreter. The crucial thing here is that it does not have a GIL so it has the same multithreaded capabilities as a vanilla Java program. There are other 'interpreted' languages such as Groovy that are implemented in Java which (AFAICT) are also able to use Java multi-threading capabilities. So even if we set aside the debate over 'compiled' versus 'interpreted', I think this is proof that there's nothing inherent that prevents an 'interpreted' language from supporting true concurrency.

The GIL (in Python anyway) is a feature. It was introduced to eliminate the possibility of a lot of nasty problems that come with multithreading. It might also have been a way to avoid creating the kind of proper memory model needed to make multithreading reliable across platforms. If you have time, you can learn more about the GIL in this video: Understanding the Python GIL

3

Somehow, it seems nobody has mentioned Erlang, the concurrent language, yet.

Erlang is interesting in regards to this question, because besides being fully interpreted (not JITted!), it provides three kinds of concurrency.

The first one is the logical concurrency, where the interpreter will occasionally switch which task it's running. The second one is true multithreading, and the third one is running different tasks of the same program on different processes, or even separate physical computers. From a code point of view, there is no visible difference.

What makes this possible (and avoids the problems with Python's GIL) is that each task has, effectively, a separate address space, with separate garbage collectors and such. They can only communicate by copying data between tasks. This is what allows it to run on separate processes/machines, and why it scales so well; there are no race conditions relating to shared mutable state, because there is no shared mutable state.

And that brings me to the actual answer to your question: most interpreted languages (in the sense you use the word, which is non-standard) are meant to be high-level, and dealing with locks and shared memory is a very low-level way of doing concurrency. So most languages that provide that are low-level.

Most high-level languages provide a higher-level abstraction for concurrency, which you wouldn't really call "language-level multithreading", in the sense of shared mutable state, even if it accomplishes the same things and is implemented in terms of it.

Go is a great example of this; it's compiled but meant to be high-level, and while you can use it with mutable state and locks, the primary kind of concurrency it encourages is sending immutable data to tasks.

So it's not a matter of whether it's possible or not; it's about what those languages are designed for.

0

There is no reason why an interpreted language shouldn't be multi-threaded, except that your interpreter needs to be multithreaded as well, or you are in the same situation as a multithreaded compiled language on a single core machine (without hyperthreading).

A singlethreaded interpreter could switch between the threads of your interpreted language, but you wouldn't be able to take advantage of multiple cores. Multiple threads can be useful even on a single threaded machine, for example to perform other tasks while one thread performs an http request.

As an example, a Java bytecode interpreter may need to load a Java class. That should be done only once. If you had a multithreaded interpreter, and they decided in the same nanosecond to load the same class, then they would need to synchronise so that one interpreter thread loads the class, and the other interpreter waits for it to finish. And that's what actually happens because a JVM is supposed to run multithreaded.

But it does come at development cost. So the interpreter for your favourite language might have to be modified very carefully to be able to run multithreaded. Just like your processor had to be designed very carefully to run multithreaded without problems.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.