Why does Python need both a compiler and an interpreter?

Question

I can understand the fact that Java needs both a compiler and an interpreter. It compiles source code to bytecode and then a virtual machine (on Windows, on Linux, on Android, etc.) translates that bytecode to machine code for the current architecture.

But why does Python need both a compiler and an interpreter? Since Python is not platform independent, why not just use interpretation? As far as I know, you cannot execute a Python program (compiled to bytecode) on any Windows or Linux machine without modification. Or am I wrong?

Answer 1

As far as I know, you cannot execute a Python program (compiled to bytecode) on every machine, such as on windows, or on linux without modification.

You are incorrect. The python bytecode is cross platform. See Is python bytecode version-dependent? Is it platform-dependent? on Stack Overflow. However, it is not compatible across versions. Python 2.6 cannot execute Python 2.5 files. So while cross-platform, its not generally useful as a distribution format.

But why Python needs both a compiler and an interpreter?

Speed. Strict interpretation is slow. Virtually every "interpreted" language actually compiles the source code into some sort of internal representation so that it doesn't have to repeatedly parse the code. In python's case it saves this internal representation to disk so that it can skip the parsing/compiling process next time it needs the code.

Answer 2

I can understand the fact that Java needs both a compiler and an interpreter.

It doesn't. There's nothing in the Java Language Specification that says that Java needs to have a compiler. There's also nothing in the Java Language Specification that says that Java needs to have an interpreter.

Whether to use an interpreter, a compiler, or a combination of the two, is completely left to the discretion of the implementor.

In fact, there are implementations of Java which compile straight to machine code, e.g. the GNU Compiler for Java gcj. Technically speaking, the Oracle OpenJDK Java compiler also compiles to machine code, specifically, JVM byte code. Now, you might say, wait a minute, that's not machine code! But, there are software interpreters for x86 machine code, and there are hardware CPUs that can execute JVM byte code, so what makes one "native" and the other not?

Note that JVM byte code sits outside of the Java Language Specification, just like x86 machine code does.

and then a virtual machine (on Windows, on Linux, on Android, etc.) translates that bytecode to machine code for the current architecture.

Again, that is purely up to the implementor.

The original Sun JVM did never translate, it always interpreted. The current Oracle OpenJDK JVM interprets, and only those parts which get executed often are compiled. The Maxine Research VM always JIT compiles. The Excelsior.JET implementation compiles once, ahead of time. The IKVM.NET JVM compiles to CIL byte code. The Android Runtime compiles ahead-of-time, once, during installation. (Also, the Android Runtime doesn't understand JVM byte code, it uses Dalvik byte code, which is a completely different language.)

But why does Python need both a compiler and an interpreter?

Again, it doesn't. There is nothing in the Python Language Specification that says that Python needs to have a compiler. There is also nothing in the Python Language Specification that says that Python needs to haven an interpreter.

Note that actually, Python is never interpreted. All existing Python implementation always compile Python to a different language. That language may or may not then, in turn, get interpreted, but that language is a different language from Python. Python doesn't get interpreted.

why not just use interpretation?

Because Python isn't designed to be easily interpreted by machines. It is designed to be easily interpreted by humans. OTOH, CPython byte code, is designed to be easily interpreted by machines. So, it makes sense to write code in a language designed for humans and interpret in a language designed for machines, and in order get from one to the other, you have to compile.

As far as I know, you cannot execute a Python program (compiled to bytecode) on any Windows or Linux machine without modification.

Yes, you can. The CPython VM is available for both Windows and Linux, as is PyPy, Jython and IronPython.

---

Languages don't have to be compiled or interpreted. Languages just are. In fact, a language can perfectly exist without having any interpreter or compiler! For example, Konrad Zuse's Plankalkül which he designed in the 1930s was never implemented during his lifetime. You could still write programs in it, you could analyze those programs, reason about them, prove properties about them … you just couldn't execute them. (Well, actually, even that is wrong: you can of course run them in your head or with pen and paper.)

Now, any particular implementation of a language may use a compiler (or even multiple compilers), an interpreter, or any combination. But that is a trait of the implementation, not the language. Every language can be implemented with a compiler, and every language can be implemented with an interpreter.

Note, however, that you can't run a program without an interpreter. A compiler simply translates a program from one language to another. But that's it. Now you have the same program, just in a different language. The only way to actually get a result of the program is to interpret it. Sometimes, the language is an extremely simple binary machine language, and the interpreter is actually hardcoded in silicone (and we call it a "CPU"), but that's still interpretation.

You might also be interested in this answer of mine, which explains the differences and the different means of combining interpreters, JIT compilers and AOT compilers and this answer dealing with the differences between an AOT compiler and a JIT compiler.

Answer 3

It's true that the bytecode is not suitable as a distribution format, but that doesn't mean it's useless. Aside from improving startup time on a given machine, after the first run, interpreting bytecode is also far simpler than interpreting an AST or, god forbid, interpreting line-by-line.

Bytecode is a more low-level, more regular, more compact (both semantically and in terms of memory layout) representation of the code. The order of operations is already spelled out, local variable names have been resolved to a simpler form (integer indices). No complicated syntax to follow, just one simple instruction after another. Additionally, less state is needed: for line-by-line interpreting you basically need to keep a whole parser around, and an AST interpreter blows up the call stack with its tree traversal, whereas a bytecode interpreter just needs a small stack for temporary values and locals.

These and other factors conspire to make bytecode interpreters significantly faster than other interpreters.