Why Java is a factor 2-3 slower than equivalent C++ program? [closed]

Question

I know there is an opinion that programs written in Java and running under JVM as as fast as C++ programs, after introducing just-in-time (JIT). I see many cases when Java is terribly slow for simple programs compared to C++, i.e. about factor 2-3 slower than equivalent C++ programs.

Just give you a small example: This is simple benchmark code to merge particles in clusters:

https://github.com/chekanov/hepjet

Go to "scjet_cpp" and type "make; make run". This runs C++ code. Then go to "scjet_java" and type "make; make run". This runs the same code written in Java.

Th first C++ code finishes merging particles within 50 msec (i7 desktop). Exactly the same Java code finishes for 170 ms. There is nothing special about the code, i.e. identical implementation in C++ and Java.

Any comment?

Answer 1

Your measurement is probably flawed.

1. If you're just running both applications and measure the time from start to end in the terminal, you are not measuring execution time, but also startup time. For Java application, this means that those 170 ms. also contain JIT compilation.

2. Moreover, 50 ms. vs. 170 ms. is not representative. Doing a task in a loop and obtaining 50 s. vs. 170 s. will be more relevant.

3. You can't just take a single project and say: "This language is better, because this particular project executes faster with this language." A single project is not representative enough. Maybe authors knew C++ more than Java. Maybe they missed opportunities to optimize code.

4. Compilers you use may make a huge difference. Some will create code with terrible performance. Others will use clever optimization techniques.

Also, what's your goal? To be able to say: "Java sucks, everyone should use C++?" This is not particularly constructive. A more constructive approach is to build an application using the technology your team knows best (say Java), then find the bottleneck and find that it can be removed only by moving this part of the project to low-level language (including Assembler).

Answer 2

There is no one answer to this, largely because it's not always true. In fact, arguably it's never really true. You can't measure the speed of a language, only the speed of some particular implementation (and rarely even of the implementation as a whole, only of its execution speed on some particular piece(s) of code.

Java code can be slower than C++ code--sometimes by a factor much larger than 2 or 3.

Java code can also be competitive with C++ code--but often requires substantially more memory to do so.

For a few, very specific, things Java can be faster than C++ that's written similarly.

That said, yes, on average code written in Java will run slower than (roughly) equivalent code written in C++. If your primary concern is with execution speed, Java has a number of disadvantages.

The main one is that it depends heavily on a JIT compiler for optimization--but since the user is waiting while the JIT compiler runs, most JIT compilers don't include the most expensive optimizations. Since a C++ compiler typically runs with only a developer waiting (or nobody waiting, in the case of server-based builds with CI systems and such) it's much more reasonable for the compiler to include every optimization in the book.

A second factor is closely related: C++ vendors see their customers as being extremely concerned with speed. Java vendors almost certainly see their customers as being more concerned with things like features and fast development than with achieving absolutely the highest possible execution speed. That's not to say they can or do neglect execution speed entirely, just that they don't emphasize it to nearly the same degree as C++ vendors.

Garbage collection can have an effect as well. Java (normally) uses a concurrent, copying collector. This works reasonably well across a wide variety of work loads, and most JVMs have some "tuning knobs" to help out when the default setting aren't optimum.

Nonetheless, testing seems to indicate that a JVM requires substantially more memory than equivalent C++ code to even hope to run at (approximately) the same speed. Especially when you're running in a memory-constrained situation, the overhead from garbage collection can increase substantially. This typically stems less from how often the garbage collector runs (in itself), than from the fact that when it runs, many objects are still "alive". A copying collector works by copying "live" (reachable) objects into a new location, so ideally you want nearly all objects to be "dead" (unreachable) before it runs. With less memory, it not only runs more often, but each iteration takes longer because fewer objects have aged enough to "die", and more objects remain reachable (and therefore need to be copied).

There's also a difference in the amount of work that's done at run time vs. compile time. C++ places a high emphasis on doing as much as possible at compile time. For example, all the cool template "stuff" happens entirely at compile time (at the expense of horribly long compile times in some cases). Despite similar syntax, much (most?) of Java's enforcement of types in its generics system happens at run-time. Java also includes (and many Java programs use) such things as reflection, which simply aren't present in C++.

Java also adds a number of convenience features such as auto-boxing that make it fairly easy to add quite a bit of overhead to seemingly simple operations in ways that aren't immediately apparent. C++ has some of these as well, but (it seems to me) they're both fewer in number, and their detrimental effects are typically somewhat less drastic (but my opinion on that may well be affected by the fact that I know C++ quite a bit better, so to me, similar problems may seem much more obvious in C++ than in Java).

Bottom line: it's difficult to point to a single, specific reason that Java programs are slower than C++ programs. Although it's true much more often than not, the reasons for its happening vary widely, and in many cases it seems to be more the "death of a thousand nicks" than being the result of a single factor.

Answer 3

As to why Java can be slower well:-

• The JVM needs to interpret pseudo machine code, C++ blindly executes native machine code.
• The JVM manages memory, C++ its up to the programmer to manage memory which is more efficient but also more error prone.
• The JVM needs to start up, then load and interpret the various classes; with a C++ program load is basically copying the instructions from disk to memory, in many cases (with DLL, .so, shared libraries etc.) the program will have already been loaded and is sitting in memory ready to go. This is actually the biggest performance hit with Java and makes the language totally unsuitable for short running independent executions.

While java may be up to three times slower it is often almost as fast as C++. You have to ask yourself if this really matters when you are running on a 3ghz processor which is spending most of its time waiting for memory to be loaded (or worse disk IO).

The other side of the equation (highly subjective, but, backed by years of experience) is that Java programs can be developed up to three times faster. The toolsets are more functional, there is a vast selection of really good libraries to choose from, and a limited number of gotchas to look out for (no if (a = b), no mixing up pointers with the pointed to).

Personally where I feel the need for speed I code in plain C. Its a language I know well and the performance constrained parts of a system tend to be small and well defined.

So its C for speed. Java for production code. Groovy or Python for everything else!

Answer 4

To avoid speaking on false premise here, I would like to see at least part of the disassembly generated from the hot spots (tight loops where the CPU spends most of the time) from the two programs, before I would say anything substantial here.

For Oracle JVM, the following option can be used to save a copy of the machine code generated from JIT:

https://wikis.oracle.com/display/HotSpotInternals/PrintAssembly

For C++ running on x86, I recommend AMD CodeAnalyst on Windows, and objdump on Linux.

One should take close look at:

• The function calls (those that haven't been inlined)
• The loops (to see how they are nested, and how many times they are executed)
• The use of floating point instructions
  • Whether it uses the old x87 stack or the new SSE2 scalar/vector floating point instruction.
  • Use of expensive floating point instructions, and evidence of any optimization techniques done manually by each program's original author.
  • Use of library calls for handling math functions

If you do not already know the hot spots of the program, use a Low Overhead CPU Profiler to identify them first. Make sure not to confuse with high-overhead profilers. They are meant for different purposes.

Answer 5

I'm very glad that I see professional answers. You are all smart! Indeed, there is a small check you can do to test that this long runtime in case of Java is due to JIT compilation. This is how to see this:

Envelope these lines inside scjet_java/KT.jet as

for (int i=0; i<1000; i++){
    KT KT = new KT(0.6, 1, -1, 5.0);
    KT.setDebug(true);
    KT.buildJets(list);
    KT.printJets();
}

This will run particle clustering 1000 times. You will see that first run takes about 75 msec (i7,64 bit), while the next ones take only 15 ns for one clustering step. 15 ns is exactly the time that is needed for C++ code inside scjet_cpp/main.cpp to run over the single event!

So, the runtime is similar in C++ and Java code. I do not care much about 10-20% difference, if any at this level.