3

Developing Big Data processing pipelines and storage, you probably come across software which is more or less a part of the Hadoop ecosystem. Be it Hadoop itself, Spark/Flink, HBase, Kafka, Accumulo, etc.

Now all of these have been very well implemented, offering fast and high-quality solutions to the developers needs. Still, especially with the Big Data usage patterns in mind, a huge amount of object allocations and deallocations happen. It is probably worthwhile to use a non-garbage collected language, like C++.

Another reason I could find for myself, why Java applications are so popular in this domain, is the distributed deployment. One key characteristic of Big Data applications is the size, they don't fit on a single machine. The JVM allows really simple deployment (just copy the bytecode around). But is this really an argument? Looking at our own cluster, the hardware is quite similar and I would assume that this holds true for most companies. So even compiled machine code should be easy to move around to all machines.

For me personally, the biggest reason would probably be DRY (don't repeat yourself). It started in Java and libraries and frameworks grew around it. They work very well and nobody is willing to invest in rewriting the whole stack in a different programming language for (if at all) marginal gain.

Maybe someone of you has a deeper insight than me?

7
  • It is perhaps worth noting that Spark at least has effectively written its own memory management layer. Jan 30, 2019 at 18:52
  • 4
    Do you have profiled your workloads and have hard evidence that object allocation and deallocation is an actual bottleneck? Which garbage collector did you use? There are dozens of them, and some of them might be better for certain use cases. Note that for most modern high-performance generational copying tracing garbage collectors, allocating temporary objects is O(1), just bumping a pointer, identical to allocating a stack value in, say, C++. And garbage collecting the young generation is O(#live objects), so deallocation of short-lived objects is 100% free. Jan 30, 2019 at 20:09
  • 1
    My intention for this question was much more general. Because of the specific use case for Big Data processing, two important factors come into play: performance and scalability. Looking at the software out there, almost always a JVM language was chosen and I am just curious what the deciding points were. I am explicitly not arguing that Java is slower than C++. We have a bit of profiling data for our applications and under heavy workloads we can see the garbage colloctor stalling the whole pipeline, we tested different collectors, but that would be a question for SO.
    – flowit
    Jan 30, 2019 at 20:22
  • 4
    Your assumption that garbage collection is very expensive over time may not be correct. Jan 31, 2019 at 1:19
  • 4
    Guesswork: From what I can see from the engineering POV on the topics I have been involved in, I'd say Java/C#/C++/whatever "doesn't matter" from a performance POV for Big Data, because, from what I can see, the performance characteristics are dominated by the distributed nature of the problem, and getting a handle on the data sizes involved. (i.e. network/disk systems) If the actual processing of the data on the single nodes is slightly more efficient / faster / less latency, doesn't seem to be the key point.
    – Martin Ba
    Jan 31, 2019 at 12:03

4 Answers 4

10

Hadoop was originally written in Java, because it was used to "fix" problems in Nutch, which also was written in Java. Nutch, in turn, was written in Java because it was a write once run anywhere solution.

As for whether C++ or another language would have been a better choice, that's definitely up for debate. With modern architectures, I'd trust Java or C#'s garbage collector over a random developer's judgement. For most applications, we don't need to be heavily concerned with resource usage, beyond normal best practices, unlike the early days of computing where every bit was important and needed to be managed.

However, Big Data is definitely an outlier for that approach. I still would have a developer who understood how Java's garbage collection worked code in Java than trust a developer in C++ to know how to do garbage collection well.

That said, this will almost always get into a debate about Java and C# developers being spoiled by their frameworks, and as a C# developer, I'd always rather have a library written and tested by a team of professionals (or a library written and tested and used by the masses) than try to do it myself. Instead of knowing how to manually allocate memory and manage it (which I can do in C, but haven't since school) I'd rather just understand how the C# garbage collector works.

1

The comment that "performance" is an important factor is odd, because "performance" is not a single monolithic thing.

The two most obvious different types of "performance" constraint are:

  1. Throughput (long-running servers, jobs that take a long time to complete, etc.)

    GC is typically good for this, since any overhead is amortized.

  2. Latency (programs that need to start up quickly, short-running event handlers that should complete quickly, etc.)

    GC can be problematic for this, since it may take a while to reach a steady state, or we may otherwise be interested in time periods over which collection overhead cannot be amortized.

When I say collection overhead, I don't really mean "overhead compared to manual memory management", since that would imply GC is always slower. I mean just the cost of running a garbage collector at all, which can be unevenly distributed in time (compared to manual memory management which one might expect to be interleaved fairly uniformly - or at least consistently - with the program logic).

Anyway, if you think latency is more important for "Big Data processing pipelines", you should absolutely consider re-writing Hadoop. Personally, I suspect throughput is more important, and that GC is a perfectly rational choice for this.

1
  • Great answer. I would add that GC is more diverse than people often think. Copy collectors such as the new/young generation in Hotspot are extremely fast for allocation and reclaiming memory, much faster than approaches that require compaction of fragmented memory. They come with a cost that they are inefficient in terms of memory usage, however.
    – JimmyJames
    Oct 8, 2020 at 14:50
0

For me, garbage collection is a solution to DRY out memory-management code. All those malloc and free calls can be automatically handled by garbage collection fairly systematically and efficiently. Sure there are some edge cases, but unless you can profile garbage collection as the most important bottleneck in your application, I don’t see why you’d bother switching to a manual memory management language like C++. At a certain point, you have to trust that libraries/frameworks/systems do what they say they do, and appropriately/efficiently.

Garbage collection systems have been extensively developed and tested. Given my own lackluster knowledge of memory management, I think I would rather trust experts who have collaborated with researchers with mathematical proofs to certain garbage collection strategies, rather than trust a developer to say “yeah I think we should free up that memory here because we won’t need those objects later”, then inevitably cause a re-allocation of those same objects later because they forgot they had already freed them.

If you have some very niche part of the application that needs very specific and detailed memory management, for example, couldn’t you just write that as a microservice or small library in the desired language, then link it back to your primary language? I don’t know what it’s called in the desktop library world, but you can certainly call C++ compiled libraries through Java if you really needed that level of micromanagement.

But I imagine that since Hadoop hasn’t done that yet (to my knowledge), memory management actually isn’t a problem for this ecosystem.

3
  • 3
    Modern C++ heavily encourages the usage of various kinds of smart pointers to avoid the need for manual memory management. By keeping track of who owns the memory, the language can automatically deallocate it when it goes out of scope.
    – 8bittree
    Jan 31, 2019 at 17:50
  • @8bittree Fair enough, it’s been 8 years since I programmed in C/C++, and we never were really into the “current standard” as far as university education goes, so my knowledge is at least a decade out of date! Jan 31, 2019 at 20:05
  • 2
    Using simple malloc/free is definitely considered outdated and substandard in modern C++. Do not evaluate the language on their level of risk; it's like evaluating a modern car by looking at the model T.
    – Aganju
    Feb 14, 2019 at 15:59
-5

tl;dr: People are already rewriting entire swaths of the big data ecosystem in C++. If you aren't thinking of rewriting your code in C++ or Rust, you should.

You are already seeing applications completely re-written from the ground up, moving from Java to C++. A sample listing:

But this revolution won't be limited to C++. You will also begin to see people porting code from Java to Rust.

The whole purpose for which Java was developed — to write-once, run-anyway — makes little sense when the world has moved from a heterogenous Unix/WindowsNT/Linux/MacOS universe of the 1990s into which it was born, to a modern "Linux everywhere" server market of the 2020s.

I'd argue that C++ right now is the solid bet. Rust for those who like to place bets on the dark horse. But Java? Even with the new modern JVMs and garbage collectors, it's an artifact of a paradigm that just doesn't exist in on-prem datacenters or in the cloud.

Java may still make some sense for client applications which need to run on Windows, Mac OS X and Linux, but for the servers? Even Microsoft Azure is majority Linux now.

Historically, even by the early 2010s there was little reason to put all our big data eggs in a Java basket. But software programming is a behavior. It takes years or even decades to shift the prejudices and biases of minds and the motor memory of typing fingers.

Java was used in the 2010s because it's what people learned over the preceding few decades. Yet it was already an anachronism by then. Now, with servers sporting 50-100+ CPUs, it just a really, really obvious anachronism.

Let's look at those core counts, especially. People need to do more than just simple "ports" of applications. They need to reconsider re-writing from the ground up for a world of multi-core, multi-CPU NUMA architecture servers.

They need to adopt highly asynchronous programming models. Design for more autonomy between processes than typical multi-threading paradigms. Otherwise cross-CPU traffic is going to lead to noisy contention, locks and stalls.

Beyond that, you can break away from the "virtual machine" sandbox thinking to take full advantage of every underlying hook your operating system can deliver. If you can squeeze 2% or 5% more out of your box here-and-there, you will. Because big data systems are, or should be, designed to be utterly "greedy."

Moving from Java to C++ means you can look at native OS capabilities in your programming, like io_uring and eBPF in Linux. Most developers have yet to embrace and integrate such capabilities into their application designs and their day-to-day thinking.

C++ (or Rust) also means you will need a deeper level of application developer and system architect than a typical Java body shop. You may need some hired guns that really grok the modern OS and hardware engines they are working on. Will this be beyond many people's comfort zones? Hell yes. And it's why some companies are hesitant to even go there. They know the technical depth of their bench just isn't there.

For those who are used to writing lowest-common denominator highest portability code, this is going to be a challenge. Like learning how to put the pedal to the metal on the unbounded Autobahn when they've lived their whole lives in a gated community with a 25 kph speed limit. You need to have a gut check: do you have a thrill for that kind of speed? Some people will simply retreat back to the gated community. It's safe. It's understood.

As another example, thinking ahead now a few years: we are on the verge of a revolution of byte-addressable solid state storage when persistent memory will be broadly available in public clouds. Rather than reading multi-kilobyte "disk pages," which are an artifact of the rotating media past. When that happens, people will wake up and realize: "Why are you still fetching 4k blocks? That makes no sense!" Once we get to byte-addressable persistent storage, you will need to totally rethink read and write paths, data caching, disk optimization and so on.

Which do you think is going to give you access to that underlying efficiency in current and emergent OS and hardware capability sooner? C++, Rust or Java?

(Full disclosure: I work at ScyllaDB.)

1
  • 1
    The real anachronism is anything that requires application developers to integrate low-level OS and hardware capabilities in "their application designs and their day-to-day thinking". That kind of thing is receding further and further into the past and not coming back. It's not some people that will reject it, it's nearly everyone, and that represents not a failure on their part but a failure on the part of the people who try to sell their inability or unwillingness to abstract away low-level concerns as a feature. Oct 8, 2020 at 12:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.