tl;dr: People are already rewriting entire swaths of the big data ecosystem in C++. If you aren't thinking of rewriting your code in C++ or Rust, you should.
You are already seeing applications completely re-written from the ground up, moving from Java to C++. A sample listing:
But this revolution won't be limited to C++. You will also begin to see people porting code from Java to Rust.
The whole purpose for which Java was developed — to write-once, run-anyway — makes little sense when the world has moved from a heterogenous Unix/WindowsNT/Linux/MacOS universe of the 1990s into which it was born, to a modern "Linux everywhere" server market of the 2020s.
I'd argue that C++ right now is the solid bet. Rust for those who like to place bets on the dark horse. But Java? Even with the new modern JVMs and garbage collectors, it's an artifact of a paradigm that just doesn't exist in on-prem datacenters or in the cloud.
Java may still make some sense for client applications which need to run on Windows, Mac OS X and Linux, but for the servers? Even Microsoft Azure is majority Linux now.
Historically, even by the early 2010s there was little reason to put all our big data eggs in a Java basket. But software programming is a behavior. It takes years or even decades to shift the prejudices and biases of minds and the motor memory of typing fingers.
Java was used in the 2010s because it's what people learned over the preceding few decades. Yet it was already an anachronism by then. Now, with servers sporting 50-100+ CPUs, it just a really, really obvious anachronism.
Let's look at those core counts, especially. People need to do more than just simple "ports" of applications. They need to reconsider re-writing from the ground up for a world of multi-core, multi-CPU NUMA architecture servers.
They need to adopt highly asynchronous programming models. Design for more autonomy between processes than typical multi-threading paradigms. Otherwise cross-CPU traffic is going to lead to noisy contention, locks and stalls.
Beyond that, you can break away from the "virtual machine" sandbox thinking to take full advantage of every underlying hook your operating system can deliver. If you can squeeze 2% or 5% more out of your box here-and-there, you will. Because big data systems are, or should be, designed to be utterly "greedy."
Moving from Java to C++ means you can look at native OS capabilities in your programming, like io_uring and eBPF in Linux. Most developers have yet to embrace and integrate such capabilities into their application designs and their day-to-day thinking.
C++ (or Rust) also means you will need a deeper level of application developer and system architect than a typical Java body shop. You may need some hired guns that really grok the modern OS and hardware engines they are working on. Will this be beyond many people's comfort zones? Hell yes. And it's why some companies are hesitant to even go there. They know the technical depth of their bench just isn't there.
For those who are used to writing lowest-common denominator highest portability code, this is going to be a challenge. Like learning how to put the pedal to the metal on the unbounded Autobahn when they've lived their whole lives in a gated community with a 25 kph speed limit. You need to have a gut check: do you have a thrill for that kind of speed? Some people will simply retreat back to the gated community. It's safe. It's understood.
As another example, thinking ahead now a few years: we are on the verge of a revolution of byte-addressable solid state storage when persistent memory will be broadly available in public clouds. Rather than reading multi-kilobyte "disk pages," which are an artifact of the rotating media past. When that happens, people will wake up and realize: "Why are you still fetching 4k blocks? That makes no sense!" Once we get to byte-addressable persistent storage, you will need to totally rethink read and write paths, data caching, disk optimization and so on.
Which do you think is going to give you access to that underlying efficiency in current and emergent OS and hardware capability sooner? C++, Rust or Java?
(Full disclosure: I work at ScyllaDB.)