I believe a lot of Fortran is used in HPC, but not sure if that's only for legacy reasons.

Features of modern programming languages like garbage collection or run-time polymorphism are not suitable for HPC since speed matters so not sure where C# or Java or C++ come in.

Any thoughts?


9 Answers 9


I have seen a lot of Java used for HPC in areas where (1) there is little legacy code, and (2) development time and code quality matter. Typical application domains are finance, data mining or bio-informatics.

It really depends on the application (there is life outside linear algebra), but the performance of recent JVMs is often on par with C code. Sometimes faster when the JVM is able to perform at runtime clever optimisations that static compilers (C, Fortran) cannot do. And definitely faster when there's a lot of symbolic computation.

Given a fixed amount of time for program development, the resulting Java code is consistently faster than C code. HPC in Java definitely makes sense when code is developed or modified frequently. Another important feature is code mobility over different hardware.

You'll find references in http://ateji.blogspot.com/2010/09/java-for-high-performance-computing.html

Regarding the Fortran assumption that two address are unique, we're working on a static analysis tool that will enable similar optimizations for code in high-level languages, but without the "Bad Things May Happen" bit. Contact me if interested.

  • 14
    Nitpick: JIT optimizations are available to static compilers if you're willing to do a little work. Both GCC and MS Visual Studio support Profile Guided Optimizations which optimize using saved runtime data. It's a little misleading to suggest there are optimizations "that static compilers (...) cannot do". Jan 13, 2011 at 16:58
  • 5
    I don't know why this is the accepted answer, nothing in this post holds any semblance of truth. C based languages will always outperform Java, as Java is a Virtual machine hinged on another language inherently. Furthermore, anything you can achieve in Java you can achieve in C with less overhead. C based languages will never cease to be the 'performant' language.
    – Mike
    Sep 25, 2014 at 14:29

In my years of experience, up to 5 years ago, it has always been Fortran and C. Which one depended mostly on whether the people came more from engineering or more from CS school of thought (I don't know how to put this better, okey? :-)

In what we were doing Fortran was almost exclusively used.

From what I read around nowadays, with the new updates to the Standard F2003/08 and with the introduction of Co-Arrays, it seems to be gaining momentum again.

Also, one, if not somewhat biased article - The Ideal HPC Programming Language


I think for real pedal to the metal, the only real choice is Fortran. The reasoning is that the most important thing for the exploitation of low level ILP (Instruction Level Parallism) is memory address disambiguation. The defacto rules in Fortran allow the compiler to determine that two address are unique (and hence the order of loads and store, or even stores and stores can be interchanged without risk of generating incorrect code). C leaves too much scope for overlapping pointers for the compiler to extract as much low level parallelism from the code.

Also, array alignment, w.r.t cache lines, and SSE/AVX boundaries is important to the generation and execution of efficient loops. If arrays are passed via common blocks, the compiler/loader can assure that all arrays start on the same address alignment boundaries, and more efficient SSE/AVX loads and stores can be utilized. The newer hardware can handle unaligned memory accesses, but because the memory access isn't aligned properly partial use of cache lines results in lower performance. Even if a C programmer properly aligns all his arrays, is there a mechanism to communicate this to the compiler?

To summarize, the two most important issues, are the independence of memory addresses, and the recognition by the compiler that the accessed data structures have the same "natural" alignment that the hardware wants. So far Fortran does the best job on those two tasks.

  • 2
    I recently did a little experiment, find the pop count of a string of 64000 bits, represented as a unsigned long long array. I used the exact same algorithm using a lot of interesting boolean and packed arithmetic stuff. In C with -O3 it took 10clocks per long long, whereas with fortran Intel Fortran 10.1, with default optimization it was 6.5! And every programmer think C is superior for bit twiddling! Fortran defacto assumptions allow more efficient low level instruction coding to be safely generated. Dec 27, 2010 at 18:11
  • 4
    That should read "The defacto rules in Fortran allow the compiler to ASSUME that two address are unique...". The manuals all tell you that the compiler is allowed to assume this, and warn you IN DETAIL that Bad Things May Happen if you violate that assumption. Dec 27, 2010 at 18:24

Just some anecdotal note. I haven't done any high performance computing myself.

For calculations (number crunching), Fortran and C. Yes it is for legacy reasons:

  • Ample availability of public domain source code and recipes.
  • Both support MPI.
  • Both languages are compiled.
  • Compilers for both languages are provided by all HPC OSes and vendors.
  • Vectorizing compilers are available.
  • Both requires crazy level of tweaking to get high performance when ported to a different cluster (different memory size, number of CPUs etc)
    • This actually explains why the open source code is important: tweaking is necessary, therefore the original recipe must be written in a language that is good for manual tweaking.

The current trend for number crunching is to write program generators that automate the source code tweaking in order to optimize performance given the cluster characteristics. These generators often output in C.

A second trend is to write in some specialized dialect of C for specific GPUs or Cell BE.

For non-numerical work, such as programs that process data from a database (but not the database itself), it is much cheaper to run on clusters of "commodity" machines without the expensive customized networking equipments. This is usually called "High Throughput Computing". And Python is the #1 language here (using the famous Map Reduce). Prior to Python, batch processing projects can be written in any language, and are usually dispatched by Condor.

  • 1
    Could you elaborate a bit on the "crazy level of tweaking" part?
    – Rook
    Dec 4, 2010 at 15:32
  • The computing center hires graduate students to rearrange the MPI calls to make it run faster.
    – rwong
    Dec 4, 2010 at 15:36
  • (?) First word here, but I guess practices differ.
    – Rook
    Dec 4, 2010 at 15:37
  • It was a climate modeling research center.
    – rwong
    Dec 4, 2010 at 15:47

Fortran, for some good and some not-so-good reasons. For heavy math crunching, a good reason is there are extensive libraries (BLAS, LAPACK) of tried-and-true subroutines, all written in Fortran (though those can be called from C and C++).

A not-so-good reason is the supposed performance advantage of Fortran over C/C++. Optimizers are pretty good, and few people understand that the benefit of optimizing a piece of code is proportional to the percent of time it is busy, which in almost all code is almost zero.

Another not-so-good reason is a culture gap between CS and non-CS programmers. Scientific programmers tend to be taught bad habits in Fortran, and to look down on the CS programmers and the bad habits they have been taught, and who look down on the former.

  • "culture gap between CS and non-CS programmers. Scientific programmers tend to be taught bad habits in Fortran, and to look down on the CS programmers and the bad habits they have been taught, and who look down on the former." Partly this is just that they are concentrating on different aspects of the problem. Fortran means FORmula TRANslation, and it is pretty efficient at translating math formulas into code. For the sorts of programming CS types usually do, other languages are superior. Dec 27, 2010 at 23:13
  • 1
    @Omega: You're right. The Fortran-taught folks tend to have no concept of formatting, loathe "implicit none", and cram the code together because they still deal with 72-character lines and think making understandable code is for wimps. The CS-taught people create monster pyramids of classes laced with polymorphisms, notifications, and abstractions, when something simple would do the job. So they deserve each other :) Dec 28, 2010 at 2:11
  • 8
    the quote used to be "the physicists are solving tomorrows problems on yesterdays hardware - while the CS guys are solving yesterdays problems on tomorrows hardware" Jun 16, 2011 at 16:11
  • @Martin: I think maybe I heard that somewhere. It sure rings true. Jun 16, 2011 at 23:26
  • Martin: So, the hardware guys are the most efficient :) Aug 6, 2011 at 9:10

I've been working on some VERY calculation-intensive code in (gasp!) C#.

I'm building a GPGPU implementation of FDTD for optical modeling. On a small (128 processor) cluster, many of our simulations take weeks to run. The GPU implementations, however, tend to run about 50x faster - and that's on a consumer-grade NVidia card. We now have a server with two GTX295 dual-processor cards (several hundred cores), and are getting some Teslas soon.

How does this pertain to your language? In the same way that the C++ FDTD code we were using before was CPU-bound, these are GPU-bound, so the (very small) horsepower difference of managed vs native code doesn't ever come into play. The C# app acts as a conductor - loading OpenCL kernels, passing data to and from the GPUs, providing the user interface, reporting, etc. - all tasks that are a pain in the ass in C++.

In years past, the performance difference between managed and unmanaged code was significant enough that it was sometimes worth putting up with C++'s terrible object model to get the extra few percent of speed. These days, the development cost of C++ vs C# far outweighs the benefits for most applications.

Also, most of your performance difference isn't going to come from your choice of language, but from the skill of your developer. A few weeks ago, I moved a single division operation from the inside of a triple-nested (3D array traversal) loop, which reduced execution time for a given computational domain by 15%. That's a result of processor architecture: division is slow, which is one of those faces that you just need have picked up somewhere.

  • 1
    c++ has an object model? But it sounds like you should have gone with a script language to write your controllers in - if C# is better than C++ because of dev speed, then python (or lua, etc) is similarly better than C#.
    – gbjbaanb
    Jul 11, 2011 at 13:23
  • 3
    @gbjbaanb Not necessarily. This implementation is GPU-bound, but moving to a scripting language could very easily change that. C# is compiled and has a very nice optimizer. Compiled, strongly-typed languages are your friends! Less strict scripting languages tend to cause increased development time for any reasonably complex project.
    – 3Dave
    Jul 11, 2011 at 15:45
  • 1
    It’s been seven years. I’ve learned a lot. c++ is pretty awesome, C# is also awesome, I really like python and: CPU perf still matters.
    – 3Dave
    Jun 18, 2018 at 4:25

Fortran is most common, primarily due to legacy (people still run old code) and familiarity (most people who do HPC are not familiar with other kinds of languages).

Features of modern programming languages like garbage collection or run-time polymorphism are not suitable for HPC since speed matters so not sure where C# or Java or C++ come in.

That is not true in general. Classical HPC was mostly doing linear algebra with machine-precision numbers. However, modern HPC is increasingly using supercomputers for a wider variety of crunching, like symbolic calculations with arbitrary mathematical expressions instead of machine precision numbers. This places quite different characteristics on the tools you use and it is not uncommon to use programming languages other than Fortran because symbolic computation can be prohibitively difficult without GC and other kinds of optimizing compiler such as OCaml's optimizing pattern match compiler.

For example, read this paper by Fischbacher et al. which says "the authors have strong reason to believe that this may well be the largest symbolic calculation performed so far".

  • Fortran is common because many people use supercomputing time to run simulations of physical systems, such as global weather forecasting, and the implementation of the required algorithms in Fortran very clear and concise.
    – Sharpie
    Jan 13, 2011 at 16:33

Basically, all the programs which do the actual work of number crunching are still FORTRAN (the old blas, lapack, arnoldi etc are still the one used)... However, when it comes to higher level structure... people are increasingly using C++.

The complexity of the simulation involve huge code and to get any kind of benefit out of writing one is to make it reusable. Also, the concepts used have also become very complex. It almost madness to represent that information using FORTRAN. That's where C++ comes in since it inherently supports Object Oriented Design. However, Run-Time Polymorphism is rarely preferred. People instead almost always use Static Polymorphism (which is implement in C++ with template meta-programming)

Also, now the compilers are really good, hence a lot of optimization is left to the compilers.


There are two kinds of problems that need to be addressed in HPC applications: one is the number crunching itself and the other is managing of computations. The first one is usually approached with code written in Fortran, C or C++ because of speed and because of the fact that there are already a lot of scientific algorithms written in this languages. The steering of computations is more conveniently implemented in higher level languages. Python is a "glue" language of choice for handling application logic and calling extensions implemented in compiled languages. Java is frequently used by projects in which managing networking and distributed computing is essential.

Not the answer you're looking for? Browse other questions tagged or ask your own question.