Why do many programming languages lack standard vector types?

Question

Many languages such as C or even C++ or C# or Java have no natively supported vector (SIMD) types or functionality. In such languages, one would have to either use non-standard extensions or third-party libraries to access vector types/instructions, or make due without them and just hope that their compiler is smart enough to auto-vectorize their code.

Parallelizing calculations can be a huge performance win, by using vector instructions that perform on multiple values at the same time instead of serial instructions that perform the calculations one at a time.

I cannot see how:

• It would be non-trivial for a compiler or runtime to check if vector instructions are available on the targeted processor and if not then simply fallback on non-vector instructions.
• Auto-vectorized code would be as dependable as hand-vectorized code.
• Semantics would differ that much on various processors, as vector types are just N of some type that can be operated on with arithmetic operators similar to regular types.

Why do many languages not provide a standard method to hand-vectorize code? Are there other reasons I am not considering? One could compare the hypothetical standardized SIMD types to optimization hints such as the C register or inline keyword; just a way to make vectorizable calculations easily identifiable to compilers.

Answer 1

I think it's as simple as:

The set of supported vector instructions is highly variable depending on the CPU type. So you are asking for languages to make assumptions about which CPUs the language can run on. Sure, there are some commonalities, but there are also many factors that are not common.

Any vectorized algorithm is likely to be written as a specialization of an existing scalar algorithm. It makes no sense for the compiler to de-vectorize the vectorized algorithm when the scalar replacement already exists.

To write a fast vectorized algorithm, you have to know which operations are fast. Even if the language supports many vector operations, you still want a different vectorization depending on whether you are running on CPU architecture A (where X,Y,Z are native and A,B,C are emulated) or CPU architecture B (where X,Y,A are native and Z,B,C are emulated). Running the B algorithm on architecture A may severely reduce the performance due to the overhead of emulating Z; it might even be slower than the scalar algorithm. In particular, emulating all the vector operations is likely to perform worse than the scalar algorithm.

---

It is more common for a language to offer higher-level operations that are easy to auto-vectorize - such as element-wise operations and reductions (e.g. sum) on arrays. C++ has std::valarray and some languages such as APL have it natively. Compilers for any language may also recognize for loops over arrays with no dependencies.

Answer 2

I try to give another perspective.

Many languages such as C or even C++ or C# or Java have no natively supported vector (SIMD) types or functionality.

Every definition of "native" is different, but C# has had some Vector types for a while. They are found in these 2 namespaces:

System.Numerics and System.Runtime.Intrinsics

But you are right, they were not there from the beginning and the vector types might often be a bit behind in support. (However, they enable you to write one method that gets hardware accelerated on both x64 and ARM64)

To your question: Vector types do not bring much to the table in the big picture. Sure, some scientific algorithms are a bit quicker. But in an application, large parts of the code will just do some user interface stuff, validation, network, parsing or painting.

Also: Vector types just make things faster. They do not enable you to do stuff that you otherwise could not so. (Like CMPXCHG does) So if you are a language designer, I can totally see that comprehensive vector types would be bottom-of-backlog.

In short:

• Vector types are hard & high maintenance, especially if you want to offer them across CPU architectures
• Large vectorized computations happen in very few programs
• Smaller vectorizations over dozens of elements do not impact perceived performance

So why langes lack the support? Because means are limited and the designers chose to spent the efforts on more promising things.

Answer 3

It is not just about hardware support and the ability of the compiler to recognize patterns that can be vectorized.

Consider what it means for a language implementation to support vector types natively. You would have byte, short, int, long, float, double and... vector? That would be an entirely different beast.

It is not fixed length, it is not fixed format, it is just a concept. A bunch of numbers that cannot be handled on the processor level like the other native types can, which map nicely to registers and processor instructions. A vector requires a lot of setup before the actual instruction can be executed and that setup (putting each number in the right place) requires knowledge about the algorithm you are implementing. It would not be helpful to just replace all values in the hardware array with another instance of your native vector type before executing the next operation. That would severely defeat the purpose of SIMD: doing things efficiently.

For the same reason a "simple" array is not a native type in any language although it may look that way. It is up to you, the programmer, to tell what should be done with the elements in the array. Sometimes you want to replace just one, then you want to iterate over all. These are not native operations from the array perspective, like adding two instances.

So the hardware support may be there, using it efficiently requires more logic, on a higher level, than a compiler could offer.

Answer 4

Many languages such as C or even C++ or C# or Java have no natively supported vector (SIMD) types or functionality.

Many processors lack vector instructions, so building support into a language could mean either never supporting those processors, or requiring compilers for those processors to implement those instructions in software. Also, vector instructions tend to be very hardware-specific; creating a common vector processing model that works the same way across a variety of different processors would be complicated at best.

But perhaps the biggest reason that languages don't have built-in support for vectors types is that there's no clear need for it. Languages themselves tend to be pretty bare-bones when it comes to useful functionality; they add utility via libraries. Adding vector support can be done easily enough by adding a library, so there's little reason to add it as a built-in feature of the language.

Answer 5

If you want really fast code, you use a language like C or C++. If speed is not essential you might use Python.

And if you want really fast code, you might want SIMD instructions. So Python + SIMD would be an odd combination, you want C or C++ with SIMD.

And the two most common C and C++ compilers, gcc and Clang, both have processor independent small vector instructions. For example you can declare a vector of double of size 72 bytes, and it just works (Clang on ARM for example will use four 16-byte vectors and either another double or half of another vector). Intentionally using an odd sized example, because it is processor dependent. And it works if the processor has no vector instructions at all.