1

Many languages such as C or even C++ or C# or Java have no natively supported vector (SIMD) types or functionality. In such languages, one would have to either use non-standard extensions or third-party libraries to access vector types/instructions, or make due without them and just hope that their compiler is smart enough to auto-vectorize their code.

Parallelizing calculations can be a huge performance win, by using vector instructions that perform on multiple values at the same time instead of serial instructions that perform the calculations one at a time.

I cannot see how:

  • It would be non-trivial for a compiler or runtime to check if vector instructions are available on the targeted processor and if not then simply fallback on non-vector instructions.
  • Auto-vectorized code would be as dependable as hand-vectorized code.
  • Semantics would differ that much on various processors, as vector types are just N of some type that can be operated on with arithmetic operators similar to regular types.

Why do many languages not provide a standard method to hand-vectorize code? Are there other reasons I am not considering? One could compare the hypothetical standardized SIMD types to optimization hints such as the C register or inline keyword; just a way to make vectorizable calculations easily identifiable to compilers.

10
  • 1
    I think because there's no benefit to having ones that don't work on your processor, and every processor has completely different ones. Also, it's more work.
    – user253751
    Mar 24, 2023 at 20:32
  • 1
    Only very basic semantics are the same on different processors. Yes, they all have add and subtract. What about shuffle? bit deposit/extract? type conversion operators? for that matter, which types are supported? even the odd ones, like bfloat16? Here's a list of x86 vector instructions, should they all be supported?
    – user253751
    Mar 24, 2023 at 20:44
  • 1
    @user253751 Do most languages even natively support bfloat16 independent of vector types? I do not believe so. 'Why would you write a vectorized algorithm that would compile into a non-vector algorithm?' Typically you would not, but the reason is so that it can run at all on systems that do not have vector instructions, instead of breaking. It is a matter of portability. Mar 24, 2023 at 20:50
  • 2
    Something else to add: for every application that would benefit from these instructions, are 100 applications that schedule dentist appointments or generate insurance quotes or other unimaginably boring thing. There isn't much bang for the buck focusing on SIMD Mar 28, 2023 at 4:13
  • 1
    I don't see how this is definitively answerable without asking all the language creators. It may be as simple as they weren't interested in adding them. Mar 28, 2023 at 16:38

5 Answers 5

5

I think it's as simple as:

The set of supported vector instructions is highly variable depending on the CPU type. So you are asking for languages to make assumptions about which CPUs the language can run on. Sure, there are some commonalities, but there are also many factors that are not common.

Any vectorized algorithm is likely to be written as a specialization of an existing scalar algorithm. It makes no sense for the compiler to de-vectorize the vectorized algorithm when the scalar replacement already exists.

To write a fast vectorized algorithm, you have to know which operations are fast. Even if the language supports many vector operations, you still want a different vectorization depending on whether you are running on CPU architecture A (where X,Y,Z are native and A,B,C are emulated) or CPU architecture B (where X,Y,A are native and Z,B,C are emulated). Running the B algorithm on architecture A may severely reduce the performance due to the overhead of emulating Z; it might even be slower than the scalar algorithm. In particular, emulating all the vector operations is likely to perform worse than the scalar algorithm.


It is more common for a language to offer higher-level operations that are easy to auto-vectorize - such as element-wise operations and reductions (e.g. sum) on arrays. C++ has std::valarray and some languages such as APL have it natively. Compilers for any language may also recognize for loops over arrays with no dependencies.

4
  • Oh, so std::valarray, although is not explicitly vectorized, is very likely to be vectorized in practice in implementations that support it? Mar 24, 2023 at 21:12
  • 3
    @user16217248 I am fairly sure that's the point of it. I don't know which implementations currently do that. You can see it is easy to vectorize but the program also works fine if it is not vectorized which is pretty much what you want.
    – user253751
    Mar 24, 2023 at 21:13
  • To expand on this great answer: support for vectorized instructions differs drastically between microarchitectures and even between different models on a microarchitecture. Sometimes, such instructions are removed from product lines. Unless you target a specific processor (march=native style) a compiler cannot safely assume the existence of many vectorized instructions, or would have to test for them at runtime and emit fallback code paths. However, there is a large common base of vectorized instructions that is supported by all modern x86-64 CPUs, and Windows 11 is helping raise that level.
    – amon
    Mar 24, 2023 at 21:41
  • @amon I don't think the answer is great :)
    – user253751
    Mar 24, 2023 at 21:42
5

I try to give another perspective.

Many languages such as C or even C++ or C# or Java have no natively supported vector (SIMD) types or functionality.

Every definition of "native" is different, but C# has had some Vector types for a while. They are found in these 2 namespaces:

System.Numerics and System.Runtime.Intrinsics

But you are right, they were not there from the beginning and the vector types might often be a bit behind in support. (However, they enable you to write one method that gets hardware accelerated on both x64 and ARM64)

To your question: Vector types do not bring much to the table in the big picture. Sure, some scientific algorithms are a bit quicker. But in an application, large parts of the code will just do some user interface stuff, validation, network, parsing or painting.

Also: Vector types just make things faster. They do not enable you to do stuff that you otherwise could not so. (Like CMPXCHG does) So if you are a language designer, I can totally see that comprehensive vector types would be bottom-of-backlog.

In short:

  • Vector types are hard & high maintenance, especially if you want to offer them across CPU architectures
  • Large vectorized computations happen in very few programs
  • Smaller vectorizations over dozens of elements do not impact perceived performance

So why langes lack the support? Because means are limited and the designers chose to spent the efforts on more promising things.

2
  • 1
    Vector classes are meaningless to the compiler. The question is about types native to the language that the compiler could recognize as such. Mar 25, 2023 at 8:18
  • 5
    If you use the mentioned Vector structs, then the .net 7 JIT compiler will use intrinsics where available. So in this specific case, they are far from meaningless. Just look at this disassembly, vaddps sounds liek a vector instruction For all intents and purposes, this is the C# standard way to work with vector instructions. It's not quite as smooth as int but close. Mar 25, 2023 at 9:35
1

It is not just about hardware support and the ability of the compiler to recognize patterns that can be vectorized.

Consider what it means for a language implementation to support vector types natively. You would have byte, short, int, long, float, double and... vector? That would be an entirely different beast.

It is not fixed length, it is not fixed format, it is just a concept. A bunch of numbers that cannot be handled on the processor level like the other native types can, which map nicely to registers and processor instructions. A vector requires a lot of setup before the actual instruction can be executed and that setup (putting each number in the right place) requires knowledge about the algorithm you are implementing. It would not be helpful to just replace all values in the hardware array with another instance of your native vector type before executing the next operation. That would severely defeat the purpose of SIMD: doing things efficiently.

For the same reason a "simple" array is not a native type in any language although it may look that way. It is up to you, the programmer, to tell what should be done with the elements in the array. Sometimes you want to replace just one, then you want to iterate over all. These are not native operations from the array perspective, like adding two instances.

So the hardware support may be there, using it efficiently requires more logic, on a higher level, than a compiler could offer.

1
  • SIMD vectorization uses fixed-length vectors and these are what the asker is asking about. (Which length? Depends on the processor architecture and data type!)
    – user253751
    Mar 27, 2023 at 9:26
1

Many languages such as C or even C++ or C# or Java have no natively supported vector (SIMD) types or functionality.

Many processors lack vector instructions, so building support into a language could mean either never supporting those processors, or requiring compilers for those processors to implement those instructions in software. Also, vector instructions tend to be very hardware-specific; creating a common vector processing model that works the same way across a variety of different processors would be complicated at best.

But perhaps the biggest reason that languages don't have built-in support for vectors types is that there's no clear need for it. Languages themselves tend to be pretty bare-bones when it comes to useful functionality; they add utility via libraries. Adding vector support can be done easily enough by adding a library, so there's little reason to add it as a built-in feature of the language.

-4

If you want really fast code, you use a language like C or C++. If speed is not essential you might use Python.

And if you want really fast code, you might want SIMD instructions. So Python + SIMD would be an odd combination, you want C or C++ with SIMD.

And the two most common C and C++ compilers, gcc and Clang, both have processor independent small vector instructions. For example you can declare a vector of double of size 72 bytes, and it just works (Clang on ARM for example will use four 16-byte vectors and either another double or half of another vector). Intentionally using an odd sized example, because it is processor dependent. And it works if the processor has no vector instructions at all.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.