Say if i have two floating point numbers
double a = 0.1
double b = 0.1
Then a, and b, after going through the EXACT SAME set of complicated arithmetic operations, on the same computer, then compare. will a==b always return true in this case?
2
-
1You don't say what processor you are using or what the operations you are applying. Some floating point processors will apply rounding with a "random" value of a least-significant-bit, causing different bit values at different times. You don't supply enough information for us to be able to answer this question.– BobDalgleishCommented Jun 15, 2020 at 15:50
-
@BobDalgleish While it would make sense for a definitive answer, I, for my part, am looking into this for knowing if my algorithm/code will work or not, in general, on the same processor, while this processor, on the other hand, could always be a different one. And if differences depending on calculation type could exist, this would be part of the answer.– LuckyLuke SkywalkerCommented Jun 28, 2023 at 13:01
Add a comment
|
5 Answers
It depends on your compiler, and what exactly you mean by "the exact same operations".
For example, if the compiler is allowed to use fused multiply-add (FMA) then the result of a*b + c*d is not defined - it could be fma (a, b, c*d) or fma (c, d, a*b), or a*b + c*d — all three are equally valid and possibly different.
And one line in your source code can be compiled in different ways. Say this multiplication a*b + c*d is contained in a function that is inlined, then several calls from several places could produce different results. In C, C++, Objective-C that's perfectly legal.
And note that FMA is faster and has only one rounding error instead of two, so it is absolutely preferable. It can make a dramatic difference to the speed of floating-point heavy code. And note that since you start with two different variables, you cannot go through the same operations.
answered Jun 12, 2020 at 8:09
-
"if the compiler is allowed to use fused multiply-add (FMA) then the result of ab + cd is not defined" Even if not defined, its behavior should be consistently used for either variable, no? Why would the compiler see two equal lines of code (barring the specific float variable being used) and decide to approach these two instructions differently? That would imply the compiler itself has some non-deterministic behavior, which is pretty much exactly what you don't want a compiler to behave like.– FlaterCommented Jun 12, 2020 at 8:29
-
@Flater optimizations may depend on context, not only a single line of code. For example, some compilers try to emit a mix of instructions which will allow the processor to run these instructions in parallel automatically. In such case, the compiler may choose between two possibilities depending on previous instructions since it knows instruction A can run in parallel with previous instruction X while B can't. Commented Jun 12, 2020 at 9:27
-
1As an example, if the processor has one unit that can process an fma, one unit that can multiply, and one that can add, then you would issue one fma, one multiply and one add for a = bc+d, e = fg+h. If you have a loop with one such operation, the compiler can unroll the loop and produce different code for the even and odd operations. Commented Jun 12, 2020 at 13:42
-
And given the compilation is the exact same, would the pc always return the same number, also when being (necessarily) inaccurate, or could the same operation on the same number in the same pc result in two different results? That’s the answer the op (and me as well) were/are searching for. Commented Jun 28, 2023 at 12:52
Yes, a==b will return true*. People are sometimes surprised by the results of floating point operations and think that there is some element of randomness involved. There's not, unless you call a random number generator. Finite precision floating point arithmetic is completely deterministic, it's just that it doesn't match our expectations for arithmetic with infinite precision integers or reals.
*unless a and b have become NAN (not a number). A value of NAN is not equal to any other value including another NAN. This is one of those elements of floating point math that is completely deterministic, but violates casual expectations.
answered Jun 12, 2020 at 3:48
-
That’s the answer i was looking for. Very helpful for future projects. Nonetheless, despite trust, any sources you can list regarding this thesis, may those be about low level computer arithmetic or other, ? Commented Jun 28, 2023 at 12:55
Effectively, yes.
I can imagine some more esoteric hardware might dispatch the operations differently (one to a dedicated floating point processor, one to a general purpose processor which produce slightly different results) but that is so unlikely that I probably wouldn't design for it.
That said, I'd still do some sort of epsilon check rather than the raw equality. Just because they're going through the exact same set of operations today doesn't mean they always will.
Yes, if you mean machine instructions.
Getting from your preferred source-language to machine instructions is an arbitrarily involved process balancing time to compile, size of the code, and expected speed of execution, where allowed transformations depend on language semantics and requested/banned optimisations.
Inlining, fast-math, when to store (and potentially discard excess precision), fusing instructions (mostly fused multiply-add) with different context can result in different machine code.
answered Jun 12, 2020 at 13:29
-
I had the great idea of an FPU with non-deterministic rounding. If the correct result x is between a and b, then it would be rounded more likely to the closer value. That would be excellent for code where you don't know how sensitive it is to rounding errors. If you run your code five times and results are the same to 14 decimals, then you can assume the results are correct to 14 decimals. If the results are identical on each run, you have no idea if they are correct or not. Commented Jun 12, 2020 at 13:46
-
@gnasher729 Something similar can sometimes be done running the same code with different rounding-modes: up, down, to zero, to infinity, to even, to odd. FENV_ACCESS is useful for something. Commented Jun 12, 2020 at 14:02
-
To some degree, yes. What you lose is that the average rounding error is zero. If you add 10,000 numbers with “round down” then you can expect an error around 5,000u. Commented Jun 12, 2020 at 14:58
As an answer to your question, then yes the floats will effectively be the same. As mentioned by Telastyn you might want check the equality because the future is always unsure and changes happen quite often.
To give some perspective/reference on the matter of floating point you can check:
