6

My understanding is that it's rarely a good idea to compare two floating point numbers due to inherent inaccuracies. However, my understanding of those inaccuracies are that they are deterministic and will always show up the same way given two identical inputs and identical transformations, hardware errors notwithstanding. So, if a collection of fp values are gathered from the same source and manipulated in the exact same ways, is it ever appropriate to test them for equality?

0
7

If by equality you mean bit by bit comparisons I have a time when that would be acceptable: when confirming that you've stored the numbers without error.

Any IEEE 754 number has a value that another IEEE 754 number of the same length (precision, whatever) can take on and be identical to it.

So the issue here isn't that IEEE 754 floating point values can never be bit by bit equal. It's that in very many other cases being or not being bit by bit equal is meaningless. That's simply because like any number that isn't allowed to go on forever the shorter numbers end up pretending they're the same thing. 3.14 is not π but we like to pretend it is. When it turns out not to be π don't blame IEEE 754.

Just like π messes with our base ten decimal numbers some base ten decimal numbers mess with IEEE 754. For example 0.1 is too big to fit in any IEEE 754 number for the same kind of reason you can't fit π on a single sheet of paper. In base 2 a 0.1 goes on forever. What trips people up is they are used to π behaving this way. They aren't used to 0.1 behaving that way.

Now if you're asking are there cases when IEEE 754 numbers map to real world values without inaccuracies I have an example of that: inches. Traditional inches (not those weird decimal inches your drafting teacher told you about) are base 2. You divide them in half, or a fourth, or an eighth, etc.

Unlike most things that get measured with IEEE 754, inches have an error free representation in the IEEE 754 floating points. But then people enter them, or display them, as base 10 numbers with a decimal point and weirdness happens. That's not IEEE 754's fault. It's because you store it one way and look at it another.

Scientists typically chose IEEE 754 not because it's good at bit by bit comparisons. They chose it because it makes good use of the bits available to create a "good enough" representation of what they're measuring. We understand good enough when we stop π at 3.14 but not so well when packing 64 bits in a IEEE 754 float. Which is why bankers (who work with decimal money) should never choose IEEE 754 floats.

So if anyone left you thinking that IEEE 754 isn't completely predictable they were pulling your leg. But many have come to expect unpredictability from people who work with IEEE 754.

3
  • Re: "if anyone left you thinking that IEEE 754 isn't completely predictable they were pulling your leg." You can use this in a way to get completely predictable results isn't the same as this is completely predictable. IEEE 754 leaves things unspecified (like rounding of transcendental functions) which may and do differ between implementations in various and unpredictable ways. Jun 9 '21 at 19:32
  • 2
    ‘According to the pigeon hole principle any IEEE 754 number has a value that another IEEE 754 number of the same length (precision, whatever) can take on and be identical to it.’ – that’s nonsense. Jun 10 '21 at 9:15
  • 2
    @user3840170 that is not nonsense, it says IEEE 754 values are not magically incapable of being equal to each other.
    – user253751
    Jun 29 '21 at 14:16
6

I never use == for FP -- and I also use == for FP all the time:

  • I never use it in production code that does anything domain-specifically meaningful with the values.

    • Like, test if the thing you calculated equals what the user entered; check if a measured value from subsystem A is the same measured value as the one from subsystem B, etc.
  • I use == all the time (but note: not always) when Unit Testing functions manipulating floating point values.

    • As you note IEEE754 manipulations/calculations are deterministic (within a given subsystem at least) and if you have a (pure) function that has some float input and some float output, you can - most often - equality compare the expected UT results just fine.
    • Or, if you store a float value in a file/database, and if you require roundtrip accuracy, then comparing with == is what you actually wanna do. (Yeah and handle NaN etc specially)
3
  • 3
    Even for unit tests, an exact comparison may couple the test too tightly to the implementation. For example changing the order of operands (e.g. a + b + c to a + c + b) might give a different result, even though this result would still be correct for all intents and purposes. Changes in hardware or compiler versions might also cause hard-coded comparisons to fail, which will make the tests fragile.
    – JacquesB
    Jun 9 '21 at 11:43
  • @JacquesB - good point. Guess how well this holds depends on the system(s) one's working on. Still: Unless your testing complicated calculations, it may be a good thing to have your UT break and have to revisit stuff if (rather simple) UT manipulations start to give different results after some changes.
    – Martin Ba
    Jun 9 '21 at 15:27
  • 1
    We are pentium of borg. You will be approximated.
    – user10489
    Jun 12 '21 at 5:42
3

Exact equality can be used when you know that the values are exact numbers. E.g. in C syntax:

 for (double i = 0.0; i <= 10.0; i++) ...

this will step through the exact values 0.0, 1.0, ... 1.0.

Integers within a certain range are exactly represented in IEEE floating-point, and this can be relied upon.

Another class of number that is exactly represented are arithmetic combinations of powers of two, both fractional and integral within the limits of the mantissa. For instance 0.5, 0.125, 0.625, 2.125, 7.25 and other such numbers that are sums of powers of two. This is actually the more general class which includes the aforementioned integer representations. We can understand 7.25 as being represented exactly because it's a sum of 1.0, 2.0 and 4.0 and 0.25.

These loops should be fine:

for (double i = 0.125; i <= 10.625; i += 0.5) ...

for (double i = 0.375; i <= 6.0; i *= 2.0) ...
2

My frank answer is – "just use range tests and be done with it." Anytime you test an FP value for "literal equality," you are making assumptions about their underlying representation ... but your source-code doesn't clearly reflect that. Therefore, someone who's debugging your code might not realize what you had in mind ... why you wrote that bug in that way.

"There's no Return On Investment (ROI) here ..."

It's trivial to simply define a boolean function, e.g. isFPequal(x,y), which not only incorporates the desired test, but also makes it extremely obvious to anyone else who's reading your code exactly what's going on.

"Obvious" is A Very Good Thing.™

2
  • Yeah. And because it's so "trivial" to do proper range tests, each UT framework out there has a whole section dedicated to the topic. I'm not necessarily disagreeing with your core statement, but "nothing" is trivial with floating point :-P
    – Martin Ba
    Jun 9 '21 at 15:23
  • Does this change if I'm targeting a virtual machine which guarantees representation?
    – CtrlAltF2
    Jun 9 '21 at 16:31
1

I've done it once--deciding if a value had changed and needed to be saved. A false inequality simply meant one extra db write.

0

The obvious: You use == and != to check if two numbers are equal according to floating-point rules. This is not a bitwise comparison. NaN ≠ NaN, and +0 = -0.

If you add floating-point numbers to a hash-table you need to decide what to do with NaN. Naively using == will allow you to add NaN multiple times, without being able to look It up. And you need the same hash value for +0 and -0.

“Same sequence of operations gives same results” is a bit dubious. The C language allows operations to be performed with higher precision. You can allow fused multiply-add which gives results that are faster, more precise, and slightly different. Underflow can be treated in different ways. You can hope that some things give identical results, like 3.06 in a JSON document and in your source code. If the code doing the conversion is good but not identical, you’ll get the same result 99.9999999999% of the time, but not 100%.

Now if you make sure that your compiler behaves well, you will be fine. And of course you need to be aware that different calculations can give different results.

2
  • 1
    "... like 3.06 in a JSON document and in your source code. If the code doing the conversion is good but not identical, you’ll get the same result 99.9999999999% of the time, but not 100%. ..." -- this is the kind of statement that spreads confusion - there may well be reasons that two FP parsers come up with slightly different values, but there are equally as many reasons that two parsers (especially parsers) come up with the same value for 100% of all representable values, because if they implement the same algorithm, all decimals should end up with the same FP representation.
    – Martin Ba
    Jun 9 '21 at 9:30
  • 1
    @MartinBa There is a reason properly implementing std::to_chars() and std::from_chars() for floating-point in C++ is a thoroughly thorny problem. Jun 9 '21 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.