Unit test a generic floating point equality function

Question

I've written a function that tests two floating point numbers for approximate equality (see this Code Review question). I'd like to unit test my function, but I'm not positive of the best way to do this. Obviously I could pick some arbitrary numbers that should be equal within the threshold, but it seems a lot more useful to me to test that actual computations that should be equal but fail a naive equality test (due to rounding errors) are considered equal by my function.

Is that valid, or should I just pick my magic numbers and move along? Are there standard test cases/examples that people have historically used? I tried to find something, but all I found was a bunch of references explaining why I shouldn't use exact equality in floating point unit tests, which I already know.

As an example, I could write a test like this (using gtest):

template <typename FP>
class FloatEquality {
  protected:
    FP left, right, diff;
    std::size_t ulps;

    virtual void SetUp()
    {
        left = 2.0;
        right = 2.1;
        diff = .2;
    }
};

TYPED_TEST_CASE_P(FloatEquality);

TYPED_TEST_P(FloatEquality, MagicNumbers)
{
    EXPECT_TRUE(nearlyEqual(this->left, this->right, this->diff, this->ulps));
}

REGISTER_TYPED_TEST_CASE_P(FloatEquality, MagicNumbers);
using FloatingPointTypes= ::testing::Types<float, double>;
INSTANTIATE_TYPED_TEST_CASE_P(FloatingPoint, FloatEquality, FloatingPointTypes);

These numbers are obviously not a great choice, but they exemplify the types of magic numbers I could choose here that would be able to check all of my boxes and give me good code coverage, but don't seem that meaningful.

I did end up finding one example of someone unit testing this, but that is the magic number approach. The numbers appear to be reasonably well chosen, but it still feels like we aren't quite testing the right thing.

Answer 1

It's not going to hurt to build an actual computation that would fail. Division and multiplication can be used to reliably produce the kind of values you are looking for. However, once you've calculated interesting values to compare, what's the point of recalculating them each time? Do you want to unit test the calculations?

One thing to consider is that numbers like 0.2 cannot be represented exactly in floating point. Using something like that as your threshold value could potentially produce some unexpected results. Numbers such as 0.5, 0.25. 0.125 are exact in floating point. You might want to come up with unit tests check these situations where the threshold itself is a estimate.

Answer 2

The answer depends on what you consider the unit which is being tested in this case. The way you explain the situation looks like you consider to break your "equality checker" into smaller parts and test them separately (though you'll still face your "floating point dilemma" in one of the parts). In this case of breaking the unit into parts you'll need to test some parts of the whole calculation.

If you don't want to break the unit apart then is it ok use what you call "magic numbers". Anyway, one can never be sure unit tests cover all cases, there's always a shade of incompleteness. You just pick your "normal" and "corner-case" data and test against it.

The basic level computations should have been tested by a CPU manufacturer. There's no need to retest them.

Answer 3

Additionally, you should probably test some pathological cases like if a user sets the epsilon to a NaN or Inf. I could see someone using a calculation to generate their epsilon, and that could end up being denormal, so I'd make sure you deal with that case, as well. I think it's fine to use some well-constructed magic numbers. You can always make them named constants, and then they aren't magic numbers anymore!

Answer 4

What is making this hard to test is the lack of fidelity on the definition or contract of this floating point comparison operation.

You need to go back and beef up the definition of the function. Consider the expectations of consuming clients, and document that as the intended behavior.
Then you can test that it meets the criteria. As long as the definition of the comparison operation remains fuzzy you won't be able to properly test it.

You will probably find that in providing a proper specification for this operation, you will have to mention some magic numbers, like infinity and such. This should guide your testing endeavors.