I am trying to make an effective modf() function that uses floats instead of doubles. An embedded application I am working on only supports the float type.

My modf() function works as expected with the exception of float rounding errors.

Consider the float number 12345.67. The current version of my modf() function simply subtracts the integer part from the input to get the fraction part. What I expected was 12345.67 - 12345. = 0.67 , but it doesn't work right.

The original mantissa for 12345.67 is this:
100 0000 1110 01.10 1010 1110 <- 0x40E6AE decimal point = 13

Separated into integer and fraction components is this:
100 0000 1110 01.00 0000 0000 <-- Integer 0x40E600
000 0000 0000 00.10 1010 1110 <-- Fraction

My modf() function normalizes the fraction to:
010 1011 1000 0000 0000 0000 <-- 0x2B8000

Because of the influx of 13 zeros from the right, the number is not 0.67 as expected, but instead, something like 0.6600178.

I need to fix the rounding error so that it is closer to 0.67. The binary version of 0.67 is this:
010 1011 1000 0101 0001 1111 <-- 0x2B851F
Notice that the last 3 nibbles have data now. The first half is the same.

If I change the 12th bit from the right to a 1, I get something like 0.67002
010 1011 1000 1000 0000 0000 <-- 0x2B8800

This is good enough for my needs, but if I did that when the input float was actually 0.67 to start with, I would have the wrong value.

So I guess the question is What is the proper way to apply bias to a float in this situation?

  • Several sources on the Internet say that if you cast the original float to a long, and then subtract that from the original float, you will get the correct fractional part. Otherwise, have a look here: web.archive.org/web/20121030234640/http://… – Robert Harvey Mar 4 '17 at 19:00
  • These numbers look very, very dubious to me. Especially the claim that changing a single bit would change a floating-point value by 0.01. One bit could be 1/64 or 1/128, but not 0.01. I'd advice you to check your numbers. – gnasher729 Mar 4 '17 at 21:09
  • Yes, in its simplest form, the routine is essentially: – user264480 Mar 6 '17 at 4:44
  • @Robert Harvey Hit return too quick. Yes, in its simplest form, the routine is essentially: <br>IntPart=(float)((unsigned) InFloat); <br>Frac=InFloat - IntPart;<br>This works for small integers, but there is extra code to handle large integers such as 1E20 for example. – user264480 Mar 6 '17 at 4:55
  • @gnasher729 By toggling the third new zero to a 1 when more than half the mantissa is shifted away seems to be working. I've had a lot more problems with rounding because floats are so small. Would still like to hear how others are handling rounding like this. – user264480 Mar 6 '17 at 4:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.