27

I was reading an article from Microsoft regarding Widening Conversions and Option Strict On when I got to the part

The following conversions may lose precision:

  • Integer to Single
  • Long to Single or Double
  • Decimal to Single or Double

However, these conversions do not lose information or magnitude.

.. but according to another article regarding data types,

  • Integer type can store from -2.147.483.648 to 2.147.483.647 and

  • Single type can store from

    • 1,401298E-45 to 3,4028235E+38 for positive numbers,
    • and -3,4028235E+38 to - 1,401298E-45 for negative numbers

.. so Single can store much more numbers than Integer. I couldn't understand in what situation such conversion from Integer to Single may lose precision. Could someone explain, please?

87

Single can store much more numbers than Integer

No, it can't. Both Single and Integer are 32 Bit, which means that both can store the exact same amount of numbers, namely 232 = 4294967296 distinct numbers.

Since the range of Single is clearly larger than that, it is immediately obvious (because of the Pigeonhole Principle) that it cannot possibly represent all numbers within that range.

And since the range of Integer is exactly the same size as the maximum amount of numbers that both Integer and Single can represent, but Single can also represent numbers outside of that range, it is clear that it cannot possibly represent all numbers inside the range of Integer.

If there are some numbers of Integer that cannot be represented in Single, converting from Integer to Single must be capable of losing information.

  • 3
    +1 for this great explanation of why that has to be the case, even though the question was actually when ("in what situation") it happens... – doubleYou Jun 2 at 11:25
  • 21
    @doubleYou: 4261412864 of the 4294967296 Integers (99.2%) cannot be represented as Single, so "when" is "pretty much always". – Jörg W Mittag Jun 2 at 17:50
  • 2
    If you want to be more precise, Single can only represent 4,278,190,079 different numbers. A Single value represents a number if and only if the stored exponent is not 255, which means that there are 255 * 2^24 Singles which represent numbers. Out of these, two of them represent the same number (namely, zero), and the others all represent different numbers. – Tanner Swett Jun 2 at 20:23
  • 10
    en.wikipedia.org/wiki/Single-precision_floating-point_format explains the limitations nicely for IEEE754 binary32. Integers in [-16777216,16777216] (2^24 = the significand width) can be exactly represented. Larger numbers are rounded to the nearest multiple of 2, 4, 8, ... depending on how large they are. – Peter Cordes Jun 3 at 5:04
  • 14
    “which means that both can store the exact same amount of numbers” — It does not mean that. It would only mean that if both types have the exact same number of ways of storing each number. And this isn’t the case; for instance, Single has two ways of storing zero. So Single can in fact represent fewer distinct numbers than Integer. – Konrad Rudolph Jun 3 at 15:01
28

Floating point types (such as Single and Double) are represented in memory by a sign, a mantissa and an exponent. Think of it as scientific notation:

Sign*Mantissa*Base^Exponent

They - as you may expect - use base 2. There are other tweaks that allow for representing infinity and NaN, and the exponent is offset (will come back to that), and a shorthand for the mantissa (will come back to that too). Look for the standard IEEE 754 which covers its representation and operations for more details.

For our purposes we can imagine it as a binary number "mantissa", and an "exponent" that tells you where to put the decimal separator.


In the case of Single, we have 1 bit for he sign, 8 for the exponent and 23 for the mantissa.

Now, the thing is, we will store the mantissa from the most significant digit. Remember that all zeroes to the left are not relevant. And giving that we are working in binary, we know that the most significant digit is a 1※. Well, since we know that, we do not have to store it. Thanks to that shorthand, the effective range of the mantissa is 24 bits.

※: Unless the number we are storing is zero. For that we will have all the bits set to zero. However, if we try to interpret that under the description I gave, you would have a 2^24 (the implicit 1) multiplied by 1 (2 to the power of the exponent 0). So, to fix it, exponent zero is a special value. There are also special values to store infinity and NaN in the exponent.

As per the exponent offset - aside from avoiding the special values - having it offset allows to place the decimal point before the start of the mantissa or after its end, without the need to have a sign for the exponent.


This means that for large numbers, the floating point type will put the decimal point beyond the end of the mantissa.

Remember that the mantissa is a 24 bit number. It will never represent a 25 bit number... it does not have that extra bit. Thus, the single cannot distinguish between 2^24 and 2^24+1 (these are the first 25 bit numbers, and they differ on the last bit, which is not represented in the single).

Thus, for integers the range of the single is -2^24 to 2^24. And trying to add 1 to 2^24 will result in 2^24 (because as far as the type is concerned, 2^24 and 2^24+1 are the same value). Try it Online. This is why there is a loss of information when converting from integer to single. And this is also why a loop that uses a single or a double could actually be an infinite loop without you noticing.

  • This isn't a perfect explanation of the implicit leading 1 bit in the significand. It's implied by the biased-exponent field being non-zero. Subnormals (aka denormals) including +-0.0 have a leading 0 bit of their significand. I guess you could simplify to just consider 0.0 a totally special case, but 0.0 actually follows the same encoding rules as other subnormals. – Peter Cordes Jun 3 at 5:11
25

Here is an actual example of when converting from Integer to Single may lose precision:

The Single type can store all integers from -16777216 to 16777216 (inclusive), but it cannot store all integers outside of this range. For example, it cannot store the number 16777217. For that matter, it cannot store any odd number greater than 16777216.

We can use Windows PowerShell to see what happens if we convert an Integer to a Single and back:

PS C:\Users\tanne> [int][float]16777213
16777213
PS C:\Users\tanne> [int][float]16777214
16777214
PS C:\Users\tanne> [int][float]16777215
16777215
PS C:\Users\tanne> [int][float]16777216
16777216
PS C:\Users\tanne> [int][float]16777217
16777216
PS C:\Users\tanne> [int][float]16777218
16777218
PS C:\Users\tanne> [int][float]16777219
16777220

Notice that 16777217 got rounded down to 16777216, and 16777219 got rounded up to 16777220.

12

Floating point types are similar to "scientific notation" in physics. The number is split up into a sign bit, an exponent (multiplier) and a mantissa (significant digits). So as the magnitude of the value increases the step size also increases.

Single precision floating point has 23 mantissa bits, but there is an "implicit 1", so the mantissa is effectively 24 bits. Therefore all integers with a magnitude up to 224 can be represented exactly in single precision floating point.

Above that successively fewer numbers can be represented.

  • From 224 to 225 only even numbers can be represented.
  • From 225 to 226 only multiples of 4 can be represented.
  • From 226 to 227 only multiples of 8 can be represented.
  • From 227 to 228 only multiples of 16 can be represented
  • From 228 to 229 only multiples of 32 can be represented
  • From 229 to 230 only multiples of 64 can be represented
  • From 230 to 231 only multiples of 128 can be represented

So of the 232 possible 32 bit signed integer values only 2 * (224 + 7 * 223) = 9 * 224 can be represented in single precision floating point. That is 3.515625% of the total.

8

Single precision floats have 24 bits of precision. Anything over that is rounded to the nearest 24-bit number. It might be easier to understand in decimal scientific notation, but keep in mind actual floats use binary.

Say you have 5 decimal digits of memory. You can choose to use those like a regular unsigned int, allowing you to have any number between 0 and 99999. If you want to be able to represent larger numbers, you can use scientific notation and just allocate two digits to be the exponent, so you can now represent anything between 0 and 9.99 x 1099.

However, the biggest number you can represent exactly is now only 999. If you tried to represent 12345, you can get 1.23 x 104, or 1.24 x 104, but you can't represent any of the numbers in between, because you don't have enough digits available.

  • 3
    Using decimal digits is a nice idea that makes it easier to understand, but the last paragraph is a bit misleading: actually you can represent numbers higher than 999, and your example shows it: 12300 would be 1.23 x 10<sup>4<sup>. What you mean is that starting from that number there are gaps. Would you mind rephrasing it a bit? – Fabio Turati Jun 3 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.