Floating point types (such as Single and Double) are represented in memory by a sign, a mantissa and an exponent. Think of it as scientific notation:
They - as you may expect - use base 2. There are other tweaks that allow for representing infinity and NaN, and the exponent is offset (will come back to that), and a shorthand for the mantissa (will come back to that too). Look for the standard IEEE 754 which covers its representation and operations for more details.
For our purposes we can imagine it as a binary number "mantissa", and an "exponent" that tells you where to put the decimal separator.
In the case of Single, we have 1 bit for he sign, 8 for the exponent and 23 for the mantissa.
Now, the thing is, we will store the mantissa from the most significant digit. Remember that all zeroes to the left are not relevant. And giving that we are working in binary, we know that the most significant digit is a 1※. Well, since we know that, we do not have to store it. Thanks to that shorthand, the effective range of the mantissa is 24 bits.
※: Unless the number we are storing is zero. For that we will have all the bits set to zero. However, if we try to interpret that under the description I gave, you would have a 2^24 (the implicit 1) multiplied by 1 (2 to the power of the exponent 0). So, to fix it, exponent zero is a special value. There are also special values to store infinity and NaN in the exponent.
As per the exponent offset - aside from avoiding the special values - having it offset allows to place the decimal point before the start of the mantissa or after its end, without the need to have a sign for the exponent.
This means that for large numbers, the floating point type will put the decimal point beyond the end of the mantissa.
Remember that the mantissa is a 24 bit number. It will never represent a 25 bit number... it does not have that extra bit. Thus, the single cannot distinguish between 2^24 and 2^24+1 (these are the first 25 bit numbers, and they differ on the last bit, which is not represented in the single).
Thus, for integers the range of the single is -2^24 to 2^24. And trying to add 1 to 2^24 will result in 2^24 (because as far as the type is concerned, 2^24 and 2^24+1 are the same value). Try it Online. This is why there is a loss of information when converting from integer to single. And this is also why a loop that uses a single or a double could actually be an infinite loop without you noticing.