11

I was told most modern computers follow the same floating point standard, does this mean they will all get the same float answer for a given math operation if the inputs are the same?

I ask because i am researching into making an RTS game on a network, and syncing hundreds of unit's positions sounds like a bad way to go.

So if i send just inputs only, i need to guarentee all clients get the same result by having them run the simulation from those inputs.

I read that older RTS games used fixed point arithmetic, but i don't know if that is still required on modern computers if they all adhere to the same standard? I was also told that although imprecise, the result of floating point is deterministic for the same input (which i presume means any computer following the same standard gets the same imprecise result?).

Do computers still have deviations even if they follow the same float point standard?

I am writing this game in C# not sure if that matters though, thought i'd mention it anyway.

3
  • Even if they did, I wouldn’t use floats for that
    – Telastyn
    Dec 10, 2018 at 4:51
  • What do you mean ? Why not?
    – WDUK
    Dec 10, 2018 at 4:54
  • Use of floats may be undesirable anyway because behaviour might depend on the position on the map. Minecraft's Far Lands were a more notable example: movement, rendering, and terrain generation would get glitchy as you moved far away from the spawn point.
    – amon
    Dec 10, 2018 at 10:30

5 Answers 5

20

Do computers still have deviations even if they follow the same float point standard?

Unfortunately, yes, especially when you use C# (or another JIT compiled language). The problem which occurs here is that the JIT compilation stage on some processor architectures produces code which uses more CPU registers than on other architectures. This can lead to situations where on some machines, extended floating point precision is used for certain operations, whilst on other machines not. This means for every iterative calculation using doubles, there is a chance to produce different accumulated rounding errors.

That is not a hypothetical problem, I have first-hand experience with such deviations in contemporary engineering simulation software, on more or less modern hardware. This issue makes it really hard to create reliable regression tests for complex floating point calculations which produce exactly the same result on all machines involved.

2
  • This. Some root causes: IEEE Std 754 includes optional "should" clauses (e.g. NaN handling) and permits design alternatives (e.g. underflow detection). In as far as language bindings support the floating-point standard, they may still give leeway to the compiler when evaluating floating-point expressions, e.g. FLT_EVAL_METHOD in ISO C/C++. Transcendental functions (e.g. sin, exp, log) are largely unregulated by both the IEEE floating-point standard and programming language standards. A simple library version upgrade (e.g. a new glibc version) could cause results to differ.
    – njuffa
    Dec 10, 2018 at 19:21
  • I've hit it myself in a game. The rocket flew fine on my laptop, would not fly on my desktop, completely identical installations. Dec 11, 2018 at 6:14
4

Floating Point Errors

Every floating point number accumulates imprecision as it is used for calculation. This is a simple fact of using an imprecise format to calculate in. The calculations are also sensitive to the order of calculation, commutativity is not guaranteed, ie: (a + b) + c may or may not be the same as a + (b + c).

Additionally processors do not necessarily have the same mantissa length as the memory standard. This can generate interesting behavior as the 32/64/128 bit float occasionally operate as if they have more bits.

Fixed-Point Errors

That being said fixed-point arithmetic can also accumulate errors. The difference is that fixed point numbers are clear about what precision is lost, and depending on the chosen operations can avoid rounding errors altogether. They also are commutative (a + b) + c = a + (b + c).

Which?

Which one to use depends entirely on what properties you need.

Floating point numbers:

  • give a vast range of values that become very fine grained close up, and progressively further apart at the extremes.
  • are sensitive to order of calculation
  • accumulate rounding errors over time.
  • can have erratic behaviour due to hardware/memory float size mismatch.

Fixed Point numbers:

  • give a smaller range of numbers with the same distance between any two consecutive numbers.
  • are less sensitive to the order of calculation
  • are clearer about rounding errors
  • can be worked with to minimise/avoid rounding issues.
6
  • 1
    "fixed point numbers are clear about what precision is lost" - floating points are clear too, the difference is rather fixed point inaccuracies are more intuitive to ordinary life numbering Dec 10, 2018 at 5:44
  • 1
    So only fixed point guarantees all computers regardless of hardware etc will experience the same errors/precision loss?
    – WDUK
    Dec 10, 2018 at 5:47
  • 1
    Essentially, yes, because you can specify that your fixed point numbers are 32 or 64 bits, and they will be on all systems. Floating point numbers may be 32 or 64-bit, but the hardware may actually use 48 or 96 bits to do the calculation and convert to 32 or 64 bits at the end resulting in differences between different types of hardware. Dec 10, 2018 at 5:53
  • @whatsisname While the floating point specifications are quite clear, you cannot easily tell me what rounding issues I will encounter in this sum: (a + b * c) / d - e. Excepting obvious issues like NaN, division by zero, or overflow/underflow it is possible for this expression to be incorrect. Add to that the impedence between memory and register in terms of precision and even a simple load/store from memory of the "same" floating point value will change the answer.
    – Kain0_0
    Dec 10, 2018 at 7:11
  • @Kain0_0: you're right, I can't easily tell you what I'll encounter, because I am not a floating point expert. That is exactly what is meant when I said "more intuitive to ordinary life numbering". When you say fixed point is "clear" and floating point is not, you make it sound as though floats are just seemingly randomly inaccurate. Dec 10, 2018 at 22:04
1

The answers given are light on relevant specifics (sadly a common problem with C# discussion). The problem isn't a trivial one, and much is written about it (it's often called "floating point determinism", a name which inevitably leads to people pointlessly arguing about what "determinism" means). Long story short it seems that a+b or a*b will always give the same answer, and it seems like the same executable will give the same results too (there might be caveats), the biggest problem seems to be what the compiler will do, namely arranging operations differently or even using different operations, which is going to be a problem if your game runs on different platforms. A way around this (for compiled languages) might be to use compiler options for a strict IEEE-754 compliance, which should avoid a reshuffling of operations that would ruin consistency across platforms, but another problem, again only when comparing different builds on different platforms, is that trigonometrical functions like say atan2() might give slightly different results due to different implementations, as in the code isn't the same at all so it might produce minute differences.

So if all your players are on Windows x86 64 bits running the same binary build (not really an option with C# because of JIT), you probably have nothing to worry about. Otherwise you might have problems, in which case you might want to check IEEE-754 compilation options, implement your own trigonometric functions or even do it all in fixed point, although even though the latter is a good approach it might not be necessary. This being said an advantage of using fixed point arithmetic for an RTS is that precision is consistent across the map, instead of having overkill precision around the origin and less and less precision towards the edges. But fixed point arithmetic isn't an easy drop-in replacement, you must carefully consider the format as well as what happens with each operation to maintain precision yet avoid overflows. There isn't one answer, Warcraft III uses fixed point arithmetic, Supreme Commander uses floating point arithmetic, but the use of C# should nudge one towards fixed point.

Some relevant discussion that goes more into details:

https://stackoverflow.com/questions/20963419/cross-platform-floating-point-consistency

https://stackoverflow.com/questions/57931671/portable-consistent-floats

https://www.yosoygames.com.ar/wp/2013/07/on-floating-point-determinism/

https://gafferongames.com/post/floating_point_determinism/

https://gamedev.stackexchange.com/questions/14776/how-are-deterministic-games-possible-in-the-face-of-floating-point-non-determini

1

You still need to be careful.

Conversion between floating point and decimal may give different results in very rare cases (depending on the quality of the compiler and standard library).

Standard library functions like sin, cos, exp may give slightly different results.

long double is quite likely to give different results, because it may be 64, 80, or 128 bits on current compilers.

Behaviour of denormalised numbers may be different.

Single precision operations may be executed in double precision, producing different results.

Fused multiply-add will give better, but different results.

Setting floating-point rounding mode and precision may or may not be respected, producing different results.

There are some macros that you can check to find out about compiler behaviour, for example if it will contract fp operations or perform them with higher precision than the source code says.

And practically every compiler has different compiler options to force the behaviour you want.

-2

There’s the question why you would want to guarantee identical results, since identical results give no guarantee at all that your results are useful.

You could have a numerically unstable algorithm that gives two identical but completely nonsensical results in different computers. If there are differences, but results are the same within 13 digits, that is much more trustworthy.

There are very few situations where reproducibility is really important: in a layout engine, or lossless compression/decompression. Using fixed point is very likely to be misguided.

1
  • I did not downvote your answer, but it seems the case described by the OP is exactly "one of those few situations where reproducibility is really important". In an RTS game, a small rounding error can make the difference between "two objects collided" or not.
    – Doc Brown
    Dec 10, 2018 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.