2

Using this calculator, I can see how the decimal ".8" and the expression ".7 + .1" have different representations:

.8      = 1.1001100110011001100110011001100110011001100110011010 *2-1
.7 + .1 = 1.1001100110011001100110011001100110011001100110011001 *2-1
                                                              ^^  

But what mechanism causes "0.8" to be printed for the top value? E.g. alert(.8) in JavaScript. Why does it not print something like "0.800000011920929"?

Is this a feature of IEEE 754 or the programming language implementing it?

  • 2
    What would you expect to be printed? – user7043 Aug 7 '14 at 17:23
  • 3
  • @BartvanIngenSchenau You think? OP seems to be fully aware of the inaccuracies of floating point. – user7043 Aug 7 '14 at 18:00
  • @delnan it is a good document. While the asker might understand FP arithmetic, people coming to this question from google might not. I think it helps the big picture. – user22815 Aug 7 '14 at 18:06
  • 2
    @Snowman Eh, I don't think it's a good resource for most people. Two thirds of it proves theorems and discusses nitty gritty details that are very useful for numeric analysis but don't affect most people and are completely impenetrable when one doesn't already have a good grip on the representations and its weaknesses. The remaining third has also been stated with equal or greater clarity in many other places. – user7043 Aug 7 '14 at 18:11
6

The paper How to Print Floating-Point Numbers Accurately by Guy L. Steele Jr. and Jon L White describes one approach to the problem of printing numbers.

Quoting from that paper:

What is the correct number of digits to produce if the user doesn’t specify? If a system prints too many digits, the excess digits may be “garbage,” reflecting more information than the number actually contains; if a system prints too few digits, the result will be wrong in a stronger sense: converting the decimal representation back to binary may not recover the original binary value.

Which echos delnan's comment: "What would you expect to be printed?"

As far as I can tell, how IEEE 754 numbers are printed is a feature of the programming language.

The Steele and White paper explain some techniques for printing floating point numbers accurately, and they, or something similar, may be implemented in JavaScript.

  • 1
    I agree, my understanding is that IEEE 754 governs the binary representation and how math works. Converting to other formats, rounding in the conversion process, etc. is implementation-defined. – user22815 Aug 7 '14 at 18:10
  • OK so looks implementation specific. Natural follow-up: Is there a short decimal which is "lossy" in a REPL of any well-known language? E.g. enter > .123 and get back something like .122999999352. Can this ever occur? – Steve Clay Aug 7 '14 at 18:45
  • 1
    @Steve Clay - I futzed around in the '80s with an early version of Xlisp by David Betz. It was very simple to modify, written very cleanly in C. It's the language that was later incorporated into AUTOCAD. IIRC, when I typed 0.3 into the REPL it'd echo the gory approximation on my PC. I can't remember if it used the underlying compilers library, or its own print function. – gbulmer Aug 7 '14 at 18:56
3

Most implementations round floating point numbers by default to 6 significant digits (removing any trailing zeros) when converting them to strings, as that seems to match the expectations of most users.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.