While investigating the accuracy of floating point numbers, I've seen in a few places a statement similar to

"float and double are (designed for/used often in) engineering and scientific calculation"

From my understanding, the strength of floats and doubles are the amount of memory they use for their (good, but not perfect) precision.

I feel like I'm almost getting an understanding from this answer

"floating point numbers let you model continuous quantities"

I still am not convinced I understand. Engineering and Science both sound like fields where you would want precise results from your calculations, which, from my understanding, floating points do not give. I'm also not sure I follow what a "continuous quantity" is, exactly.

Can somebody expand on this explanation and perhaps give an example?

  • 2
    See floating-point-gui.de Oct 21, 2014 at 18:42
  • 47
    Engineering and Science both sound like fields where you would want precise results from your calculations, which, from my understanding, floating points do not give. In both Science and Engineering you only care about precision up to a certain point. Using infinite precision for every calculation is often unnecessarily expensive. What sets floating point apart from fixed point is that you don't have to commit to a certain number of decimal places - you can have really small quantities with a lot of decimal places or really large quantities with limited precision.
    – Doval
    Oct 21, 2014 at 18:50
  • 24
    To add to the points made above, not only do you not care about precision beyond a certain point, you can't get arbitrarily precise results because many of your inputs are measured quantities that have some inherent error.
    – user7043
    Oct 21, 2014 at 19:24
  • 2
    It's also wroth pointing out that it's not a given that the rounding errors will continue to accumulate either. It depends on what you're doing and how you're doing it; there's an entire field dedicated to that.
    – Doval
    Oct 21, 2014 at 19:52
  • 10
    Floating point isn't "random precision", the errors for various operations are predictable and well-known, and the errors for an algorithm can be worked out. If they're low enough (and in particular if your backwards errors are smaller than the uncertanties in your input variables) then you can be certain that your results are good (or at least that any problems with them aren't caused by floating-point error).
    – hobbs
    Oct 21, 2014 at 22:13

9 Answers 9


Computation in science and engineering requires tradeoffs in precision, range, and speed. Fixed point arithmetic provides precision, and decent speed, but it sacrifices range. BigNum, arbitrary precision libraries, win on range and precision, but lose on speed.

The crux of the matter is that most scientific and engineering calculations need high speed, and huge range, but have relatively modest needs for precision. The most well determined physical constant is only known to about 13 digits, and many values are known with far less certainty. Having more than 13 digits of precision on the computer isn't going to help that. The fly in the ointment is that sequences of floating point operations can gradually lose precision. The bread and butter of numerical analysis is figuring out which problems are particularly susceptible to this, and figuring out clever ways of rearranging the sequence of operations to reduce the problem.

An exception to this is number theory in mathematics which needs to perform arithmetic operations on numbers with millions of digits but with absolute precision. Numerical number theorists often use BigNum libraries, and they put up with their calculations taking a long time.

  • 3
    Good answer. While the underlying functions may be perfectly continuous, which would require perfect precision to exactly model, the reality is that everything in science and engineering is an approximation. We'd rather have decent, useful approximations and accomplish something than infinite precision, for which we'd wait forever for many operations to complete. Oct 21, 2014 at 19:55
  • 4
    @JonathanEunice You cannot exactly model the reality. The input for the model comes from measurements and you will probably never able to measure things so precisely that a native real number in modern computer/software (at the time) would limit it. In other words, you can have perfect model, software or mathematical, it doesn't matter. E.g. Calculate a volume of a box. a*b*c easy stuff, however you need to measure the dimensions which you cannot do with absolute certainty, thus you don't really need infinite precision of calculation anyways, just enough to be bound by measurement error.
    – luk32
    Oct 21, 2014 at 22:23
  • 2
    @luk32 We violently agree about most of those points. One can model some thing exactly (volume of a sphere, e.g.), but can never measure exactly. And reality never perfectly fits a perfect model. Better to get slightly imprecise, useful values/models than wait for perfect measurements or computations--something that will always be one step away. Oct 21, 2014 at 23:32
  • 2
    “The crux of the matter is that most scientific and engineering calculations need high speed, and huge range” If I give you long time, you still cannot compute exactly because algorithms to compute exactly are widely unknown. And first of all, we cannot even represent numbers exactly. This is just a problem we do not know how to solve, neither quickly nor slowly. Oct 22, 2014 at 8:47
  • @MichaelGrünewald, we can't represent real numbers exactly, but we are able to solve problems to a close enough approximation that we can build structures a couple thousand feet high, identify genes in DNA, and rendezvous a satellite with a comet after two years in space. To paraphrase Randy Newman, that may not be exact, but it is all right. In fact we can represent rationals exactly using arbitrary precision libraries (subject to limitations of memory). Oct 22, 2014 at 16:07

What alternative do you propose?

Continuous quantities are represented using real numbers in mathematics. There is no data type that can encode every possible real number (because reals are uncountable), so that means we can only pick a subset of those real numbers that we're most interested in.

  • You can pick all computable reals, which is similar to what computer algebra systems (CAS) do. The problem is that it becomes rapidly unfeasible as your expression tree grows larger and larger. It's also very slow: try solving a huge system of differential equations in Mathematica symbolically and compare against some other floating-point based implementation and you'll see a dramatic difference in speed. Additionally, as Jörg W Mittag and kasperd have pointed out: you don't even have decidable equality/comparison operations.

  • You could use exact rational numbers, but that doesn't really work for many applications because you need to calculate square roots or cosines or logarithms etc. Furthermore, there is also a tendency for rationals to become increasingly complex and thus requiring more space to store and time to process as you perform more and more calculations on them.

  • You could also use arbitrary-precision decimals, but then even something as simple as division won't work because you get infinitely repeating digits. You can also run into the issue of increasing complexity as you perform more similar to rational numbers, though to a lesser extent.

So you'd be forced to use approximations at some point, in which case that's exactly where floating-point numbers do best. Floating-point numbers are also of fixed width (unlike all the other 3 data types mentioned earlier), which prevents the complexity increase as you perform more and more calculations on them.

  • 1
    One of the best answers, I overlooked it before writing mine. Oct 22, 2014 at 8:56
  • 8
    Plus, there is the slightly inconvenient fact that you cannot even tell if two computable reals are equal. Oct 22, 2014 at 9:32
  • 1
    Wouldn't using all computable reals run into a problem with comparisons? I'm pretty sure you can't compare arbitrary computable reals without solving the halting problem.
    – kasperd
    Oct 22, 2014 at 16:45
  • @kasperd: I think that would depend in some measure upon what operations one was allowed to use in the computation, though I'm not sure how rich a set of computation types one could have and still guarantee that any two arbitrary results that could be produced in a finite number of operations could be compared in bounded time. Algebraic types would almost certainly meet that criterion, but I don't know if ln(x) and exp(x) functions could be added and still meet it.
    – supercat
    Oct 22, 2014 at 21:48
  • You can support arbitrary precision arithmetic (add, multiply, subtract, divide), irrationals (like √2), well known transcendentals (like Pi and e), trig functions, etc. using continued fractions. See Gosper's algorithm in HAKMEM. When finished, you can perform lazy evaluation to get a floating point approximation to the desired precision. Apr 24, 2015 at 17:52

Your proposition about science is wrong, Engineering and Science other then Math don't work with exact precise results. They work with a precision factor which is built into how many digits you show.

The key term you need to understand here is: significant figures. The significant figures of a number are those digits that carry meaning contributing to its precision.

Which basically means if I state that something is 12 centimeters longs, it can actually be somewhere between 11,5 and 12,5 centimeters long. If however I state that something is 12,00 centimeters long it can be somewhere between 11,995 and 12,005 centimeters long.

Just as an illustration, if you take a measurement tape and measure your living room. Even though you may find that it 6 meters 25 centimeters wide, you know that your tape measurement wasn't accurate enough to tell anything about the millimeter-accuracy or nano-meter-accuracy.

  • @leftaroundabout what do you mean math (as in mathematics) isn't science? In my book it is.
    – Pieter B
    Oct 22, 2014 at 10:50
  • 2
    @PieterB: Maths isn't Science. It's Philosophy. Science it the act of forming an understanding of our physical world. Philosophy is the act of understanding how ideas work in an ideal world.
    – slebetman
    Oct 22, 2014 at 12:07
  • I think science usually prefer to work with explicit confidence ranges rather than significant figure.
    – Taemyr
    Oct 22, 2014 at 12:10
  • @slebetman Besides that it has nothing to do with my point in my post, if math is a science or not I can't help to drop a quote : Nature is innately mathematical, and she speaks to us in mathematics. We only have to listen. Because nature is mathematical, any science that intends to describe nature is completely dependent on mathematics. It is impossible to overemphasize this point, and it is why Carl Friedrich Gauss called mathematics "the queen of the sciences."
    – Pieter B
    Oct 22, 2014 at 12:17
  • That quote being from here. A good read and plenty to discuss about, but not here since indeed it has nothing to do with your post or this question. Oct 22, 2014 at 14:32

Note that floating point numbers are basically the same as scientific and engineering notation, the standard way for humans to write numbers in math and science. In these fields, there's not a great need for extreme precision, but there's often a huge range.

To pick a random example from my physics homework, I recently had to work with the mass of an electron, which is roughly 9.11 * 10^-31 kg. I don't care a lot about the precision; it could easily be 9.12 for all I care. But I care about the exponent and don't want to have to write out 0.0000...911 kg, so I use scientific notation.

Similar reasoning applies in scientific and engineering computing: there's a huge range, but we don't want to have to store and work with very large numbers, so we store a normalized value and an exponent, which is smaller and faster to work with.


Floating-point numbers also have several properties that lend themselves well to computing certain types of scientific results. Most notably, precision is inversely proportional to magnitude, just like in scientific notation, so you can represent both small differences close to zero and larger differences much farther away.

Goldberg's paper is probably the most famous analysis of the properties of floating-point numbers (and should be required reading if you care about this sort of thing), but Kahan's papers I think do a better job of explaining the rationale behind many of the subtle design issues.

In particular, Kahan's diatribe about Java's implementation of floating point, while quite inflammatory, makes several good points about why IEEE-754 semantics are useful, and Much Ado About Nothing's Sign Bit explores the rationale for signed zero in considerable depth.

  • I've not read Kahan's entire paper yet, but he seems more polite than I would be. Java could have had numerics which were more useful and performed faster than what it actually has if it had added a real type which would take three stack entries to store, and would represent the machine's natural computational precision; the value could be stored as an 80-bit float + 16 bits padding a 64-bit float + 32 bits padding, or 64 bit mantissa, 16 bit exponent, and 16 bits for sign and flags [for non-FPU implementations].
    – supercat
    Oct 21, 2014 at 23:17
  • Specify that float and double are storage formats, and real is the computational format. In many systems with no FPU, working with a mantissa, exponent, and flags that are on word and half-word boundaries would be faster than having to unpack and repack doubles with every operation.
    – supercat
    Oct 21, 2014 at 23:19

TL;DR We do not know how to compute most functions with perfect precision, there is therefore no point representing numbers with perfect precision.

All of the answers so far miss the most important point: we cannot compute exact values of most numbers. As an important special case, we cannot compute exact values of the exponential function — to cite only the most important irrational function.

Naive answer to the naive question

It seems your question is rather “there is exact arithmetic libraries, why don't we use them in place of floating point arithmetic?” The answer is that exact arithmetic works on rational numbers and that:

  • Archimede's number — the pedantic name of π — is not rational.
  • Many other important constants are not rational.
  • Many other important constants are not even known to be rational or not.
  • For any non-zero rational number x the number exp(x) is irrational.
  • Similar statements hold for radicals, logarithms, and a wealth of functions important to scientists (Gauss's distribution, its CDF, Bessel functions, Euler functions, …).

The rational number is a lucky accident. Most numbers are not rational (see Baire's theorem) so computing on numbers will always bring us out of the rational world.

What is computing and representing a number?

We may react by saying “OK, the problem is that rational numbers were not such a great choice to represent real numbers.” Then we roll up our sleaves fork Debian and devise a new representation system for real numbers.

If we want to compute numbers we have to pick a representation system for real numbers and describe important operations on them — i.e. define what computing means. Since we are interested in scientific computing, we want to represent accurately all decimal numbers (our measures), their quotients (rational numbers), values of the exponential functions and some funny constants, like Archimede's number.

The problem is that the only way to perfectly represent numbers in such a system is to use symbolic form, that is, not to compute anything at all and work with algebraic expressions. This is a rather crippled representation of real numbers, because we cannot reliably compare two numbers (which one is greater)? We cannot even easily answer the question “Is the given number equal to 0?”.

If you look for more precise mathematical definitions and problems, look for rational numbers, transcendental numbers, best approximations, and Baire's theorem, for instance.

  • I think this is a great answer, just not to this question, insofar I am not confident that the asker will understand the points you are making. That and you are being quite glib with the inexact representation of \Real or \Complex numbers by a finite digital representation (regardless of dynamic or static bit width). That is all entirely true, but beside the point. Kudos for not robotic-ally citing Goldberg. :) And Baire's theorem is not part of the usual rhetoric found on Programmers or StackOverflow.
    – mctylr
    Oct 22, 2014 at 21:49


1) The authors make the assumptions that "engineering and scientific calculation" measure real-world physical quantities

2) Physical quantities are continuous, and exactly as you state "floating point numbers let you model continuous quantities"

.. and the rest of my answer is summed up nicely by Rufflewind, so I'm not going to repeat that here.


Floating point numbers provide relative accuracy: they can represent numbers that are at most a small percentage (if you want to call something like 0.0000000000001% a percentage) away from any accurate number over a wide range of numbers. They share this trait with a slide rule, though the latter does not get better than something like 3 digits of accuracy. Still, it was quite sufficient for working out the static and dynamic forces of large structures before digital computers became commonplace for that, and that's because the material constants also show some variation, and picking constructs that are reasonably benign against material and construction differences will tend to make the maximum loads and weak points reasonably identifiable.

Now "accuracy" is a useful feature for many numbers representing measurements and/or magnitudes of physical properties.

Not everything in science/engineering belongs in that category. For example, if you are using number theoretic transforms for multiplying large numbers or Galois fields for manipulating error correction polynomials, there is no such thing as a small error: any single bit error during processing will lead to results that are quite indistinguishable from completely random noise.

Even in those areas one can work with floating point numbers (like using complex FFTs for doing convolution) if one keeps track of the accumulation of errors and makes sure that the floating point errors do not accumulate enough magnitude to possibly even flip a single bit in the actual entities that they are an approximations of. For such approximations, fixed point processing would likely be more appropriate but floating point units in the field tend to provide faster operation and a larger number of usable bits.

Also programming languages like C or Fortran make it surprisingly hard to access basic operations like mixed precision multiplication and division or a carry bit for addition/subtraction, and those are basic building blocks for going beyond limited precision integers.

So if you can map operations to floating point numbers, you tend to have reasonably powerful hardware at your disposal these days and you can reasonably well specify your algorithms in one of today's general purpose programming languages.


I think this can be answered by addressing what application float/double data types are not suitable for.

When you need to make sure that you can represent a number accurately with a specific number of digits, then floating point numbers are inappropriate, because they represent the numbers as powers of 2, instead of powers of 10, as is how we represent numbers in the real world.

So one domain where floating point data types should not be used is that of finance*. For the core system of e.g. a bank, it would be completely unacceptable if an amount that should have been $100000.01 suddenly becomes $100000.00 or $100000.02.

Such a problem could easily occur when using floats, especially if the number was the result of one or more calculations, e.g. calculating the sum of all transactions in an account.

Engineering and scientific calculation are domains where these relatively small rounding errors are acceptable. Users are normally aware that all numbers have a limited precision, and they often work with a number of significant digits. But most importantly they have a well defined relative precision, i.e. they have provide the same number of significant digits, both for very large numbers, and for very small numbers.

* I did once work on a financial application where floats had been used to represent values, and as a consequence, rounding errors were introduced. Fortunately, this specific bug wasn't critical at all, the users did complain about calculation errors in the program. And this led to a different, far worse, effect: the users started loosing faith in the system.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.