TL;DR We do not know how to compute most functions with perfect precision, there is therefore no point representing numbers with perfect precision.
All of the answers so far miss the most important point: we cannot compute exact values of most numbers. As an important special case, we cannot compute exact values of the exponential function — to cite only the most important irrational function.
Naive answer to the naive question
It seems your question is rather “there is exact arithmetic libraries, why don't we use them in place of floating point arithmetic?” The answer is that exact arithmetic works on rational numbers and that:
- Archimede's number — the pedantic name of π — is not rational.
- Many other important constants are not rational.
- Many other important constants are not even known to be rational or not.
- For any non-zero rational number x the number exp(x) is irrational.
- Similar statements hold for radicals, logarithms, and a wealth of functions important to scientists (Gauss's distribution, its CDF, Bessel functions, Euler functions, …).
The rational number is a lucky accident. Most numbers are not rational (see Baire's theorem) so computing on numbers will always bring us out of the rational world.
What is computing and representing a number?
We may react by saying “OK, the problem is that rational numbers were not such a great choice to represent real numbers.” Then we roll up our sleaves fork Debian and devise a new representation system for real numbers.
If we want to compute numbers we have to pick a representation system for real numbers and describe important operations on them — i.e. define what computing means. Since we are interested in scientific computing, we want to represent accurately all decimal numbers (our measures), their quotients (rational numbers), values of the exponential functions and some funny constants, like Archimede's number.
The problem is that the only way to perfectly represent numbers in such a system is to use symbolic form, that is, not to compute anything at all and work with algebraic expressions. This is a rather crippled representation of real numbers, because we cannot reliably compare two numbers (which one is greater)? We cannot even easily answer the question “Is the given number equal to 0?”.
If you look for more precise mathematical definitions and problems, look for rational numbers, transcendental numbers, best approximations, and Baire's theorem, for instance.