Yes, it's true. Both strings and decimals, as well as all other types, are represented in binary.
However, the problem lies in the significance of the binary. You cannot simply read a string as a decimal, or at least you shouldn't if you were looking to get the decimal representation of that decimal.
Take for example this string in ASCII: "1.5"
In binary, this is represented by an array of bytes of size 4 or greater.
1 . 5 \0
31 2E 35 00
If this were to be interpretted as a double, the value would be:
4.07659398311e-315
What does this have to do with the number 1.5? They share the same binary message, but since binary is only as good as its interpretation, this double is meaningless. This interpretation is complicated further when you consider that some systems prefer to save decimal information in big-endian or little-endian formats, meaning you're also dependent on the system you're working on.
In order to convert this into a double, you'd have to parse this binary as it was intended. As it turns out, the equivalent conversion of 1.5 in hexadecimal is 0x3FF8000000000000.
If you prefer precision, then you generally tend to retain only the string form in your program. Similarly, if you need to use this value in calculations, you keep the float or double form. So what happens if you need both? You keep both in your program, using whichever form is best suited for precision or calculation. Long past are the days when keeping both in memory might be an issue (save for few exceptions where speed is critical, such as in the case of drivers).
If you wanted to determine whether or not a string representation of a number is a power of 2, the calculation to do so would be more complicated than the simple act of converting it first to a number. Once it is a number, determining whether or not it is a power of two is trivial, with any binary string containing exactly one 1 (in the case of decimal, you should check also for binary strings containing exactly one 0 for negative decimal values).