Formatting should be done as late as possible.
Inside your application, you should represent data in a format that makes it easy to work with. E.g. you would probably represent numbers with a numeric type, not with strings. Currency values will often need a custom type in order to track different currencies (e.g. USD and EUR), and to store fractional values losslessly (floating point numbers are completely unsuitable for currencies).
Your application has various interfaces to the external world. These interfaces might connect the application to other software systems, or to (human) users. Each interface will have its individual requirements how data should be formatted. Therefore, the conversion should happen at the boundaries of your system, within the interface: convert to the internal format when data enters the system, and convert it back just before the data leaves the system.
For user interfaces, localization becomes important. Different users have different expectations how currencies or dates are formatted, or how text is sorted. In my locale, I would expect the value “1234567.89 USD” to be formatted as “1 234 567,89 $”. Different locales would expect “$12,34,567.89” or “$ 1'234'567,89”. It is not possible to handle all that variety throughout your application, and there is no common format understood by all locales. As a result, you have a choice of either ignoring locales, or performing any localization after your processing – usually inside the user interface code.