So, is slower performance, of programming languages, really, a bad thing?
Everything else being equal, going faster is a good thing. After all, nobody really wants to wait longer for some results, and once that result is done it can free up resources for other things.
But not everything else is equal. For starters, it's also important to produce the right result, or at least right-enough. (If completely wrong results are allowed, you can produce those very quickly indeed and they will be of exactly zero value to anyone.) If a change to a somewhat slower language makes it more likely that the right result will be produced, that's typically a great trade-off. Higher-level languages have an advantage over lower-level ones here, as their richer set of models usually makes it easier to express a complex problem without overwhelmingly much explicit detail.
It's also usually important to manage the cost of producing the software in the first place, of adding new features as desired, and of keeping it working as the underlying systems change. Higher-level languages usually allow for faster programming turnaround, and there's a lot of value in keeping the costs of programming within budget. Indeed, keeping costs there down allows more different things to be achieved overall, which is generally a good thing.
The final key point to note is that it is not necessary to use just one language, and that many software systems have a majority of their components being not performance critical. Using a low-level language to produce high-performance components for the critical bits is sensible, while leaving the less critical parts to a high-level language (so as to minimize the cost of producing them) is eminently sensible. What's more, the features that make a good low-level language (the ability to control precisely what the machine does) are not the features that make a good high-level language (the ability to infer the details from much smaller descriptions): they are diametrically opposed, so being able to couple them together and use them for their strengths and avoid their weaknesses, that's a great thing indeed.
Which components should get the high-performance treatment? The optimization? Measure them. Profile them. Find the truth rather than guessing. Focus your effort where it does most good.