Been there many times.
There are some ways to handle it, no easy one though.
You almost said it yourself: they only care you deliver as fast as possible. Show the stakeholders a business case where a X month refactoring round will reduce the time it take to develop new rounds - and try to estimate some ROI - return of investment, after a year or so, for the refactoring.
Or, if this is very hard, there are other economical arugments that might help - "Given we are running a prototype in production, we need to hire two more engineers to maintain it. This cost can be avoided if we take on a refactoring project.".
If you have internal cost centers etc. make sure the operational people supporting this application charges more for it, than similar "production worthy" applications.
If your team has some control, it might be possible embedd refactoring in the new feature deveopment phases. Say, they want to input complex hierarchies into an already shitty data model - include refactoring as a tech requirement to even make any changes to the data model. A similar case: if you ask your plumber to change the radiator, if the pipe is too rusty to function properly, he will prob. change that too, since its his profession to know what to do. The same thing (should) goes for software devs.
If you hit the wall, gather stats for your arguments. Tag each bug/operational issue that is reported that could have been avoided by production worthy code. Then you could show a business case by reducing the bugs with so-and-so many %, and increase the uptime/stability by another %.
However, there is always a fact that software developer want to make the best possible solution, when only "good enough" will do. If you can't make up economical arguments for refactoring, it's probably not worth it and should not be done.