Refactoring is like picking up your room.
If you keep things tidy, you have a linear overhead, proportional to the amount of productive work you're doing on the code, O(n) in algorithmologist terms. Assuming you spend 10% of your time refactoring (or keeping your room tidy), that 10% is a given, and it will remain constant over time.
If, however, you toss your dirty clothes in a corner, and keep doing it, the amount of time you are going to spend picking up your room grows as the mess becomes more complex. Assuming that each individual piece of dirty laundry contributes exponentially to the required cleanup time, you are now in an O(en) situation.
Anyone who has ever digged into the concept of algorithmic complexity will observe that there is a break-even point somewhere, that is, there is an optimal amount of dirty laundry to accumulate; how much that is depends on the constant factors that are discarded in big-O notation. Another factor is the value of your work over time: if your work is worth a lot now, but cheap next week (i.e., there is a deadline this friday for this project and three more, but after that, you'll be mostly idle), the equation might turn out in favor of not refactoring.
And then there's the complexity critical mass. At some point, the mess ('critical mess', if you will) gets so bad that it seems easier to just burn down the entire room and buy new clothes. In reality it usually isn't, but it appears that way, and psychological effects will make it ten times harder to tackle the thing.
And obviously, if you step into a project that is a giant multiply-redundant mess already, you have limited choice.
TL;DR: If in doubt, refactor. You should have really good evidence before deciding not to.