Consider this. When you "find annoying things (...) to clean up" and you make an executive decision to do so, you're cutting the rest of your team out of a priorities discussion and decision. You're letting your agenda trump everyone else because of your privileged relationship with the code. I don't think that's nice. From experience, it also leads to team/shareholder resentments.
Instead, create an issue/task for the clean-up/refactoring. While it's fresh in your mind, list the reasons it's important: estimates of increased stability, ease-of-maintenance, that sort of thing. Maybe include an estimation of effort depending on how your team works. Then in your next task selection/assignment/priorities meeting, present your refactoring task and position it against other tasks. As a team, decide when it should be completed.
Please don't think I'm telling you to throw out good sense in the name of principles. Use your head. If there's something ugly in the function you're editing, it's not a new refactoring task. Fix it and check everything in. If renaming the property you're working with to something more sensible affects a couple extra source files, it's not a new refactoring task. Fix it and check everything in. If, on the other hand, you don't like the way another developer (Mitch, I hate that guy) did something in a function you're not editing and said function appears to be working fine, leave it alone for now. Create a refactoring task and present your case to your team.
If refactoring is always down-voted by your team in favor of new features, start looking for another job. It's easier to find a job when you already have one.