I've tried planning things in advance, but I can't seem to really
foresee everything until I start hammering out some code.
It is tempting to think that perfect planning will give you perfect software design/architecure, however, it turns out that's categorically false. There's two big problems with this. Firstly, "on paper" and "the code" rarely match, and the reason is because it's easy to say how it should be done as opposed to actually doing it. Secondly, unforeseen changes in requirements become apparent late in the development process that couldn't have been reasoned about from the onset.
Have you heard of the Agile movement? It's a way of thinking where we value "reacting to change" as opposed to "following a plan" (among other things). Here's the manifesto (it's a quick read). You can also read up about Big Design Up Front (BDUF) and what the pitfalls are.
Unfortunately, the corporate version of "Agile" is a bunch of bogus (certified scrum masters, heavy process in the name of "Agile", forcing scrum, forcing 100% code coverage, etc), and usually results in asinine process changes because managers think Agile is a process and a silver bullet (of which it is neither). Read the agile manifesto, listen to people who started this movement like Uncle Bob and Martin Fowler, and don't get sucked into the nonsense version of "corporate Agile".
In particular, you can usually get away with just doing TDD (Test Driven Development) on scientific code, and there's a good chance your software project will turn out pretty darn well. This is because successful scientific code mostly has ultra-usable interfaces, with performance as a secondary (and sometimes competing) concern, and so you can get away with a more "greedy" design. TDD kind of forces your software to be ultra-usable, because you write how you want things to be called (ideally) before you actually implement them. It also forces small functions with small interfaces that can quickly be called in a simple, "input"/"output" fashion, and it puts you in a good position to refactor in case requirements change.
I think we can all agree that
numpy is successful scientific computing software. Their interfaces are small, super usable, and everything plays nicely together. Note that
numpy's reference guide explicitly recommends TDD: https://docs.scipy.org/doc/numpy-1.15.1/reference/testing.html. I've used TDD in the past for SAR (Synthetic Aperature Radar) imaging software: and I can also assert that it works extremely well for that particular domain.
Caveat: The design part of TDD works less well in systems where a fundamental refactoring (like deciding that you need your software to be highly concurrent) would be hard, like in a distributed system. For instance, if you had to design something like Facebook where you have millions of concurrent users, doing TDD (to drive your design) would be a mistake (still okay to use after you have a preliminary design, and just do "test first development"). It's important to think about the resources and the structure of your application before jumping into the code. TDD will never lead you to a highly available, distributed system.
How can I avoid always feeling like if I completely rebuilt my program
from scratch I'd do it much better?
Given the above, it should be somewhat evident that a perfect design is actually impossible to achieve, so chasing a perfect design is a fools game. You can really only get close. Even if you think you can redesign from scratch, there are probably still hidden requirements that haven't shown themselves. Furthermore, rewrites take at least as long as it took to develop the original code. It almost certainly won't be shorter, since it's likely that the new design will have unforseen problems of its own, plus you have to re-implement all the features of the old system.
Another thing to consider is that your design only really matters when requirements change. It doesn't matter how bad the design is if nothing ever changes (assuming it's fully functional for the current use cases). I worked on a baseline that had a 22,000 line switch statement (the function was even longer). Was it terrible design? Heck yea, it was awful. Did we fix it? No. It worked just fine as it was, and that part of the system never really caused crashes or bugs. It only got touched once in the two years I was on the project, and someone, you guessed it, inserted another case into the switch. But it's not worth taking the time to fix something that is touched so infrequently, it just isn't. Let the imperfect design be as it is, and if it aint broke (or constantly breaking) then don't fix it. So maybe you could do better...but would it be worth a rewrite? What will you gain?