Required reading: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
With that said, for professional software development, it is rare that a complete start from scratch is ever a good idea. About the only scenario where a complete rewrite makes sense is when you need to make an ancient software, dependent on a discontinued old platform, work on modern software. Even within that scenario, a total rewrite is still unnecessary.
A complete rewrite being a good idea means that 100% of your work from before is complete garbage, with nothing worth keeping. Writing 100% complete garbage is never something you want to be doing on a regular basis.
When you do your job well, most of the real guts of the value adding parts of your work should be in reasonably modular, packaged groups: classes, libraries, whatever. Usually, much of the cruft of an initial stab of a project and source of confusion is how everything is hooked together. Replacing all that glue code, and reorganizing and refactoring your code base, can let you keep 90% of the meat and potatoes of your code, but reorganized into a more sensible project.
But I feel most projects are more exploratory than anything. In addition to moving requirements, we are constantly learning new methodologies, patterns and "tricks".
Your perceptions don't reflect the reality of the working world. Few projects are completely exploratory. I work doing R&D for medical devices, working on software to do things that have never been done before, and only a sliver of a given project is actually exploratory. The majority, like any other application, is more mundane like saving and loading data files/databases, communicating with hardware, drawing charts and making PDF reports. Someone working on a shopping cart or CRM application will have even less exploratory work to do.
So in short, a complete rewrite is pretty much always a bad idea. Identify whats good and bad, keep the good, refactor and reorganize the bad, and keep moving towards a good product.