I seem to identify two common problems:
- Shifting requirements
- Bad database design.
What seems strange to me is that you have thorough performance problems which imply large rewrites- that is extremely unusual. If you are not profiling, when you find a performance problem it is common to try and solve the performance problems blindly, which is unlikely to work and very, very frustrating (the feeling you get is that you have rewritten everything, but things are still slow- this normally means that you haven't rewritten the important part). If you learn to use a profiler, you will find easily the true performance problems in your code and you will probably solve them easily.
Now, once that is solved, let's address the rest.
Shifting requirements are often unavoidable and often very costly. Database model design is vital.
What can you do:
- Learn and practise database modelling. Learn from your mistakes. Read some books. One I would recommend which might work for you is The Data Model Resource Book, which contains typical database models for business applications. It is not a substitute for modelling theory, but it will come in handy
- Devote more time to requirements gathering. Clients seldom can describe their data model in detail on their own- you need to help them- examples from the book I mentioned earlier might come in handy.
- Keep your database as normalized as possible. Normalized database designs are "simpler"- although they might involve extra tables, they tend to be much easier to refactor
- Use the single-responsibility principle, DRY... and other, least important software design principles :) It will drive you towards smaller, easier-to-refactor code.