For me it is often more productive to err on the side of optimizing in hindsight, just slapping in a basic implementation which is easy to reason about in terms of correctness and revisit as needed with profiler in hand. I prefer to iterate towards faster solutions when possible/practical because I like to see my system get fleshed out sooner even if I have to go back and drill down and tune it in key areas.
But I need to put big emphasis on the practical part, since it's not like you can write a software whose design revolves around interacting with teeny objects in a scalar, one-at-a-time fashion, and expect to find much room to optimize that without changing the design. There's no use having a race car if there's only 10 meters worth of road for it to drive over, and designing overly granular interfaces that deal with teeny things one-at-a-time can corner you like this with a design that leaves no breathing room for optimization.
So it helps when you anticipate a performance-critical area, which I do believe you can often anticipate without measuring*, to design sufficiently coarse, not granular, interfaces.
- As said I believe you can anticipate where the performance-critical parts are in your system reasonably well, even if you may not be able to anticipate the details fully until measuring. All you have to do is ask basic questions like, "where are we going to be looping over a million things"? Well, if you are implementing a GUI system, it's obvious where you'll be looping over a million things, and that'll be in the GUI drawing functions where you can potentially be looping over a million pixels to process. It's not unreasonable to deduce, in foresight, that this area is probably going to be a performance-critical so that you design it with sufficient breathing room to optimize and optimize it in the future.
For example, instead of having your system revolve around interacting with granular
Pixel objects, design it to revolve around interacting with collections of
Image objects which could represent millions of pixels at once. Similarly instead of
Particle objects, interact with a
ParticleSystem. Instead of
Creature, interact with
Creatures. Instead of a callback which is designed to process one teeny thing at a time (one pixel at a time, e.g.), have a callback which is designed to process a range of things at a time (a range of pixels at one time). These types of things will leave you so much more breathing room to optimize without changing the design.
That said, this advice is oriented towards people like yourself who want to move on and tackle big projects. If you are, say, in the mindset of designing a general-purpose library of data structures whose intentions are to be as widely applicable as possible, and that's basically the end product, then you might save more time putting thought into how to make it as efficient as possible upfront -- of course still measuring and tuning and iterating as you go, but not just slapping in a basic implementation if you're pretty sure you're just going to have to rewrite it.
In those cases, efficiency and applicability are related concepts, since if your library is skewed in performance characteristics and lacks well-rounded data structures, people might not use them so much or you might end up feeling the introduce more and more data structures when tuning the former ones might have sufficed. So there it can sometimes really pay to just try to get the most efficient version you can mostly upfront. I've found over the years as I've gotten better at optimizing code and especially for locality of reference that I can get away with using fewer and fewer data structures, ending up with more well-rounded data structures I can use in a wider range of areas instead of ones with skewed performance characteristics that are narrowly applicable. The end result is much less code to maintain and enhanced productivity, even though that came about from an optimization mindset.