The number one thing should always and forever be readability. If it's slow but readable, I can fix it. If it's broken but readable, I can fix it. If it's unreadable, I have to ask someone else what this was even supposed to do.
It is remarkable how performant your code can be when you were only focused on being readable. So much so I generally ignore performance until given a reason to care. That shouldn't be taken to mean I don't care about speed. I do. I've just found that there are very few problems whose solutions actually are faster when made hard to read.
Only two things take me out of this mode. When I see a chance at a full blown big O improvement when n is big enough that anyone would care, and when I have tests that show real performance problems. Even with decades of experience I still trust the tests more than my math. And I'm good at math.:
- When I see a chance at a full blown big O improvement, even then only when n is big enough that anyone would care.
- When I have tests that show real performance problems. Even with decades of experience I still trust the tests more than my math. And I'm good at math.
In any case, avoid analysis paralysis by making yourself think you shouldn't try a solution because it might not be the fastest. Your code will actually benefit if you try multiple solutions because making the changes will force you to use a design that makes it easy to change. A flexible code base can be made faster later where it really needs it. Choose flexible over speed and you can choose the speed you need.