You would typically use memoization where you have established that the problem that your function is trying to solve has optimal substructure. This means that the larger problem can be broken well into a series of small subproblems. It sounds as if you were successful in realizing this and that you solved this in a recursive way, which is also a way of breaking down a large problem into subproblems.
If you have working code and it satisfies all requirements including non-functional requirements on performance and scalability then you don't necessarily need to improve the performance of the code by using memoization. Delaying the release of working software to optimize is generally the spirit of "Avoid premature optimization" in my humble opinion.
Would it be dangerous to use lru_cache() from now?
It depends on how you define danger. If you have a working recursive solution without memoization then the only real danger here is possibly adding more complexity which increases the risk of introducing a bug into your logic. The other risk is that you might delay the release of the software to work on an optimization that isn't necessary based on clearly defined non-functional requirements.
I might even argue though that if you have a long running recursive function then there might actually be danger in NOT using memoization!
What I mean by this is that for a sufficiently large dataset you run the risk of blowing the stack or running out of memory. You should do adequate stress testing to ensure that your recursive solution is fine.
Should I not care at all about which pieces of my code can be optimized (even if they are obvious to spot, and easy to optimize)?
If it is an easy optimization then by all means go for it. Premature optimization avoidance isn't so much a hard and fast rule as it is a guideline for living ones life as a developer. It is inspired by some other subjective guidelines like YAGNI and KISS.
Should I ignore memoization on obvious cases completely, or perhaps note it (e.g. #TODO comments) so that it's easier later on when time for optimization comes?
Absolutely. I think TODO's are a great way to leave little post-its for yourself and other developers about where something could be improved or done better. That is also a matter of personal preference though. Pro-TODO and Anti-TODO developers though have very strong opinions on either side and they both make good points.