I've worked in Agile environments for many years. However, one of the things that I've always felt uncomfortable about is the idea that each story should map to an end-user request.....i.e., an enhancement that is clearly visible to someone using the application.
However, a significant part of the work in real-life software engineering is done behind-the-scenes and under-the-hood. For example: switching out one ORM or DBMS for another (and the resulting code changes required); or enhancing the performance of the application via implementing more efficient data structures and algorithms; or making the application more scalable/extensible by replacing massive hard-coded
if blocks with a Factory pattern; and so on.
The problem has typically been that, when I would bring these issues up in Sprint planning, product/project managers would throw out the terms "technical debt" and "refactoring" and reiterate that these things should not be the concern of Sprint planning - that instead we need to focus only on user-requested changes.
I admit that a lot of what I mentioned above could be classified as "technical debt". However, it is my experience that, when something gets classified thusly, it goes into a black hole and is forgotten amidst the never-ending torrent of user requests.
How, then, do we address this? How do we free ourselves of this tyranny? Is there a legitimate way to create stories in Agile that are not centered on things that are immediately visible to end users? And if so, how to we "word" these?