In Agile development, how does one account for tasks that are invisible to users? [duplicate]

Question

I've worked in Agile environments for many years. However, one of the things that I've always felt uncomfortable about is the idea that each story should map to an end-user request.....i.e., an enhancement that is clearly visible to someone using the application.

However, a significant part of the work in real-life software engineering is done behind-the-scenes and under-the-hood. For example: switching out one ORM or DBMS for another (and the resulting code changes required); or enhancing the performance of the application via implementing more efficient data structures and algorithms; or making the application more scalable/extensible by replacing massive hard-coded if blocks with a Factory pattern; and so on.

The problem has typically been that, when I would bring these issues up in Sprint planning, product/project managers would throw out the terms "technical debt" and "refactoring" and reiterate that these things should not be the concern of Sprint planning - that instead we need to focus only on user-requested changes.

I admit that a lot of what I mentioned above could be classified as "technical debt". However, it is my experience that, when something gets classified thusly, it goes into a black hole and is forgotten amidst the never-ending torrent of user requests.

How, then, do we address this? How do we free ourselves of this tyranny? Is there a legitimate way to create stories in Agile that are not centered on things that are immediately visible to end users? And if so, how to we "word" these?

Answer 1

In teams I've worked on, we've always created a special person for that, called The Developer. This way, there's an Identity using the story at some point.

So, for example, we've got a story in the current Sprint related to migrating databases. The story is listed as: "As a Developer, I want to move the database from its current server to the new server so the application performs better."

While its not 'truly' agile, i've had great success with this method, as it calls out things as stuff we could work on and avoids the technical debt black hole. Additionally, it presents a business benefit to the stakeholders; They now understand in less-technical terms what the benefit of doing this work is.

Answer 2

Stories don't always have to be relevant to the end-user. For example

As the accountant, I want to switch to a cheaper database solution

In addition, performance and other technical issues can certainly be relevant to the end-user:

As a user, I want to execute the operation in less than 2 seconds for 90% of the cases.

Or

As a user, I want to be able to connect to the system even when there are 9000 other users connected.

Certainly, there are cases where Adam Wells' answer is appropriate, too, to use a "developer user".

Technical debt and refactoring can many times be bundled into other stories - they are mostly implementation details, after all, and invisible to the end-user as well as the PO.