is it ok that development team create refactoring tasks in the product backlog?

Question

I'm aware of software development principles and best practices. However, in most cases I entered a project that was already on its phase 2, and that already had a significant amount of code written. In addition, in some cases the code and architecture of the project was causing several problems to the application (performance issues, difficulty for maintenance or adding features, lack of coding patterns, in some cases even security issues, etc).

So, because of this we noticed that new features were taking too long to be implemented, due to the complexity of the architecture and difficulty to maintain it.

In most of these projects we use SCRUM as a base for the development methodology; therefore we have constant collaboration with the customers, that help prioritizing and defining many aspects of the tasks.

My question is: since normally the requests for changes or new features come from the customers, is it ok that our team create stories or tasks within the product backlog, regarding refactoring of the code? Or such thing should be done gradually inserted into existing stories?

I'm asking this because in several projects a significant refactoring was really necessary, because the current situation was causing a negative impact on the customer (features taking too long to be developed); also in some cases this need for refactoring was noticed in the middle of the project (no chance of considering this in the project initial phases).

Answer 1

Within the framework of Scrum, any stakeholder is allowed to bring in new backlog items at any time. As the team is also a stakeholder, they can also bring in backlog items.

The main risk with backlog items from the development team is that they, in my experience, tend to be using rather technical language. This can make it hard for the product owner to see the business value of those stories and cause them to be given a low priority.
If the team wants to bring in a backlog item, they should make an effort to write it in terms that are understandable to the PO and preferably also to the customer and that indicate what advantage the customer would get when the proposed work is done. This increases the chance that the added value will be correctly recognized.

Answer 2

Sometimes it's worth remembering that the rules of scrum based development aren't carved in stone next to a flaming bush.

They're a framework for certain types of software development that have proved useful. They're really useful for framing the interactions between interested parties in the software process.

Sometimes, though, they're not quite adequate for what you need. Once you accept that then the problem, more or less, solves itself.

In your case, my suggestion would be this. Write a paper on why you want to refactor and an outline for what you want to do.

A key thing is to make the case. You outlined some items in your question: slow delivery of features, maintainability. That sort of thing. These are business affecting issues. In my world, I'd be expecting metrics here e.g. bug counts or feature delivery time trends. YMMV.

I'd also included an outline of the refactoring involved and some idea of the overall cost and how you want to approach the refactor e.g. explicit tasks, extensions to feature tasks or a combination of both.

With this in hand, you sit down with the relevant parties and explain the problem and the proposed solution. Either the case is made and you include the refactor or it's not and you don't. The key requirement though is that the communication is transparent with all the interested parties.

I would definitely recommend that the actual deliveries be included as part of your normal sprint process. This may mean having to think quite creatively about how you chunk up the code changes. Fundamentally, scrum works because it normalises communication so you want to get back to well understood processes asap. It's not always possible but the more you do outside of this the more risky things get and the more care you need to take.

Answer 3

Scrum is focused on achieving the backlog tasks. It doesn't really matter who writes them as long as the PO prioritises them.

However. In practice if you only ever put new features in the backlog, your code will become more 'hacky' over time. As scrum, rightly, focuses on achieving the goal as quickly as possible.

Whether, refactoring as you go and slowing down new features pays off vs gradually increasing cost of adding features as the system becomes more complex is a difficult question.

But more importantly its a business question. You have to make sure that whom ever is paying the developers wages is happy that the right balance is being struck.

Work out how long the product is expected to be in service and try and get an idea of the total scope of the features it will eventually have. This will guide you and the devs in deciding what should and shouldn't be refactored