59

I regularly review the technical debt tickets from my backlog, to prioritize them and remove those which are no longer relevant (fixed by some other development, obsolete...) Among those with high priority, we take 2 or 3 in each sprint, and this way our codebase is healthy for the moment.

The problem is that all those tickets that are still relevant but have not been prioritized, represent a big part of the backlog (50%), and my PO insists that they should be removed, the same way that he deletes regular Story tickets that he knows won't be prioritized in the next semester, in order to have a "lean and healthy backlog".

I acknowledge the fact that with our current "tech debt velocity", we won't be able to take most of them in the next semester, but it frightens me to delete tickets that are pointing to spots in our code that may rot if not fixed, lending further developments more difficult (well, you all know the point of tech debt and why it is important).

So my question is: should I prune the tech debt tickets with lower priority?

11
  • 23
    If you delete these tickets in your backlog, does that mean that they are not tracked/documented at all? – Alex Jun 29 '20 at 16:48
  • 80
    This strikes me as removing the wet floor sign so that there won't be any more wet floor. – Zibbobz Jun 29 '20 at 18:56
  • 5
    How expensive are these tickets to make and how likely is it that you create a duplicate by mistake? E.g., are these made by an automated code scanner like SonarQube or are people manually making these based on issues they know about or something else entirely? – Captain Man Jun 29 '20 at 19:52
  • 6
    Maybe if you start describing the backlog to your PO as "thick and lustrous", it will become more acceptable. – Paul D. Waite Jun 30 '20 at 12:07
  • 5
    Do you have another column you can move them to, just to make PO happier with the backlog? Maybe create a "Wishlist" column, or simply a "Tech Debt" column. Personally, I think your PO doesn't understand the purpose of Backlog. It's the stuff you don't have a specific timeframe for. It can be very large, if you need it to be. – Dan Jones Jun 30 '20 at 19:27

10 Answers 10

10

I would say that there isn't a single best answer for this problem. You've got a few overall strategies, you would probably be best doing all of them. But the information should never be deleted, but it can be transformed.

Keep them in the issue tracker with the rest of the project

For high priority or fresh debt I would keep this strategy, once tickets start getting older or are deemed very low priority I would move on to the next two strategies

Pros:

  1. Keeps the context to one area making it easier to manage the technical debt
  2. Gives them high visibility

Cons:

  1. Religious management and filtering of these will become important (it may be a large time commitment)
  2. If there's a very large number, the small unimportant ones may drown out more important ones

Keep them in the issue tracker separated from the project

Move older tickets or less important tickets into a separate project and pull them into the main project as required for working on them.

Pros:

  1. Keeps the backlog clean and sane
  2. Makes it easy to determine what is high priority vs low priority by changing their visibility

Cons:

  1. This second project becomes a dumping ground and can easily fill up, it will still need to be managed
  2. A lot of these tickets will be moved here and never look at again, begging the question if they'll ever get done?

Move the information into code

By using a special commenting style, ie // TECHDEBT: .... you can mark the areas required for clean up directly in your code base instead and these can be fixed up when there is spare time in the project

Pros:

  1. The information about the debt is located where the debt roughly is
  2. Doesn't pollute the issue tracker
  3. Can create a good culture around identifying debt
  4. If it's apparent that there is a lot of notes about technical debt in a particular area, it makes it easier for a developer to flag that something needs to be done (the collection is more important than the individual)
  5. A developer may fix this when otherwise working on the feature (thanks Heinzi for adding this)
  6. When cleanup of the debt has been performed the note should naturally be removed (thanks Steve for adding this)

Cons:

  1. Can create noise in the code itself
  2. May still never get fixed
  3. From a project management perspective, the work may never get prioritised, so it may never get fixed
  4. The information is now split across two different places, making it harder to find and more likely to file a duplicate ticket
3
  • 2
    Additional "pro" for the code comment: when the code in question is deleted or transformed beyond where the issue is relevant any more, the comment is naturally deleted. The ticket likely wouldn't even be noticed at that point. I guess arguably this is just a corollary of (1), that the issue report is located with the offending code. – Steve Jessop Jul 2 '20 at 2:14
  • I think that's a great point @SteveJessop and worth pointing out, I'll amend the answer to include this – Mattisdada Jul 2 '20 at 2:45
  • 4
    Another "pro" for the code comment: The comment will be seen by someone who needs to work on that code for other reasons (e.g. to implement a new feature). Since that person needs to familiarize themselves with that part of the code anyway, it often makes sense to "kill two birds with one stone" and also do the code cleanup, because it reduces overhead: Only one person needs to invest time reading the code and (after cleaning the debt and implementing the new feature) testing the code. – Heinzi Jul 2 '20 at 12:01
74

You are considering deleting the records of genuine problems with the codebase because the product owner wants a shorter backlog?

For me, the only reason to delete (close) an item in the backlog is because you decide it will never be implemented, not because it won't be implemented for a while. Also, in an agile environment, priorities may change quickly and the backlog can be re-ordered. If you have trimmed the list only to what you can do in the near future, you lose the ability to bring lower priority items up the list.

Maybe you should re-assess the tech debt issues if they represent such a large proportion of the backlog; you might be able to close a proportion of them as "won't do".

I am not sure what is concerning the PO to be honest. A healthy backlog contains a mix of items with lower priorities naturally floating down at the bottom. If it's really a problem, just filter the backlog or even create a second list (still logically a single backlog but split into 2 lists for mangeability). Maybe the PO is already doing something like this; I doubt he is actually deleting stories on the basis that they are not part of the current commitment?

6
  • 34
    Also, such low priority fixes are usually a great way for new people joining the team to learn about the codebase. – Simon Richter Jun 29 '20 at 21:03
  • 1
    yeah, joining a new company with just big projects, but when you begin working it, and learned the "small debt" from other programmer when you are balding your head after several hour confused. Not a good experience at all. – encryptoferia Jun 30 '20 at 8:01
  • 9
    I would like to add that in all my ticket systems, even a "will never be implemented" ticket still doesn't get deleted. It gets marked as "not going to happen", which is a closed state that can be reported against, and shows up in the duplication prevention system so we don't re-address it in a year without knowing that it was already brought up. – Logarr Jun 30 '20 at 14:53
  • Ideally you'd re-assign x% of each sprint to time for people picking off such tickets on their own recognisance. – Asteroids With Wings Jun 30 '20 at 19:10
  • "I doubt he is actually deleting stories on the basis that they are not part of the current commitment" You must be lucky to never have POs that don't understand the process. – Dan Jones Jun 30 '20 at 19:32
36

I agree with Dan Saunders's answer, but I'm going to go one step further.

I agree that the only reason to close a request for a new feature or a modification to an existing feature is if the change will never be implemented. However, for bugs and technical debt, the only way to close the issue is to "fix" it. There are different ways to fix such issues. One would be resolving it - fixing the bug or refactoring the tech debt away. The other would be removing it - deprecating the feature that the bug or the tech debt exists in from the system. For bugs, reports that don't represent bugs (that is, the actual result of the execution steps is indeed the expected result, even if it's not that way to the user) can also be closed. Still, it may be worth understanding why the user thought it was a bug and correcting it at a root cause.

The most significant factor for keeping all known bugs and technical debt in an issue tracker is visibility and transparency. Users can run across bugs as the system is changed, and changes may increase the likelihood or impact of a bug to the intended operation of the system. Likewise, developers may run across technical debt when working with impacted functionality. If you can track bugs and technical debt into parts of the system, you can make additional work to resolve them known to stakeholders earlier and build it into the plan to develop and release the changes. By keeping all of the known work in a single tool, you can let all of the stakeholders make informed decisions.

Backlog size was a much bigger problem before electronic tooling that comes with the ability to search and filter records. Today, especially in a more remote and highly distributed world, there is little reason for anyone to be using index cards and sticky notes. We should no longer be constrained by physical space on a wall. Electronic tools range from lightweight task tracking to heavyweight project management. It's pretty straight-forward to set up filters that hide these lower priority issues without closing or deleting them in most tools that I've worked with, yet making them visible to the right people at the right time.

5
  • 8
    And for bonus points, even if you have a bug that will never be fixed, you still want it in the system - so that when the bug gets reported again and again, you don't have to waste all that time on investigating the bug again and again. There are always bugs that aren't worth fixing - but you should still know about them, and have the workarounds ready. – Luaan Jun 30 '20 at 7:46
  • @Luaan Absolutely. Being able to generate a list of "known issues" has been immensely valuable to me. If a customer finds an issue, being able to say "we knew about it, we investigated, and we didn't prioritize it for these reasons" goes a long way. Having the issue in the issue tracker with the right content to be able to search or filter for it can help capture your notes about investigations and potential workarounds to help others in the future. – Thomas Owens Jun 30 '20 at 16:23
  • 1
    There is one more consideration I'd add to this answer: new features and tech debt exist in a different sense: features are plans, they can change and vanish if the PO changes his mind, while techincal issues exist independently form people's wishes, and their existence is not a matter of discussion but a fact. Knowledge about facts is useful. The developers need to be aware of them, even if they'll never work on them. It might be a good thing to provide a "clean" (filtered) view to the PO, who doesn't want that noise around. – Rad80 Jun 30 '20 at 20:15
  • @Rad80 I'd have to think about that more, but I'm hesitant in relying on such a filtered view. Bugs and technical debt can be blockers from implementing features and knowing that information is good. If so, the PO may want to prioritize a new feature with no blocking tech debt or bugs over one that would need a lot of tech debt paydown first. – Thomas Owens Jun 30 '20 at 20:42
  • 1
    @ThomasOwens good point. However , since the PO doesn't want that info, I assumed they'd rely on the defelopers to tell them about tech debt issues with the stories when they arise. If the PO is able to assess that autonomously then they need to see that info. Just to be clear, I do not reccommend hiding the existence of the issues from the PO, just give them and the developers different default views tailored on their needs. – Rad80 Jun 30 '20 at 21:19
6

Does your bug tracking software allow you to mark a bug according to its priority? If so, then you can ask your boss if it's OK to create a "lean and healthy backlog" by simply viewing the database with filtering based on priority.

Is the bug tracker publicly visible? If so, then maybe the problem is purely marketing. Your competitor may be telling prospective customers that you have thousands of bugs, so your product is obviously bad. Maybe point out that users like you more because you're open about bugs, rather than hiding bugs -- which costs users time and effort because they can't get information.

If you're contemplating deleting a lot of bug reports, then one thing to ask yourself is this. Before I deleted them, would I consider it worthwhile to copy them all somewhere so that the information was preserved? If so, then this is a problem of perception or database design, not a problem that should be solved by a database purge. You don't want to maintain two databases, a big one and a small one. That's just an inefficient way of maintaining a single database with a binary priority field in it.

1
  • 2
    For the first paragraph alone, this is probably the best answer here. Prioritizing and categorizing tickets is the entire point, and taking that and trimming all the low-priority tickets because they 'will never be solved' misses that point entirely. – Zibbobz Jun 30 '20 at 13:40
6

Yes, you likely should prune your backlog of un-prioritized technical debt, as you won't fix it (similar to YAGNI).

Think of fixing technical debt as an investment opportunity. Something developers can invest time in, and hope that it pays off in the future, and hope that it pays at a higher rate of return than other things you could have invested time in.

Many times we invest days or weeks into "fixing" technical debt, with no demonstrable benefits. Or the rate of return on the investment is low, and we miss out on other opportunities that would benefit the team/code more. Often times, we end up cleaning up regions of code that don't get modified again for years, or refactor code that needs to be re-written weeks later when we have to accommodate a new feature that moves the product in a direction we didn't anticipate.

Furthermore, keeping items in a backlog, has cost a non-zero cost. Either you ignore the items in your backlog, and it keeps growing so large as to not be groom-able. Or, you do a good job and groom it weekly team, in which case you're wasting minutes of everyone's time talking about it each week.

The tough reality is, you need to write good code the first time. As soon as you accumulate technical debt, you're never likely to have time to directly tackle it. You may be able to chip away at it, by making sure every modification you make to the code, cleans the regions it touches (Robert C. Martin's "Clean Code"). Perhaps one of the most important parts of good software design, is to be able to compartmentalize these regions of technical debt within a code base.

That being said, never confuse technical debt with bugs (and possible bugs). Bugs must be prioritized, investigated, and fixed or documented as features (#5 of the Joel Test). The lack of testing is not technical debt, its just not yet discovered bugs.

3
  • I think it's generally a bad idea to follow the otherwise excellent advice of Uncle Bob on this subject. Apart from really small code improvements, I tend to avoid refactoring/cleaning modifications at the same time as developing new features. In my experience it makes the reviews harder, and increases the chance of undetected regressions even in high test coverage codebases. – Gua-naiko-che Jun 30 '20 at 7:38
  • 1
    @Gua-naiko-che It's helpful to make the two kinds of changes in two (or more) separate commits. You should have some idea of what you're trying to do and what it's going to require, and do the cleaning/redesign in preparation of doing those changes. This should make your actual changes easier to understand and safer, and it focuses your refactoring efforts to areas of code that are actually being changed. I disagree with many things Uncle Bob recommends, but it's far more frequent that people missed what he's trying to say - be careful when learning through a 3rd party. – Luaan Jun 30 '20 at 7:57
  • 1
    @Gua: Another practical reason for refactoring while you're in there anyway, is that it's cheaper. A ticket like, "I spotted an opportunity to refactor X" is all very well, but by the time you get to it you've forgotten how the code works, so you have to remember and then refactor. If you're changing the code, then you had to spend the time to remember how it works anyway, so now's your chance. That said, I'm not sure I'd even class "I spotted an opportunity to refactor X" as technical debt. "We've repeatedly lost time making changes in this area due to failure to refactor" is technical debt. – Steve Jessop Jul 2 '20 at 1:34
4

I expected to see someone reference Joel Spolsky's old article.

Since you say that these tickets haven't been prioritized it sounds like no one's looking at them, and in that case it's ok, leave them, but to quote Joel:

Every minute you spent writing down, designing, thinking about, or discussing features that are never going to get implemented is just time wasted.

If you find yourself spending time reading through these tickets with any frequency, then they're in the way -- get rid of them.

3

It makes no sense to have a "lean and healthy" backlog for a codebase which isn't equally "lean and healthy". The backlog's job is to reflect the state of what must/could/should/would (not) change in the product. A good backlog isn't pretty, it's representative and up-to-date. It informs all its users on the oughts and ought nots.

Generally, tech debt items in the backlog will be added by developers for the PO to read, thus making information flow the "wrong" way unlike other items. Maybe your PO doesn't want to read them? It doesn't sound like you have an acute tech debt problem though.

The thing which is supposed to be pretty is the list of closed issues.

You should probably investigate why your PO wants the backlog to be pretty. There might be a misunderstanding somewhere. Perhaps the PO is reporting something to a higher level manager using the backlog, which is probably wrong.

2

You and your PO have conflicting uses for the backlog, which need to be reconciled.

You want a list of every good idea that anyone has ever had. Perhaps above a certain threshold of good, since you're calling it "tech debt" rather than just "potential improvements".

Your PO wants something that you can look through on a regular basis to choose tickets: a realistic and manageable list of what actually needs doing. Your increasing tech debt is not quite the same as "low priority", though, because to the PO a low priority ticket in the backlog means something that's still worth periodically reviewing and perhaps raising its priority. Your PO strongly believes that for these tickets it's not worth doing even that.

You are also concerned that something in the backlog might secretly be really important, since it points to an area that might rot. If you had time to do all of these tickets, then maybe you could deal with every potential point of failure that you've identified. But you do not have time, which is why you prioritize. The things that are persistently not prioritized have a low(er) chance of causing serious failure, and hopefully a genuinely low chance of causing failures that will not be caught in testing in the event that they do happen.

So, by not prioritizing them you have decided not to do them. "We're not going to do this any time soon" means, "we're not going to do this ever, unless it gets worse". It doesn't literally mean that, but be honest with yourself: that's what happens.

So, own that decision. Other than maybe a small sample pulled out for one reason or another (like, it's mid-afternoon, you're going on vacation, and there are no small tickets left in the sprint) you're not going to do these tickets any time soon. Unless you link them to higher-priority symptoms you very likely aren't going to do them ever. You should not be grooming these nuisances every sprint.

If your backlog doesn't satisfy both needs then that is an issue with your issue-tracking system, that needs to be solved in a way your issue-tracking system supports:

  • Tag them "slush pile" and exclude the tag from the view used for the majority of your planning. Then include it back in when someone joins the team and you're filtering the entire backlog for "tickets that a novice could tackle before lunch" to get them started.

  • Close them with a "won't fix" resolution, so that they're still searchable if they contain any useful insights how to address the issue identified. OK, so it's embarrassing if you eventually re-open or duplicate a "won't fix" ticket and actually fix it, because that means you were "wrong" when you predicted you wouldn't. But should it be more embarrassing than never fixing hordes of tickets you predicted you would fix?

  • Move them to another "code improvements" project. You can occasionally look at this to assess whether it's time to focus a sprint on some developer quality-of-life issues that will improve future efficiency, or on some ticking time-bomb issues (like Y2K, or the tiresome inevitability that for every dependency you have, some day upstream will release a critical security fix which is not back-ported to the version you're using).

  • Define more priority levels, hide the lowest level from the default backlog view, and make a rule that you do not waste your time looking at those tickets unless you run into them for some reason other than, "they're on the list".

In all of these approaches, the basic idea is that you keep them on your backlog (used for recording everything you know about the state of the product), and remove them from your PO's backlog (used for sprint planning).

Anyway I suspect the root issue here is that you take "backlog" to mean, "every issue we've identified and not fixed", whereas the PO takes "backlog" to mean "todo list that I actually have to think about". Those are both worth having, but one is a filtered view of the other. Feel free to argue with your PO which of those two things deserves the name "backlog" in a well-regulated Scrum methodology. But that might be one for the pub rather than work time.

If you move this junk out of sight and the PO still objects to it, then it's possible that there's some political reason for rigging some "code healthiness" metric. Then maybe you actually need to drill into whether it's right or wrong to include this stuff in that metric. Beware Goodhart's law: "When a measure becomes a target, it ceases to be a good measure" (or, "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes"). Especially beware that knowing the law does not make your own KPIs immune to it. This is why you don't pay bonuses for increased velocity (and if you do, what you get is ticket point inflation). You shouldn't pay bonuses for hiding bugs, either.

0

A technical debt must be paid or forgiven. The reasons for the latter can be anything from implementation complexity, revised importance of said feature, movement of feature to another version(not even a very strong reason), etc.

Of all the reasons a debt should be forgiven, too many post-it notes shouldn't be one of them. If it means having a backlog of your current backlog, that makes more sense than deleting a report that might eventually find its way back to the backlog when history repeats itself.

You can deescalate, but don't remove until you know it's never going back on that board.

-11

Remove them.

If developer hits the smelly code, he can create a tech debt ticket once again.

9
  • 22
    "Those who cannot remember the past are condemned to repeat it." (George Santayana). 1. Someone raises a ticket to describe a problem. 2. Someone else de-prioritises (deletes) that ticket. 3. Go To 1. – Phill W. Jun 29 '20 at 11:22
  • 11
    but why would they bother? "Well - I could put this tech debt card in, but it will probably just get deleted again so what's the point?" - this manager's tactic is to make him look good by having a 'clean' backlog but will make the code worse, which in the long run, will make everyone look bad – NKCampbell Jun 29 '20 at 17:35
  • 1
    @PhillW. but, it's a developer against a manger and Jira has a REST API.... hippity hoppity via scripting this backlog is my property.^^ – Frank Hopkins Jun 29 '20 at 22:39
  • 2
    @Andy If you keep finding the same backlog bug and writing duplicate tickets for it, should you really have removed the original backlog bug to begin with? This would just obfuscate how frequently you run into the issue, and make re-prioritizing issues later down the line that much more difficult. – Zibbobz Jun 30 '20 at 13:39
  • 2
    @Andy If having lots of work to do is off-putting, I find it hard to believe anyone would ever work :D – Zibbobz Jun 30 '20 at 14:24

Not the answer you're looking for? Browse other questions tagged or ask your own question.