Test driven reduction of technical debt

Question

Suppose you have been put in charge of an already existing project. As you are starting to familiarize yourself with the repository, you notice a few technical debt issues (insufficient test coverage, conflicting standards, missing documentation, etc). As someone who is going to be working on this project and maintaining this code, you add this to the backlog of the project with a goal to slowly reduce it over several releases.

Your policy on this technical debt is:

• This technical debt can be detected and measured by unit tests.
• You want to use unit tests to detect and measure this technical debt instead of relying on code reviews where humans can miss things.
• For each development, is acceptable for the amount of technical debt to stay the same or decrease. It is not acceptable for the amount of technical debt to increase.
• The technical debt that currently exists is expected to be slowly reduced and eliminated over several releases.

(The above may apply to only a subset of the total technical debt instead of all of it; for the purpose of the question we are considering only the technical debt where the above is true.)

There are a few approaches that I have identified, but they have downsides:

• Set up unit tests for different sections of the code that will pass when the technical debt is cleared. This has the downside that for a long period of time releases will have to be performed with failing unit tests.
• Set up unit tests that will pass if amount of technical debt is below a certain threshold. This has the downside that if the technical debt level is decreased, it is possible for more technical debt to "sneak in" before the threshold is reduced.
• Set up unit tests, but whitelist certain objects or sections of the code so that the tests always pass for the whitelist. This has the disadvantage that if a development interacts with the whitelist, the tests will pass even if the developments make the whitelist objects worse.

What is the best approach to take in this situation?

Answer 1

Technical debt takes on many forms not detectable by unit tests.

For example:

• API function names written in Swahili. Team only speaks Esperanto.
• Codebase uses KORBA. No one supports KORBA.
• Contractor provided serializer API works flawlessly but doesn't follow style guide.

You may be thinking, oh wait, style guide! That we can check for conformity with a static analysis tool. Well no. Because any decent style guide asks for more then just a green light from a plug in. Relieving technical debt is about making humans happy. Not the CPU.

You may find some ways to automate tests for technical debt. But you're largely missing the point. The best tool to fight technical debt is a code review. If someone goes the extra mile while fixing their ticket and they just happen to change the names of some functions to things you understand tell them thank you.

If you do succeed in building a suite of automated technical debt tests please understand that what you’ve created is also code. Code that will have its own technical debt. So be careful how deep down this rabbit hole you go.

Technical debt should always been seen as a judgment call. There’s no perfect score. There’s good enough and there’s “aren’t you done yet?”

---

Come to think of it, there is an automated test for technical debt that provides a very useful metric without incurring much debt of it's own. I've used it for years. Goes like this:

grep -r "//TODO" . | wc -l

Answer 2

This sounds rather like SonarQube, which lets you set policies for both test coverage and number of automatically detectable "code smells".

SonarQube can also integrate with revision control so it can tell whether a line is "new code".

This has the downside that for a long period of time releases will have to be performed with failing unit tests

This is generally considered very bad - you make the "fails unit test" signal useless, so people have to check the details of which fail it was every time.

Set up unit tests that will pass if amount of technical debt is below a certain threshold

This is a pragmatic policy that will allow you to get things done. At the same time, set up a recurring event every few months to go back and check the number. Decide what the target is, how quickly to move towards the target, and how much time you can devote to improving issues.

Set up unit tests, but whitelist certain objects or sections of the code so that the tests always pass for the whitelist

This may be necessary if certain areas are particularly bad and can't be fixed, but again you probably want to schedule a review to come back, fix the issues, and remove the whitelist.

Answer 3

This answer is a frame challenge to the quality of your proposal, since you are inherently asking us to vet your proposed solution.

The proposed solution suffers from what I can only infer to be naivete. This is only an educated guess, but your idea makes me think of the kinds of ideas me and close friends of mine put forward when we were in-between the junior and mid-level stages of our developer careers.

I make this remark in relation to things such as:

• The assertion that unit tests are somehow an adequate observation tool for technical debt (see comments already made by others), and the fact that you ran with this idea before first confirming its correctness, viability, or specific scope (you mention that it might not apply to all debt, but in absence of any concrete evidence I can only conclude that this is an assumption on your part, not something you've verified and identified).
• The omission of any consideration for the complexity of expressing said technical debt as an "amount" (what unit would you use?)
• The notion that one can somehow declare and immediately enact that the accrual of technical debt should cease at a chosen point in time - in a team/company that until this point has operated in a way that technical debt was consistently generated and ignored.
• The omission of any intent towards tackling the business development processes that led to the technical debt in the first place.

Technical debt is inherently defined as flaws in the code that are not directly observed by the behavior of the code, i.e. problems that are ancillary and detract from the developer quality of life.

One can make an argument that performance issues are the exception here if the requirements never prescribed an expected level of performance and the current performance level is considered "operational but annoying".

Once you prescribe what constitutes acceptable performance, and you mandate that the developers at all times adhere to it, it has become a requirement.

Semantically, this means that failure to meet these requirements can be tested (in a way that you couldn't before since it wasn't explicitly quantified), but any failure would no longer be considered technical debt - instead it would not be meeting the acceptance criteria.
You may find this a meaningless semantical distinction, but when your question talks about technical debt without any concrete example of that debt; it is impossible for readers to infer what kind of debt you are focusing on and whether it's already (or can be made to be) part of your concrete requirements.

---

The above example on performance testing aside, I feel that your current approach has considerable holes in it that detract from the value you will get out of the effort you're going to be putting in.

Technical debt is generally accrued due to a business process defect. Maybe the developers are not experienced enough and lack a mentor. Maybe the company sets deadlines too tight. Maybe the people in charge actively dismiss the validity of good practice guidelines.
Whatever the reason, it is a human process that has failed, and if you only focus on tackling the result (i.e. code quality) and not the business defect that lies at its root, then you are treating the symptom, not the disease.

In the role of development manager, I wouldn't be giving your current proposal much weight. It is entirely built on assertions that are contentious (unit tests can measure technical debt), lack forethought (the complexity of expressing technical debt as an amount), or lack root cause analysis (the business process that caused the problem in the first place).