0

Suppose you have been put in charge of an already existing project. As you are starting to familiarize yourself with the repository, you notice a few technical debt issues (insufficient test coverage, conflicting standards, missing documentation, etc). As someone who is going to be working on this project and maintaining this code, you add this to the backlog of the project with a goal to slowly reduce it over several releases.

Your policy on this technical debt is:

  • This technical debt can be detected and measured by unit tests.
  • You want to use unit tests to detect and measure this technical debt instead of relying on code reviews where humans can miss things.
  • For each development, is acceptable for the amount of technical debt to stay the same or decrease. It is not acceptable for the amount of technical debt to increase.
  • The technical debt that currently exists is expected to be slowly reduced and eliminated over several releases.

(The above may apply to only a subset of the total technical debt instead of all of it; for the purpose of the question we are considering only the technical debt where the above is true.)

There are a few approaches that I have identified, but they have downsides:

  • Set up unit tests for different sections of the code that will pass when the technical debt is cleared. This has the downside that for a long period of time releases will have to be performed with failing unit tests.
  • Set up unit tests that will pass if amount of technical debt is below a certain threshold. This has the downside that if the technical debt level is decreased, it is possible for more technical debt to "sneak in" before the threshold is reduced.
  • Set up unit tests, but whitelist certain objects or sections of the code so that the tests always pass for the whitelist. This has the disadvantage that if a development interacts with the whitelist, the tests will pass even if the developments make the whitelist objects worse.

What is the best approach to take in this situation?

8
  • 2
    How would a unit test measure technical debt? Apr 17 at 14:57
  • @GregBurghardt To give an example, suppose you have a wrapper function for a database call that will verify the database and object, log the query and cache the result. It is possible to write tests that will check which objects are using the wrapper function and which are calling the database directly.
    – EdG
    Apr 17 at 15:13
  • 3
    When you force a team to primarily work on "technical debt" issues which can be detected by some automatic tests and/or static analysis tools, chances are high they will start to ignore the interesting and often more important technical debt issues: semantical issues which are based on some human understanding of the code base.
    – Doc Brown
    Apr 17 at 16:01
  • @DocBrown This wouldn't necessarily be the primary technical debt that is worked on, but I feel that if it is possible to set up automated tests and checks on technical debt, it's better to enhance the human code review process with the automated checks than to leave the human process unassisted.
    – EdG
    Apr 17 at 16:14
  • 5
    I'm struggling with the focus on unit tests and technical debt. Consider Martin Fowler's definition of technical debt and the technical debt quadrants. How do unit tests address the breath of technical debt? For example, how do unit tests help you when the technical debt is using the wrong third-party package or tool, for example? There are plenty of forms of technical debt that just cannot be covered in unit testing (or any kind of testing, really).
    – Thomas Owens
    Apr 17 at 18:11

3 Answers 3

3

Technical debt takes on many forms not detectable by unit tests.

For example:

  • API function names written in Swahili. Team only speaks Esperanto.
  • Codebase uses KORBA. No one supports KORBA.
  • Contractor provided serializer API works flawlessly but doesn't follow style guide.

You may be thinking, oh wait, style guide! That we can check for conformity with a static analysis tool. Well no. Because any decent style guide asks for more then just a green light from a plug in. Relieving technical debt is about making humans happy. Not the CPU.

You may find some ways to automate tests for technical debt. But you're largely missing the point. The best tool to fight technical debt is a code review. If someone goes the extra mile while fixing their ticket and they just happen to change the names of some functions to things you understand tell them thank you.

If you do succeed in building a suite of automated technical debt tests please understand that what you’ve created is also code. Code that will have its own technical debt. So be careful how deep down this rabbit hole you go.

Technical debt should always been seen as a judgment call. There’s no perfect score. There’s good enough and there’s “aren’t you done yet?”


Come to think of it, there is an automated test for technical debt that provides a very useful metric without incurring much debt of it's own. I've used it for years. Goes like this:

grep -r "//TODO" . | wc -l
2
  • This is not meant to be an exhaustive solution that removes the need for humans to do code reviews, it's more meant to provide an element of redundancy in case the human doing the code review is tired/distracted/other and would miss problems that can be caught by an automated method.
    – EdG
    Apr 17 at 15:58
  • @EdG: The issue isn't that automation can't help reviews; the issue is that you're relying on automation to spit out a number that you will base the validity of your approach on. That is effectively equivalent to reviewing the code. You are indirectly arguing for a process that outputs the same kind of expression of confidence that a code review would render.
    – Flater
    Apr 17 at 23:34
2

This sounds rather like SonarQube, which lets you set policies for both test coverage and number of automatically detectable "code smells".

SonarQube can also integrate with revision control so it can tell whether a line is "new code".

This has the downside that for a long period of time releases will have to be performed with failing unit tests

This is generally considered very bad - you make the "fails unit test" signal useless, so people have to check the details of which fail it was every time.

Set up unit tests that will pass if amount of technical debt is below a certain threshold

This is a pragmatic policy that will allow you to get things done. At the same time, set up a recurring event every few months to go back and check the number. Decide what the target is, how quickly to move towards the target, and how much time you can devote to improving issues.

Set up unit tests, but whitelist certain objects or sections of the code so that the tests always pass for the whitelist

This may be necessary if certain areas are particularly bad and can't be fixed, but again you probably want to schedule a review to come back, fix the issues, and remove the whitelist.

2
  • Does SonarQube offer a benefit over the second of the solutions in the original post? Having an event to remind you to check your threshold is okay, but it's not much effort to update the threshold as part of the release process and using SonarQube just for a reminder feels a bit heavy-handed.
    – EdG
    Apr 17 at 15:28
  • I think you've confused two things: SQ is a product that does various sorts of checking, and provides a dashboard with the results, while I was suggesting using a calendar for the reminder. If "every release" works for you then great.
    – pjc50
    Apr 17 at 16:20
1

This answer is a frame challenge to the quality of your proposal, since you are inherently asking us to vet your proposed solution.

The proposed solution suffers from what I can only infer to be naivete. This is only an educated guess, but your idea makes me think of the kinds of ideas me and close friends of mine put forward when we were in-between the junior and mid-level stages of our developer careers.

I make this remark in relation to things such as:

  • The assertion that unit tests are somehow an adequate observation tool for technical debt (see comments already made by others), and the fact that you ran with this idea before first confirming its correctness, viability, or specific scope (you mention that it might not apply to all debt, but in absence of any concrete evidence I can only conclude that this is an assumption on your part, not something you've verified and identified).
  • The omission of any consideration for the complexity of expressing said technical debt as an "amount" (what unit would you use?)
  • The notion that one can somehow declare and immediately enact that the accrual of technical debt should cease at a chosen point in time - in a team/company that until this point has operated in a way that technical debt was consistently generated and ignored.
  • The omission of any intent towards tackling the business development processes that led to the technical debt in the first place.

Technical debt is inherently defined as flaws in the code that are not directly observed by the behavior of the code, i.e. problems that are ancillary and detract from the developer quality of life.

One can make an argument that performance issues are the exception here if the requirements never prescribed an expected level of performance and the current performance level is considered "operational but annoying".

Once you prescribe what constitutes acceptable performance, and you mandate that the developers at all times adhere to it, it has become a requirement.

Semantically, this means that failure to meet these requirements can be tested (in a way that you couldn't before since it wasn't explicitly quantified), but any failure would no longer be considered technical debt - instead it would not be meeting the acceptance criteria.
You may find this a meaningless semantical distinction, but when your question talks about technical debt without any concrete example of that debt; it is impossible for readers to infer what kind of debt you are focusing on and whether it's already (or can be made to be) part of your concrete requirements.


The above example on performance testing aside, I feel that your current approach has considerable holes in it that detract from the value you will get out of the effort you're going to be putting in.

Technical debt is generally accrued due to a business process defect. Maybe the developers are not experienced enough and lack a mentor. Maybe the company sets deadlines too tight. Maybe the people in charge actively dismiss the validity of good practice guidelines.
Whatever the reason, it is a human process that has failed, and if you only focus on tackling the result (i.e. code quality) and not the business defect that lies at its root, then you are treating the symptom, not the disease.

In the role of development manager, I wouldn't be giving your current proposal much weight. It is entirely built on assertions that are contentious (unit tests can measure technical debt), lack forethought (the complexity of expressing technical debt as an amount), or lack root cause analysis (the business process that caused the problem in the first place).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.