A metric to measure the cost of cut and paste programming

Question

I'm hunting a metric to measure the cost of cut-and-paste programming. We already use tools to detect duplicated code - but if we don't clean up our duplicated code, how much will it cost us?

We know that the DRY principle, Don't Repeat Yourself, is a core principle of Extreme Programming. How can we actually measure the impact of non-DRY code on our projects?

I'm referring to the simplest duped code problem of the same expressions in two or more places. Other cases of duplication such as the same logic written differently in different places is trickier to detect.

I'm hunting a metric, a solid number that I can offer management. I am aware of the "WTF" metric of Robert Martin, but that is not precise enough.

What is the purpose of insisting on a metric? Management is insisting on this metric and they won't be dissuaded. I'm really afraid that they may mandate that duplicated code be reduced on each sprint. Because we are so pressed to deliver, developers will find the duplicated code and hack it so it does not get detected. This would be a ticking time bomb!

Answer 1

Duplicated costs extra effort to understand, to maintain (e.g., modifying the duplicates), and to find (you can't fix the duplicates if you don't know where they are).

I have experience building duplicate code detection tools (see bio). My tool has found 10-20% duplication (sometimes a LOT more) in pretty much every big system it is pointed at. This means every programmer is looking at 10-20% more code than is technically necessary.

If you believe programmers spend 50% of their time just looking at code, and 10-20% of that is unnecessary, then some 5-10% of a programmer's time is wasted by the existence of cloned code. If your programmer costs $100K burdened, that $5K-$10 per year per programmer. If have 10 programmers, that's $50-100K/year of wasted engineering costs and corresponding delay in delivery of useful results.

Clone detection tools seem pretty valuable from this point of view. It does take effort to make the clones go away, so the savings from having a clone detection isn't quite as high.

OTOH, it is really hard to find a software engineering metric that suggests you can get any obvious percentage improvement by doing things differently. (Software Reuse arguably delivers 20-30% improvement). Clone detection/removal is up there pretty high in general.

YMMV.

Answer 2

So I was going to comment, instead of answer, but then I decided I wished to be a little more verbose.

Management insisting on reduced duplication in each sprint? Sign me up! I am usually fighting with them to allow us to spend time doing things like that, reducing technical debt sprint by sprint rather than simply piling it all up for some kind of bug-stomping fest, or god forbid, a massive production issue later!

You said sprint, so I am going to make a few assumptions here. I am a Certified Scrum Master, for what that may or may not be worth to you. So, Sprint: 1) you are using SCRUM or a similar framework. 2) You have planning meetings for each sprint, and commit to work in these planning meetings, after estimating it as a team.

You say you are pressed for time. Do you have an overall deadline that MUST happen, regardless of the team's commitment each sprint and how much they can deliver? If not, then when you estimate your stories at each sprint planning, try to take into account the cost of doing it 'right', IE not duplicating the logic in question, and instead wrapping it in some form of pattern that will make it reusable. Not only will you make management happy, but you will make yourselves happier as well! No more changing multiple copies of things when a business rule changes. No more wondering what where you have to go to find where you did it before, instead its part of an ongoing centralized library of tools at your disposal.

The key is: Nothing is free. If you have significant duplication (and I would say you do if even the management is seeing it as a problem), then explain that it takes time to fix that. The only way you have the time to fix it correctly is if they back off a bit on the time schedules, and let you do it. If you have control over commitment (you as in the developers) then commit to what you can do, realistically, and properly! If you don't have control over comitting to work (I have been there.) then talk to the PO or whomever DOES control the roadmap and release dating and explain the situation, the likely problems from not fixing it, and get them to agree.

If all of that fails, and management insists on duplication being reduced, but doesn't also guarantee the extra time that is needed to do it, you are between a rock and a hard place. The only thing the developer in me has for that is 'take the time you need to do it right'. I have never ever ever been reprimanded, by even my most unreasonable managers, for being late but getting it done right, over getting it done on time but having it blow up in a customers face. Sounds like now is the time to regroup and talk about it though, regardless.

Answer 3

Two problems here.

One is the total lack of any reliable metrics. Lines of Code can be measured accurately -- but is a meaningless and unusable figure, Total Cost of Ownership is meaningful and useful but can only be measured after the patient is dead.

Second. Re-factoring and reducing repetition that has crept into your could base is a real cost now. Any costs associated with maintaining a bloated code base (and if its well tested with stable requirements these costs may be zero) belong in the future and, probably, to another manager.

While you should strive to avoid writing the same code again and again (if only to save wear and tear on the keyboard) once its there you are pretty much stuck with it. Its almost always the right management decision to favor working and tested code over "pretty" code.

Answer 4

Prediction of the future is always difficult and I am not aware of any precise formula to calculate the cost of code clones. What we do in our code quality audits is that we calculate the clone overhead by putting the size of the system in relation to the size of a hypothetical variant of the system without any clones. That allows you estimate how much time or money you could save if there were less clones. But please note that this is just a very rough estimation and I think it's way not precise enough to be used for defining any quality goals.

The question remains of how to calculate the size of the redundancy-free system. This is in itself an estimation but can be done quite straightforward. For each clone class (code fragment and its copies), remove the size of the copies from the original system size. In addition you have to be aware of overlapping copied fragments. Any more detail here would require me to know more about the clone detection approach you use.