How to profile before optimising for readability

Question

When optimising for speed, profiling first can help focus the effort on the parts of the code that bring most benefit. Since speed can be measured objectively, the benefit can also be measured objectively by profiling before and after making a change.

Is there any analogous approach when optimising for readability? I'm not expecting to find a method of saying objectively that one version of the code is more readable than another. Measuring by profiling before and after a change is therefore out of scope. Instead I'm looking for ways of identifying parts of the code that would most benefit from an improvement in readability. How to make any improvement will still be a human judgement call.

For example, even if a section of code is already more readable that other sections, it may still be the most beneficial place to make a further readability improvement if it is read more often.

In thinking this through I've considered the following as possibilities:

• Measuring how many other parts of the code depend on this part
• Measuring how frequently this part is edited
• Measuring how frequently parts of the code that depend on this part are edited
• Measuring how much time is spent per change to this part of the code

What I'm trying to indirectly measure is "How much time is spent reading this part of the code?". I'm trying to think of indirect approaches because I can't think of a currently practical way of directly measuring how long is spent reading code. I want ways of estimating this using data that is already being recorded, rather than something impractical/intrusive like setting up eye tracking for everyone who works on the codebase.

I'm assuming that code that is read often is worth making more readable, and that code that is read slowly is worth making more readable. Estimating the total length of time spent reading each part of the code seems like a good way of capturing both cases.

Measuring which parts of the code are dependent on which other parts requires knowledge of the specific programming language(s) being used. Measuring how frequently part of the code is changed or how much time is spent per change could potentially be done in a language agnostic way using version control data. Time spent per change probably can't be measured in any meaningful way for individual changes, but I include it in case there is some way of estimating it as a long term average (perhaps from the average lifetime of raised issues that went on to involve a change to that part of the code?).

I can imagine version control based results being very tailored to the particular team that happened to be working on the code during the period of measurement, but if the team working on the code in future will be mostly the same people I can see this as being useful. In some cases tailoring to a specific team may even be preferable.

My guess is that building such a profiler will be more effort than it is worth for an individual team working on a specific project. Ideally I'd like to find an approach that can be as widely used as possible, making it worthwhile as an open source project to be used by anyone. For this reason I'd lean more towards language agnostic approaches, but I'm open to arguments that a language specific approach would be worth the effort.

Answer 1

The idea is compelling, but I think it needs some more work to find valid measurements for readability.

To understand a piece of software, you need to understand the complete code dependency tree from the entry point down to the lowest level of your supporting libraries. Of course, reading through the complete tree isn't practical because this easily sums up to millions of code lines.

I think we have to distinguish between two different aspects of readability:

Implementation Readability

That's what comes to your mind first. When you read the implementation of a method, how easily do you understand what the code does?

This has some importance, as it can reduce the "understanding time" of a single piece of code by some factor of maybe 10 or so. But if you still have to traverse the complete tree, it's not enough to make the job feasible.

Abstraction Readability

When you see a method call, how easily do you understand what this call does without looking into the method's implementation?

That's the real time-saver because it cuts the code-traversal tree. A good abstraction allows you to ignore the implementation of a call (and sub-implementations and so on), assuming that this call does its job. With good abstractions, you can understand the complete sub-tree (as far as it's important for your application layer) without completely reading through it. E.g. HashMap.put(key,value) is a good abstraction where using it never created the need to look at its implementation (and the implementations it depends upon and so on).

Discussion

Let's discuss indicators and their relation to Implementation and Abstraction Readability:

Measuring how many other parts of the code depend on this part: This indicator increases the probability that this code is relevant for my current task, so it aggravates any abstraction or implementation readability problem here. In itself, it's a sign that this code is useful enough to be called in many places, so the abstraction can't be completely weird.

Measuring how frequently this part is edited: This assumes that editing is (at least partly) done because of bad readability, to improve readability. If it's a local change, it can only affect implementation readability. Edits meant to improve abstraction readability tend to touch many distinct places (e.g. one method definition and many method usages). However, the edits just indicate that some older version suffered of bad readability, and don't tell anything about the current version's readability. One type of edit I'd immediately correlate with poor implementation readability is adding comments or renaming variables without changing the core. That's something I often do when trying to understand some piece of code.

Measuring how frequently parts of the code that depend on this part are edited: This can be a hint that the abstraction implemented here is easily misunderstood by the developers, so it's a (weak?) sign for bad abstraction readability.

Measuring how much time is spent per change to this part of the code: Of course, time is influenced by readability, but I guess that other factors like complexity of the task solved in this code dominate.

I'd like to suggest one measurement that IMHO best captures abstraction readability:

Measuring how often a developer navigates to the source of a given method call (e.g. F3 key in Eclipse). At least 90% of the times I use this key, I do this because the call wasn't self-explanatory, indicating a poor abstraction readability.

Answer 2

How about encouraging people to add a todo comment whenever they feel some code needs improvement? Maybe a button can be added to downvote a method. Then you can easily rank methods by downvotes and find the worst and most commonly viewed code that needs the most attention.