How to comprehend abstraction in code?

Question

When looking at a new codebase I like to start from a bottom-up approach.
Where I comprehend one file and then move up to the next abstraction.
But often times I find myself forgetting what the lower-level abstraction is doing.

So I'll be at this point where I find myself in an almost endless loop of going back to the files that I've previously fully comprehended, and then trying to relearn them; whilst trying to juggle numerous other abstractions that connect to each other in my head.

Is there a better strategy for dealing with this situation?

Should I just forget about the lower-level details and take them as a given? But even then, many times a previous understanding of the lower-level abstraction is needed to understand what the current abstraction is doing.

Answer 1

Programming concretely is the impulse to pull details towards you so you can nail them all down in one place. We all start this way and it's hard to let go.

Programming abstractly is most definitely "forgetting about the lower-level details". Sometimes even high level details. You push details away and let something else deal with them. The sneaky thing is you've been doing this all along. Do you really understand what all happens between print "Hello world" and it showing up on your screen?

The number one thing to demand as you struggle to let go of these details is good names. A good name ensures you will not be surprised when you look inside. This is why you weren't surprised that print put something on your screen and didn't really care how. foo "Hello world" would have been a different story.

Also, levels of abstraction should be consistent. If you're at a level that is about calculating pi you shouldn't also be worried about how to display pi. That detail has leaked into an abstraction where it doesn't belong.

Lower, higher, or sideways, details, that aren't about the one thing I'm thinking about in this one place, can either go away altogether or at least hide behind a good name.

So if you're really struggling bouncing from file to file I'll lay odds someone has stuck you with bad names or leaky abstractions.

I fix this by reading with my fingers. Once I have decent tests around this mess I tease responsibilities apart, give them clear names that avoid surprises, and show it to someone else to make sure I'm not living in a fantasy world.

Apparently I'm not alone when it comes to working this way:

Whenever I work on unfamiliar code I start extracting methods. When I do this, I look for chunks of code that I can name - then I extract. Even if I end up inlining the methods I’ve extracted later, at least I have a way of temporarily hiding detail so that I can see the overall structure.

Michael Feathers - Orange Code

Answer 2

At the bottom, there are some updates to how this fared for me every quarter of the year or so, I think they're valuable.

Good naming. Or, if it's someone else's code, trying to attribute good names / responsibilities based on even bad names on that system's classes / functions so it makes sense in my head. Once it does, the low-level implementations become way easier to remember.

That's all I have. There are a lot of purists on this site who will swear-by god-knows what patterns or objects of whatever types, but good naming will get you far. I've done more than well by myself by creating minimally documented / well-named / well decoupled code and it never came back to bite me, even if my code was used in a lot of places, by a lot of people, but the one thing I did right was waste a lot of time on good naming, good comments and schematics that explained the flow of my code. Low-level implementation is necessary to understand if you desire to expand on my code in a deep way. Well written code can be expanded in reasonable ways, so, it's ok that someone or you don't understand / remember the low-level implementations.

If you're interested in a bit of controversy that people in my original field as well as me know to be the truth, but, if you listen to what is written down here, you'll learn to both agree and disagree to this answer, read on ahead:

---

But there's another issue at hand here - purists. You'll hear well-worded answers and ideologies that are reasonable and completely logical, in fact, there's nothing wrong with them. But you don't have to follow them, in fact, they might be playing into your disadvantage.

My friends worked with big systems and they just laugh off people who care a tad bit too much about conventions and patterns and for good reason, I'd do it too - I can find my reasoning for this from my main field of data analysis, since I'm not such an experienced developer: Most of the things you think matter, don't matter and there's a strong correlation to your ego in this sense. Often times an individual, due to his ego, will have obtained knowledge that he most likely misunderstood due to his biases that are now re-inforced by the an authority that he thinks just said "the same thing I did". This is a very well known trap which you should never fall into. This doesn't mean he's not using it rightly or for the greater good, but often times, what these people will do is promise that whatever they're saying is the golden prize.

So what can you do?

Explain your code to a co-worker and ask them if it makes sense from a high-level point of view.

That's all that matters. Of course anyone that's reading someone else's code will always have an alt-tab fiesta to see certain things' implementation, but that doesn't matter, if whoever's reading your code has the high-level understanding of your system and understands "why things happen" (again, without necessarily knowing, fully "how they happen"), then you're golden.

This is not me saying go ahead and write crap code that's not performant or doesn't respect anything, but what I'm saying is:

1) It's okay to forget. In time, you'll get better at reading code you're working with. If the code you're reading demands you know the low-level implementations at a good level, then it's code badly written and it plays into what I said before: does a co-worker understand you?

2) The world is filled with a lot of very intelligent people who are not very smart. They're also often times very emotional and they're prone to bias-reinforcing from outside forces. They are very good at what they do, but what they, as actors of spreading information forget is: ideas / information, even if backed by "logic" have the context of the one sending them, which is crucial in understanding whether or not that information is useful to you too. What makes sense for you could make sense for others and they'd love it but information shouldn't be taken as absolute and one, again, should consider, or at least try to figure out the context where it came from and check against his own context to see if it matches. It's really the same as billionaires giving us these "bits of knowledge to get ahead" - it surely is easy to say the things they say, but harder to implement, of course, this is a dumb example but you get the idea.

In short: write code that's understandable and realize that it's still debatable where we need as many patterns / classes and refinery as some say. There's very smart people on both sides of the argument and it should only reinforce the idea of doing whatever works for your team in a reasonable manner -- don't be stuck on small details that don't matter, you'll figure them out later, remember, you live in an extremely competitive world where timing is the most important thing:

Timing in startups success.

Allocate your time and resources in a meaningful, greedy way.

---

Here's an edit, 6 months later:

It's been an insane journey. I never thought that just separation / good naming & documentation can basically allow you to plug anything in & out of your codebase. I had to re-write a lot of code to bring it up to speed with the new changes and I did a good chunk of it in 2-3 days. I can safely say that I didn't follow SOLID everywhere due to lack of knowledge, nor best practices and I can tell they're in my technical debt, but not by a lot. Separate, name well & document, it'll allow you to change code in no time when you eventually realize how dumb you were.

Don't misunderstand me: if you write your code tightly coupled, you'll be in for a lot of pain, whether or not you hate SOLID, even understanding & applying it at a base level allows for great decoupling which, honestly, is the only thing that OOP really helps with. OOP was supposed to also be about code re-use and while that happens here and there, you don't really get to re-use a lot of objects you create, so, focus on making sure your system is well-separated. Once you reach maturity and let's assume Uncle Bob comes and takes lead on your project, he'll say "Ok, this is dumb as hell but at least everything is separated, well named & documented so at least I know what this is all about" (I hope). For me, it works. My LOC constantly changes, but at the time of writing, it's 110k lines of code, 110k lines of performing code that works in harmony for a single person is a lot.

---

Here's an edit, 3 months later, on an 8 months code I'm refactoring:

It all makes sense. I can now take what I wrote back then, conceptually and reforge the code anew, with new ideas because I perfectly understand what's going on and why it works because of the schematics / good naming and comments. I wrote some code a long time ago that I didn't care about naming well and such and it's pain to go through. I'm now thinking what the next step of explaining my code could be.

Answer 3

Some added thoughts to supplement the good answers which are already here:

Our brains are very, very good at remembering information which is meaningful to us, and very, very bad at remembering random, disconnected, meaningless data. So when reading code, you will retain far more if you understand not only what the code is doing, but why.

If the author left behind comments or other code-level documentation, you are lucky! If the author is available for consultation, even better! (In many cases, the author's e-mail address will be available. Don't be afraid to write people you don't know personally and politely ask if they are willing to answer questions; they might say yes!)

If the project's source control history is available, that is also a helpful tool. Make good use of git blame (or the like) to find out when an obscure passage was added and how it was modified. Often, the history will clearly reveal why the code is the way it is.

If you have none of the above, you are in the position of an archaelogist who has discovered some unfamiliar artifact from a lost civilization and must infer what its maker was thinking. This is an arduous process; don't expect it to be fast or easy.

What is worse is that sometimes, there may be no good reason why the code is written a certain way. While you should never jump to that conclusion, it may be the case. Perhaps the author just wrote the first thing that came into their mind. Perhaps the code was edited multiple times by different people, with very different styles and intentions. It may even turn out (and I have seen this more than once) that the code only ever worked "by accident"; in other words, the way it works is not the way the author intended it to work.

In such cases, you are in for a rough time. Do not expect that you can read the code through once as you would read a newspaper. You will have to go back and forth many times.

Answer 4

How it should work: You have a class or some classes, and they have some documentation, and you just trust that they work as described by the documentation. You don’t actually have to understand them.

So you look at the level that you need to understand, and you make sure you understand it. You trust that everything else just works. Only if you actually need to understand the next lower level, either because it doesn’t seem to work, or it doesn’t support functionality that you need, only then do you start reading and understanding (and possibly fixing or extending) the next lower level.

Answer 5

Consider:

1. Make a list of data structures and the abstraction levels that modify those data structures.
2. Make diagrams that show the relationships between abstractions.
3. Make a list of the abstractions with short descriptions (perhaps one sentence) for each abstraction.

This will give you a quick reference for remembering the layers of abstraction, and what talks to what.