Did Dijkstra intend for code modularization, when he wrote about separation of concerns?

Question

First, I read an excerpt Edsger W. Dijkstra's 1974 paper "On the role of scientific thought":

Let me try to explain to you, what to my taste is characteristic for all intelligent thinking. It is, that one is willing to study in depth an aspect of one's subject matter in isolation for the sake of its own consistency, all the time knowing that one is occupying oneself only with one of the aspects. We know that a program must be correct and we can study it from that viewpoint only; we also know that it should be efficient and we can study its efficiency on another day, so to speak. In another mood we may ask ourselves whether, and if so: why, the program is desirable. But nothing is gained —on the contrary!— by tackling these various aspects simultaneously. It is what I sometimes have called "the separation of concerns", which, even if not perfectly possible, is yet the only available technique for effective ordering of one's thoughts, that I know of. This is what I mean by "focusing one's attention upon some aspect": it does not mean ignoring the other aspects, it is just doing justice to the fact that from this aspect's point of view, the other is irrelevant. It is being one- and multiple-track minded simultaneously.

I see modern separation of concerns talk about modularizing your code. However, reading the quote above, I understand this as focusing your mind onto one particular task at a time, while not focusing on other aspects. This does not mean to me necessarily that code needs to be separated into modular chunks.

That is, say there is a code in front of you that in one file has the concepts of view, repository, controller, event handling, factory, etc. all in one file.

For a brief example, here's some code that has data access, and view (output):

$sql = "SELECT * FROM product WHERE id = " . db_input($id);
$row = db_fetch_array(db_query($sql)); 
<option value="<?=$row['id']?>"<?= $row['ver'] == $row['ver'] ? '  selected="selected"' : '' ?>>Version <?=$row['ver']?></option>

Using modern OO I could place data access in its own file using Repository pattern, the View code can go into its own file template, and I can wire those up together to communicate via a controller (or Action or Request Handler), and I can add a factory to create and wire up various dependencies. And I can have a configuration file that defines those factories. Surely it's a step away from single-file-everything.

My question on separation of concerns is like so: reading Dijkstra's quote, I got an idea that perhaps he did not necessarily mean for separation of concerns to be "modular separation of code (into files or their own functions/methods/etc)", and that he meant more so to focus one's mind upon an aspect of the program, without burdening yourself focusing on other important yet not-currently-to-be-considered aspects, regardless of whether they are physically separated in code, or not.

Why then are we burdening ourselves with physical modular code separation and design patterns? Will it be not enough to just focus oneself onto an aspect, regardless of how your code is structured?

I am not talking about writing up the most horrible spaghetti code and then only considering an aspect of it, that would likely be a burden. But in the end, what I am going towards, is, why perform the physical code separation, why split the code up into separated files or chunks (methods), when it is not necessary for mentally focusing yourself on an aspect?

Should separation of concerns remain a mental exercise, rather than physical?
In other words, should there be a disconnect between the mental (focus on) and the physical (code on paper) aspects of programming?

Answer 1

Dijkstra is making an explicit statement about how to think. Program (and process) modularization - and it's desirability - is perhaps a result of that thinking, but the key point he is making is how to assess a problem. The program is typically the solution to a problem, and by advocating a "separation of concerns", he is offering wise advice. The best example of this is perhaps "optimization". The joke was: "When planning to optimize a program, your first strategy should be: Don't." In other words, you want to focus on first making the program correct. Making it fast and fancy is a concern that should be separated - but also not completely removed either.

Answer 2

Separation of concerns is an abstract way of thinking that consist in considering separately things that do not have to be related.

Modularisation (separating unrelated group of functions into modules), encapsulation (hiding internal details of the modules), and abstraction (separating the general from the specific, and the idea from its implementation) are all means to implement this way of thinking in the domain of software design.

Answer 3

I would suggest that, whilst the paper is of historic interest, what Dijkstra meant by the term "separation of concerns" over 40 years ago is not particularly relevant today. Nowadays it is used extensively in reference to modulization.

There is a wealth of evidence that modulization is hugely beneficial and that those benefits far outweigh the "burdens" it places upon us. Whatever Dijkstra meant back then, doesn't change the fact that small chunks of code, each focused on just one thing, leads to code that is easier to write, read, understand, maintain and test.

Answer 4

I can give you a personal example of separation of concerns that I think is comparable to Dijkstra's concepts. When I analyze a particular subject matter in software I construct three views.

1. First I consider the data. The data represents the logical predicates of the problem. Classes are constructed to abstract entities in the real world, with their attributes being the parameters of the abstraction. Associations between classes represent functional mappings between the class instances. There is no code involved in the thinking at this point and no notion of any processing. Just a static view of the logic involved in the subject matter.
2. Second I consider the dynamics. Any class that has a non-trivial lifecycle is modeled as a finite state machine. This involves considerations of sequencing execution and synchronization. Again, no code just working out how things interact and sequence.
3. Third I consider the processing. Here, the actual algorithmic work that must be done upon state transitions or in other synchronous operations.

In the end, one obtains a three faceted view of the subject matter which can then be formulated as code in whatever groupings are convenient for the code itself and its maintenance. The three facets are not just a mental exercise. I produce written descriptions of all facets. Why? Because if the subject matter is large enough, I cannot hold even one complete facet in short term memory. If the subject matter is small, almost any approach will work because you can hold it all in your head.

The motivation for separating concerns is to accomodate the short term memory limitations of humans. We simply can't carry everything in our heads at once, although computer programmers tend to be more capable than most other folks at the number of concepts they can manipulate in their short term memory. To be effective, separating concerns must systematically exclude one or more aspects of a problem in order to focus on some other particular aspect. Of course, excluding one aspect does not make it disappear from consideration. There must be a means to combine all problem aspects to achieve a solution. Experience shows that often the end result of separating and recombining yields a more understandable solution than a single giant leap where many aspects might be jumbled together. This is particularly the case when the size of the problem is large or intricate.

Answer 5

Separation of concerns is a logical concept that will propagate into your code organization model no matter how you implement it. It is true that a code file is just a technical detail, one way to keep you software manageable. A single file with a good editor that allows collapsing an expansion of regions might work for you as well (for a while). Or a relational database that stores classes and methods in a parent-child fashion in separate tables could work as a storage medium. But text files are hard to beat in a world where source code needs to be

• portable
• accessible by lots of different external tools
• accessible by multiple programmers
• versionable and comparable
• scale well with operating systems that happen to be very good at file handling

The bottom line is that we humans are not very good at thinking about or dealing with different things at once. So we need a model that allows thinking about and working on one thing at a time without the danger of ruining some other part we are not considering at that time. So we build, laying one brick at a time, making sure the bricks we laid down earlier will not interfere with bricks laid down later. And if we want to change a brick later, things must not collapse. That is a model that works for our one-track minds.

This is not how fungus or algae grow though... How is that for a humbling fact?

Answer 6

Mark Dijkstra's words:

even if not perfectly possible, [separation of concerns] is yet the only available technique for effective ordering of one's thoughts

So separation of concerns is an ideal.

Code modularization is an approach for approaching that ideal for some types of concern in programming.

(Please do not confuse "module" with "source file". A module is anything that separates its inside from the outside by some interface, e.g. a stand-alone function. The burden involved can be very small.)

Answer 7

I believe the specific answer to Dijkstra's quote has been addressed, however, since you state "Using modern OO I could place data access in its own file" and ask "Should separation of concerns remain a mental exercise, rather than physical?" let me draw your attention to where modern OO principals direct us.

One should follow the SOLID principals in developing using OO. Here's a nice link for them, but, the TLDR on "separation of concerns" is mostly in the S in SOLID: The Single Responsibility Principle or SRP.

This, most definitely, is a physical exercise, not a mental one. For your specific example, MVC (or it's siblings MVVM and MVP) directs one to physically separate the concerts of Model, View, and Controller/Presenter/ViewModel into separate files. I've seen some MVVM implementations where these are implemented into separate assemblies to further constrain one's tendency to "mix up concepts".

But. It goes beyond just simple "this is a view and this is a model", if you follow Uncle Bob's view on this.

One needs to also consider the source of the requirements for any particular OO element. If you're mixing, say, what the Customer wants with what the Operations Personnel wants, you're also violating the SRP. Or, to put it as Uncle Bob does: A Class should have one and only one reason to change.

I highly recommend that you research this further using the links supplied or do a web search for "solid principles".