Why is it a good idea for "lower" application layers not to be aware of "higher" ones?

Question

In a typical (well-designed) MVC web app, the database is not aware of the model code, the model code is not aware of the controller code, and the controller code is not aware of the view code. (I imagine you could even start as far down as the hardware, or perhaps even further, and the pattern might be the same.)

Going the other direction, you can go just one layer down. The view can be aware of the controller but not the model; the controller can be aware of the model but not the database; the model can be aware of the database but not the OS. (Anything deeper is probably irrelevant.)

I can intuitively grasp why this is a good idea but I can't articulate it. Why is this unidirectional style of layering a good idea?

Answer 1

Layers, modules, indeed architecture itself, are means of making computer programs easier to understand by humans. The numerically optimal method of solving a problem is almost always an unholy tangled mess of non-modular, self-referencing or even self-modifying code - whether it's heavily optimized assembler code in embedded systems with crippling memory constraints or DNA sequences after millions of years of selection pressure. Such systems have no layers, no discernible direction of information flow, in fact no structure that we can discern at all. To everyone but their author, they seem to work by pure magic.

In software engineering, we want to avoid that. Good architecture is a deliberate decision to sacrifice some efficiency for the sake of making the system understandable by normal people. Understanding one thing at a time is easier than understanding two things that only make sense when used together. That is why modules and layers are a good idea.

But inevitably modules must call functions from each other, and layers must be created on top of each other. So in practice, it's always necessary to construct systems so that some parts require other parts. The preferred compromise is to build them in such a way that one part requires another, but that part doesn't require the first one back. And this is exactly what unidirectional layering gives us: it is possible to understand the database schema without knowing the business rules, and to understand the business rules without knowing about the user interface. It would be nice to have independence in both directions - allowing someone to program a new UI without knowing anything at all about the business rules - but in practice this is virtually never possible. Rules of thumb such as "No cyclical dependencies" or "Dependencies must only reach down one level" simply capture the practically achievable limit of the fundamental idea that one thing at a time is easier to understand than two things.

Answer 2

The fundamental motivation is this: You want to be able to rip an entire layer out and substitute a completely different (rewritten) one, and NOBODY SHOULD (BE ABLE TO) NOTICE THE DIFFERENCE.

The most obvious example is ripping the bottom layer out and substituting a different one. This is what you do when you develop the upper layer(s) against a simulation of the hardware, and then substitute in the real hardware.

The next example is when you rip a middle layer out and substitute a different middle layer. Consider an application that uses a protocol that runs over RS-232. One day, you have to change the encoding of the protocol completely, because "something else changed". (Example: switching from straight ASCII encoding to Reed-Solomon encoding of ASCII streams, because you were working over a radio link from downtown L.A. to Marina Del Rey, and you are now working over a radio link from downtown L.A. to a probe orbiting Europa, one of the moons of Jupiter, and that link needs much better forward error correction.)

The only way to make this work is if each layer exports a known, defined interface to the layer above, and expects a known, defined interface to the layer below.

Now, it is not exactly the case that lower layers know NOTHING about upper layers. Rather, what the lower layer knows is that the layer immediately above it will operate precisely in accordance with its defined interface. It can know nothing more, because by definition anything that is not in the defined interface is subject to change WITHOUT NOTICE.

The RS-232 layer doesn't know whether it is running ASCII, Reed-Solomon, Unicode (Arabic code page, Japanese code page, Rigellian Beta code page), or what. It just knows that it is getting a sequence of bytes and it is writing those bytes to a port. Next week, he might be getting a completely different sequence of bytes from something completely different. He doesn't care. He just moves bytes.

The first (and best) explication of layered design is Dijkstra's classic paper "Structure of the T.H.E. Multiprogramming System". It is required reading in this business.

Answer 3

Because the higher levels maywill change.

When that happens, whether because of requirements changes, new users, different technology, a modular (i.e. unidirectionally layered) application should require less maintenance and be more easily adapted to fit the new needs.

Answer 4

IMO, it's very simple. You can't re-use something that keeps referencing the context it's used in.

Answer 5

I think that the main reason is that it makes things more tightly coupled. The tighter the coupling the more likely to run into issues later. See this article more info: Coupling

Here is an excerpt:

Disadvantages

Tightly coupled systems tend to exhibit the following developmental characteristics, which are often seen as disadvantages: A change in one module usually forces a ripple effect of changes in other modules. Assembly of modules might require more effort and/or time due to the increased inter-module dependency. A particular module might be harder to reuse and/or test because dependent modules must be included.

With that being said on reason to have a tigher coupled system is for performance reasons. The article that I mentioned also has some information about this too.

Answer 6

Layers should not have two-way dependencies

The advantages of a layered architecture are that the layers should be usable independently:

• you should be able to build a different presentation layer in addition to the first one without changing the lower layer (e.g. build an API layer in addition to an existing web interface)
• you should be able to refactor or eventually replace the lower layer without changing the top layer

These conditions are basically symmetrical. They explain why it's generally better to have only one dependency direction, but not which.

The dependency direction should follow the command direction

The reason why we prefer a top down dependency structure is because the top objects create and use the bottom objects. A dependency is basically a relationship that means "A depends on B if A can't work without B". So if the objects in A use the objects in B, that's how dependencies should go.

This is in a way somewhat arbitrary. In other patterns, such as MVVM, control easily flows from the bottom layers. For example you can set up a label whose visible caption is bound to a variable and changes with it. It's normally still preferable to have top down dependencies, though, because the main objects are always the ones the user interacts with, and those objects do the majority of the work.

While from the top down we use method invocation, from the bottom up (typically) we use events. Events allow dependencies to go top down even when the control flows the other way around. The top layer objects subscribe to events on the bottom layer. The bottom layer does not know anything about the top layer, which acts as a plug in.

There are also other ways of maintaining a single direction for example:

• continuations (passing a lambda or a method to be called on and event to an async method)
• subclassing (create a subclass in A of a parent class in B which is then injected in the bottom layer, a bit like a plugin)

Answer 7

I would like to add my two cents to what Matt Fenwick and Kilian Foth have already explained.

One principle of software architecture is that complex programs should be built by composing smaller, self-contained blocks (black boxes): this minimizes dependences thus reducing complexity. So, this unidirectional dependency is a good idea because it makes it easier to understand the software, and managing complexity is one the most important issues in software development.

So, in a layered architecture, the lower layers are black boxes that implement abstraction layers on top of which the upper layers are built. If a lower layer (say, layer B) can see details of an upper layer A, then B is no longer a black box: its implementation details depend on some details of its own user, but the idea of a black box is that its content (its implementation) is irrelevant for its user!

Answer 8

Just for fun.

Think of a pyramid of cheerleaders. The bottom row is supporting the rows above them.

If the cheerleader on that row is looking down they are stable and will stay balanced so that those above her don't fall.

If she looks up to see how everyone above her is doing, she will loose her balance causing the whole stack to fall down.

Not really technical, but it was an analogy I thought might help.

Answer 9

While ease of understanding and to some degree replaceable components are certainly good reasons an equally important reason (and probably the reason that layers were invented in the first place) is from the software maintenance viewpoint. The bottom line is that dependencies cause the potential to break things.

For example, suppose A depends on B. Since nothing depends on A, developers are free to change A to their hearts content without having to worry that they could break anything other than A. However, if the developer wants to change B then any change in B that is made could potentially break A. This was frequent problem in early computer days (think structured development) where developers would fix a bug in one part of the program and it would raise bugs in apparently totally unrelated parts of the program elsewhere. All because of dependencies.

To continue with the example, now suppose A depends on B AND B depends on A. IOW, a circular dependency. Now, anytime a change is made anywhere it could potentially break the other module. A change in B could still break A, but now a change in A could also break B.

So in your original question, if you are on a small team for a small project then all this is pretty much overkill because you can freely change modules at your whim. However, if you are on a sizeable project, if all modules depended on the others then everytime a change was needed it could potentially break the other modules. On a large project, knowing all the impacts could be difficult to determine so you'll likely miss some impacts.

It gets worse on a large project where there are many developers, (e.g. some who only work layer A, some layer B and some layer C). As it becomes likely that each change has to be reviewed/discussed with members on the other layers in order to make sure your changes don't break or force rework on what they are working on. If your changes do force changes on others, then you have to convince them that they should make the change, because they aren't going to want to take on more work just because you have this great new way of doing things in your module. IOW, a bureaucratic nightmare.

But if you limited dependencies to A depends on B, B depends on C then only layer C people need to coordinate their changes to both teams. Layer B only needs to coordinate changes with the Layer A team and the layer A team is free to do whatever they want because their code doesn't affect layer B or C. So ideally, you'll design your layers so layer C changes very little, layer B changes somewhat and layer A does most of the changing.

Answer 10

The most basic reason that the lower layers shouldn't be aware of the higher layers is that there are many more kinds of higher layers. For instance, there are thousands and thousands of different programs on your Linux system, but they call the same C library malloc function. So the dependency is from these programs to that library.

Note that "lower layers" are actually the middle layers.

Think about an application that communicates through the outside world through some device drivers. The operating system is in the middle.

The operating system does not depend on details within the applications, nor within the device drivers. There are many kinds of device driver of the same type and they share the same device driver framework. Sometimes kernel hackers have to put in some special case handling into the framework for the sake of a particular hardware or device (recent example I came across: PL2303-specific code in the usb-serial framework of Linux). When that happens, they usually put in comments about how much that sucks and should be removed. Even though the OS calls functions in the drivers, the calls go through hooks that make the drivers look the same, whereas when drivers call the OS, they often use specific functions directly by name.

So, in some ways, the operating system is really a lower layer from the perspective of the application and from the perspective of the application: a kind of communication hub where things connect and data is switched to go the appropriate pathways. It helps the design of the communication hub to export a flexible service that can be used by anything, and not to move any device or application specific hacks into the hub.

Answer 11

Seperation of concerns and divide/conquer approaches can be another explaination for this questions. Seperation of concerns gives the ability of portability and in some more complex architectures, it gives the platform independent scaling and performance advantages.

In this context, if you think about a 5-tiered archictecture (client,presentation,bussiness,integration and resource tier) lower level of architecture shouldn't be aware of the logic and bussiness of the higher levels and vice versa. I mean by lower level as integration and resource levels. Database integration interfaces provided in integration and real database and webservices (3rd party data providers) belongs to resource tier. So that suppose you will change your MySQL database to a NoSQL document DB like MangoDB in terms of scalability or whatever.

In this approach, bussiness tier doesn't care how the integration tier provides the connection/transmission by the resource. It only looks for data access objects provided by the integration tier. This could be expanded to more scenarios but basically, seperation of concerns could be the number one reason for this.

Answer 12

Expanding on Kilian Foth's answer, this direction of layering corresponds to a direction in which a human explores a system.

Imagine that you are a new developer tasked with fixing a bug in the layered system.

Bugs are usually a mismatch between what customer needs and what he gets. As the customer communicates with the system through UI, and gets result through UI (UI literally means 'user interface'), the bugs are reported in terms of UI too. So, as a developer, you don't have much choice but to start looking at UI too, to figure out what happened.

That is why having top-down layer connections is necessary. Now, why don't we have connections going both ways?

Well, you have three scenarios how that bug could ever occur.

It could occur in UI code itself, and so be localised there. This is easy, you just need to find a place and fix it.

It could occur in other parts of the system as a result of calls made from UI. Which is moderately difficult, you trace a tree of calls, find a place where the error occurs, and fix it.

And it could occur as a result of a call INTO your UI code. Which is hard, you have to catch the call, find its source, then figure out where the error occurs. Considering that a point you start at is situated deep down in a single branch of a call tree, AND you need to find a correct call tree first, there could be several calls into the UI code, you have your debugging cut out for you.

To eliminate the hardest case as much as possible, the circular dependencies are strongly discouraged, layers connect mostly in top-down fashion. Even when a connection the other way is needed, it is usually limited and clearly defined. For example, even with callbacks, which are a sort of reverse connection, the code being called in callback usually provides this callback in the first place, implementing a sort of "opt-in" for reverse connections, and limiting their impact on understanding a system.

Layering is a tool, and primarily aimed at developers supporting an existing system. Well, connections between layers reflect that, too.

Answer 13

Another reason I would like to see explicitly mentioned here is code reusability. We've already had the example of the RS232 media that gets replaced, so lets go one step further...

Imagine you're developing drivers. Its your job and you write quite a bunch. Protocols might probably start repeating at some point, as might physical media.

So what you'll start doing - unless you're a great fan of doing the same thing over and over again - is to write reusable layers for these things.

Say you have to write 5 drivers for Modbus devices. One of them uses Modbus TCP, two use Modbus on RS485 and the rest go over RS232. You're not going to reimplement Modbus 5 times, because you're writing 5 drivers. Also you're not going to reimplement Modbus 3 times, because you have 3 different Physical layers below you.

What you do is, you write a TCP Media Access, an RS485 Media Access and Possibly an RS232 Media Access. Is it smart to know that there's going to be a modbus layer above, at this point? Probably not. Next driver you're going to implement might also use Ethernet but use HTTP-REST. It would be a shame if you had to reimplement the Ethernet Media Access in order to communicate via HTTP.

One layer above, you're going to implement Modbus just once. That Modbus layer once again, will not know of the drivers, that are one layer up. These drivers, of course will have to know that they're supposed to talk modbus, and they're supposed to know that they're use Ethernet. Howevever implemented the way I just described it, you could not only just rip out a layer and replace it. you could of course - and that to me is the biggest benefit of all, go ahead and reuse that existing Ethernet layer for something absolutely unrelated to the project that originally caused its creation.

This is something, we probably see every day as developers and that saves us tons of time. There are countless Libraries for all sorts of protocols and other things. These exist because of principles like the dependency direction following the command direction, which allows us to build reusable layers of software.