Suppose a large-scale project is being developed in Python 3.7. Some layered architecture is chosen: "clean architecture", "onion" or "hexagonal". The Dependency rule in it only allows inward-looking dependencies; http://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html.

In some languages this rule is supported by build tools (e.g., gradle, maven or sbt), which define explicit dependencies between sub-projects; e.g., https://stackoverflow.com/questions/27984854/enforcing-layered-architecture-in-java. This, however, doesn't seem possible in Python 3. The only solutions I've come up with, like a separate Python project or a microservice per layer, seem overly complicated.

Specifically, suppose the Application layer can depend on the Domain layer, but not vice versa.

Questions:

  • What can be a solution in Python 3.7 to maintain this dependency structure?
  • How could modules in Domain be technically disallowed to import from Application modules ?

Extra: My current solution is to emulate sub-projects by defining top-level packages APP and DOMAIN. This capitalisation serves as a flag in import statements.

  • If you do find a way to enforce this within a module, know that it would be distinctly unpythonic. Among many other encapsulation-breaking things, Python gladly lets anyone inspect "private" data in a class because, as the credo goes, "we're all consenting adults here". – Alex Reinking Nov 14 at 14:49
  • 1
    The most practical way of doing this would be to separate the code into multiple modules/projects, though. Code review would ensure that the projects do not reference each other in an illegal way. – Alex Reinking Nov 14 at 14:51
  • So there would be e.g. a separate PyCharm project per layer? – Tupolev._ Nov 14 at 14:53
  • 1
    It's good to leverage the available tools to some extent, but rather then relying solely on mechanisms that could help enforce the architecture, think about how to communicate the intent and the reasoning behind it to the developers working on the project, because, regardless of the language, you can't really enforce everything you'd like - a well-meaning developer that needs to get stuff done can probably find a way around it. What will make a difference is the underlying understanding - if a developer can evaluate the impact of a change, then he/she can make informed decisions about it. – Filip Milovanović Nov 14 at 16:07
  • @FilipMilovanović Probably, "enforce" is not a good term. I've edited the post. My key concern is how to preserve a clean architecture and avoid a Big Ball of Mud on a large-scale Python project. – Tupolev._ Nov 14 at 16:31
up vote 1 down vote accepted

Using separate projects would be the only way to enforce this separation, but it might be sufficient to simply guide the programmer away from unintended dependencies.

Python here has the problem that it is a dynamic language: you can depend on an interface without ever mentioning it, as everything is duck typed.

Possible strategies include:

  • taking extra care to perform dependency injection rather than importing functionality, and then testing your code by replacing the outer layers with mocks and test drivers. This will come naturally if you're trying to follow a BDD-ish approach.

  • making the implicit interfaces explicit, by starting to use type annotations with a type checker such as MyPy. You can then write stub files for your external interfaces, and the type checker should complain if you happen to depend on functionality that wasn't mentioned in the interface. Unfortunately, annotating the source code to a level where the type checker can work well takes a lot of effort, especially since many variables will have the Any type that is not subject to further checks.

  • I agree - it probably makes more sense in Python to rely on conventions and testing. If the project grows too large, then technical solutions, like microservices and separate projects, could be added. I've also found a Python tool, github.com/seddonym/layer_linter, currently in beta, for checking dependencies between program layers. – Tupolev._ Nov 16 at 11:45
  • @Tupolev._ While linters can help, looking at the import graph is not quite good enough: you might also have implicit dependencies via function arguments. You will have to mix and match various approaches so that they make sense for your circumstances, and possibly at some point the best approach is to not use Python (but e.g. C# or some other statically typed language). Note that microservices imply substantial complexity, and that simply dividing your layers into separate library projects may be all that is needed. – amon Nov 16 at 12:11

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.