0

I'm doing some work with my team to refactor/rearchitect some parts of our existing codebase which consists of two separate Django apps hosted in one common project repository. We're starting work on a new app and at this point want to break down the project into more useful components.

Currently, our presentation layers (Django templates and views, etc) are separated from their domain logic and DB models by residing in different paths in the code. Further, the shared code they both use exists in another distinct path.

We're considering further partitioning these parts into their own distinct source control repositories and having the CI system assemble the output project automatically. The suggestion is this:

  • Shared domain logic repository (core unit tests here)
  • App-specific domain logic repository (domain unit tests here)
  • App-specific presentation layer repository (functional tests here)
  • General app presentation layer repository
  • Host layer (out-of-the-box Django host, possibly its own repo)

It's been suggested to keep the 2 app-specific layers together. Pros to splitting them up is mainly about separation of concerns and the idea that the core logic doesn't need to know anything about how it's presented, making it good for unit tests and even a CLI wrapper. The con is that as separate repositories the commit history would now be split up and could make tracing changes more difficult.

Will splitting up a large project along these lines result in code that is easier to maintain and reuse or will it add unnecessary complexity? Are there other considerations for how to divide things up?

UPDATE:

It seems like splitting up the app-specific layers into their own repositories does not make sense so we're not going to do that. Those two parts are tightly coupled and no value seems to come from managing them independently.

The shared domain logic is a core library with code common to all projects and it already exists as an isolated piece of code. The new project is for a different part of the institution and requires being in its own repository (because the development may be taken over by another team in the future and they have no interest in the other two projects). The option is to either create a copy of the common code in the other repo OR to migrate it to its own repo and have the build process pull it in per build. Duplicating the code would work but removes the value of reuse.

The general app presentation layer is the common authentication, templates, HTML, etc used by all three projects. Again, we can duplicate it into the new repository, that is the easier thing to do but it removes more common code reuse so that when a change is made it will now have to be made in (at least) two separate locations.

UPDATE 2:

The discussion here clarified a few things:

  • Minimizing the number of distinct source control repositories is preferred.
  • If a library truly is an independent entity only then does it belong in its own repo.
  • There are other approaches which can accomplish separation and reuse while keeping things together.

With this insight the distinct parts can be broken into just two repositories. The core library with the most generic common code, and which isn't "owned" by any single project, will go into its own repository. The rest of the project(s) can remain in their own repositories. A template can be set up to quickly clone the general structure for each new project.

3
  • 1
    nooooooooooooooooooooooo this will just add unnecessary complexity! And you'll never be able to figure out which versions of each thing are supposed to go together. – user253751 Mar 23 at 18:05
  • I'd think the CI build process (Jenkins in our case) could manage the dependencies. Each repo is built as a Python package consumed by the subsequent higher-up package repository. – Nick Gotch Mar 23 at 18:31
  • 1
    If you don't create generic component for reuse don't bother having specific repos. When you describe a "layer" it means that there's always coupling. Each layer is coupled to the layer beneath. One thing that springs to mind is to create multiple projects into one repo and let the application be the composite of those modules. But those projects will depend on each other and add complexity you don't need and it doesn't solve any real problem – Bart Mar 23 at 18:38
2

You wrote

shared domain logic is a core library with code common to all projects and it already exists as an isolated piece of code

Ask yourself, how much isolation does this lib have:

  • does it have its own life cycle, apart from the projects / programs / layers using it?

  • is it developed, tested, versioned and deployed completely on its own, and the other projects use only those stable, tested and deployed versions, no intermediate ones (like a third-party lib)?

  • could there be a team on its own which is responsible for this library alone (like a third-party vendor), not directly involved in the development of the other projects?

If you can answer all of these three questions with "yes", then it makes sense to put the core library into a separate repo. Otherwise, do yourself a favor and keep everything in one.

4
  • The issue with keeping it in the same repo is that the new project being developed needs to go into its own repo because it is for another division of the institution. If we don't make this library into a standalone package the only other way to share this code with the new project is to copy it over. Maybe that is the answer; it just seems like a waste to have to maintain the same core logic in more than one place. – Nick Gotch Mar 24 at 0:30
  • My answers here would be yes, yes, and no (the core library will receive contributions from those developing each individual project). – Nick Gotch Mar 24 at 0:35
  • 1
    @NickGotch: when I wrote "the other projects use only those stable deployed versions", I really meant "used in development, comparable as if the lib way developed and deployed to you by a third party vendor". So none of the individual projects will ever require to use intermediate, non-released versions of that lib. If that's the really case, the lib can be handled as a product of its own, then a separate repo makes sense. Note, when you manage it this way, you will copy the release lib versions, always, into each of the individual projects. Still you maintain it in one place. – Doc Brown Mar 24 at 5:48
  • Okay, got it, this makes sense. We'll make a package from the one core functionality library (a Python package) and have the two(+) other projects import it during the build through the requirements file (it'll be internally hosted but that's a technical detail we can work out). Great, thank you, I think I know the way to go. – Nick Gotch Mar 24 at 12:42
3

We're considering further partitioning these parts into their own distinct source control repositories and having the CI system assemble the output project automatically. The suggestion is this:

I would stick with a single repository in this case because it sounds like these separate repos must stay in sync with each other.

Pros to splitting them up is mainly about separation of concerns and the idea that the core logic doesn't need to know anything about how it's presented

What does the repo structure have to do with separation of concerns? How will pulling code into different repos affect your current issues with separation of concerns? You don't need separate repos to ensure the business logic and ui logic are separated. The code can still be heavily coupled even in separate repositories.

Will splitting up a large project along these lines result in code that is easier to maintain and reuse or will it add unnecessary complexity?

Again, how does the repo structure affect maintainability? If your code is already highly coupled and unmaintainable how will pulling into different repos change that?

Currently I see no advantages to separating the repo given your post. I see several disadvantages caused by the added complexity.

Are there other considerations for how to divide things up?

Yes. You should divide repos by what is expected to be out-of-sync. An easy way to tell is what things have separate release schedules. For example, back-end and apps are almost always separate repos on separate release schedules.

You've used the words Shared domain logic which suggest there is significant shared logic in your codebase. It sounds like this shared domain logic will affect the release schedule of client specific code.

EDIT in response to comments

so the idea is to pull those out into separate repos which the CI can build as a single output

If it will produce a single output then it needs to be one repo. This means for sure the repos need to stay in sync.

4
  • We'll have 3 independent projects with completely different customers, needs, and domain logic. These seem like things that should be separate so that when we build one project we aren't forced to update the others. The common logic used between them is what we want to share, so the idea is to pull those out into separate repos which the CI can build as a single output. – Nick Gotch Mar 23 at 18:27
  • @NickGotch - if it's a single output it needs to be a single repo. – sevensevens Mar 23 at 18:37
  • To give a little more background, this is a university and the code is educational technology. The idea is to take the core code which has an understanding of students, courses, etc, and put that in a repository which can be built into a Python package consumed by each course-specific application. There are definitely parts of the domain that are common across all projects and a lot else which isn't. – Nick Gotch Mar 23 at 18:39
  • I'm definitely getting the feeling the 2 app-specific layers (domain logic and presentation) should stay together. There may be a case when we'd want just one part of it but I don't think that's enough to justify separation. – Nick Gotch Mar 23 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.