When to separate a project in multiple subprojects

Question

I'd like to know if it makes sense to divide the project I'm working on in two repositories instead of one.

From what I can say:

• Frontend will be written in html+js
• Backend in .net
• The backend doesn't depend on frontend and the frontend doesn't depend on the backend
• The frontend will use a restful api implemented in the backend.
• The frontend could be hosted on any static http server.

As of now, the repository has this structure:

root:

• frontend/*
• backend/*

I think it's a mistake to keep both project in the same repository. Since both project do not have dependencies between each others, they should belong in individual repositories and if needed a parent repository that has submodules.

I've been told that it's pointless and that we won't get any benefit from doing that.

Here are some of my arguments:

• We have two modules that don't depend between each others.
• Having source history of both projects in the long term may complicate things (try searching in the history for something in the frontend while you have half of the commits that are completely unrelated to the bug you're looking for)
• Conflict and merging (This shouldn't happen but having someone pushing to the backend will force other developer to pull backend changes to push frontend changes.)
• One developer might work only on the backend but will always have to pull the frontend or the other way around.
• In the long run, when it will be time to deploy. In some way, the frontend could be deployed to multiple static server while having one backend server. In every case, people will be forced to either clone the whole backend with it or to make custom script to push to all servers the frontend only or to remove the backend. Easier to just push/pull only the frontend or backend than both if only one is needed.
• Counter argument (One person might work on both projects), Create a third repo with submodule and develop with it. History is kept seperated in individual modules and you can always create tags where version of backend/frontend do really work together in sync. Having both frontend/backend together in one repo doesn't mean that they will work together. It's just merging both history into one big repo.
• Having frontend/backend as submodules will make things easier if you want to add a freelancer to the project. In some case, you don't really want to give full access to the codebase. Having one big module will make things harder if you want to restrict what the "outsiders" can see/edit.
• Bug introduction and fixing bug, I inserted a new bug in the frontend. Then someone fix a bug in the backend. With one repository, rolling back before the new bug will also rollback the backend which could make it difficult to fix. I'd have to clone the backend in a different folder to have the backend working while fixing the bug in the frontend... then trying to remerge things up... Having two repository will be painless because moving the HEAD of one repo won't change the other. And testing against different version of backend will be painless.

Can someone give me more arguments to convince them or at least tell me why it is pointless (more complicated) to divide the project in two submodules. The project is new and the codebase is a couple of days old so it's not too soon to fix.

Answer 1

At my company, we use a separate SVN repository for every component of the system. I can tell you that it gets extremely frustrating. Our build process has so many layers of abstraction.

We do this with Java, so we have a heavy build process with javac compilation, JibX binding compilation, XML validation, etc.

For your site, it may not be a big deal if you don't really "build it" (such as vanilla PHP).

Downsides to splitting a product into multiple repositories

1. Build management - I can't just checkout code, run a self-contained build script and have a runnable / installable / deployable product. I need an external build system that goes out to multiple repos, runs multiple inner build scripts, then assembles the artifacts.
2. Change tracking - Seeing who changed what, when, and why. If a bug fix in the frontend requires a backend change, there are now 2 divergent paths for me to refer back to later.
3. Administration - do you really want to double the number of user accounts, password policies, etc. that need to be managed?
4. Merging - New features are likely to change a lot of code. By splitting your project into multiple repositories, you are multiplying the number of merges needed.
5. Branch creation - Same deal with branching, to create a branch, you now have to create a branch in each repository.
6. Tagging - after a successful test of your code, you want to tag a version for release. Now you have multiple tags to create, one in each repository.
7. Hard to find something - Maybe frontend/backend is straightforward, but it becomes a slippery slope. If you split into enough modules, developers may have to investigate where some piece of code lives in source control.

My case is a bit extreme as our product is split across 14 different repos and each repo is then divided into 4-8 modules. If I remember, we have somewhere around 80 or some "packages" which all need to be checked out individually and then assembled.

Your case with just backend/frontend may be less complicated, but I still advise against it.

Extreme examples can be compelling arguments for or against pretty much anything :)

Criteria I would use to decide

I would consider splitting a product into multiple source code repositories after considering the following factors:

1. Build - Do the results of building each component merge together to form a product? Like combining .class files from a bunch of components into a series of .jar or .war files.
2. Deployment - Do you end up with components that get deployed together as one unit or different units that go to different servers? For example, database scripts go to your DB server, while javascript goes to your web server.
3. Co-change - Do they tend to change frequently or together? In your case, they may change separately, but still frequently.
4. Frequency of branching/merging - if everybody checks into trunk and branches are rare, you may be able to get away with it. If you frequently branch and merge, this may turn into a nightmare.
5. Agility - if you need to develop, test, release and deploy a change on a moment's notice (likely with SaaS), can you do it without spending precious time juggling branches and repos?

Your arguments

I also don't agree with most of your arguments for this splitting. I won't dispute them all because this long answer will get even longer, but a few that stand out:

We have two modules that don't depend between each others.

Non-sense. If you take your backend away, will your frontend work? That's what I thought.

Having source history of both projects in the long term may complicate things (try searching in the history for something in the frontend while you have half of the commits that are completely unrelated to the bug you're looking for)

If your project root is broken into frontend/ and backend/, then you can look at the history of those hierarchies independently.

Conflict and merging (This shouldn't happen but having someone pushing to the backend will force other developer to pull backend changes to push frontend changes.) One developer might work only on the backend but will always have to pull the backend or the other way around.

Splitting your project into different repos doesn't solve this. A frontend conflict and a backend conflict still leaves you with 2 conflicts, whether it's 1 repository times 2 conflicts or 2 repositories times 1 conflict. Somebody still needs to resolve them.

If the concern is that 2 repos means a frontend dev can merge frontend code while a backend dev merges backend code, you can still do that with a single repository using SVN. SVN can merge at any level. Maybe that is a git or mercurial limitation (you tagged both, so not sure what SCM you use)?

On the other hand

With all this said, I have seen cases where splitting a project into multiple modules or repositories works. I even advocated for it once for a particular project where we integrated Solr into our product. Solr of course runs on separate servers, only changes when a changeset is related to search (our product does much more than search), has a separate build process and there are no code artifacts or build artifacts shared.

Answer 2

Some of your arguments are valid and some are not.

We have two modules that don't depend between each others.

That is actually not entirely true. To be able to communicate, both front-end and back-end need to have a common interface (description). That makes it a weak argument in favour of having both in a common repository. But only a weak argument as it does not make that much of a difference.

Having source history of both projects in the long term may complicate things (try searching in the history for something in the frontend while you have half of the commits that are completely unrelated to the bug you're looking for)

This is a bogus argument. If you want to look up how a particular bug got fixed, you look in the bug-tracker for which commit contains the fix. And if you want to know how a particular piece of code evolved, you look at the history of a single file (or at most a handful). In either case, having other files, possibly from other modules, in the repository should not complicate things in any way.

Conflict and merging (This shouldn't happen but having someone pushing to the backend will force other developer to pull backend changes to push frontend changes.)

This is a bogus argument. I am not aware of any (half decent) VCS where you need to synchronize the entire repository before you can commit/push your changes. At most you need to synchronize the folders that contain the changed files (and often only the files themselves).

One developer might work only on the backend but will always have to pull the backend or the other way around.

This is the same bogus argument as the previous one.

In the long run, when it will be time to deploy. In some way, the frontend could be deployed to multiple static server while having one backend server. In every case, people will be forced to either clone the whole backend with it or to make custom script to push to all servers the frontend only or to remove the backend. Easier to just push/pull only the frontend or backend than both if only one is needed.

Depending on how people envision that the deployment will be done, this can be a valid argument. If the deployment will be done by unpacking a zip file/tarbal on the server, then it does not matter how your repositories are organized. If the deployment will be done by checking out (part of) a repository on the server, then it can be a good idea to use separate repositories for modules that get deployed separately.

Counter argument (One person might work on both projects), Create a third repo with submodule and develop with it. History is kept seperated in individual modules and you can always create tags where version of backend/frontend do really work together in sync. Having both frontend/backend together in one repo doesn't mean that they will work together. It's just merging both history into one big repo.

This is a valid argument, but it is not that strong.

Having frontend/backend as submodules will make things easier if you want to add a freelancer to the project. In some case, you don't really want to give full access to the codebase. Having one big module will make things harder if you want to restrict what the "outsiders" can see/edit.

This is a valid argument.

Bug introduction and fixing bug, I inserted a new bug in the frontend. Then someone fix a bug in the backend. With one repository, rolling back before the new bug will also rollback the backend which could make it difficult to fix.

That is a bogus argument, because it would mean that after two bugfixes to one module, you would not be able to revert the first one. Any half-decent VCS will allow you to roll back just about any old commit (although it will often mean that you make a new commit that reverses those changes, sometimes even for the top of HEAD).

I'd have to clone the backend in a different folder to have the backend working while fixing the bug in the frontend... then trying to remerge things up... Having two repository will be painless because moving the HEAD of one repo won't change the other. And testing against different version of backend will be painless.

This is actually quite a good argument. Having two repositories makes it easier to test the scenarios where the deployed front- and back-ends might become (slightly) out of sync.

Answer 3

This post is a bit old but I would like to contribute. Whilst your back-end doesn't really know about the front-end the front-end needs to have requests matching the back-end's API. If you consider your back-end as a REST API, then you could define an interface file such as a swagger YAML interface. Now there are really 3 projects, which you can individually divide into different repos as you see fit:

• API definition
• Back-end
• Front-end

The API definition is a dependency in the other two projects, lets say you were using maven as a dependency injection tool. Then it depends how strictly you want to do versioning. You could bump up the version of the API definition project every-time you make a change to ensure the projects are always in a compatible state but requires more overhead, or you can use something like SNAPSHOTS in maven, and only do the versioning once you are happy with the interface which is less overhead but you may often have incompatibilities. But as long as you enforce the API definition in your front and back-end you will be fine splitting up the projects into different repositories.

These issues are more about managing dependencies. Even if the projects are not split up and are in the same repository, it's pretty easy for the website to be put in a state where the front end and back end are out of sync. Really the only way to stop this is to actually define the contract between the two, but you want to do it in a way which doesn't couple the implementations of the front and back-end, much like you would code to an interface instead of an implementation in OO programming.

Also to preemptively handle the criticism that this creates an overhead of maintaining this interface file, swagger for instance can even produce code stubs for different programming languages and frameworks such as JAX-RS. So you can produce an interface in your chosen technology and then implement this interface. It's also adds very nice documentation to your back-end, making it easier for front-end developers to do their job.