0

I was tasked with implementing a networking stack. The software is self-contained with no external dependencies - it only needs the C++14 standard library, and a POSIX platform.

In my opinion there's nothing preventing it from being successfully implemented in a single repository as the logical separation (layers, classes, etc.) can still be laid out cleanly without the need to manifest as physical separation (a separate project-directory-with-a-Makefile per class, for example). However, my superiors insisted on splitting the project into multiple smaller repositories and I have a hard time evaluating their rationale.

One of their arguments is that "back in 2003 we have tried writing projects in single repositories and it didn't work, it needed a week to compile!". This does not convince me.

How would splitting the project up into multiple subprojects help in this scenario? If you have one 50k lines project, you have to compile 50k lines. If you have ten 5k lines projects you still have to compile 50k lines, but now you can't let the build system exploit the dependencies between entities to compile in parallel because you have serialised the build (first build package A, then package B, then package C, etc.).

Also, you can't just have class A have B* member and, at the same time, have class B have a A* member without either having them in the same project (to let the compiler see their headers), or creating yet another package containing only the headers for those classes. But this is madness! Instead of having one project you now have three... because of reasons. (As a side note, maybe this is caused by the fact that me and my team are forced to cram everything into Maven packages? I don't see how that helps us for exactly the "A*-B*" problem stated above.).

What is more, putting everything into packages stored on a central server (again, we are forced to use Maven) has the unfortunate effect of other developers affecting my work at the least expected moments. When I'm fixing a bug I expect my environment to remain constant - but when I build a project, and the build system automatically fetches new versions of all its dependencies the environment changes, and a bug that my colleague has just introduced shadows the one I'm working on because it happens earlier in the execution path... and now I'm losing time because I need to fix my local environment. I don't see how that helps us.

Again, this issue can be easily fixed by performing only offline builds without reaching out to the online repository but then I have to manually go into each project directory and run the build to make the packages available in local repository on ever developer's machine.
However, this has clear disadvantages in languages such as C++ because I have to remember in which order the packages must be built - otherwise, the size of objects produced by a library may be different from what the consumer of the library expects (it's the simplest example of what can go wrong: I build package A with an old version of package B, then I build package B but the class it contains now has one additional int member... and oops, we're out of sync and go straight to the SIGSEGV land); if I was allowed to make the project a monorepo even a simple Makefile would hand the dependencies correctly.

On the other hand, a single project has clear advantages (in my opinion):

  • I can tell the test team "The last known good version is 0f52ab27199ffc, use that for testing because we may have since broken X, I'm not yet sure, but it behaves strangely after we did Y."
  • I can freeze my environment for the time I need to fix a bug (simply by branching out from the master branch).
  • I can fully utilise available tools to speed-up builds (e.g. invoke make -j N to launch N jobs in parallel).
  • I can let the compiler loose on the codebase with -O3 and let it optimise the resulting binary without placing obstacles in its way (e.g. it's much easier to inline a function that's available as source, than one that's available as a symbol in some .so).
  • I can easily debug the program - no "Oh, I forgot to add -g to project foo." or "Make breakpoint pending on future shared library load?" in GDB.

Are my arguments valid, or am I misguided and splitting a project into a bunch of .sos is really the way to go ("because that's how we do it in our Java projects")?

  • Basically, because if you split your module well. You will not need to modify all of them each time your doing something so you will be only recompiling the one you're working one while working with the last "dev release" of the others. And it force you to organize your code and design better, and that's for everyone's good. – Walfrat Feb 27 '18 at 8:36
  • Comments are not a place for discussion, but... that's not an answer. Monolithic repository does not imply monolithic design. Even with monolithic repository I can compile just what I need (in fact, it's even easier), and design a modular program. – Mael Feb 27 '18 at 8:45
  • 2
    @Mael: you seem to be aware that Monolithic repos don't imply monolithic design, but most of the issues you mentioned are IMHO monolithic design issues. Like your problems with automatic dependency detection across multiple projects, or the example of cyclic dependencies. Your Maven problem is a different one and should be attacked separately, it has IMHO not much to do with "one repo" "vs multiple ones". – Doc Brown Feb 27 '18 at 9:26
  • 1
    ... but don't get me wrong, that does not mean that "multiple repos" is the way you should go, there are valid arguments against this approach, but I don't think you have identified them yet. I recommend you get a copy of Lakos book "Large-Scale C++ Software Design". It does not contain much about version control, but it teaches you the problems of physical design issues of C++ programs, any why logical separation is not enough. – Doc Brown Feb 27 '18 at 9:32
  • 2
    Another argument outside of the programmation aspects could be simply peoples. Even if you're are able to design a proper modulable application in a monolithic project, they just might think that it's better to keep it splitted so no one, neither you or others developpers in the future might take shortcuts available in a monolithic project. The kind of shortcut that add a lot of technical debt later on. So you're basically taking back the lack of skill of others. – Walfrat Feb 27 '18 at 10:23
3

There are two questions in your post, and they should IMHO be answered separately. Assumed one is going to create a somewhat "large" software system using C++,

  • should one create different, physically separated C++ (sub)projects, each one with its own makefile, separated versioning, separated tests? Or is logic separation enough?

  • in case one uses physically separated projects, should one use also different repos for it?

From my experience, the answer to the first question is "for systems of a certain size, it pays off to split them up". Otherwise, build and link times will kill your project. Especially link times of most C++ linkers tend to behave at least quadratic, so expect the link time for 10 projects with 5kLOCs to be much smaller than for one huge project with 50kLOCs. There is no parallel build which will prevent you from this. I cannot tell you exactly what "certain size" will mean in your case, but I heavily recommend Lakos' book for getting more background information about this, the whole topic is far too broad to be explained here in one or two paragraphs.

Of course, for splitting a large project into smaller ones one will have to invest some thoughts about managing the dependencies automatically and how to incorporate different makefiles. But I am sure there are solutions available for every major C++ build environment, Google and CMake will be your friend.

The second question was already asked more than once on this site, and it is IMHO more dependend on the VCS you are using, the team's way of working and less on the programming language. See this older question, it is tagged with "Java", but the arguments are not quite different for C++.

My personal opinion with this is, if the software, as you wrote, "is self-contained with no external dependencies", using different repos and a tool like Maven is overkill, quite useless and adds lots of unnecessary management overhead to it. However, if your team or the company you are working for has a "one (sub)project - one repo" policy, and has already a solution in place for handling the additional management overhead with multiple repos, then you may be better off with sticking to that policy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.