0

I have a large C++ project I'm hoping to release publicly soon (it is currently an internal development at work).

We have a large auto documentation setup which uses doxygen and sphinx, however our documentation includes many images (useful as our tool deals primarily with image processing, geometries, and rendering).

Currently I'm just keeping the images sitting in a shared folder, and reference them when I build a copy of the docs. But when we publicly release the project I'd like to somehow provide them such that anyone could easily build the documentation by simply setting the BUILD_DOCS cmake option to ON, and not have to worry about sourcing files from all over the place. But at the same time, I don't want to include them in the repository itself as I doubt most people would be building the docs from scratch and so would never need the images at all.

One solution I was thinking of was to keep the images in a separate repository that the main repo simply contains as a submodule. Then I can configure the CMake to simply go fetch the contents of that submodule if someone is building the docs. I do something similar already with vcpkg which I have included as a submodule to simplify the build process (but still allow people to bypass using it)

Is that a sensible option? Or is there a better way of dealing with this? The advice I've gotten most often is to just not care about it but I really like how small my repository size is and adding these images would immediately grow the repo an order of magnitude in size.

3
  • Which order of magnitude is the total size of those assets, and how many files do you have approx.? Are you talking about several KB, MB or GB? BTW, have a look at opensource.com/life/16/8/how-manage-binary-blobs-git-part-7
    – Doc Brown
    Jun 6, 2023 at 15:32
  • 1
    Its a few dozen files, though that amount is subject to change (with some images being swapped and some new ones being added). Currently, the amount is ~30MB in size. Which again I know is a rather smaller amount now, but its both much larger than the code-base itself and if/when images are changed it can quickly grow out of hand
    – Chris Gnam
    Jun 6, 2023 at 15:36
  • Well, a few dozen files and 30MB is nothing which should worry about, just keep them in your current repo. When it will become a few hundred files and 300MB, then it is time to start thinking about. But you may consider to to search this site for "git binary files", there are a lot of former questions here about this topic, and our community expects askers to show some research effort before they ask.
    – Doc Brown
    Jun 6, 2023 at 15:41

1 Answer 1

1

I don't want to include them in the repository itself as I doubt most people would be building the docs from scratch

The main criterion of what belongs in the same repository is not whether someone will make direct use of everything. Opinions on a monorepo aside, it's a valid example of how not everyone will touch everything that's in such a monorepo.

The main criterion is whether it's important that the included items share a versioning lifecycle. Do you want to ensure that a specific version of the docs (and generation tools) refers to a specific version of your code? If this is paramount, then there is sufficient justification for adding it to the same versioned repository.

If, however, the document generation version is separate from the codebase version, then there's no point in adding them to the same repository.

One solution I was thinking of was to keep the images in a separate repository that the main repo simply contains as a submodule.

The question then becomes how you know which version of repo A should be referenced by repo B? Let's say you have 20 repo A versions (= codebase), and 3 repo B versions (= images, or all of the document generation tooling). Which version of repo A references which version of repo B?

I really like how small my repository size is and adding these images would immediately grow the repo an order of magnitude in size

Differential commits mean that you don't really suffer from having the images in your repo, other than the initial checkout.

If the images change frequently, then that's the cost of doing business. The assumption here being that you'd only add it to the repo if keeping the version between code and documentation synchronized is more important to you than some bandwidth usage.

2
  • This answer is right in general, but I think it lacks some remarks about what will happen when a Git repo is filled with many large binary files, and maybe links to some useful older Q&As on this site which deal exactly with that problem (which has been discussed here and on Stackoverflow several times over the years). Of course, the OP's project, storing everything in the repo does not seem to be a technical problem yet.
    – Doc Brown
    Jun 7, 2023 at 6:23
  • @DocBrown: I'd argue the consideration for binary files hinges on these files changing with relative frequency. If it's a slow-growing set of document assets for styling purposes, I wouldn't be too fussed about it tbh.
    – Flater
    Jun 7, 2023 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.