0

I work at a company which has it's source code managed using subversion. Our code base is huge (800MB) but we want to work with branching. For what I noticed is that when branching, SVN just duplicates every file which makes it almost impossible to branch because of the space.

I would like to separate our library and other not often changed files, so it can be loaded in each branch and stored only once. How can I structure our repository so that branching doesn't copy every single file in the repository?

I was looking at svn:externals, but I don't know if that's the way to go.

  • I'm more of a git guy myself and I couldn't believe this to be true either. It's weird because the files in my branches folder and trunk folder aren't hardlinks they are actual files which take up actual space. Maybe this is because the svn host I'm using (beanstalkapp) does something weird? – Martijn Nosyncerror Aug 7 '15 at 13:35
  • 1
    @snowman If you check out the root, you get one directory per branch on the client. I think that's where the OP's confusion comes from. But AFAIR you can check out individual branches. – CodesInChaos Aug 7 '15 at 13:38
  • Yes I got that, and I would understand if these files were symlinks to original files. But the total size of my project folder is much bigger than one branch folder for example – Martijn Nosyncerror Aug 7 '15 at 13:43
  • Oops sorry, mind pointing me where to ask this question? It's kind of a big issue. – Martijn Nosyncerror Aug 7 '15 at 13:58
  • Why is it a big issue? As others have pointed out, the space is only consumed on the client, not the server. And there are many techniques for minimizing that space, the simplest being to delete the branches once you've merged them to trunk. You can also use --depth and --set-depth to control what gets pulled to the client. Or you can create an archived-branches directory where you svn mv branches after merge (although, really, deleting them is far better). – kdgregory Aug 7 '15 at 16:06
4

Some caveats:

  1. You really shouldn't be storing libraries in your source-control system. Especially third-party libraries. Depending on the language and tools that you're using, there are a lot of ways to avoid doing so, but that's another question.
  2. Recognize that the Subversion repository uses the equivalent of links internally, so the only time a file is actually duplicated is when it's changed. If you commit a file in version 100 of the repository, every branch based on that version will reference the same file (unless and until you commit a new version).
  3. You don't want Subversion to create symlinks on the client. You want each branch to be self-contained, so that you don't accidentally change a shared file. However, be aware that Subversion will manage symlinks that you create.

With those in mind, and with the goal of moving toward a world where you don;t keep your libraries in the source repository, I recommend creating separate directories at the top of your repository:

/libraries
/project-1
    /trunk
    /tags
    /branches

This is one of the under-appreciated features of Subversion: you can check out any arbitrary subtree of the repository into any arbitrary directory on your local filesystem.

So, how do you deal with separation of libraries and source? That again depends on your language and tools. It can be as simple (and ugly, and barely maintainable) as a lib directory in your project that contains symlinks back to the libraries directory.

Better is to use a build tool that gets its library references from a configuration file. This will let you easily add libraries into the repository, and lets different project versions use different sets of libraries.

  • "You really shouldn't be storing libraries in your source-control system" - I think that is not what you really meant. You mean not to store shared libraries in the same tree or repository as other source code, but still in your SCS, right? – Doc Brown Aug 7 '15 at 17:00
  • 1
    @DocBrown - no, I meant exactly what I said. In my view, libraries are dependencies used by your source code, and artifacts produced from your source code, but they are not source code in themselves. You may have a need to store libraries-as-dependencies in a locally-managed repository, but I believe that even there a source code management system with its "forget nothing" design is inappropriate. – kdgregory Aug 7 '15 at 17:14
  • 1
    If updating a dependency can break software, then there is at least some potential value in having that dependency in source control, especially if you have continuous integration. Whether it's a net positive or not varies, but I find that in any project which requires "building" the final deliverable, there's usually at least one dependency for which it makes sense to have a "local" copy. – Ixrec Aug 7 '15 at 17:38
  • 1
    @Ixrec - "local" copy, sure, but why in the source control repo? Perhaps I've been spoiled by the Java ecosystem, where Maven Central has been a reliable resource for ten years or so. But even when I was writing C code back in the 80s, we kept our source separate from our builds. – kdgregory Aug 7 '15 at 17:58
  • @kdgregory Good point about Maven. I'm used to environments that are a little less robust and self-contained when it comes to dependency management, and some dependencies that are utterly nightmarish to build (blame it on C++). – Ixrec Aug 7 '15 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.