Builds are slow and take time. We can get MSBuild to parallelize them, but only on a single machine, not across a cluster. Why hasn't anyone come up with clustered build solutions in the .Net space? I know such solutions exist in the C++ space (e.g., IncrediBuild, Electric Cloud, etc.) but we .Net developers are forced to build huge solutions on individual machines. Anyone know if there's some unsolvable technical difficulty in this?

Just curious.

  • 1
    I don't know what exactly you define as "slow" and "huge". But I'm pretty certain it's peanuts compared to what the larger, very template-heavy C++ projects take. You don't need to wait for thousands of template instanciations, the associated compiletime computations, countless optimization passes, native code generation, etc. that make C++ compile times so infamous.
    – user7043
    Apr 22, 2011 at 19:29
  • I've seen Visual Studio Solutions that take +60 minutes to compile; on a modern machine.
    – Nate
    Apr 22, 2011 at 19:32
  • Code-weaving usually slows down compile time quite significantly. PostSharp, CodeContracts...
    – GregC
    Apr 22, 2011 at 19:42
  • 2
    @Nate: C# code? A C# project that I worked on (~600k lines spread across around 150 projects and 20-30 solutions) built from scratch in about 4 minutes. In my experience, it wouldn't be worthwhile having an IncrediBuild-like system for C#/.NET Apr 23, 2011 at 8:07
  • 2
    @Dmitri: Well, I've also worked on C++ projects that took over 4 hours to build, so yeah, I'm OK with 4 minute build times :-) That's a rebuild-from-scratch as well, just building a single project or two typically takes seconds... Apr 23, 2011 at 8:30

2 Answers 2


" Why hasn't anyone come up with clustered build solutions in the .Net space?"

If compilation were to be clustered:

  1. The compiler will have to first break the compilation into different threads which can be done on indepenent machines.
  2. Send the code to different machines for compliation.
  3. Wait till all machines have completed their task
  4. Get the results from all machines in the cluster together.
  5. Put together the results and link them up.

steps 1 and 5 are dependant on the nature of the project : Too many interdepencies and the number of distribuable tasks is drastically reduced. Step 2 and 4 depend on the status of your network. The time taken to send and recive the object files adds to the compile time and beats the benefit of clustering. Depending on the project you probably may end up with the same time for compiling on a single machine as you would for a cluster.

  • 1
    If you're saying that it's not worth it, I totally disagree :) Regarding your points, 1. is already being done locally so could be done remotely (e.g., with MSBuild), 2. is really a non-issue, I use DropBox and it does peer-to-peer on my LAN so, basically, it's instant, 3. is why we are doing it in the first place, 4. is same a 2. i.e. a non-issue, and 5. is the only really 'tough' thing. Apr 23, 2011 at 7:39
  • I am saying that it wont be worth it except for very large projects. Compilation is a CPU intesive task. It would have been a geat idea in the days of yore but not today. What appears to us as "instant" may not really be for the computer. What I understand is that network transfer is much slower than single CPu compilation.
    – DPD
    Apr 23, 2011 at 10:25

Pretty much any CI system does that. If you're talking specifically about on the dev machine: Automated Build Studio is one such example.

That said, most of the products I've worked on/with never require a full rebuild on the dev machine as the code is usually loosely coupled enough and the devs rarely need to make changes to that many different assemblies between checkins.

  • If it's loosely coupled, you're still limited by the # of cores on your machine, not by the number of cores on all your machines. Apr 23, 2011 at 7:40
  • @Dmitri Nesteruk: Yes, but if it's loosely coupled, an incremental build won't perform a whole rebuild, hence you only have to build a couple files/projects instead of the whole application - thus, distributed compilation isn't really necessary. Apr 23, 2011 at 17:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.