Among the hats I wear is overseeing packaging and installation of the software and the development tool set for one of my company's business units. We build everything (almost; see below) from source from scratch nightly using a custom-built, fully-automated process. Anything not burped out by the build process doesn't ship, no exceptions.
We do this for a lot of reasons:
Safety. Building from source mitigates the risk that a provided binary has had something malicious added. If we suspect software-induced foul play, we can go back and audit the sources.
Accountability. If we change versions of a package, we'll know about it because we put the distributions (usually as the blob, not broken out into individual files) into our VCS.
Maintainability. As we correct bugs or make enhancements, we write patches that go into our VCS with the distributions and get applied as part of the build process. As we upgrade to later versions, we remove patches if what they do has been incorporated into the source and review the remainder for relevance. Patching also adds accountability we wouldn't have otherwise ("that broke after we applied the autodefrobulator patch").
Insight. Because we go through the process of building sources into packages, we get a good understanding of the dependencies and what quirks there are in the build process. Because builds are automated, the known-correct process for building the package is scripted and can be read and understood. In at least one case, we've ditched something because the process of figuring out how to make it build caused us to look closely at the source, when we discovered that the entire thing was poorly written.
There are two things we don't build:
Operating Systems. The benefit we would get from building the entire thing from source isn't worth the administrative effort. We get our OS media from sources we trust, usually directly from the vendor, and treat that as a unit. Most of our OS installations are done from a central server, so we keep that up to date with the latest patches.
Commercial Products. There are a handful of things we use that are either unavailable or prohibitively expensive in source form. In those cases, we keep the binary distributions, one for each architecture, in our VCS with everything else. In some cases, when the vendor provides it in something other than our preferred installation format, we'll actually have a build process that repackages it as we want it. In all cases, even if the build process is "do nothing, here's your finished package," these go through our automated build process.
Is it practical and sustainable?
Absolutely. There is a certain level of labor and equipment required to get it started and the kinks worked out, but our process has been set-and-forget. I haven't done a detailed accounting of it, but I'm pretty sure we spend a lot less time downloading and compiling software because the engineering staff can go look in a well-known place for what they need. Those of us that do the actual packaging have become are good enough at it that most new additions are available after the next nightly build.
We're not doing anything on the scale of spinning a Red Hat distribution, which involves 3,000+ packages. We're an order of magnitude smaller than that, but my experience has been that once the figure gets beyond a certain point, consolidating everything into a central system is worthwhile.
It seems that the sheer number of dependencies to maintain and
build from source for some of the major open source projects would
quickly complicate things.
If your builds are done using a packaging system that can handle dependencies and conflicts (e.g., RPM), it's not as difficult as you might think. We do go through "dependency hell" when we build something that has a lot of prerequisites that are new to us, but once it's done, it's done. After that, the nightly build takes care of making sure everything is in the right order. Understanding the dependencies forces us to think about the overall architecture and how new additions fit in. We'll reconsider something if it's going to turn into a rat's nest.
Addendum based on comments:
Multiple Versions: I go to great lengths to avoid situations where we need multiple versions of the same thing installed, but sometimes the ugly train stops at your station and you have to board it if you want to get anywhere. Python has been a bit of a thorn in my side in that respect, because we get a few things from other parts of the company where they've used features from later versions than the one we like to use. Fortunately, Python is built so multiple versions can be installed in parallel, so we do that and write patches for the "outside" software to force it to use the right one. On other occasions, I've had to do full repackagings so software requiring a special version of other software has the what it needs bundled in and isolated from the rest of the system.
Upgrades: I don't do them unless they need to happen. Once nice thing about building from sources is that you can pick and choose things from later versions if you need them and turn them into patches to your existing version. I do this with bug fixes.
