72

I heard Google has a giant private (internal) repository of all of their code and their employees have access to it so that when they are developing things they don't have to reinvent the wheel. I'd like to know more about it!

Is there anyone here from Google that can describe it in a bit more detail, or do you know a bit more about it? I'm interested in knowing mainly about how it's organized and how they can make it easy for an employee to find something in such a giant codebase as it must be.

Improve this question
2

3 Answers 3

Reset to default
45

Here is a video explaining how it is organized: Development at the Speed and Scale of Google

Ashish Kumar presents how Google manages to keep the source code of all its projects, over 2000, in a single code trunk containing hundreds of millions of code lines, with more than 5,000 developers accessing the same repository.

Improve this answer
1
  • 10
    I know the other answer has more upvotes, but this video has all the data from the other answer and then some. If you (the reader) want a summary, read Chris's answer, otherwise if you have an hour to spare watch this video!
    – Ricket
    Feb 1, 2011 at 21:55
69

For the most part, Google uses a Perforce setup. However, there are internal tools for getting other tools like git to work on top of it. (How they accomplish this I don't know.) Large, open-source projects like Android and Chromium have separate repositories however.

Built on top of Perforce are a lot of stellar internal apps. For example, there are tools to make building, testing, and code reviews nothing short of magical.

Partly because of this 'magic' and testing culture, Google doesn't really use branching. Everybody checks into 'main'. For any project you can see the source, build it, and run the unit tests without any specialized knowledge. This is huge. When I was at Microsoft each product required the sacrifice of a different animal to build and running their tests would be out of the question.

Also, Google has a company-wide style guide for the major languages we use. if you have access to another team's source code, what would be the point if the formatting is all wonky!

As for searching, you might be familiar with Google Code Search. There is a special version of that, along with other top-secret code searching tools that make navigating code much easier.

In short, Google has a very engineering-centric culture which understands the value of tools and developer productivity.

Improve this answer
2
  • 1
    I've been using the style guide you referenced for years, it's great! But isn't that for Google-originated open-source projects? Is there a different guide for internal projects?
    – Dennis
    May 13, 2014 at 14:46
  • 1
    The "top-secret" code-searching tool is being made open source over at github.com/google/kythe - it is a subset and has no UI (any more, the example one is no longer maintained afaik) but I think their goal is to make Kythe as complete as their internal tool is.
    – mmlac
    May 14, 2017 at 19:36
5

It is huge:

(as of jan 2015)

  • Total number of files*: 1 billion
  • Number of source files: 9 million
  • Lines of code: 2 billion
  • Depth of history: 35 million commits
  • Size of content: 86 terabytes
  • Commits per workday: 45 thousand

* The total number of files includes source files copied into release branches, files that are deleted at the latest revision, configuration files, documentation, and supporting data files.

They use an internal tool called Piper, itself relying on google infrastructure.

Source: Why Google Stores Billions of Lines of Code in a Single Repository

Improve this answer
2
  • @CodesInChaos This info is taken from the video in the slide at 3:22. The slide contains more explanations about how those numbers are calculated.
    – Benjamin Crouzier
    Sep 23, 2016 at 14:18
  • 5
    Wait, wouldn't that mean an average of 2 lines per file?
    – Hashim Aziz
    Aug 31, 2019 at 2:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.