This is a question about the internal workings of Mercurial.

I vaguely remember a fellow programmer explaining to me how mercurial works out which changesets differ between two repositories and that it had a speedy solution to the problem.

Does anyone know how this works or can point me towards a document describing the details?


Algorithm works in the following way. You have two repository: local and remote. They both contains a DAG of changelists.

The goal of the discovery protocol is to find one set of node common, the set of nodes shared by local and remote.

One of the issue with the original protocol was latency, it could potentially require lots of roundtrips to discover that the local repo was a subset of remote (which is a very common case, you usually have few changes compared to upstream, while upstream probably had lots of development).

The new protocol only requires one interface for the remote repo: known(), which given a set of changelists tells you if they are present in the DAG.

The algorithm then works as follow:

  • We will be using three sets, common, missing, unknown. Originally all nodes are in unknown.
  • Take a sample from unknown, call remote.known(sample)
    • For each node that remote knows, move it and all its ancestors to common
    • For each node that remote doesn't know, move it and all its descendants to missing
  • Iterate until unknown is empty

There are a couple optimizations, first is instead of starting with a random sample of missing, start by sending all heads, in the case where the local repo is a subset, you computed the answer in one round trip.

Then you can do something similar to the bisecting strategy used when finding faulty changesets. Instead of random samples, you can try picking nodes that will maximize the number of nodes that will be classified with it (since all ancestors or descendants will be marked as well).

The end result worked pretty well.

The code is pretty readable if you're curious: http://selenic.com/hg/file/tip/mercurial/setdiscovery.py

| improve this answer | |
  • This makes sense to me, thanks. I assume remote.missing(sample) was meant to be remote.known(sample), and after the first few optimised queries one can binary chop the unknowns paths. – Robin Sep 2 '13 at 8:43
  • Oh yeah, let me fix the typos. – tonfa Sep 3 '13 at 7:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.