Is there any way to clone a repository from the web incrementally?

Question

I'm on dialup in lousy place (yes, it still happens in 2011), and trying to clone a huge repository. It starts without problem, but every time the dialup disconnects (which is unavoidable, it seems), the !#%$* hg rolls everything back and I'm left again with an empty directory.

Is there a solution other than doing it on a remote PC and then downloading the whole thing by FTP or something?

Answer 1

In a bash-like shell you could do something like this:

$ hg init myclone
$ cd myclone
$ for REV in `seq 10 10 100` ; do hg pull -r $REV <REMOTEREPO>; done

Starting at 10, each pull downloads the next 10 revisions, up to 100. In case of a lost connection, adjust the first argument to seq to match what you've already pulled.

Answer 2

Depending on how flaky your connection is, there are two options for performing initial clones.

First, you can try so-called “streaming clones”. These minimize Time To First Byte, but do generally require a bit more data to be transferred.

Here’s how to do a streaming clone:

$ hg clone --uncompressed https://~~~~

Your second option will be a hg clone –-rev operation, followed by a number of incremental pulls. This behaves similarly to cloning a repository in some distant past and doing occasional updates.

$ hg clone --rev 5 https://~~~~

Answer 3

Based on the suggestions here,

I created a repo that did this.

https://github.com/nootanghimire/hg-clone-bash

It's optimized for a single repo, but i guess you can fork and work on it! :)