Incremental backups with git bundle, for all branches

Question

What is the easiest way to do incremental backups of a git repository with git bundle?

If I just wanted to backup a single branch, I could do something along these lines:

git bundle create foo last-backup..master
git tag -f last-backup master

But what if I want to backup everything (including all branches)?

---

To answer the questions in the comments:

Strictly speaking, I do not need to use the usual Git bundles, as long as the solution satisfies the following properties:

• Each incremental backup is a single file. I can store it somewhere and subsequent incremental backups do not need to modify this file.

• The size of the file is approximately equal to the total size of Git commits since the previous backup. Changes in binary files are also efficiently stored.

• A full backup + all incremental backups since then contain everything that I need to automatically reconstruct the repository, including all branches.

(As a naive example, simply constructing a tar archive with recently-changed files in the git repository fails to satisfy the second requirement if, for example, automatic garbage collection has occurred.)

And ideally I would also like to have a system that is idiot-proof:

• I can take virtually any full backup of my Git repository, plus all recent incremental backups, and I can simply "pull" everything from the backups and the repository will be up-to-date. In particular, it does not matter if there is a partial overlap between the full backup and incremental backups.

Git bundles satisfy all this very nicely if I only need to handle one branch.

Answer 1

(We discussed this problem with Jukka, this is the outcome.)

Preliminaries:

1. Have the last backup available as backup.bundle
2. Have a remote backup that points at backup.bundle

Making a backup:

1. git fetch backup – just to make sure we're up to date
2. git bundle create newbackup.bundle ^backup/A ^backup/B A B C
   • This means we create a bundle that excludes all the stuff that was already in the bundle
   • It's easy to generate the needed ^backup/A-style arguments from refs/remotes/backup/
   • Likewise the A-style arguments are from refs/heads
3. copy newbackup.bundle to wherever you keep your backups
4. replace backup.bundle with newbackup.bundle so you know where to start the next incremental backup

Recovering:

1. Have a repository that is either empty or represents an old version of your repository
2. For every backup file, in sequence:
   1. git remote rm recovery
   2. git remote add recovery <name-of-bundle>
   3. git fetch recovery – you need to name the remote for this to work
3. Now you should have every branch available in refs/remotes/backup

Answer 2

Try using --since with --all.

Create the first backup:

git bundle create mybundle-all --all

Do an incremental backup:

git bundle create mybundle-inc --since=10.days --all

The incremental should contain all commits on all branches that have happened in the past 10 days. Make sure the --since parameter goes back far enough or you might miss a commit. Git will also refuse to create the bundle if no commits have happened in that time-frame, so plan for that.

Answer 3

You could do

git clone --mirror <your_repo> my-backup.git

It will create a bare repo with all refs.

Then you could periodically do git push --mirror <my-backup>.

Answer 4

Seems opqdonut solution will not work, cause ^backup/A ^backup/B only points to last incremental backup. And actually need to exclude refs from all previous incremental backups.

Need to create remotes for each of previous bundles.

UPD: No, it should work, see Jukka comment below.