I\'m working with a repository with a very large number of files that takes hours to checkout. I\'m looking into the possibility of whether Git would work well with this kin
Based on this answer by apenwarr and this comment by Miral I came up with the following solution which saved me nearly 94% of disk space when cloning the linux git repository locally while only wanting one Documentation subdirectory:
$ cd linux
$ du -sh .git .
2.1G .git
894M .
$ du -sh
2.9G .
$ mkdir ../linux-sparse-test
$ cd ../linux-sparse-test
$ git init
Initialized empty Git repository in /…/linux-sparse-test/.git/
$ git config core.sparseCheckout true
$ git remote add origin ../linux
# Parameter "origin master" saves a tiny bit if there are other branches
$ git fetch --depth=1 origin master
remote: Enumerating objects: 65839, done.
remote: Counting objects: 100% (65839/65839), done.
remote: Compressing objects: 100% (61140/61140), done.
remote: Total 65839 (delta 6202), reused 22590 (delta 3703)
Receiving objects: 100% (65839/65839), 173.09 MiB | 10.05 MiB/s, done.
Resolving deltas: 100% (6202/6202), done.
From ../linux
* branch master -> FETCH_HEAD
* [new branch] master -> origin/master
$ echo "Documentation/hid/*" > .git/info/sparse-checkout
$ git checkout master
Branch 'master' set up to track remote branch 'master' from 'origin'.
Already on 'master'
$ ls -l
total 4
drwxr-xr-x 3 abe abe 4096 May 3 14:12 Documentation/
$ du -sh .git .
181M .git
100K .
$ du -sh
182M .
So I got down from 2.9GB to 182MB which is already quiet nice.
I though didn't get this to work with git clone --depth 1 --no-checkout --filter=blob:none file:///…/linux linux-sparse-test
(hinted here) as then the missing files were all added as removed files to the index. So if anyone knows the equivalent of git clone --filter=blob:none
for git fetch
, we can probably save some more megabytes. (Reading the man page of git-rev-list
also hints that there is something like --filter=sparse:path=…
, but I didn't get that to work either.
(All tried with git 2.20.1 from Debian Buster.)