How to import git repositories with large files?

Question

Given that GitHub doesn\'t allow to push files larger than 100 MB, it is not possible to git clone and push a repository with large files into GitHub enterprise. The push fa

Answer 1

The easiest way I found was taking advantage of git filter-branch and the BFG Repo-Cleaner by rtyley (I used version 1.12.12):

1. Prerequisite: you need to have git lfs installed

2. Create a new repository on GitHub Enterprise. You'll import your external Git repository to this new repository.

3. Clone the repository you want to migrate to a local folder:

$ git clone --mirror git@oldgithost:repo
$ cd repo.git
# mirror into a local directory

4. Rewrite the history to lfs-track your large files¹:

$ git filter-branch --tree-filter 'git lfs track "*.{zip,jar}"' -- --all
# writes the patterns to lsf-track into .gitattributes

5. Use the BFG to extract the relevant files into Git LFS

$ java -jar ~/usr/bfg-repo-cleaner/bfg-1.12.12.jar --convert-to-git-lfs '*.zip'
$ java -jar ~/usr/bfg-repo-cleaner/bfg-1.12.12.jar --convert-to-git-lfs '*.jar'
# Convert large files (I couldn't find a one-liner for multiple patterns)

6. Push to your GitHub enterprise remote:

$ git push --mirror https://hostname/ghuser/repo.git
# Pushes the mirror to the new GitHub Enterprise repository

7. Delete temporary dir:

$ cd ..
$ rm -rf repo.git

---

Notes

¹ Due to the high I/O, it is recommended to rewrite the history into a temporary directory off-disk with the -d option, e.g. on tmpfs.

Answer 2

You can now use git lfs migrate built-in command to both assess which files is best to migrate and also do the actual history-rewriting.

See git-lfs migration tutorial for more details.