Clean up large files on git server

Question

Someone accidentally committed some large (multi-GB) binaries to my self-hosted gitlab repository, and now every time someone tries to pull from the repository the server ge

Answer 1

Had the same problem and the process to get it resolved was quite involved.

We run the community-maintained sameersbn/gitlab 11.4.5 in a Docker container. I didn't want to install bfg there, but opted to perform the changes locally.

# Install the bfg tool, ex. on MacOS via homebrew
brew install bfg

# Clone repo locally
cd ~/Development
git clone --mirror ssh://git@server.com:22/some/dir/myrepo.git

# Clean the repo
bfg --delete-files \*.pdf myrepo.git
cd myrepo.git
rm -rf .git/refs/original/
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now

# Upload to container-host, e.g. via FileZilla

# Connect to the container-host via ssh

# Rename the original directory in the container, to have a backup
docker exec -it gitlab /bin/bash
mv /home/git/data/repositories/some/dir/myrepo.git /home/git/data/repositories/some/dir/myrepo.git.mybackup
exit

# Copy from container-host into container
docker cp /root/Documents/myrepo.git gitlab:/home/git/data/repositories/some/dir/myrepo.git

# Fix permissions in container
docker exec -it gitlab /bin/bash
cd /home/git/data/repositories/some/dir/myrepo.git
find . -type f -print0 | xargs -0 chown git:git
chown -R git:git /home/git/data/repositories/some/dir/myrepo.git
chmod 770 /home/git/data/repositories/some/dir/myrepo.git

# Re-create the "hooks" subdir with some symlinks in the repo
cd /home/git/gitlab/bin
./rake gitlab:shell:create_hooks

# Clear Redis cache (unclear if needed)
./rake cache:clear
exit

# Clone the changed repo locally again, also tell everyone who got a copy to clone again (history is broken now)

# Then do a commit to the repo, to hit the hook and trigger a size recheck