GIT Rebase Fatal on Multiple 0.5GB Binary Files

Question

[This question is essentially reopening git crash during rebase which never had an answer]

I'm attempting to a rebase from my 'secc' branch as:

$ git rebase main
First, rewinding head to replay your work on top of it...
fatal: Out of memory, malloc failed (tried to allocate 553656577 bytes)         # about 0.5 GB
$ git rebase --abort
No rebase in progress?

The failure is related to the fact that both branches and their common ancestor have three .dat files each of which is 0.5 GB.

How can I do a rebase in this situation?

Additional info:

• A 'git merge main' works just fine.
• Augmenting .gitattributes with '*.dat merge=keepTheirs' did not prevent the fatal.
• The *.dat files do differ.
• I'm willing to remove the *.dat files to rebase the others and then add back the *.dat. But how?
• I'm using git 1.7.9.4

Answer 1

In the comments you've said that doing the same with a computer that contains more, in this case 32GB, memory has resolved the issue. Based on that, I would conclude that you just had too little memory available to do this on the machine you first tried it with.

Answer 2

You won't know if your machine is big enough until failing on the 'git rebase' but by that point your directory is in a munged state. During the rebase another branch was checked out (main) so that the 'secc' changes could be applied to it. Recover, and proceed with:

git checkout secc

Having failed on the rebase, as you've noted you've got two options:

1. If rebase is not required, go with 'git merge main'
2. Get a bigger machine and retry 'git rebase main'

You don't have a practical option to ignore the 0.5GB files, do the rebase, and then get those giga files back.

Answer 3

Try putting these in .git/info/attributes:

# whatever gets them...
yourfile binary -delta merge=binary
*.yourext binary -delta merge=binary

that'll cause merge conflicts if those files change, and you'll have to figure out what to do with them. check the gitattributes and rebase doc for other merge strategies, I'm not even going to name the dangerous ones here.

I'm not sure it's the actual merge trying to get the whole thing in core from running, but it seems worth a try.