Reduce increasing time to push a subtree

Question

I'm using git in my project, which consists of several sub-projects. Each sub-project is "linked" in the main project using the git-subtree command. That's my way to realize "svn externals" in git. I'm using it since some weeks, but the time pushing my changes from the subtree to the remote location increases during every commit. It looks like this, when I'm pushing the changes using the command

git subtree push -P platform/rtos rtos master

git push using:  rtos master

1/    215 (0)2/    215 (1)3/    215 (2)4/    215 (3)5/    215 (4)6/    215 (5)7/    215 (6)8/    215 (7)9/    215 (8)10/    215 (9)11/    215 (9)12/    215 (10)13/    215 (11)14/    
....

20 more lines

....

(204)209/    215 (205)210/    215 (206)211/    215 (207)212/    215 (208)213/    215 (209)214/    215 (210)215/    215 (211)To https://github.com/rtos/rtos.git
   64546f..9454ce  9a9d34c5656655656565676768887899898767667348590 -> master

Is there any way to "clean up" the subtree and therefore reduce the time pushing the changes?

Answer 1

Note that if you decide to switch to git submodule, you now can, since git1.8.2 (2013-03-08) track the latest commits of a submodule repo.

See git externals.

"git submodule" started learning a new mode to integrate with the tip of the remote branch (as opposed to integrating with the commit recorded in the superproject's gitlink).

That could be make for quicker push, while benefiting from the additional information a submodule has over a subtree (i.e a lightweight record of a specific commit of the submodule)

You can update that submodule to the latest of a given branch with:

git submodule update --remote

This option is only valid for the update command.
Instead of using the superproject's recorded SHA-1 to update the submodule, use the status of the submodule's remote tracking branch.

Answer 2

Try using the --rejoin flag, so that after the split the subtree is correctly merged back to your main repository. This way each split needs not to go through all history.

git subtree split --rejoin --prefix=<prefix> <commit...>

From the original subtree documentation:

After splitting, merge the newly created synthetic history back into your main project. That way, future splits can search only the part of history that has been added since the most recent --rejoin.

Answer 3

No, unfortunately not. When you run git subtree push, it will recreate all commits for this subtree. It has to do that, as their SHA depends on the previous commit and needs those SHAs to be able to link the new commits to the old ones. It could cache that, but it doesn’t.

I guess this is the price you pay for using subtree vs. submodules. Subtree is very stateless in your repository, which is nice on the hand but causes this long computation on the other hand. Submodules store all their information, which requires you to manage it but also makes stuff like this a lot faster.

Answer 4

Maybe this helps: I think you can tell git subtree split to only go back n commits by doing

git subtree split --prefix XXX HEAD~n..

or by specifying the commit you want to start with for example

git subtree split --prefix XXX 0a8f4f0^..

This helps reduce the time, though it's inconvenient.