How do I synchronise two remote Git repositories?

Question

I have two repository urls, and I want to synchronise them such that they both contain the same thing. In Mercurial, what I\'m trying to do would be:

hg pull         


        

Answer 1

As others I came here because this SO-question has so right header for this particular problem.
This was few years ago. I studied available answers, but nothing solved my situation.

I took a step further and created several solutions.
And my final solution posted here is for Git repositories only (sorry, I do not use Mercurial right now as it was asked in the body of the question).

I had rather difficult conditions at the start than others here.

• I can't use Git-hooks because I always have a limited access to one or the other remote repository. I consider them a burden dependency.
• Teams on both remote sides are big. They produce huge amount of commits and commits at the same Git-branches too.
• We need 24/7 and fast synchronizing solution. This drastically decreases amount of possible Git-conflicts and converts them to simple local Git-merges.
• It was important to have a capability to do Git-merges from any remote repository for any branches and by any member of both teams.
• Sometimes one remote repository needs to be completely replaced by an empty new repository at another location. And I didn't want to do initial repository filling manually.
• CI/CD managing branches should be migrated quite arbitrary.
• Not all present syncing methods recognize subtle changes right. For example, moving back in history.
• And some methods have a threatening tendency to occasionally delete branches or the entire repository.

And just like the author of this SO question, I also wanted to have

• no user interaction, at all. That is, everything should be fully automated.
• all changes in both repos for all branches should be migrated.

I quickly wrote some bash script and quickly realized that with Git this would not be enough.
The main problem is conflict solving.
Long story short, you can only solve conflicts by some conventions or by asking to repeat a conflicting merge (commit).

So, my script became a compiled application.

Later arrived some requirements.

• It was very desirable to remain some branches invisible to the other side repository.
• We cannot allow arbitrary updating for all branches. Some branches should be updated in a limited way on one or the other side.
• Some Git-servers like to create util- (garbage) branches and tags. No one wants them to be migrated.
• More importantly, some Git servers, like Gitlab or Bitbucket, like to completely block some branches and tags. This is a catastrophe.

The other thing is that people tend to forget the fact that they have two synchronized remote Git-repositories. They do it really fast and do not want to remember any rules.
Some teams may have some level of employee rotation.
A big application requires more documentation and support. And etc.

Finally, I threw out some complicated conflict solving and state analysis.
A remaining subset of logic was pretty simple, which allowed me to convert my app into a bunch of bash and gAWK scripts.

And as a result of all, my tool only synchronizes branches with defined prefixes.
You can say, let's synchronize branches that begin with the @.
Or branches that start with my-company- and your-company-.

Of course, my tool has a little learning curve.
But it is quite mature and gives a possibility to forget about some sync problems, at all.

Actually I forgot about my tool and came here two years later, because I've implemented a wish list.

My latest tool is here - git-repo-sync. I hope this will help someone else.