Update Git submodule to latest commit on origin

Question

I have a project with a Git submodule. It is from an ssh://... URL, and is on commit A. Commit B has been pushed to that URL, and I want the submodule to retrieve the commit

Answer 1

It seems like two different scenarios are being mixed together in this discussion:

Scenario 1

Using my parent repository's pointers to submodules, I want to check out the commit in each submodule that the parent repository is pointing to, possibly after first iterating through all submodules and updating/pulling these from remote.

This is, as pointed out, done with

git submodule foreach git pull origin BRANCH
git submodule update

Scenario 2, which I think is what OP is aiming at

New stuff has happened in one or more submodules, and I want to 1) pull these changes and 2) update the parent repository to point to the HEAD (latest) commit of this/these submodules.

This would be done by

git submodule foreach git pull origin BRANCH
git add module_1_name
git add module_2_name
......
git add module_n_name
git push origin BRANCH

Not very practical, since you would have to hardcode n paths to all n submodules in e.g. a script to update the parent repository's commit pointers.

It would be cool to have an automated iteration through each submodule, updating the parent repository pointer (using git add) to point to the head of the submodule(s).

For this, I made this small Bash script:

git-update-submodules.sh

#!/bin/bash

APP_PATH=$1
shift

if [ -z $APP_PATH ]; then
  echo "Missing 1st argument: should be path to folder of a git repo";
  exit 1;
fi

BRANCH=$1
shift

if [ -z $BRANCH ]; then
  echo "Missing 2nd argument (branch name)";
  exit 1;
fi

echo "Working in: $APP_PATH"
cd $APP_PATH

git checkout $BRANCH && git pull --ff origin $BRANCH

git submodule sync
git submodule init
git submodule update
git submodule foreach "(git checkout $BRANCH && git pull --ff origin $BRANCH && git push origin $BRANCH) || true"

for i in $(git submodule foreach --quiet 'echo $path')
do
  echo "Adding $i to root repo"
  git add "$i"
done

git commit -m "Updated $BRANCH branch of deployment repo to point to latest head of submodules"
git push origin $BRANCH

To run it, execute

git-update-submodules.sh /path/to/base/repo BRANCH_NAME

Elaboration

First of all, I assume that the branch with name $BRANCH (second argument) exists in all repositories. Feel free to make this even more complex.

The first couple of sections is some checking that the arguments are there. Then I pull the parent repository's latest stuff (I prefer to use --ff (fast-forwarding) whenever I'm just doing pulls. I have rebase off, BTW).

git checkout $BRANCH && git pull --ff origin $BRANCH

Then some submodule initializing, might be necessary, if new submodules have been added or are not initialized yet:

git submodule sync
git submodule init
git submodule update

Then I update/pull all submodules:

git submodule foreach "(git checkout $BRANCH && git pull --ff origin $BRANCH && git push origin $BRANCH) || true"

Notice a few things: First of all, I'm chaining some Git commands using && - meaning previous command must execute without error.

After a possible successful pull (if new stuff was found on the remote), I do a push to ensure that a possible merge-commit is not left behind on the client. Again, it only happens if a pull actually brought in new stuff.

Finally, the final || true is ensuring that script continues on errors. To make this work, everything in the iteration must be wrapped in the double-quotes and the Git commands are wrapped in parentheses (operator precedence).

My favourite part:

for i in $(git submodule foreach --quiet 'echo $path')
do
  echo "Adding $i to root repo"
  git add "$i"
done

Iterate all submodules - with --quiet, which removes the 'Entering MODULE_PATH' output. Using 'echo $path' (must be in single-quotes), the path to the submodule gets written to output.

This list of relative submodule paths is captured in an array ($(...)) - finally iterate this and do git add $i to update the parent repository.

Finally, a commit with some message explaining that the parent repository was updated. This commit will be ignored by default, if nothing was done. Push this to origin, and you're done.

I have a script running this in a Jenkins job that chains to a scheduled automated deployment afterwards, and it works like a charm.

I hope this will be of help to someone.