Jenkins: Retrieving submodules with Git

问题

Currently I've been stuck on an issue trying to retrieve the submodules of a repository from within Jenkins. My configuration is fine and I can pull repositories without any submodules just fine.

I can also pull the main components of a repo with submodules (both with authentication in the repository-name as with SSH). The problem only arises when I have to pull the submodule-components. I'm running the latest version of Jenkins and I've added a part at the bottom which is for "Advanced sub-modules behaviours". I selected "Recursively update submodules" here and ran the build several times to no avail.

When I try adding an extra build step at the bottom with shell-commands, the updating of the repositories doesn't work either. When I try these commands outside of jenkins in my terminal, this works just fine. The issue I always get in Jenkins is:

FATAL: Command "git submodule update" returned status code 1:

stdout: 

stderr: Cloning into 'thisismysubmodule'...
fatal: Authentication failed for 'https://git.thisismyrepo.com/scm/ap/thisismysubmodule.git/'

I've found this issue: https://issues.jenkins-ci.org/browse/JENKINS-20941 but I can't use the suggested solution at the bottom due to security concerns. Does anyone here have any experience with this problem or a possible solution?

回答1:

Here is a workaround using SSH agent forwarding. It worked fine for me.

1. First, edit <jenkins_home>/.ssh/config and set ForwardAgent yes
2. Then, install SSH Agent plugin for Jenkins.
3. Then, in project configuration, reset the Git credentials.
4. Finally, in project configuration, set the SSH Agent credential.

回答2:

Beta versions of the git-client and git-plug-in modules have been released now to solve this issue. To quote the JIRA issue.

The git client plugin 2.0.0-beta1 has been released to the experimental update center. It includes git submodule authentication, JGit 4.3, and requires JDK 7. It requires at least Jenkins 1.625 (the first version to mandate JDK 7)

Using the above there is an option under the Additional Behaviours section called:

Use credentials from default remote of parent repository

This solved my issue when using Jenkins 2.11 and the beta versions of the plug-in running a Windows server and slave. I have not checked other build machines. Also you must use the same authentication method, if using http you must use that for submodules as well, if use SSH you must use this for the submodules, trying to mix methods will not work correctly.

-- UPDATE --
Beta versions are not required anymore, please see the following pages:
Git Client Plugin
Git Plugin

回答3:

One solution would be to declare in the global git config file a netrc credential help, which would provide the necessary credentials for any http query coming from git.

git config --global credential.helper "netrc -f C:/path/to/_netrc.gpg -v"

(make sure to use the same account as the one used for running Jenkins)

I use an encrypted netrc file, but you can start some test with an un-encrypted one.

回答4:

Considering I have tried pretty much all available options to get this working (SSH, .netrc, hardcoded credentials,...) the only option was the one which was mentioned on the bottom of the JENKINS-20941 issue by 'andreg':

Yeah, this issue is a real pain for us too. The only way we could get it working for our Stash/Jenkins setup was to create a read-only user and to hard code this user's credentials in the reference to the submodule. Although bad practise, all users working on the git repo already have at least read-only access, so we didn't feel it was too much of a security concern. e.g. in the parent repo .gitmodules file:

[submodule "shared-library"] path = shared-library url = https://username:password@stash.yourcompany.com/scm/project/shared-library.git

and then the Jenkins job has "Recursively update submodules" selected.

回答5:

I actually just moved it to a shell command and in the shell command I told it which credentials helper to use, since I'm on windows it was wincred:

git config --global credential.helper wincred
git submodule init 
git submodule sync 
git submodule update --init --recursive

回答6:

I managed to get this working by simply adding a .netrc file with the credentials on linux. Not the most secure solution but if you need to get it working quickly it will get you going.

回答7:

This solution worked for me:

1. Make sure ssh credentials are the same for the parent repo and the submodule repo.
2. Setup credentials for the parent repo in Jenkins. (Source Code Management (choose "Git") > Repositories)

3. Setup your .gitmodule file so that it uses the ssh credentials from the parent repo:

   [submodule "foo/repository"]
      path = foo/repository
      url = ssh://jenkins_build_monkey@domain.com/repository
   

4. There is a small caveat to this solution. Anytime that someone clones the parent repo, they will need to sync the url to one that they have access to. The first time a user initializes the submodule, they need to first edit the .gitmodules file and change the url:

   [submodule "foo/repository"]
      path = foo/repository
      url = ssh://username@domain.com/repository
   

   Then, in the terminal:

   git submodule sync
   git submodule init repository
   git submodule update --remote repository
   

   And then change the url back to the jenkins build url. Unless you sync again, the submodule will use the url associated with the user.

In September 2016, Jenkins is planning to release a new feature that will allow submodules to share the credentials of the parent repository. Then a HTTP url can be used in .gitmodule instead of ssh.

来源：https://stackoverflow.com/questions/25134106/jenkins-retrieving-submodules-with-git