1. 技术文章
  2. Chain automated builds in the same Docker Hub repository

Chain automated builds in the same Docker Hub repository

由 霸气de小男生 提交于 2019-12-03 16:23:57

Recently had the same requirement to chain dependent builds, and achieved it this way using Docker Cloud automated builds:

  • Create a repository with build rules for each Dockerfile that needs to be built.
  • Disable the Autobuild option for all build rules in dependent repositories.

  • Add a shell script named hooks\post_push in each directory containing a Dockerfile that has dependents with the following code:

    for url in $(echo $BUILD_TRIGGERS | sed "s/,/ /g"); do
  curl -X POST -H "Content-Type: application/json" --data "{ \"build\": true, \"source_name\": \"$SOURCE_BRANCH\" }" $url
done

  • For each repository with dependents add a Build Environment Variable named BUILD_TRIGGERS to the automated build, and set the Value to a comma separated list of the build trigger URLs of each dependent automated build.

Using this setup a push to the root source repository will trigger a build of the root image, once it completes and is pushed the post_push hook will be executed. In the hook a POST is made to each dependent repositories build trigger, containing the name of the branch or tag being built in the requests body. This will cause the appropriate build rule of the dependent repository to be triggered.

How long is the build taking? Can you post your Dockerfile?

Option 1: is to find out what is taking so long with your automated build to see why it isn't finishing in time. If you post it here, we can see if there is anything you can do to optimize.

Option 2: Is what you are already doing now, using a 3rd party app to trigger the builds in the given order.

Option 3: I'm not sure if this will work for you, since you are using the same repo, but normally you would use repo links for this feature, and then chain them, when one finishes, the next one triggers the first. But since you have one repo, it won't work.

Option 4: Break it up into multiple repos, then you can use repo links.

Option 5: Total hack, last resort (not sure if it will work). You add a CURL statement on the last line of your Dockerfile, to post to the build trigger link of the repo with the given tag for the next step. You might need to add a sleep in the next step to wait for the push to finish getting pushed to the hub, if you need one tag for the next.

Honestly, the best one is Option 1: what ever you are doing should be able to finish in the allotted time, you are probably doing some things we can optimize to make the whole thing faster. If you get it to come in under the time limit, then everything else isn't needed.

It's possible to do this by tweaking the Build Settings in the Docker Hub repositories.

First, create an Automated Build for /step-1.Dockerfile of your GitHub repository, with the tag step-1. This one doesn't require any special settings.

Next, create another Automated Build for /step-2.Dockerfile of your GitHub repository, with the tag step-2. In the Build Settings, uncheck When active, builds will happen automatically on pushes. Also add a Repository Link to me/step-1.

Do the same for step-3 (linking it to me/step-2).

Now, when you push to the GitHub repository, it will trigger step-1 to build; when that finishes, step-2 will build, and after that, step-3 will build.

Note that you need to wait for the previous stage to successfully build once before you can add a repository link to it.

I just tried the other answer and they are not working for me, so I invented another way of chaining builds by using separate branch for each build rule, e.g.:

master              # This is for docker image tagged base
docker-build-stage1 # tag stage1
docker-build-latest # tag latest
docker-build-dev    # tag dev

in which stage1 is dependent on base, latest is dependent on stage1, dev is based on latest.

In each of the dependencies’ post_push hook, I called the script below with the direct dependents of itself:

#!/bin/bash -x

git clone https://github.com/NobodyXu/llvm-toolchain.git
cd llvm-toolchain
git checkout ${1}
git merge --ff-only master

# Set up push.default for push
git config --local push.default simple

# Set up username and passwd
# About the credential, see my other answer:
#     https://stackoverflow.com/a/57532225/8375400
git config --local credential.helper store
echo "https://${GITHUB_ROBOT_USER}:${GITHUB_ROBOT_ACCESS_TOKEN}@github.com" > ~/.git-credentials

exec git push origin HEAD

The variables GITHUB_ROBOT_USER and GITHUB_ROBOT_ACCESS_TOKEN are environment variables set in docker hub auto build configuration.

Personally, I prefer to register a new robot account with 2FA enabled on github specifically for this, invite the robot account to become a collaborator and use access token instead of password as it is safer than using your own account which has access to far more repositories than needed and is also easy to manage.

Edit:

You need to disable repository link, otherwise there will be a lot of unexpected build jobs in docker hub.

If you want to see a demo of this solution, check NobodyXu/llvm-toolchain.

标签
heroku
web-applications
Docker
build-automation
dockerhub
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!