How do I prevent two pipeline jenkins jobs of the same type to run in parallel on the same node?

Question

I do not want to allow two jobs of the same type (same repository) to run in parallel on the same node.

How can I do this using groovy inside Jenkinsfile ?

Answer 1

The answer provided in https://stackoverflow.com/a/43963315/6839445 is deprecated.

The current method to disable concurrent builds is to set options:

options { disableConcurrentBuilds() }

Detailed description is available here: https://jenkins.io/doc/book/pipeline/syntax/#options

Answer 2

Install Jenkins Lockable Resources Plugin.

In your pipeline script wrap the part in the lock block and give this lockable resource a name.

lock("test-server"){
    // your steps here
}

Use the name of whatever resource you are locking. In my experience its usually a test server or test database.

Answer 3

You got at the disableConcurrentBuilds property:

properties properties: [
  ...
  disableConcurrentBuilds(),
  ...
]

Then the job would wait the older one to finish first

Answer 4

Another way is to use the Lockable Resources plugin: https://wiki.jenkins-ci.org/display/JENKINS/Lockable+Resources+Plugin

You can define locks (mutexes) however you want and can put variables in the names. E.g. to prevent multiple jobs from using a compiler concurrently on a build node:

stage('Build') {
    lock(resource: "compiler_${env.NODE_NAME}", inversePrecedence: true) {
      milestone 1
      sh "fastlane build_release"
    }
}

So if you wanted to prevent more than one job of the same branch running concurrently per node you could do something like

stage('Build') {
    lock(resource: "lock_${env.NODE_NAME}_${env.BRANCH_NAME}", inversePrecedence: true) {
      milestone 1
      sh "fastlane build_release"
    }
}

From: https://www.quernus.co.uk/2016/10/19/lockable-resources-jenkins-pipeline-builds/

Answer 5

One of the options is to use Jenkins REST API. I researched for another options, but seems that this is only one available with pipelines functionality.

You should write script which polls Jenkins for info of current jobs running and check whether job of same type is running. To do this you should use Jenkins REST API, documentation you may find in the right bottom corner in your Jenkins page. Example script:

#!/usr/bin/env bash

# this script waits for integration test build finish
# usage: ./wait-for-tests.sh <jenkins_user_id> <jenkins_user_token_id>
jenkins_user=$1
jenkins_token=$2
build_number=$3

job_name="integration-tests"
branch="develop"

previous_build_number=build_number
let previous_build_number-=1
previous_job_status=$(curl -s http://${jenkins_user}:${jenkins_token}@jenkins.mycompany.com/job/mycompany/job/${job_name}/branch/${branch}/${previous_build_number}/api/json | jq -r '.result')

while [ "$previous_job_status" == "null" ];
do
    previous_job_status=$(curl -s http://${jenkins_user}:${jenkins_token}@jenkins.mycompany.com/job/mycompany/job/${job_name}/branch/${branch}/${previous_build_number}/api/json | jq -r '.result')
    echo "Waiting for tests completion"
    sleep 10
done

echo "Seems that tests are finished."

I've used bash here, but you may use any language. Then just call this script inside of your Jenkinsfile:

sh "./wait-for-tests.sh ${env.REMOTE_USER} ${env.REMOTE_TOKEN} ${env.BUILD_NUMBER}"

So it will wait until job completion (don't be confused with integration-test mentions, it's just job name).

Be also aware that in rare cases this script may cause deadlock when both jobs are waiting for each other, so you may want to implement some max retry policies here instead of infinite waiting.

Answer 6

Example using options block in the declarative pipeline syntax:

pipeline {

  options { 
    disableConcurrentBuilds() 
  }

...
}