How to make Pipeline job to wait for all triggered parallel jobs?

Question

I've Groovy script as part of the Pipeline job in Jenkins as below:

node {
    stage('Testing') {
        build job: 'Test', parameters: [string(name: 'Name', value: 'Foo1')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Bar1')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Baz1')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Foo2')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Bar2')], quietPeriod: 2, wait: false
        build job: 'Test', parameters: [string(name: 'Name', value: 'Baz2')], quietPeriod: 2, wait: false
    }
}

which executes multiple other freestyle jobs in parallel, because of wait flag being set to false. However I'd like for the caller job to finish when all jobs are finished. Currently the Pipeline job triggers all the jobs and finishes it-self after few seconds which is not what I want, because I cannot track the total time and I don't have ability to cancel all triggered jobs at one go.

How do I correct above script for Pipeline job to finish when all jobs in parallel are completed?

I've tried to wrap build jobs in waitUntil {} block, but it didn't work.

Answer 1

You should use pipeline parallel expression, which will wait for all spawned jobs / subtasks to complete:

stage('testing') {
    def branches = [:]

    for(i = 0; i < params.size(); i += 1) {
        def param = params[i]

        branches["Test${i}"] = {
            build job: 'Test', parameters: [string(name: 'Name', value: param)], quietPeriod: 2
        }
    }
    parallel branches
}

You can find some more examples in pipeline docs at jenkins.io

Answer 2

However @agg3l's example is not working with multiple jobs.

Map jobResults = [:]

Boolean failedJobs = false
def buildJobWithParams(def jobs_list, Map results) {
  def branches = [:]    
  for(job in jobs_list)
  {
    print job
    branches["Test-${job}"] = {
       def jobBuild = build job: job, propagate: false
       def jobResult = jobBuild.getResult()
       echo "Build of '${job}' returned result: ${jobResult}"
       results[job] = jobResult
    }
  }    
  return branches
}

stage('Run integration tests') {
      steps {
            def job_branch = buildJobWithParams(item_list, jobResults)
            print job_branch
            parallel job_branch
          }
}

The item_list has more than 1 job, however it will execute only last job multiple times.

Answer 3

Just run into the same problem, and find a working solution. Just use foreach.

stage('testing') {
    def jobs = [:]

    [1,2,3,4,5].each{
        i -> jobs["Test${i}"] = {
            build job: 'Test', 
            parameters: [string(name: 'theparam', value: "${i}")],
            quietPeriod: 2
        }
    }
    parallel jobs
}

Answer 4

This works for me. Triggers 3 jobs. Wait for them to finish. Pay attention to the extra "->" to specify groovy closure. I have one -> on the each loop, and one on the parallel line. It means that the value will be evaluated when running the parallel section.

def jobsString = "job1,job2,job3"
ArrayList jobsList = jobsString.split('\\,')

def parallelJobs2Run = [:]
jobsList.each { job ->
    echo "Going to parallel for job ${job}"
    parallelJobs2Run["${job}"] = { ->
        echo "Calling job ${job}"
        jobResults=build job: "${pathJenkinsFolder}${job}",
        parameters: [
            string(name: 'param1', value: "${value1}"),
            string(name: 'param2', value: "${value2}")
        ],
        propagate: true,
        wait: true

        // List of values: https://stackoverflow.com/questions/46262862/how-to-i-get-the-url-of-build-triggered-with-build-step-on-jenkins
        buildNumber = ${jobResults.number}
        echo "${job} Build number |${buildNumber}| result: |${jobResults.result}|"
        echo "See details on: |${jobResults.absoluteUrl}|"
    }
};
parallel parallelJobs2Run