How to wait for user input in a Declarative Pipeline without blocking a heavyweight executor

Question

I'm rebuilding an existing build pipeline as a jenkins declarative pipeline (multi-branch-pipeline) and have a problem handling build propagation.

After packaging and stashing all relevant files the pipeline is supposed to wait for user input to trigger deployment.

If i just add an input step the current build-node is blocked. As this executor is pretty heavy i would like to move this step to a more lightweight machine.

Initially i did the job as a scripted pipeline and just created two different node('label') blocks. is there a way for me to do something similar with the declarative syntax?

node('spine') { 
    stage('builder') {
        sh 'mvn clean compile'
        stash name: 'artifact', includes: 'target/*.war'
    }
}
node('lightweight') {
    stage('wait') {
        timeout(time:5, unit:'DAYS') {
            input message:'Approve deployment?'
        }
    }
    // add deployment stages
}

I tried a couple of things already:

configuring the agent on the top-level and adding an additional agent config to the propagation step, but then i have two executors blocking as the top-level defined build-node is not stopped.

Setting agent none on top-level and configuring the agents per step. then the git checkout is not present on the first node.

EDIT 1

i reconfigured my pipeline following you advice, it currently looks like this:

pipeline {
agent none
tools {
    maven 'M3'
}
stages {
    stage('Build') {
        agent { label 'spine' }
        steps {
            checkout scm // needed, otherwise the workspace on the first step is empty
            sh "mvn clean compile"
        }
    }
    stage('Test') {
        agent { label 'spine' }
        steps {
            sh "mvn verify" // fails because the workspace is empty aggain
            junit '**/target/surefire-reports/TEST-*.xml'
        }
    }
}
}

this build will fail because the workspace does not carry over between steps as they dont run on the same executor.

EDIT 2

apparently sometimes the steps run on the same executor and sometimes don't. (we spawn build slaves on our mesos/dcos cluster on demand, so changing the executor mid build would be a problem)

I expected jenkins to just run with the current executor as long as the label in the agent definition does not change.

Answer 1

See best practice 7: Don’t: Use input within a node block. In a declarative pipeline, the node selection is done through the agent directive.

The documentation here describes how you can define none for the pipline and then use a stage-level agent directive to run the stages on the required nodes. I tried the opposite too (define a global agent on some node and then define none on stage-level for the input), but that doesn't work. If the pipeline allocated a slave, you can't release the slave for one or more specific stages.

This is the structure of our pipeline:

pipeline {
  agent none
  stages {
    stage('Build') {
      agent { label 'yona' }
      steps {
        ...
      }
    }
    stage('Decide tag on Docker Hub') {
      agent none
      steps {
        script {
          env.TAG_ON_DOCKER_HUB = input message: 'User input required',
              parameters: [choice(name: 'Tag on Docker Hub', choices: 'no\nyes', description: 'Choose "yes" if you want to deploy this build')]
        }
      }
    }
    stage('Tag on Docker Hub') {
      agent { label 'yona' }
      when {
        environment name: 'TAG_ON_DOCKER_HUB', value: 'yes'
      }
      steps {
        ...
      }
    }
  }
}

Generally, the build stages execute on a build slave labeled "yona", but the input stage runs on the master.

Answer 2

Another way to do it is using the expression directive and beforeAgent, which skips the "decide" step and avoids messing with the "env" global:

pipeline {
    agent none

    stages {
        stage('Tag on Docker Hub') {
            when {
                expression {
                    input message: 'Tag on Docker Hub?'
                    // if input is Aborted, the whole build will fail, otherwise
                    // we must return true to continue
                    return true
                }
                beforeAgent true
            }

            agent { label 'yona' }

            steps {
                ...
            }
        }
    }
}

Answer 3

use agent none on top and define agent for every stage except the stage including the input step.

source: discussion in Use a lightweight executor for a declarative pipeline stage (agent none)

Updated: what do you mean by "the git checkout is not present on the first node"? please show what you've got so far for the declarative pipeline.

Answer 4

I know this thread is old, but I believe a solution to the "Edit 2" issue, besides stashing, is to use nested stages.

https://jenkins.io/blog/2018/07/02/whats-new-declarative-piepline-13x-sequential-stages/#running-multiple-stages-with-the-same-agent-or-environment-or-options

According to this page:

... if you are using multiple agents in your Pipeline, but would like to be sure that stages using the same agent use the same workspace, you can use a parent stage with an agent directive on it, and then all the stages inside its stages directive will run on the same executor, in the same workspace.

Here is the example provided:

pipeline {
    agent none

    stages {
        stage("build and test the project") {
            agent {
                docker "our-build-tools-image"
            }
            stages {
               stage("build") {
                   steps {
                       sh "./build.sh"
                   }
               }
               stage("test") {
                   steps {
                       sh "./test.sh"
                   }
               }
            }
            post {
                success {
                    stash name: "artifacts", includes: "artifacts/**/*"
                }
            }
        }

        stage("deploy the artifacts if a user confirms") {
            input {
                message "Should we deploy the project?"
            }
            agent {
                docker "our-deploy-tools-image"
            }
            steps {
                sh "./deploy.sh"
            }
        }
    }
}