How to run multiple stages on the same node with declarative Jenkins pipeline?

Question

Goal
Run multiple stages of a declarative Jenkins pipeline on the same node.

Setup
This is just a minimal example to show the problem. There are 2 Windows nodes "windows-slave1" and "windows-slave2" both labeled with the label "windows".

NOTE: My real Jenkinsfile cannot use a global agent because there are groups of stages that require to run on different nodes (e.g. Windows vs. Linux).

Expected Behaviour
Jenkins selects one of the nodes in "Stage 1" based on the label and uses the same node in "Stage 2" because the variable windowsNode was updated to the node selected in "Stage 1".

Actual Behaviour
"Stage 2" sometimes runs on the same and sometimes on a different node than "Stage 1". See the output below.

Jenkinsfile

#!groovy

windowsNode = 'windows'

pipeline {
  agent none
  stages {
    stage('Stage 1') {
      agent {
        label windowsNode
      }
      steps {
        script {
          // all subsequent steps should be run on the same windows node
          windowsNode = NODE_NAME
        }
        echo "windowsNode: $windowsNode, NODE_NAME: $NODE_NAME"
      }
    }
    stage('Stage 2') {
      agent {
        label windowsNode
      }
      steps {
        echo "windowsNode: $windowsNode, NODE_NAME: $NODE_NAME"
      }
    }
  }
}

Output

[Pipeline] stage
[Pipeline] { (Stage 1)
[Pipeline] node
Running on windows-slave2 in C:\Jenkins\workspace\test-agent-allocation@2
[Pipeline] {
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] echo
windowsNode: windows-slave2, NODE_NAME: windows-slave2
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Stage 2)
[Pipeline] node
Running on windows-slave1 in C:\Jenkins\workspace\test-agent-allocation
[Pipeline] {
[Pipeline] echo
windowsNode: windows-slave2, NODE_NAME: windows-slave1
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS

Any ideas what's wrong with the setup? I guess it's how the Jenkinsfile is parsed and executed.

Other suggestions? Maybe there is a Jenkins API to select a node based on the "windows" label when setting windowsNode initially.

Answer 1

Since version 1.3 of Declarative Pipeline plugin, this is officially supported. It's officially called "Sequential Stages".

pipeline {
    agent none

    stages {
        stage("check code style") {
            agent {
                docker "code-style-check-image"
            }
            steps {
                sh "./check-code-style.sh"
            }
        }

        stage("build and test the project") {
            agent {
                docker "build-tools-image"
            }
            stages {
               stage("build") {
                   steps {
                       sh "./build.sh"
                   }
               }
               stage("test") {
                   steps {
                       sh "./test.sh"
                   }
               }
            }
        }
    }
}

Official announcement here: https://jenkins.io/blog/2018/07/02/whats-new-declarative-piepline-13x-sequential-stages/

Answer 2

You could define stages inside a script block. Those stages are kind of sub-stages of a parent stage running in a given agent. That was the approach that I had to use in a similar use case than yours.

#!groovy

windowsNode = 'windows'

pipeline {
  agent none
  stages {
    stage('Stage A') {
      agent {
        label windowsNode
      }
      steps {
        script {

          stage('Stage 1') {        
            windowsNode = NODE_NAME
            echo "windowsNode: $windowsNode, NODE_NAME: $NODE_NAME"
          }

          stage('Stage 2') {
            echo "windowsNode: $windowsNode, NODE_NAME: $NODE_NAME"
          }
        }
      }
    }
  }
}

Answer 3

I have found that this works as you would expect

#!groovy

windowsNode = 'windows'

pipeline {
    agent none
    stages {
        stage('Stage 1') {
            steps {
                node(windowsNode) {
                    script {
                        // all subsequent steps should be run on the same windows node
                        windowsNode = NODE_NAME
                    }
                    echo "windowsNode: $windowsNode, NODE_NAME: $NODE_NAME"
                }
            }
        }
        stage('Stage 2') {
            steps {
                node(windowsNode) {
                    echo "windowsNode: $windowsNode, NODE_NAME: $NODE_NAME"
                }
            }
        }
    }
}

Answer 4

replace agent none with agent any