Publish Multiple Robot Test Results From Jenkins Pipeline

Question

I have a Jenkins 2.0 Pipeline script that runs a two separate suite of Robot tests. The script tries to publish both test suite results, however the publisher over-writes the first publish, with the last one.

node('robot') {
    ...
    publishTestResults('journey')
    publishTestResults('regression')
}

void publishTestResults(String type) {
step([
        $class           : 'hudson.plugins.robot.RobotPublisher',
        outputPath       : 'portfolio-app\\target\\robot-output\\' + type,
        passThreshold    : 100,
        unstableThreshold: 100,
        otherFiles       : '',
        reportFileName   : '*\\report*.html',
        logFileName      : '*\\log*.html',
        outputFileName   : '*\\output*.xml'
])

}

From the UI, we see two published results, but both sets are for the regression test cases. The last publish wins.

Is there any way that we can publish two sets of Robot results.

Answer 1

This will not answer your question directly, but it is a possible solution to what you are trying to accomplish.

You can use rebot to combine your two sets of Robot results into one. And then just publish the merged report. There are lots of options with rebot on how to merge reports.

Answer 2

I used a workaround that I publish results only once after all test sets are executed.

Output file and report parameters are set as **/.ext.

Not perfect but it seems it works - I have one 'Robot table' and reports, logs, ... are accessible in subfolders.

stage('Smoke tests'){
    steps {        
        bat('pybot --nostatusrc --outputdir ./robot_reports/smoke <other parameters>')
     }
}

stage('E2E tests'){
    steps {
        bat('pybot --nostatusrc --outputdir ./robot_reports/e2e <other parameters>')
      }
}

stage('Publish Robot results') {
    steps {
        script {
          step(
            [
              $class              : 'RobotPublisher',
              outputPath          : 'robot_reports',
              outputFileName      : "**/output.xml",
              reportFileName      : '**/report.html',
              logFileName         : '**/log.html',
              disableArchiveOutput: false,
              passThreshold       : "${env.ROBOT_PASS_THRESHOLD}" as double,
              unstableThreshold   : "${env.ROBOT_UNSTABLE_THRESHOLD}" as double,
              otherFiles          : "**/*.png,**/*.jpg",
            ]
          )
        }
  }
}