How to parametrize Maven surefire plugin so I can choose which TestNG suites to run

Question

I've got many test suites in TestNG. These are XML files. I want to be able to choose multiple XML suites when running integration-test from maven.

Currently I can add the suite files to pom.xml like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <suiteXmlFiles>
      <suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
      <suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
</plugin>

This solution has some limitations. I can only change a path to the test suite I've got defined in pom.xml. So in my example it always has to be two files. I'm not able to run, lets say, 5 suites or just one.

Is there a way to somehow parametrize the whole section "suiteXmlFiles" in pom.xml ?

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <suiteXmlFiles>
      ${multiple_paths_ToMySuiteFiles}
    </suiteXmlFiles>
  </configuration>
</plugin>

Running everything that matches given test group is not an option for me: I don't want to load all the suites I've got and then run just the selected tests using groups in TestNG suite. The reason being that a report that gets generated after running all the test suites with group filters is different from a report when just the selected test suites were run.

Answer 1

According to User Property of suiteXmlFiles You can use:

mvn test -Dsurefire.suiteXmlFiles=test1.xml,test2.xml

Answer 2

We also hit this issue with our tests. The current workaround that we use now is to define a property variable in the property section and inject it in sure-fire suiteXmlFiles block.

<properties>
    <!-- Default suites -->
    <batsSuiteFile>${project.build.testOutputDirectory}/BatsTests.xml</batsSuiteFile>
    <smokeSuiteFile>${project.build.testOutputDirectory}/SmokeTests.xml</smokeSuiteFile>

    <!-- Default suite files if not being specified from mvn command line -->
    <defaultSuiteFiles>${batsSuiteFile},${smokeSuiteFile}</defaultSuiteFiles>
    <suiteFile>${defaultSuiteFiles}</suiteFile>
</properties>

Then in the plugin section...

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
       <suiteXmlFiles>
          <!-- Suite file injection parameter from command line -->
          <suiteXmlFile>${suiteFile}</suiteXmlFile>
       </suiteXmlFiles>
    </configuration>
</plugin>

If nothing is specified from the command line it will fall back and default to the 2 suites specified above in the properties section. Then if you want to kick off a specified set of suite files you can do:

mvn test -DsuiteFile=test1.xml,test2.xml

Surprisingly from the maven docs you expect that the arg suiteXmlFiles should just override this from the command line and should just accept a comma delimited list of testng xmls http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#suiteXmlFiles

If anyone has any other better way please share.

Answer 3

You can use groups parameter or specify -Dgroups=... in the command line:

(TestNG/JUnit47 provider with JUnit4.8+ only) Groups for this test. Only classes/methods/etc decorated with one of the groups specified here will be included in test run, if specified. For JUnit, this parameter forces the use of the 4.7 provider This parameter is ignored if the suiteXmlFiles parameter is specified. .