There is test run duplication when using a TestNG suite XML file with the maxParallelForks gradle test setting


(Sud Ramasamy) #1

There is a bug in either gradle or TestNG when using a TestNG suite XML file and the maxParallelForks option where if you have a second TestNG test class the tests get executed twice. The sample project is located here - https://github.com/sudr/gradle-testng-fork-bug.

Here are the steps to reproduce the issue.

  1. Create a simple project with one TestNG class with a testng.xml file with the TestNG test class in the suite. See here for an example:
  1. Create a build.gradle that uses TestNG and points to the TestNG suite XML file and contains the maxParallelForks option. See here for an example:
  1. Run the gradle build and you should see that single test (PersonTest) be forked and execute.

  2. Now add a second TestNG class to the project. There is no need to add it to the TestNG suite XML. The mere fact that there is another TestNG class in the path triggers the bug.

https://github.com/sudr/gradle-testng-fork-bug/blob/master/src/test/java/org/gradle/AnimalTest.java (you’ll need to uncomment the @Test annotation)

  1. Now re-run the build and you will see that the PersonTest is now executed twice.