There is test run duplication when using a TestNG suite XML file with the maxParallelForks gradle test setting

Sud_Ramasamy · June 28, 2014, 4:48pm

There is a bug in either gradle or TestNG when using a TestNG suite XML file and the maxParallelForks option where if you have a second TestNG test class the tests get executed twice. The sample project is located here - https://github.com/sudr/gradle-testng-fork-bug.

Here are the steps to reproduce the issue.

Create a simple project with one TestNG class with a testng.xml file with the TestNG test class in the suite. See here for an example:

github.com

sudr/gradle-testng-fork-bug/blob/master/src/test/resources/testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Suite1">
  <test name="test" >
    <classes>
       <class name="org.gradle.PersonTest" />
    </classes>
  </test>
</suite>

github.com

sudr/gradle-testng-fork-bug/blob/master/src/test/java/org/gradle/PersonTest.java

package org.gradle;

import java.lang.management.ManagementFactory;

import org.testng.Assert;
import org.testng.annotations.Test;

public class PersonTest {
    
    @Test
    public void canConstructAPersonWithAName() throws InterruptedException {
        System.out.println("person started: " +pid());
        Person person = new Person("Larry");
        Assert.assertEquals("Larry", person.getName());
        Thread.sleep(2000);
        System.out.println("person passed: " + pid());
    }
    
    String pid() {
        return ManagementFactory.getRuntimeMXBean().getName();

This file has been truncated. show original

Create a build.gradle that uses TestNG and points to the TestNG suite XML file and contains the maxParallelForks option. See here for an example:

github.com

sudr/gradle-testng-fork-bug/blob/master/build.gradle

apply plugin: 'java'
apply plugin: 'eclipse'

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'org.testng', name: 'testng', version:'6.8.8'
}

test {
    useTestNG() {
        suites 'src/test/resources/testng.xml'
    }
    
    maxParallelForks=5
    
    onOutput { descriptor, event ->
        logger.lifecycle(event.message )

This file has been truncated. show original

Run the gradle build and you should see that single test (PersonTest) be forked and execute.
Now add a second TestNG class to the project. There is no need to add it to the TestNG suite XML. The mere fact that there is another TestNG class in the path triggers the bug.

https://github.com/sudr/gradle-testng-fork-bug/blob/master/src/test/java/org/gradle/AnimalTest.java (you’ll need to uncomment the @Test annotation)

Now re-run the build and you will see that the PersonTest is now executed twice.

Topic		Replies	Views
Unit tests seem to run multiple times when using TestNG with 'suites' XML and maxParallelForks > 1 Old Forum Archive	6	2909	May 26, 2014
Can I run TestNG tasks in parallel (single build project) Help/Discuss	5	3410	April 25, 2017
TestNG not forking test processes Help/Discuss	1	1624	October 5, 2015
Is it possible to run multiple testng xml suites in gradle by using TestNG options in build.gradle file Help/Discuss	1	3412	June 29, 2020
TestNG failures not (always) being reported Old Forum Archive	9	746	July 10, 2012

There is test run duplication when using a TestNG suite XML file with the maxParallelForks gradle test setting

Related Topics