Hi,
we recently upgraded from gradle 1.4 to 1.9 and along the way found this issue related to the generation of the junit XML reports.
We have organized our tests in multiple sub projects. Our tests are defined in some Excel documents and are executed with junit using a custom Junit runner (@RunWith). The custom runner reads excel files and creates instance of junit test classes such as CustomJunitTestCase extends TestCase.
With gradle 1.4, junit XML report file names use suite class names which would create such an output:
/subproject1/build/test-results/TEST-Project1Suite.xml: testsuite name=“Project1Suite” tests=“123” *
/subproject2/build/test-results/TEST-Project2Suite.xml: testsuite name=“Project2Suite” tests=“123”
When updating to gradle 1.9 the xml report file name have changed to use implementation class and are not unique anymore:
/subproject1/build/test-results/TEST-CustomJunitTestCase.xml: testsuite name=“CustomJunitTestCase” tests=“123”
/subproject2/build/test-results/TEST-CustomJunitTestCase.xml: testsuite name=“CustomJunitTestCase” tests=“123”
The issue with gradle 1.9 is that the test reports are not unique and it creates some incorrect test reporting on CI servers. After investigation, this change was introduced by GRADLE-2649 (fixed in 1.5)
As Peter Niederwieser commented in the Jira, test file names can be made unique by including both the suite and its constituents: >“I think we should show both the suite and its constituents (i.e. add one additional level). This is how suites are modelled in JUnit, how they are commonly thought of, how they are displayed by IDEs, and AFAIK also how we report them via the test listener.” http://issues.gradle.org/browse/GRADLE-2649
In our case, this is preventing us to upgrade from 1.4 at the moment.