Gradle does not generate TestNG report for longer builds

Help/Discuss
1

I am facing this issue that when there are large number of tests and test task runs for longer duration, then Gradle test task does not generate TestNG reports (emailable-report.html, testng-results.xml, as well as junitreports and other TestNG related JS and CSS files). I am expecting it to be generated under build/reports/tests/test/ path but any TestNG related files are not created.

I am using following version of dependencies:-

  • TestNG 7.10.1
  • Gradle 8.5
  • JDK 17

I am running tests with gradle clean test. Here is how I have already enabled listeners in my build.gradle to produce testng reports:-

test {
   useTestNG() {
     useDefaultListeners = true
   }
}

Additional Context:-

Intrestingly this issue does not happen if I only run a subset of smoke tests rather than running a whole regression test suite which takes longer (above 1 hour with 60+ tests). As you can see from the screenshot of build directory, there is no TestNG related XML, HTML, and other asserts.

Screenshot 2024-05-09 at 22.18.10
Screenshot 2024-05-09 at 22.18.10248×633 16.8 KB

I am attaching the link to Gradle debug logs as well:
gradle-debug-logs-gist

I am assuming that this happens because somehow Gradle task interferes with the TestNG reports creation resulting in overwriting or deletion of the TestNG reports. I am not sure whether disabling the Gradle report will resolve this issue and how should this be done?
Does anyone know if there is a way to properly generate TestNG reports with Gradle? I need this testng-results.xml for further processing but I am unable to generate it if I run a larger regression suite.

2

I was able to find out reason why TestNG reports were not being produced. It turned out, for some of my tests, NullPointerException was being thrown in the test listener causing Gradle to throw this below exception and not generating TestNG reports
org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 30.
It seems to me a more gradle specific issue where it was not able to handle this exception coming from testng listener class implementation.

On my end, I was able to resolve this issue by implementing a nullsafe mechanism so tests now don’t throw NullPointerException leading to successfull creation of testng-results.xml report and other TestNG reports.

I have mentioned the steps to reproduce and details in the issue I created at gradle:
gradle/gradle#29098 (comment)

1 Like