Gradle 2.3, TestNG6.7 with RetryAnalyzer, failed count differs in TestNG report and Gradle report

(ygsh) #1


We are using Gradle 2.3, TestNG6.7 with RetryAnalyzer. In my build.gradle I have set useDefaultListeners=true inside the useTestNG() task.

The problem is when tests executes, Gradle marks a test that executed 2 times and finally passed as failed. TestNG correctly picks this as passed. So, now I have 100% pass in TestNG report but few failures in Gradle Tests tab.

Also, Gradle status in Teamcity is able to pick that test was run 2 times with 1 failure. I see under status: Failure 1 of 2, but Gradle is choosing to mark final status as failed.

I have seen multiple posts on this topic on this forum as well as other forums but none seems to be aware of any answers

Any help is appreciated!


(ygsh) #2

Just another input - useDefaultListeners=true or false seems to have no effect on the eventual output. It always retries and always sets the output as fail if there was one instance of failure.

(ygsh) #3

Ok - I could not find any solution to this. Below is what I did on Teamcity.

Remove Gradle runner, instead use CMD with Gradle command. Then in Build finish step, add a ANT Junit XML report parser to my test report in agent machine. Since this Junit report is generated by TestNG itself, the results are accurate.

I know this is ugly, but this is what I will have to use until I can update Gradle, TestNG and Teamcity.