Gradle 2.3, TestNG6.7 with RetryAnalyzer, failed count differs in TestNG report and Gradle report


(ygsh) #1

Hi,

We are using Gradle 2.3, TestNG6.7 with RetryAnalyzer. In my build.gradle I have set useDefaultListeners=true inside the useTestNG() task.

The problem is when tests executes, Gradle marks a test that executed 2 times and finally passed as failed. TestNG correctly picks this as passed. So, now I have 100% pass in TestNG report but few failures in Gradle Tests tab.

Also, Gradle status in Teamcity is able to pick that test was run 2 times with 1 failure. I see under status: Failure 1 of 2, but Gradle is choosing to mark final status as failed.

I have seen multiple posts on this topic on this forum as well as other forums but none seems to be aware of any answers

Any help is appreciated!

Thanks,
Ygsh


(ygsh) #2

Just another input - useDefaultListeners=true or false seems to have no effect on the eventual output. It always retries and always sets the output as fail if there was one instance of failure.


(ygsh) #3

Ok - I could not find any solution to this. Below is what I did on Teamcity.

Remove Gradle runner, instead use CMD with Gradle command. Then in Build finish step, add a ANT Junit XML report parser to my test report in agent machine. Since this Junit report is generated by TestNG itself, the results are accurate.

I know this is ugly, but this is what I will have to use until I can update Gradle, TestNG and Teamcity.