Gradle build fails even Tests passes through TestNG retry mechanism

Help/Discuss
1

Hello,
I have implemented retry mechanism in my TestNG listerners and I’m running my tests from gradle. During the execution, a test will fail for the 1st run and it immediately retries and passes during the 2nd run. But, my overall gradle build shows as FAIL (since it failed in 1st run) and same will be reflected in Jenkins.

When I’m running from testNG, I can handle it correctly at testNG level(I mean, I’m able to delete the duplicate tests).

I’m new to gradle. So, could you please help me in overcoming this issue. here is my build.gradle and the output.

//Test Logging
tasks.withType(Test) {
testLogging {

    // set options for log level LIFECYCLE
    events "passed", "skipped", "failed"
    showExceptions true
    exceptionFormat "short"
    showCauses true
    showStackTraces true
    
    info.events = debug.events
    info.exceptionFormat = debug.exceptionFormat
    
    afterTest { desc, result ->
        println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
        def testdesc = desc.name
        def sResultType = result.resultType
        if (testdesc.contains('Test[0]')) {
        	println("Inside if condition")
        	if (sResultType.toString().contains('FAILURE')) {
        		println ("test FAILED")
        		//throw new StopExecutionException("skip this task") ...this is not working
        	}
        	else { 
        		println ("Test PASSED")
        	}	
        }
        else {
        	println "It is the Retry scenario"
        }
    }
	
	
    afterSuite { desc, result ->
        if (!desc.parent) { // will match the outermost suite
            def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
            def startItem = '|  ', endItem = '  |'
            def repeatLength = startItem.length() + output.length() + endItem.length()
            println('\n' + ('-' * repeatLength) + '\n' + startItem + output + endItem + '\n' + ('-' * repeatLength))
        }
    }
}

}

test {
outputs.upToDateWhen { false }
useTestNG() {
suites file(“testng_single.xml”)
}
}

#########################Output################################

c:\Eclipse_Workspace\MyProject>gradle clean test
:clean
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava
:processTestResources UP-TO-DATE
:testClasses
:test
Executing test Test[0](chrome, WINDOWS, Chrome) [scripts.EditMyAccount] with res
ult: FAILURE
Inside if condition
test FAILED

Suite > Airlines > scripts.EditMyAccount.Test[0](chrome, WINDOWS, Chrome) FAILED

java.lang.AssertionError at EditMyAccount.java:66

Executing test Test[1](chrome, WINDOWS, Chrome) [scripts.EditMyAccount] with res
ult: SUCCESS
else condition

Suite > Airlines > scripts.EditMyAccount.Test[1](chrome, WINDOWS, Chrome) PASSED

Results: FAILURE (2 tests, 1 successes, 1 failures, 0 skipped)

2 tests completed, 1 failed
:test FAILED

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ‘:test’.

There were failing tests. See the report at: file:///C:/Eclipse_Workspace/MyProject/build/reports/tests/index.html

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug
    option to get more log output.

BUILD FAILED

Total time: 1 mins 17.08 secs

c:\Eclipse_Workspace\MyProject>

2

Hi,

Did you ever find a solution to your problem?

Thanks