Make gradle fail the build when a cucumber-jvm test fails

Using cucumber-jvm and Gradle, I can’t get my build to fail when a cucumber test fails.

When running with “gradle clean test --info”, my tests are run and the build succeeds, despite obvious errors. I’m sure I’m missing something simple, but I cannot find it.



Expected: is <4>

but: was <2>

at org.hamcrest.MatcherAssert.assertThat(

at org.hamcrest.MatcherAssert.assertThat(

at net.sf.anathema.SiderealSteps.she_has_dots_in_Craft(

at ?.Then she has 4 dots in Craft(net\sf\anathema\SiderealFavoredAbility.feature:7)

Gradle Worker 1 finished executing tests. Process ‘Gradle Worker 1’ finished with exit value 0 (state: SUCCEEDED)


If you can provide a self-contained reproducible example, we can have a look.

I just had another look, and found a link to GRADLE-2615 on SO. I assume that’s my issue as well, gradle and cucumber have different interpretations of the jUnit API.

Yeah, that’s likely the culprit. Last time I checked, Cucumber didn’t honor JUnit’s ‘RunNotifier’ contract. Some tools deal better with that, others less so.

I run cucumber using javaexec and then parse the junit xml report file like so

     * Slurps the XML JUnit report file to determine whether any failures occurred & ensures the test
    protected void reportFailures() {
        final file = new File(junitReportFile)
        if (file.exists()) {
            def report = new XmlSlurper().parse(file)
            final failures
            try {
                failures = report.@failures.toInteger()
            } catch (Exception e) {
                throw new BuildException("Cuke junit report is available ($file.path) but unparseable, failures attribute is not present or is not an integer", e)
            if (failures > 0) {
                if (!ignoreFailures) {
                    throw new GradleException("There were failing tests. See the report at $junitReportFile")
                } else {
                    logger.warn("There were failing tests. See the report at $junitReportFile")
        } else {
            throw new BuildException("Cuke junit report does not exist at $file.path", null)

How do you run cucumber to get an XML report file (which JUnit knows nothing about)?

use the cucumber junit formatter by passing an arg to the cuke cli

'-f', "junit:${junitReportFile}"