There appears to be a gradle bug here as a result of the difference between a cuke scenario and a junit test.
JUnitTestEventAdapter#testStarted creates a DefaultTestDescriptor and uses this call to calculate the name of that test
// Use this instead of Description.getMethodName(), it is not available in JUnit <= 4.5
private String methodName(Description description) {
Matcher matcher = methodStringMatcher(description);
if (matcher.matches()) {
return matcher.group(1);
}
return null;
}
cucumber-jvm does not work this way, tests are really scenarios and steps that are described in gherkin feature files. For example, a feature file could be
Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers
Scenario: Add two numbers
Given I have entered 1 into the calculator
And I have entered 2 into the calculator
When I press add
Then the result should be 3 on the screen
Each scenario will be presented as a junit test and each step (the given/when/then lines) will also appear as a test.
The junit runner contains no methods, e.g.
mport cucumber.junit.Cucumber;
import org.junit.runner.RunWith;
@RunWith(Cucumber.class)
@Cucumber.Options(format={"pretty", "html:target/cucumber"}, features = "classpath:", glue = "classpath:", tags = "@beta")
public class BetaCukeRunner {
}
Given this setup, ‘JUnitTestEventAdapter#methodName’ will create a descriptor with name = null. The JUnit Description passed in looks valid as its name is the scenario name and it has children (the individual given/when/then steps).
Later on, the following code will be called in ‘JUnitXmlReportGenerator’ (ultimately from ‘JUnitTestEventAdapter#testStarted’)
@Override
protected void started(TestState state) {
TestDescriptorInternal test = state.test;
// NPE here - test.getName() is null
if (test.getName().equals(test.getClassName())) {
testSuiteReport = documentBuilder.newDocument();
rootElement = testSuiteReport.createElement("testsuite");
testSuiteReport.appendChild(rootElement);
// Add an empty properties element for compatibility
rootElement.appendChild(testSuiteReport.createElement("properties"));
outputs.put(TestOutputEvent.Destination.StdOut, new StringBuilder());
outputs.put(TestOutputEvent.Destination.StdErr, new StringBuilder());
testSuite = state;
}
}
and hence you blow with an NPE. JUnit responds to the failure by removing the failed listener and continuing on its way.
There may still be a problem with how you interpret cucumber results as 1 scenario looks likely to result in the appearance of 3 “junit” tests but not sure if that is really an issue or not. It’s not obvious from the code and I can’t progress beyond the above without patching gradle.