jacocoTestReport consistently skipped for custom test tasks but not standard test task

Help/Discuss
1

Hi,

It appears that jacocoTestReport is always skipped for custom test tasks, e.g. test tasks not named test. Here’s an example (edited for relevance) test setup:

tasks.withType(Test) {
	forkEvery = 1
	finalizedBy jacocoTestReport
}

jacocoTestReport {
	reports {
		xml.required = true
		csv.required = true
	}
}

test {
	// no special config needed
}

task testQuick(type: Test) {
	// for Gradle 9.0 compatibility
	testClassesDirs = testing.suites.test.sources.output.classesDirs
	classpath = testing.suites.test.sources.runtimeClasspath

	filter {
		excludeTestsMatching "*LongTest"
	}
}

In this case, starting from a clean build each time, jacocoTestReport always runs for test but is always skipped for testQuick:

> Task :jacocoTestReport SKIPPED
Skipping task ':jacocoTestReport' as task onlyIf 'Any of the execution data files exists' is false.

Interestingly, for a clean -> test -> testQuick order of operations jacocoTestReport is noted as up to date for testQuick, which seems very wrong.

test produces a jacoco/test.exec file, while testQuick produces a jacoco/testQuick.exec file.

I found a SO answer [1] that provided a work around, essentially forcing jacoco to always use jacoco/coverage.exec to store coverage data for all tests. This allows the coverage data to be built for test and testQuick but

  • Seems hacky - why is this necessary? Shouldn’t this Just Work ™?
  • Has the drawback that the coverage data for test gets overwritten by the data for testQuick and vice versa.

I’m very nooby when it comes to Groovy and Gradle so I’m wondering if there’s a better way to make this work. I don’t see any obvious answers in the jacoco docs [2].

Thanks in advance for any advice

  1. Gradle skipping jacoco coverage plugin - Stack Overflow
  2. The JaCoCo Plugin
2

You misinterpret things.

jacocoTestReport produces at JaCoCo report for the coverage recorded during the test task, not any task of type Test.

The testQuick task is not the test task, so when the jacocoTestReport task is to be executed it sees that there is no input and thus is skipped, as the input would come from the test task, not the testQuick task.

How to “properly” configure it highly depends on what you actually want to achieve.
Assuming that you don’t want to run test and testQuick at the same time, you could for example configure the test task depending on some project property you then specify with -P instead of creating a second task.

3

I see, thanks for the answer. You’re right, I didn’t realize that jacocoTestReport was bound specifically to the the test task.

How I would like things to work is to have the two test tasks, test and testQuick, and have jacoco reports produced for each one when they run, and have the reports be independent of each other.

For example, if we pretend the Test portion of jacocoTestReport specifies the test task (which I’m sure it does not), I could imagine a test config like:

tasks.withType(Test) {
	forkEvery = 1
}

jacocoTestReport {
	reports {
		xml.required = true
		csv.required = true
	}
}

jacocoTestQuickReport {
	reports {
		xml.required = true
		csv.required = true
	}
}


test {
	finalizedBy jacocoTestReport
}

task testQuick(type: Test) {
	// for Gradle 9.0 compatibility
	testClassesDirs = testing.suites.test.sources.output.classesDirs
	classpath = testing.suites.test.sources.runtimeClasspath

	filter {
		excludeTestsMatching "*LongTest"
	}
	finalizedBy jacocoTestQuickReport
}

The behavior I’m imagining here is that the user can run the test task and testQuick task and use the jacoco output to compare coverage between the two scenarios, among other uses.

Is that possible?

Thanks again for the answer and in advance of any further information

4

Sure, just register a second jacoco report task like you add a second test task.

5

Alternatively you could also use the JVM test suites configuration to create a second test suite instead of defining the test task manually.

Because for proper test suites, the JaCoCo plugin automatically adds the matching JaCoCo tasks already.

6

Thanks again. Excuse my noobidity, but I don’t supposed I could prevail upon you for an example for the dual test approach? I’m stuck with a chicken / egg problem trying to get an example I found to work:

tasks.withType(Test) {
	forkEvery = 1
}

tasks.withType(JacocoReport) {
	reports {
		xml.required = true
		csv.required = true
	}
}

test {
	finalizedBy jacocoTestReport
}


task testQuick(type: Test) {
	// for Gradle 9.0 compatibility
	testClassesDirs = testing.suites.test.sources.output.classesDirs
	classpath = testing.suites.test.sources.runtimeClasspath

	filter {
		excludeTestsMatching "*LongTest"
	}
	finalizedBy jacocoTestQuickReport
}

task jacocoTestQuickReport(type: JacocoReport, dependsOn: testQuick) {
	executionData(testQuick)
}

This set up fails because the jacocoTestQuickReport symbol doesn’t exist when testQuick is defined. If I move the jacocoTestQuickReport task above testQuick, then the testQuick symbol doesn’t exist.

7

There are several ways.

You can use a String with the task name for finalizedBy.

Or you can configure the testQuick task (again) after the jacocoTestQuickReport task is also created.

Or you could stop breaking task-configuration avoidance by registering the tasks instead of creating them directly like you do, because then the configuration is done later when both already are added (and also that bad usage of withType).

I also highly recommend to switch to the Kotlin DSL. By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and an amazingly better IDE support if you use a good IDE like IntelliJ IDE or Android Studio.

8

It turns out I was missing a required configuration line in the jacoco task:

task jacocoTestQuickReport(type: JacocoReport, dependsOn: testQuick) {
	sourceSets sourceSets.main
	executionData(testQuick)
}
1 Like