I’m trying to follow the new test-suites approach to testing laid out in this blog post, using Gradle 8.4. I now have both a test suite and an integrationTest suite, and they each produce their own reports.
The post suggests using the test-report-aggregation plugin to merge the two reports together, but I haven’t been able to find a way to do that. The plugin adds tasks called testAggregateTestReport and integrationTestAggregateTestReport. Is there any way to configure the reporting so that both test reports are merged? All the examples I’ve seen involve multiple subprojects, not multiple types of test tasks.
I’ve appended my build.gradle.kts file here:
@file:Suppress("UnstableApiUsage")
plugins {
application
id("test-report-aggregation")
}
repositories {
mavenCentral()
}
dependencies {
implementation("com.google.guava:guava:32.1.1-jre")
implementation("ch.qos.logback:logback-classic:1.4.6")
}
testing {
suites {
// Configure the built-in test suite
val test by getting(JvmTestSuite::class) {
// Use JUnit Jupiter test framework
useJUnitJupiter("5.10.0")
}
val integrationTest by registering(JvmTestSuite::class) {
dependencies {
useJUnitJupiter("5.10.0")
implementation(project())
implementation("org.assertj:assertj-core:3.21.0")
implementation("org.postgresql:postgresql:42.6.0")
implementation("org.testcontainers:postgresql:1.19.1")
implementation("org.testcontainers:junit-jupiter:1.19.1")
}
targets {
all {
testTask.configure {
shouldRunAfter(test)
}
}
}
}
}
}
reporting {
reports {
testing {
// Maybe correct configuration goes here?
}
}
}
tasks.named("check") {
dependsOn(testing.suites.named("integrationTest"))
}
application {
mainClass.set("com.kousenit.App") // generated by "gradle init"
}
Try adding the following configuration to your build.gradle.kts file:
reporting { reports { testing { // Aggregate the test reports from the testandintegrationTest suites aggregateReports("test", "integrationTest") } } }
This should create a new test report task called testAggregateTestReport , which will merge the test reports from the test and integrationTest suites. You can then run this task to generate the aggregated test report.
To run the aggregated test report task:
./gradlew testAggregateTestReport
This will generate the aggregated test report in the build/reports/tests/aggregated-results directory.
Can you share a more complete and working example?
What you showed is the same as leaving out reporting { reports { ... } } as that is just visual clutter and testing refers to the top-level testing extension.
And in that no aggregateReports method is available.
And afaik the test report aggregation plugin can also not out-of-the-box aggregate results from multiple test tasks, but only supports aggregating test results per test type.
I think to aggregate different types together, you need to do it manually, setting the test results to use on a TestReport task. If within one project directly, if through multiple projects like the plugin, then by using multiple lenient artifact views for the different test types to aggregate.
Ok, here is a more detailed solution. To merge the test reports from your test and integrationTest suites using the test-report-aggregation plugin in Gradle 8.4, you can add the following configuration to your build.gradle.kts file:
reporting {
reports {
testing {
// Configure the test report aggregation plugin
testReportAggregation {
// Specify the test suites to aggregate
testSuites = listOf(
testing.suites.named("test"),
testing.suites.named("integrationTest")
)
}
}
}
}
This will create a new test report task called testAggregateTestReport that aggregates the test reports from both of your test suites. You can then run this task to generate the aggregated test report:
gradlew testAggregateTestReport
The aggregated test report will be generated in the build/reports/tests/unit-tests/aggregated-results directory.
Note that the test-report-aggregation plugin is currently in an alpha state, so there may be some issues with it. If you encounter any problems, please report them to the Gradle team.
I’ve been just experimenting with this, and I got a working solution, it’s longer than I would like, but couldn’t figure out the artifactView API to get all the dependencies ignoring the test suite name attribute. This might be a good thing, because it forces us to explicitly select the individual test suites, rather than consuming everything.
plugins {
id("org.gradle.test-report-aggregation")
}
// ... Other setup as described in https://docs.gradle.org/8.13-rc-1/userguide/test_report_aggregation_plugin.html
tasks.register<TestReport>("allAggregateTestReport") {
group = "verification"
destinationDirectory.convention(java.testReportDir.map { it.dir("all/aggregated-results") })
testResults.from(testSuite("unitTest"))
testResults.from(testSuite("functionalTest"))
testResults.from(testSuite("integrationTest"))
}
fun testSuite(name: String): Provider<FileCollection> =
configurations.aggregateTestReportResults.map { configuration ->
@Suppress("UnstableApiUsage")
configuration.incoming.artifactView {
withVariantReselection()
componentFilter { id -> id is ProjectComponentIdentifier }
attributes {
attribute(
Category.CATEGORY_ATTRIBUTE,
objects.named(Category::class, Category.VERIFICATION)
)
attributes.attribute(
TestSuiteName.TEST_SUITE_NAME_ATTRIBUTE,
objects.named(TestSuiteName::class, name)
)
attribute(
VerificationType.VERIFICATION_TYPE_ATTRIBUTE,
objects.named(VerificationType::class, VerificationType.TEST_RESULTS)
)
}
}.files
}
I also had an intermediate solution which created a fake test suite, but it brought in so many extra tasks/compilations/source sets, that it was worth just registering the TestReport task with destinationDirectory manually.
For a combined JaCoCo report over all test suites of all projects I “just” create an additional outgoing configuration on all projects via convention plugin with TEST_SUITE_TYPE_ATTRIBUTE set to "all", go over all jvm test suites to add their test task’s JaCoCo destination file as BINARY_DATA_TYPE artifact to that configuration, and then I can register a JaCoCo coverage report with testType"all" to get the overall report.
You can most probably do it similarly for the test report as the underlying technique used is the same as for the JaCoCo report aggregation.