Aggregate test reports for test suites

I’m trying to follow the new test-suites approach to testing laid out in this blog post, using Gradle 8.4. I now have both a test suite and an integrationTest suite, and they each produce their own reports.

The post suggests using the test-report-aggregation plugin to merge the two reports together, but I haven’t been able to find a way to do that. The plugin adds tasks called testAggregateTestReport and integrationTestAggregateTestReport. Is there any way to configure the reporting so that both test reports are merged? All the examples I’ve seen involve multiple subprojects, not multiple types of test tasks.

I’ve appended my build.gradle.kts file here:

@file:Suppress("UnstableApiUsage")

plugins {
    application
    id("test-report-aggregation")
}

repositories {
    mavenCentral()
}

dependencies {
    implementation("com.google.guava:guava:32.1.1-jre")
    implementation("ch.qos.logback:logback-classic:1.4.6")
}

testing {
    suites {
        // Configure the built-in test suite
        val test by getting(JvmTestSuite::class) {
            // Use JUnit Jupiter test framework
            useJUnitJupiter("5.10.0")
        }

        val integrationTest by registering(JvmTestSuite::class) {
            dependencies {
                useJUnitJupiter("5.10.0")
                implementation(project())
                implementation("org.assertj:assertj-core:3.21.0")
                implementation("org.postgresql:postgresql:42.6.0")
                implementation("org.testcontainers:postgresql:1.19.1")
                implementation("org.testcontainers:junit-jupiter:1.19.1")

            }

            targets {
                all {
                    testTask.configure {
                        shouldRunAfter(test)
                    }
                }
            }
        }
    }
}

reporting {
    reports {
        testing {
            // Maybe correct configuration goes here?
        }
    }
}

tasks.named("check") {
    dependsOn(testing.suites.named("integrationTest"))
}

application {
    mainClass.set("com.kousenit.App")  // generated by "gradle init"
}

Try adding the following configuration to your build.gradle.kts file:

reporting { reports { testing { // Aggregate the test reports from the testandintegrationTest suites aggregateReports("test", "integrationTest") } } }

This should create a new test report task called testAggregateTestReport , which will merge the test reports from the test and integrationTest suites. You can then run this task to generate the aggregated test report.

To run the aggregated test report task:

./gradlew testAggregateTestReport

This will generate the aggregated test report in the build/reports/tests/aggregated-results directory.

Can you share a more complete and working example?
What you showed is the same as leaving out reporting { reports { ... } } as that is just visual clutter and testing refers to the top-level testing extension.
And in that no aggregateReports method is available.

And afaik the test report aggregation plugin can also not out-of-the-box aggregate results from multiple test tasks, but only supports aggregating test results per test type.
I think to aggregate different types together, you need to do it manually, setting the test results to use on a TestReport task. If within one project directly, if through multiple projects like the plugin, then by using multiple lenient artifact views for the different test types to aggregate.

Unfortunately, Gradle tells me there is no method named aggregateReports available. How did you resolve that?

Ok, here is a more detailed solution. To merge the test reports from your test and integrationTest suites using the test-report-aggregation plugin in Gradle 8.4, you can add the following configuration to your build.gradle.kts file:

reporting {
  reports {
    testing {
      // Configure the test report aggregation plugin
      testReportAggregation {
        // Specify the test suites to aggregate
        testSuites = listOf(
          testing.suites.named("test"),
          testing.suites.named("integrationTest")
        )
      }
    }
  }
}

This will create a new test report task called testAggregateTestReport that aggregates the test reports from both of your test suites. You can then run this task to generate the aggregated test report:
gradlew testAggregateTestReport

The aggregated test report will be generated in the build/reports/tests/unit-tests/aggregated-results directory.

Note that the test-report-aggregation plugin is currently in an alpha state, so there may be some issues with it. If you encounter any problems, please report them to the Gradle team.

When I add your suggested code, I get an error that testReportAggregation cannot be resolved in the available receivers. Where did you get that?

Are you really talking about the built-in test-report-aggregation that is available since Gradle 7.4, @margaretcika?

I ask, because as @Ken_Kousen already said, also that snippet by you cannot compile.

Also the test-report-aggregation plugin was never in alpha or incubating state, but immediately was released as stable feature in Gradle 7.4.

Again, please provide a more complete example, that is, a full working project.