Merge Jacoco coverage reports for multiproject setups

Help/Discuss
1

When using the Gradle JaCoCo plugin, it would be nice if the coverage reports for all subprojects were merged together, to make them easier to access. Could the next version handle multiproject setups better?

In the meantime, is there a manual configuration that makes gradle test jacoco perform this merging when run at the parent project level?

2

You can manually merge them using the jacoco merge task, constructing the executionData FileCollection by iterating on all your exec files.

3

Can you provide an example of how to use JacocoMerge?

4

Possibly the integration test

5

That integration test doesn’t actually appear to be merging subprojects and I couldn’t figure out a variation that seemed to work to merge my subprojects. I am admittedly mostly just a monkey pressing keys at random when it comes to Gradle, but a working example of this for a project with subprojects would be really useful.

6

The JacocoMerge is probably more correct, but a common alternative is the jacocoRootReport style task. The example below is how you can create an aggregate report and publish to Coveralls. You can dig into the usage here if you run into problems.

task jacocoRootReport(type: JacocoReport, group: 'Coverage reports') {
  description = 'Generates an aggregate report from all subprojects'
  dependsOn(subprojects.test)

  additionalSourceDirs = files(subprojects.sourceSets.main.allSource.srcDirs)
  sourceDirectories = files(subprojects.sourceSets.main.allSource.srcDirs)
  classDirectories = files(subprojects.sourceSets.main.output)
  executionData = files(subprojects.jacocoTestReport.executionData)

  reports {
    html.enabled = true
    xml.enabled = true
  }

  doFirst {
    executionData = files(executionData.findAll { it.exists() })
  }
}

coveralls {
  sourceDirs = subprojects.sourceSets.main.allSource.srcDirs.flatten()
  jacocoReportPath = "${buildDir}/reports/jacoco/jacocoRootReport/jacocoRootReport.xml"
}
7

The integration test merges two test tasks within the same project. This could easily be adapted for a multiproject build.

Eg:

task jacocoMerge(type: JacocoMerge) {
   subprojects.each { subproject ->
      executionData subproject.tasks.withType(Test)
   } 
}
8

Thanks, I’ve made some progress based on that, but now find myself running into problems like those described in https://issues.gradle.org/browse/GRADLE-2955. All of the attempts I’ve made to exclude our generated source files either don’t work or run into the problem reported in https://issues.gradle.org/browse/GRADLE-3138.

9

This test is now present in https://github.com/gradle/gradle/blob/master/subprojects/jacoco/src/integTest/groovy/org/gradle/testing/jacoco/plugins/JacocoPluginMultiVersionIntegrationTest.groovy

10

I have been searching for the answer to this question several times over the last year, each time giving up after a couple of hours of testing a zillion different suggestions - in vain. I am not deeply into Gradle, so much of it seems like black magic. I finally managed to adapt Benjamin’s suggestion into something that works for me (a project with subprojects common, client, server, and integration). And it works on Gradle 5.6.

task jacocoMergeAll(type: JacocoMerge) {
   dependsOn(subprojects.test, subprojects.jacocoTestReport)
   subprojects.each { subproject ->
      // exclude common and integration subprojects
      if (subproject.name != 'integration' &&
          subproject.name != 'common') {
        executionData subproject.tasks.withType(Test)
      } 
   } 
}

task jacocoRootReport(type: JacocoReport, group: 'Coverage reports') {
  description = 'Generates an aggregate report from all subprojects'
  dependsOn(jacocoMergeAll)

  additionalSourceDirs.from =
                   files(subprojects.sourceSets.main.allSource.srcDirs)
  sourceDirectories.from =
                   files(subprojects.sourceSets.main.allSource.srcDirs)
  classDirectories.from =
                   files(subprojects.sourceSets.main.output)
  executionData.from =
                   files("${buildDir}/jacoco/jacocoMergeAll.exec")

  reports {
    html.enabled = true
    xml.enabled = false
  }
}

The one thing I could not figure out is how to automatically avoid subprojects that do not produce a jacoco.exec (which is the case of ‘common’), so I have to have a manual test in the merge task, which is a bit ‘handheld’. Anyone who knows the trick to make gradle ignore in case there is no file?

Embed the above in the root build.gradle, and gradle test jacocoRootReport to get coverage for all subprojects in the build/reports/
 folder.

3 Likes
11

The one thing I could not figure out is how to automatically avoid subprojects that do not produce a jacoco.exec

You could do something like

task jacocoRootReport(type: JacocoReport) {
   def jacocoProjects = subprojects.findAll { it.pluginManager.hasPlugin('jacoco') } 
   jacocoProjects.each {
      additionalSourceDirs.from it.sourceSets.main.allSource.srcDirs
      ... 
   } 
}
12

I did something like this:

task jacocoMergeAll(type: JacocoMerge) {
    dependsOn(subprojects.test, subprojects.jacocoTestReport)
    def jacocoProjects = subprojects.findAll {
        def buildDir = it.getBuildDir()
        def testExecFile = file(buildDir.getAbsolutePath() + '/jacoco/test.exec')
        logger.debug('Looking for JaCoCo test.exec file {}', testExecFile)
        return testExecFile.exists()
    }
    jacocoProjects.each { subproject ->
        executionData subproject.tasks.withType(Test)
    }
}
13

I wouldn’t recommend this approach. You are looking for the files in Gradle’s “configuration” phase but the files are created in Gradle’s “execution” phase. See build phases

TLDR; this will only work in a “dirty” build scenario where a previous build has run jacoco and was not “cleaned”

I suggest you determine the projects by inspecting the Gradle model (ie tasks/plugins in the model)

14

@ciprian-radu where would you put the task jacocoMergeAll(type: JacocoMerge) script you have written? I’ve tried to put it in the root gradle file of my android project, but as soon as I sync I get the following error message

Could not get unknown property 'test' for project ':app' of type org.gradle.api.Project.

Is there something that I am missing? Do you have a full working sample somewhere that I could use a reference?

1 Like
15

Seriously, I have spent hours trying to fix jacoco after upgrading 5.3 → 5.6. The first comment on this github gist USED to work in 5.3 but not in 5.6
(perfect example of how to help people with a full working solution)


https://gist.github.com/aalmiray/e6f54aa4b3803be0bcac

I am trying to get 5.6 to work without the “can’t find test.exec” error and it seems people keep giving ‘partial’ solutions. The gist above was a complete amazing example that ‘just worked’. Does someone have a merge task that ‘just works’? I have tried way way too many things and spent way tooo much time on this due to ‘partial’ solutions of try this. or if I find it, I will post my complete solution as that would help everyone here.

I really also don’t get why this would work as ‘every’ subproject has the plugin BUT it’s just they don’t generate a test.exec file since they have no tests


def jacocoProjects = subprojects.findAll { it.pluginManager.hasPlugin(‘jacoco’) }

Instead, the other solutions of gathering up all test.exec files seems more valid to me(and what was done in the past versions like 5.3). but then @Lance said “I wouldn’t recommend this approach.” so I am very very confused. A complete jacoco merge to use in gradle 5.6 that works in gradle 6 would rock if someone has it. If I get to it first, I will post it as well, but I am getting to the point of slamming into brick wall and stopping for a while to cycle back fresh at some point.

16

@Lance I tried combining your idea with the merge report but alas I get test.exec not found in the empty projects since the jacoco plugin is in my subprojects section, even empty projects get the plugin so that doesn’t seem to work. Here is my latest try and the error


task jacocoMerge(type: JacocoMerge) {
   dependsOn = subprojects.jacocoTestReport
   def jacocoProjects = subprojects.findAll { it.pluginManager.hasPlugin('jacoco') }
   jacocoProjects.each { subproject ->
      executionData subproject.tasks.withType(Test)
   }
}

The error is

Unable to read /Users/dean/workspace/webpieces/core/output/jacoco/test.exec

NOTE: webpieces/core directory is ONLY a directory containing about 5 subprojects. It has no src directory and no tests, but because the jacoco plugin is in the subprojects {} section, it ends up getting the plugin.

We need a way like @ciprian-radu 's method that only gathers existing test.exec files and skips projects without that file.

ALSO of note is that the test task in webpieces/core directory does not fail saying there is no tests so why does jacoco fail just because there is no test.exec file. This seems like a plugin bug that they should skip merging in certain projects that have no test.exec file automatically.

17

Just to add one more solution - this worked for me, and feels pretty solid:

18

In a multi-project build, you can use the gradle-jacocolog plugin to aggregate coverage information across subprojects. The plugin is really for logging the coverage data, but it can also perform aggregations.

Configure your Jacoco like you would normally, then apply the plugin to the root project:

plugins {
    id 'org.barfuin.gradle.jacocolog' version '1.2.3'
}

After that, you can:

gradle jacocoAggregatedReport

to display the aggregated coverage. There is also a jacocoMergeSubprojects task which will just create a merge coverage file, if you want to process it in some other way.

Hope this helps!
(Full dislosure: I am the author of gradle-jacocolog.)

19

I have the same question. I don’t know where should I put the task.
There is one setting.gradle file in the root directory which should be parsed during initialization phase, and can not define task in it.

20

I didn’t read the whole thread here, but it is seldomly a good idea to use a JacocoMerge task.
This is only necessary if you want to feed JaCoCo execution data of multiple tasks to some tool that does not support handling multiple such files which most should do. The JacocoReport task for example can handle multiple execution data files without problem.

But anyway, if you want to generate an overall JaCoCo report over test from a multi-project build, nowadays there are cleaner and more idiomatic ways anway. I recommend having a look at the sample that exactly demonstrates that: Reporting code coverage with JaCoCo Sample