Jacoco and test sourceSets with no tests

Help/Discuss
1

This is a minimal example of my problem.

This is what my project looks like

├── build.gradle
├── gradle
│   └── shared.gradle
├── module1
│   ├── build.gradle
│   └── src
│       ├── integrationTest
│       │   └── java
│       │       └── m1
│       │           └── Thing1IntegrationTest.java
│       └── main
│           └── java
│               └── m1
│                   └── Thing.java
├── module2
│   ├── build.gradle
│   └── src
│       ├── main
│       │   └── java
│       │       └── m2
│       │           └── Thing.java
│       └── test
│           └── java
│               └── m2
│                   └── ThingTest.java
└── settings.gradle

The build.gradle files in the module1 and module2 have this

apply from: '../gradle/shared.gradle'

The shared.gradle has this

apply plugin: 'java'
apply plugin: 'jacoco'

repositories {
    jcenter()
}

sourceSets {
    integrationTest {
        java {
            compileClasspath += main.output
            runtimeClasspath += main.output
            srcDir file('src/integrationTest/java')
        }
        resources.srcDir file('src/integrationTest/resources')
    }
}

configurations {
    integrationTestCompile.extendsFrom testCompile
    integrationTestRuntime.extendsFrom testRuntime
}

dependencies {
    testCompile 'junit:junit:4.12'
}

task integrationTest(type: Test) {
    testClassesDirs = sourceSets.integrationTest.output.classesDirs
    classpath = sourceSets.integrationTest.runtimeClasspath
}

check.dependsOn integrationTest

jacoco {
    toolVersion = '0.7.9'
}

jacocoTestReport {
    executionData tasks.withType(Test)
}

check.dependsOn 'jacocoTestReport'

When I run ./gradlew check, I get this:


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':module2:mergeJacocoReports'.
> Unable to read /Users/rahul/src/gradle-jacoco-problem/module2/build/jacoco/integrationTest.exec

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':module1:mergeJacocoReports'.
> Unable to read /Users/rahul/src/gradle-jacoco-problem/module1/build/jacoco/test.exec

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
==============================================================================

BUILD FAILED in 1s
8 actionable tasks: 2 executed, 6 up-to-date

That is understandable, one of the modules doesn’t have an integrationTest and the other doesn’t have a test.
In this reduced example, I can tweak this manually.
However when other plugins (grails for example) contribute test types, it is not possible.

Is there a way to get jacoco to ignore these situations?

2

It’s not clear to me what requirements you have for your reports. Do you want a report of absolutely everything, including those from the other plugins that you mentioned (grails for example), or do you only want a report of the test and integrationTest coverage?

Prior to adding all tasks with type Test to the executionData, the JacocoReport task will automatically be skipped if the source doesn’t exist for the default Test task. The .exec file will be generated for the integrationTest task, but not included in a report.

In the simplest case, adding a report for your integrationTest tasks would give you a report, and everything would just work (reports for test and integrationTest tasks when they exist):

task jacocoIntegrationTestReport(type: JacocoReport) {
    executionData integrationTest
    sourceSets sourceSets.integrationTest
}
check.dependsOn 'jacocoIntegrationTestReport'

Or do you need the data reported in a specific way?

3

If a test sourceSet is empty (and no .exec file is created), I want jacocoTestReport to ignore it instead of erroring out the way it does right now.

1 Like
4

No error will be thrown if you use the separate report task like I mentioned above, but you didn’t answer the questions to determine if that’s good enough. Those specifically are:

Do you need coverage reports from tasks of type Test that are added by other plugins (if they run and have generated the *.exec)?

Do you need a single coverage report that includes both test and integrationTest results if a project has both (not one for each)?

If the answer to both of those is “no”, you can add the extra report task, remove adding all tasks of type Test to the executionData, and be done.

If the answer to one or both of those is “yes”, you need to add some complexity to the project to make the task function as desired. Your answers will impact the recommended code to solve it.

5

We have a similar situation.

We have a modules which are auto generated using gradle.
The always contain tests and integration tests in the following structure:

  • src
    • main
    • test
    • integrationTest

Some modules contain tests classes but do not contain any integration test classes.
This means, whenever we run the tests and the integration tests on the root module we get the following Jacoco *.exec files:
image
(the integrationTest.exec file is missing)

We want a single Jacoco report for the module so we perform the following in the root build.gradle file:

def jacocoExecutionDataFiles =
            tasks.withType(Test.class)
                    .getNames()
                    .stream()
                    .map { name -> "build/jacoco/${name}.exec" }
                    .collect(Collectors.toList())

jacocoTestReport {
    executionData = files(jacocoExecutionDataFiles)
}

This works fine for modules which contain some integration tests since the integration test executable is generated fine.
For modules which have no tests at all (no *.exec files exist), it also works fine.
For modules which don’t have 1 of the test types, either unit or integration, we get the following errors:
No unit tests:

* What went wrong:
Execution failed for task ':some-project:jacocoTestReport'.
> Unable to read execution data file ...\some-project\build\jacoco\test.exec

No integration tests:

* What went wrong:
Execution failed for task ':some-project:jacocoTestReport'.
> Unable to read execution data file ...\some-project\build\jacoco\test-int.exec

We would like to have some kind of flag to specify whether we want errors or only warnings on such cases. In case we set the flag to warnings, it should not fail.

6

We’ve managed to find a solution to our problem a few minutes after I posted this post.
The solution was to change the build.gradle file to:

def jacocoExecutionDataFiles = fileTree(buildDir).include("/jacoco/*.exec")

jacocoTestReport {
    executionData = jacocoExecutionDataFiles
}

This finds only existing *.exec files so if we are missing test.exec or integrationTest.exec because there are no tests, it won’t fail on us.