Jacoco global default thresholds / local modifications setup


(Thibault Kruse) #1

Hi,
comparing the palantir jacoco plugin (https://github.com/palantir/gradle-jacoco-coverage) with the gradle plugin (https://docs.gradle.org/current/userguide/jacoco_plugin.html), it seems configuring custom local thresholds is much more of a pain.

What I mean is in a gradle multi-project build, I would like to define a global policy (like 90% line coverage), but for a file inside submodule X, I want to be more lenient like (allowing 50% line for class X.java).

With palantir, this is trivial, globally I set

jacocoCoverage.fileThreshold 0.9, LINE

and in the module X, I add

jacocoCoverage.fileThreshold 0.5, LINE, "X.java"

However with the gradle plugin, I need to define a rule, so if I have a global rule

        rule {
            limit {
                minimum = 0.8
                counter = 'LINE'
            }
        }

Tnen I cannot override this rule in the submodule for class X. Redefining the rule for class X globally however breaks module encapsulation. So I find myself having to either copy/paste global policies, or to write some ugly gradle function that computes policies for each submodule.

Do I miss anything obvious? Is there any clean solution?


(Thibault Kruse) #2

Ah, I guess what happened is that Gradle copied the configuration approach from Maven, instead of copying from the palantir plugin. Would it be too late to throw away the Maven aproach and use the palantir approach instead?

The problem is not just for multi-module builds. Whenever a team wants to have some default global thresholds, and some exceptions for specific files and packages, the Maven-like approach with rules is extremely difficult to use, because multiple rules have to be defined with mutually exclusive includes and excludes.


(Thibault Kruse) #3

Also see https://jivimberg.io/blog/2018/04/26/gradle-verify-coverage-with-exclusions/ for more confusion with the current plugin.


(Thibault Kruse) #4

Just to show an example:

Consider enforcing global line coverage of 90%, except using 80% for files Foo.java. Also global branch coverage of 80%, except using 25% for package com.example.dto

With palantir, that is:

# from a shared global_defaults.gradle
fileThreshold 0.9, LINE
fileThreshold 0.8, BRANCH
 
# (sub-)project-specific coverage.gradle
fileThreshold 0.8, LINE, "Foo.java"
fileThreshold 0.25, BRANCH, "com.example.dto"

As you can see the DSL is pretty close to my natural language explanation, and it is easy and natural to maintain global defaults in one place, and local overrides in another place.
Now with the standard gradle plugin (following Maven), that would be

jacocoTestCoverageVerification {
violationRules {
rule {
    element = 'REPORT'
    excludes = ['Foo.java'] // need to exclude here

    limit {
        counter = 'LINE'
        minimum = 0.9
    }
}
rule {
    element = 'REPORT'
    includes = ['Foo.java']

    limit {
        counter = 'LINE'
        minimum = 0.8
    }
}
rule {
    element = 'REPORT'
    excludes = ['com.example.dto'] // need to exclude here

    limit {
        counter = 'BRANCH'
        minimum = 0.8
    }
}
rule {
    element = 'REPORT'
    includes = ['com.example.dto']

    limit {
        counter = 'BRANCH'
        minimum = 0.25
    }
}
}
}

The more non-defaults to be used, the worse it gets, for any new threshold value in the same category, exclusions have to be correctly set in all similar rules. It’s horrible spaghetti-references that emerge.