Hi,
comparing the palantir jacoco plugin (https://github.com/palantir/gradle-jacoco-coverage) with the gradle plugin (https://docs.gradle.org/current/userguide/jacoco_plugin.html), it seems configuring custom local thresholds is much more of a pain.
What I mean is in a gradle multi-project build, I would like to define a global policy (like 90% line coverage), but for a file inside submodule X, I want to be more lenient like (allowing 50% line for class X.java).
With palantir, this is trivial, globally I set
jacocoCoverage.fileThreshold 0.9, LINE
and in the module X, I add
jacocoCoverage.fileThreshold 0.5, LINE, "X.java"
However with the gradle plugin, I need to define a rule, so if I have a global rule
rule {
limit {
minimum = 0.8
counter = 'LINE'
}
}
Tnen I cannot override this rule in the submodule for class X. Redefining the rule for class X globally however breaks module encapsulation. So I find myself having to either copy/paste global policies, or to write some ugly gradle function that computes policies for each submodule.
Do I miss anything obvious? Is there any clean solution?
Ah, I guess what happened is that Gradle copied the configuration approach from Maven, instead of copying from the palantir plugin. Would it be too late to throw away the Maven aproach and use the palantir approach instead?
The problem is not just for multi-module builds. Whenever a team wants to have some default global thresholds, and some exceptions for specific files and packages, the Maven-like approach with rules is extremely difficult to use, because multiple rules have to be defined with mutually exclusive includes and excludes.
Just to show an example:
Consider enforcing global line coverage of 90%, except using 80% for files Foo.java. Also global branch coverage of 80%, except using 25% for package com.example.dto
With palantir, that is:
# from a shared global_defaults.gradle
fileThreshold 0.9, LINE
fileThreshold 0.8, BRANCH
# (sub-)project-specific coverage.gradle
fileThreshold 0.8, LINE, "Foo.java"
fileThreshold 0.25, BRANCH, "com.example.dto"
As you can see the DSL is pretty close to my natural language explanation, and it is easy and natural to maintain global defaults in one place, and local overrides in another place.
Now with the standard gradle plugin (following Maven), that would be
jacocoTestCoverageVerification {
violationRules {
rule {
element = 'REPORT'
excludes = ['Foo.java'] // need to exclude here
limit {
counter = 'LINE'
minimum = 0.9
}
}
rule {
element = 'REPORT'
includes = ['Foo.java']
limit {
counter = 'LINE'
minimum = 0.8
}
}
rule {
element = 'REPORT'
excludes = ['com.example.dto'] // need to exclude here
limit {
counter = 'BRANCH'
minimum = 0.8
}
}
rule {
element = 'REPORT'
includes = ['com.example.dto']
limit {
counter = 'BRANCH'
minimum = 0.25
}
}
}
}
The more non-defaults to be used, the worse it gets, for any new threshold value in the same category, exclusions have to be correctly set in all similar rules. It’s horrible spaghetti-references that emerge.
Hey Thibault,
I’m aware that this is a old topic, but we’re currently facing the same problem and it seems that there is nothin new “build in”, so we build our own solution that I’d like to share.
First of all we build a jacoco report in the root project containing the executions of all submodules:
tasks {
// ...
jacocoTestReport {
executionData.setFrom(fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec"))
subprojects
.filter { it.name !in listOf("integrationtests") }
.forEach {
this@jacocoTestReport.sourceSets(it.sourceSets.main.get())
this@jacocoTestReport.dependsOn(it.tasks.test)
}
reports {
xml.isEnabled = true
xml.destination = file("$buildDir/reports/jacoco/report.xml")
}
}
// ...
}
Then we have a helper method to parse the XML report:
import org.w3c.dom.NodeList
import org.xml.sax.InputSource
import java.io.StringReader
import javax.xml.parsers.DocumentBuilderFactory
import javax.xml.xpath.XPathConstants
import javax.xml.xpath.XPathFactory
fun calculateCoverage(): Map<String, Float> {
val xmlFile = File("$buildDir/reports/jacoco/report.xml").readText()
val dbFactory = DocumentBuilderFactory.newInstance().apply {
setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false)
}
val dBuilder = dbFactory.newDocumentBuilder()
val doc = dBuilder.parse(InputSource(StringReader(xmlFile)))
val xpFactory = XPathFactory.newInstance()
val xPath = xpFactory.newXPath()
(xPath.evaluate("/report/counter", doc, XPathConstants.NODESET) as NodeList).let { nodes ->
return (0 until nodes.length).map { i ->
val node = nodes.item(i)
val type = node.attributes.getNamedItem("type").nodeValue
val missed = node.attributes.getNamedItem("missed").nodeValue.toInt()
val covered = node.attributes.getNamedItem("covered").nodeValue.toInt()
val total = missed + covered
val coverage = covered.toFloat() / total.toFloat()
return@map (type to coverage)
}.toMap()
}
}
I assume that it will be less code with groovy and XmlSlurper
and finally a custom coverage verification task project level:
tasks {
// ...
val verifyCoverage by registering {
dependsOn(jacocoTestReport)
doLast {
val expectedCoverage = 0.85f
val coverage = calculateCoverage()
if ((coverage["LINE"] ?: 0f) < expectedCoverage) {
throw GradleException("Expecting $expectedCoverage LINE coverage but was ${coverage["LINE"]}")
}
}
}
check.get().dependsOn(verifyCoverage)
// ...
}
I know that this is no very nice solution, but this way you can have a global verification in root project and more fine grained verifications using jacocoTestCoverageVerification
task in every submodule.
1 Like
We implemented this in a slightly simpler way.
Shared coverage.gradle:
jacocoTestCoverageVerification {
violationRules {
rule {
element = 'CLASS'
limit {
minimum = 1
}
}
}
}
Project specific build.gradle:
jacocoTestCoverageVerification {
violationRules {
// To exclude a specific class from the global rule
rules.findAll { it.element == 'CLASS' }.each {
it.excludes = [
'com.example.Foo'
]
}
// Rule specific to the Foo class
rule {
element = 'CLASS'
includes = [
'com.example.Foo'
]
limit {
counter = 'INSTRUCTION'
minimum = 0.16
}
limit {
counter = 'LINE'
minimum = 0.3
}
limit {
counter = 'COMPLEXITY'
minimum = 0.5
}
limit {
counter = 'METHOD'
minimum = 0.5
}
}
}
}