Jacoco code coverage report varies between local and sonar

Old Forum Archive
1

Hi Folks,

I have the beginnings of jacoco code coverage in my gradle (1.6) multi-module project. I’m using the jacoco plugin syntax and from a local build, I’m getting what appear to be correct outputs of code coverage (comparable to cobertura reports). Where things start to get a little strange is when I run my build against sonar (3.2). What I’m seeing is drastically lower code coverage levels. For a module on my local build, I have around 79% line coverage. The same module reported in sonar is around 40%.

My local command is:

gradle clean build jacocoTestReport

And my command to run sonar:

gradle classes sonarAnalyze check --info

Below shows how I’m configuring things. Hopefully someone has some insight on what’s going on here. Thanks!

In my parent gradle file, I declare sonar. What’s interesting is the value for the sonar.jacoco.reportPath doesn’t seem to be used, I can change the value to anything and it still runs a report to sonar.

sonar {
      server {
        url = "http://sonar.acme.net:8020"
    }
      database {
        url = "jdbc:mysql://sonar.acme.net:3306/sonar"
        driverClassName = "com.mysql.jdbc.Driver"
    }
      project {
        dynamicAnalysis = "reuseReports"
        withProjectProperties { props ->
            props["sonar.core.codeCoveragePlugin"] = "jacoco"
            props["sonar.jacoco.reportPath"] = "$reportsDir/jacoco/jacocoTest.exec"
        }
          withGlobalProperties { props ->
            props["sonar.projectKey"] = "content-dam"
            props["sonar.projectName"] = "content-dam"
        }
    }
}

Then within the sub-project gradle file:

jacoco {
    toolVersion = "0.6.2.201302030002"
}
  test{
      jvmArgs '-XX:-UseSplitVerifier'
      jacoco {
        append = false
        destPath = file("$buildDir/jacoco/jacocoTest.exec")
        classDumpPath = file("$buildDir/jacoco/classpathdumps")
    }
}
  jacocoTestReport {
      group = "Reporting"
    description = "Generate Jacoco coverage reports after running tests."
    additionalSourceDirs = files(sourceSets.main.allJava.srcDirs)
      reports {
        xml.enabled true
        csv.enabled false
        html.destination "${buildDir}/jacocoHtml"
    }
      classDirectories = fileTree(dir: 'build/classes/main', include: 'com/**')
    sourceDirectories = fileTree(dir: 'scr/main/java', include: 'com/**')
}
2

I think you should configure Sonar JaCoCo settings for the same project(s) that you are actually running JaCoCo for. Maybe the different percentages are caused by different base sets of sources (e.g. single project vs. multiple projects, or includes such as ‘com/**’ used for the plain report but not for Sonar). You should be able to determine this by comparing the reports. Also I recommend to switch to the new ‘sonar-runner’ plugin.

3

Thanks for the help, I think I’ve narrowed this down to a rather simple conclusion. When I omit the classes task in my command, in other words:

gradle sonarAnalyze --info

I get the expected coverage levels. I put classes back:

gradle classes sonarAnalyze --info

I get the incorrectly lowered coverage levels. So… looking closer, I’m not sure why I’ve been including the classes task in my command. Searching around I don’t see other examples of people doing this. I’m inclined to remove this task from my command and call it a day on this :slight_smile:

4

This might not be part of the issue, but by default the sonarAnalyze task doesn’t build classes or run any tests. This works fine with the default sonar quality profile that only runs static source code analyzers. If the sonarway with findbugs profile is selected in sonar it will fail on a clean build without depending upon compileJava or some other task that depends upon compileJava like classes/test/check/build (either an explicit sonarAnalyze.dependsOn classes) or done on the command line. Tests are also not run by default either, but would be caught by check/build or a sonarAnalyze.dependsOn test for each subproject.

gradle clean build jacocoTestReport

gradle classes sonarAnalyze check

That would clarify why you potentially would get different behavior with the test output from the two above commands.

Your follow-up post has me a bit confused as to how the behavior would cause different coverage levels, other than the tests potentially being inconsistent with the classes most recently compiled in the second case (in the first case no classes should be compiled or tests run, so they would always be “consistent”).

-Spencer