Cobertura Plugin and Zero Unit Test Data

Old Forum
1

I am a newbie to Gradle, Cobertura, and Sonar and I am trying to get them to play together nicely. I noticed that sometimes on my Sonar dashboard, my unit test success is 100.0% for 34 tests and other times it just reports 0 tests. After some amount of digging, I found that

This works fine gradle build gradle cobertura sonar

This causes sonar to report 0 tests gradle build cobertura sonar

Why does it matter if I call them together or separately?

And upon some further digging, I found that sometimes my gradle generated unit test reports at build/reports/tests/index.html are there and sometimes they aren’t. And in fact

This works fine gradle build gradle cobertura

This causes the unit test reports to disappear (or to never be created?) gradle build cobertura

Any ideas would be greatly appreciated!

Thanks, WhatWouldHoneyBadgerDo

Here is my version information

$ gradle -v
  ------------------------------------------------------------
Gradle 1.0-milestone-8
------------------------------------------------------------
  Gradle build time: Monday, February 13, 2012 11:53:32 PM UTC
Groovy: 1.8.4
Ant: Apache Ant(TM) version 1.8.2 compiled on December 20 2010
Ivy: 2.2.0
JVM: 1.6.0_31 (Sun Microsystems Inc. 20.6-b01)
OS: Linux 2.6.9-89.0.11.ELlargesmp amd64

Here is my build script

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'war'
apply plugin: 'sonar'
  sourceCompatibility = 1.6
version = '1.0'
implementationTitle = 'NameChangedToProtectTheInnocent'
  /*
createWrapper creates gradle-wrapper.jar, gradle-wrapper.properties, gradlew, gradlew.bat
*/
task createWrapper(type: Wrapper) {
    gradleVersion = '1.0-milestone-8'
    distributionBase = Wrapper.PathBase.PROJECT
    archiveBase = Wrapper.PathBase.PROJECT
}
  repositories {
    mavenCentral()
}
  dependencies {
    compile("log4j:log4j:1.2.16")
    compile("org.apache.httpcomponents:httpclient:4.1.3")
    compile("org.springframework:spring-web:3.1.1.RELEASE")
    testCompile("junit:junit:4.10")
}
  jar {
    manifest {
        attributes 'Implementation-Title': implementationTitle, 'Implementation-Version': version
    }
}
  sonar {
    project {
        coberturaReportPath = file("$buildDir/reports/cobertura/coverage.xml")
    }
}
  /*
Apply cobertura plugin for code coverage in sonar
This will add the 'cobertura' task which instruments the code,
executes the tests and outputs to build/reports/cobertura.
*/
buildscript {
    def githubBase = 'https://raw.github.com/valkolovos/gradle_cobertura/master/ivy'
    apply from: "${githubBase}/gradle_cobertura/gradle_cobertura/1.0-rc4/coberturainit.gradle"
}
2

Although I don’t completely understand how dependencies are mapped at the command line, this sounds suspiciously similar to the dependency order issue. Gradle does not guarantee the order that a task’s dependencies will be executed.

Cobertura must instrument the code before running the tests for results to appear. So the execution order must be: compile -> instrument -> test -> cobertura.

You should be able to check this in your build output.

3

Thank you for your reply mzbrand. I think you may be on to something. I looked at the execution order and noticed that when I run gradle build cobertura, the cleanTest task is executed after the test task. But when I run gradle build and then gradle cobertura, the cleanTest task is executed before the test task. That probably explains why all my unit test data is getting wiped out. I’m surprised no one else using the cobertura plugin has run into this issue. Now I just have to figure out how to control the order.

Thanks! WhatWouldHoneyBadgerDo

gradle build cobertura

:compileJava
:processResources
:classes
:war
:assemble
:compileTestJava
:processTestResources
:testClasses
:test
Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Instrumenting 1 file
Cobertura: Saved information on 10 classes.
Instrument time: 2093ms
Flushing results...
Flushing results done
Cobertura: Loaded information on 10 classes.
Cobertura: Saved information on 10 classes.
:check
:build
:cleanTest
:coberturaXml
Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 10 classes.
Report time: 238ms
:cobertura
Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 10 classes.
Report time: 462ms

gradle build >gradle cobertura

:compileJava
:processResources
:classes
:war
:assemble
:compileTestJava
:processTestResources
:testClasses
:test
:check
:build
  :cleanTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test
Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Instrumenting 1 file
Cobertura: Saved information on 10 classes.
Instrument time: 718ms
Flushing results...
Flushing results done
Cobertura: Loaded information on 10 classes.
Cobertura: Saved information on 10 classes.
:coberturaXml
Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 10 classes.
Report time: 326ms
:cobertura
Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 10 classes.
Report time: 386ms
4

I found that the Cobertura plug-in didn’t handle dependencies as well as I would have liked, so I implemented a new version of it. It takes a more sophisticated approach toward dependency management to ensure that the build order is followed correctly. I’d be happy to contribute this if you think it would help.

5

Thanks mzbrand, I really appreciate your offer. I do have a decent workaround (call gradle build and gradle cobertura separately from CI server), but if it is easy enough for you to share your version, I’d be interested to see it.

Thanks, WhatWouldHoneyBadgerDo

6

@mzbrand: I also had to create a custom cobertura plugin for Gradle (working on getting it open-sourced). Seems like we should collaborate :slight_smile:

7

I need to look over my code and make sure it is in good shape. My understanding of how to write plugins has evolved a bit since I wrote it. I’ll try to post it by Monday.

Eric, I’d love to collaborate. Once it was finished it wasn’t a lot of code, but I really had to think through the workflow logic. I like the idea collaborating one what we’ve discovered and publishing a more robust implementation.

8

@mzbrand: Right on! Let me know where it’s hosted if you open-source it. You can also email me at emwendelin at gmail.

9

Thanks for the reminder. I forgot got into long meetings and forgot to post this as promised. :slight_smile:

I have two classes, listed below. I think this was my first custom plugin, so forgive any stylistic issues (there are several changes I’d already make). I’m also a Cobertura newbie, so I’d be very interested in feedback.

Mike

package org.gradle.cobertura
  import org.gradle.api.GradleException;
import org.gradle.api.Project
import org.gradle.api.Plugin
    class CoberturaPlugin implements Plugin<Project> {
   Project target
 def coberturaExtension
      @Override
 void apply(Project target) {
  this.target = target
  this.coberturaExtension = new CoberturaPluginExtension(target)
  target.extensions.cobertura = coberturaExtension
      target.configurations {
   cobertura
  }
    target.with {
   apply plugin: 'java'
        dependencies {
    cobertura 'net.sourceforge.cobertura:cobertura:1.9.3'
    cobertura files(coberturaExtension.instrumentationDir)
   }
        // pass information on cobertura datafile to your testing framework
      coberturaExtension.testTasks.each{testTaskName->
    getTasksByName(testTaskName, false).each{testTask->
     testTask.configure {
      dependsOn 'instrumentCobertura'
      systemProperties['net.sourceforge.cobertura.datafile'] = coberturaExtension.dataFile
     }
    }
   }
     check.dependsOn 'cobertura'
        task('instrumentCobertura', instrumentCoberturaTask)
   task('cobertura', coberturaTask)
  }
 }
   def instrumentCoberturaTask = {
  dependsOn 'compileJava'
  doLast {
   logger.info "Instrumenting cobertura to: $coberturaExtension.instrumentationDir.absolutePath"
     ant.taskdef(resource:'tasks.properties',
        classpath: target.configurations.cobertura.asPath)
   ant.'cobertura-instrument'(toDir: coberturaExtension.instrumentationDir.absolutePath,
     datafile: coberturaExtension.dataFile.absolutePath) {
    if (target.sourceSets.main.output.classesDir.exists()) {
     fileset(dir: target.sourceSets.main.output.classesDir) {
      include(name: "**/*.class")
     }
    }
          // This is causing Spring unit tests to fail.
It's a known issue.
    project.sourceSets.all {
     // place your instrumented classes at the FRONT of the list of TEST runtime
     //
dependencies (including the original compiled project classes ) so that they get preference at test time
     runtimeClasspath = project.configurations.cobertura + runtimeClasspath
    }
   }
     }
 }
     def coberturaTask = {
  group = 'Verification'
  description = 'Run Cobertura analysis'
  dependsOn 'instrumentCobertura'
  dependsOn 'test'
  doLast {
   logger.info "Writing cobertura report"
   if (project.extensions.cobertura.srcDirs) {
    logger.info "Creating Cobertura reports for " + project.extensions.cobertura.srcDirs
     logger.info "Reports will be located in the directory " +
       project.extensions.cobertura.reportDir
     ant.'cobertura-report'(destdir: project.extensions.cobertura.reportDir,
       datafile: project.extensions.cobertura.dataFile){
     project.extensions.cobertura.srcDirs.each {dir->
      fileset(dir: dir){
       include(name: "**/*.java")
      }
     }
    }
   } else {
//
  throw new GradleException("property srcDirs must be set to use Cobertura")
    logger.warn "Cobertura cannot run becuase no source directories were found."
   }
  }
 }
}

package org.gradle.cobertura
  import org.gradle.api.Project;
  class CoberturaPluginExtension {
 def project
 File dataFile
 File reportDir
 def instrumentationDirName = "cobertura-instrumentation"
 File instrumentationDir
 def testTasks = ['test']
 def srcDirs
  def dataFilePath = "reports/cobertura/cobertura.ser"
    CoberturaPluginExtension(Project project) {
  dataFile = new File(project.buildDir, dataFilePath)
  reportDir = dataFile.parentFile
  instrumentationDir = new File(project.buildDir, instrumentationDirName)
  srcDirs = project.sourceSets.main.java.srcDirs.findAll{it.isDirectory()}
 }
    def testTask(String taskName) {
  testTasks << taskName
  }
}