Jacoco Unit and Integration Tests coverage - individual and overall


(arun_sangal) #1

Hi,

I have a project (ProjectA) which contains some unit tests and integration tests. Following is the structure.

ProjectA –

– -- src/java

(java source code)

– -- test/java (Junit unit tests)

– -- test/resources (resources required for Junit Unit tests)

– -- src/java-test (Integration tests)

– -- conf

(contains .xml files required for building/testing/compiling purposes)

I run the following commands – All of them works but I have a doubt on how the configurations that I have in build.gradle / GRADLE_HOME/init.d/*.gradle files are affecting what I’m getting. It seems like I’m missing something and not getting where I want what.

  1. gradle clean build – it works fine

  2. gradle clean build jacocoTestReport – it works fine.

  3. gradle clean build integrationTest jacocoTestReport – it works fine (if I have a tomcat instance up and running in another putty window for the same ProjectA).

After the 3rd bullet operation is complete, I see the extra folder “build” and its subfolders (other than what’s checked out from source/version control) following in my Jenkins jobs workspace.

under – Jenkins_Workspace /build

– -- classes (contains .class files for the following which are mentioned as one of the sourceSets section)

– -- – -- integrationTest

– -- – -- main

– -- – -- test

– -- resources (this contains all the files .properties/.xml files which were under “conf” folder in source control.

– -- reports (contains .xml/.html files for PMD/CheckStyle/Findbugs and Tests results for either Unit or IT Tests but NOT both).

– -- – -- checkstyle

– -- – -- findbugs

– -- – -- pmd

– -- – -- jacoco

– -- – -- tests

(this is plural i.e. it’s not “test” which is defined as one entry in sourceSets)

– -- jacoco (This contains 2 .exec files i.e. test.exec and integrationTest.exec both have different file size)

– -- – -- test.exec

– -- – -- integrationTest.exec

– -- jacocoHtml

(This folder contains lots of folders (containing .html files) and mainly “index.html” under it.

– -- – -- some_xxx_folders

– -- – -- – -- some_xx_folder’s_.html files

– -- – -- index.html

– -- – -- other etc files/folders

– -- test-results

(This contains some .xml files BUT only for either Unit tests or Integration tests).

i.e. if I run “gradle clean build”, then you’ll see Unit test related .xml files and

if I run “gradle clean build integrationTest”, then Unit test .xml files are overwritten

and the .xml files I see are only related to/generated by integrationTest task.

Following is one of the common gradle (GRADLE_HOME/init.d/some.common.gradle file)

//
//Extra file can hold global Gradle settings so that these dont have to be inserted in project
//specific build.gradle file.
//Filename: extraN.common<anyname>.gradle
  allprojects {
   apply plugin: 'java'
   apply plugin: 'pmd'
   apply plugin: 'findbugs'
   apply plugin: 'checkstyle'
   apply plugin: 'jacoco'
   apply plugin: 'sonar-runner'
     tasks.withType(Compile) {
     options.debug = true
     options.compilerArgs = ["-g"]
   }
     sourceSets {
      main {
         java {
            srcDir 'src/java'
         }
         resources {
            srcDir 'conf'
         }
      }
      test {
         java {
            srcDir 'test/java'
         }
         resources {
            srcDir 'test/resources'
            srcDir 'conf'
         }
      }
      integrationTest {
         java {
            srcDir 'src/java-test'
         }
         resources {
            srcDir 'conf'
         }
      }
   }
     def sonarServerUrl = "dev.sandbox.server.com"
     sonarRunner {
      sonarProperties {
         property "sonar.host.url", "http://$sonarServerUrl:9000"
         property "sonar.jdbc.url", "jdbc:h2:tcp://$sonarServerUrl:9092/sonar"
         property "sonar.jdbc.driverClassName", "org.h2.Driver"
         property "sonar.jdbc.username", "sonar"
         property "sonar.jdbc.password", "sonar"
           properties ["sonar.sources"] += sourceSets.main.allSource.srcDirs
         //properties ["sonar.tests"] += sourceSets.test.java.srcDirs
         properties ["sonar.tests"] += sourceSets.integrationTest.allSource.srcDirs
        }
   }
     checkstyle {
        configFile = new File(rootDir, "config/checkstyle.xml")
        ignoreFailures = true
        //sourceSets = [sourceSets.main, sourceSets.test, sourceSets.integrationTest]
        sourceSets = [sourceSets.main]
   }
       findbugs {
        ignoreFailures = true
        sourceSets = [sourceSets.main]
   }
     pmd {
        ruleSets = ["basic", "braces", "design"]
        ignoreFailures = true
   }
       jacoco {
      toolVersion = "0.6.2.201302030002"
      reportsDir = file("$buildDir/customJacocoReportDir")
   }
     task testReport(type: TestReport) {
      destinationDir = file("$buildDir/reports/allTests")
   }
     test {
        jacoco {
            //destinationFile = file("$buildDir/jacoco/jacocoTest.exec")
            destinationFile = file("$buildDir/jacoco/test.exec")
            //classDumpFile = file("$buildDir/jacoco/classpathdumps")
            classDumpFile = file("$buildDir/build/classes/test")
        }
   }
     jacocoTestReport {
         group = "Reporting"
         description = "Generate Jacoco coverage reports after running tests."
         reports {
                xml{
                    enabled true
                    destination "${buildDir}/reports/jacoco/jacoco.xml"
                }
                csv.enabled false
                html{
                    enabled true
                    destination "${buildDir}/jacocoHtml"
                }
        }
        additionalSourceDirs = files(sourceSets.main.allJava.srcDirs)
        //additionalSourceDirs = files([sourceSets.main.allJava.srcDirs, xxxx, 'xxxxyyyy' ])
   }
}

“build.gradle” file looks like

import com.tr.ids.gradle.CustomFileUtil
  apply plugin: 'CustomSetup'
apply plugin: 'java'
apply plugin: 'customJarService'
apply plugin: 'customWarService'
  sourceSets {
   main {
      java {
         srcDir 'src/java'
      }
   }
   test {
      java {
         srcDir 'test/java'
      }
      resources {
         srcDir 'test/resources'
         srcDir 'conf'
      }
   }
   integrationTest {
      java {
         srcDir 'src/java-test'
      }
   }
}
  // Read dependency lists from external files. Our custom plugin reads a dep-xxx.txt file for compile/test/war related .jar file entries
// where each entry is like: groupid:artifactid:x.x.x
// and these artifact jars are available in Artifactory
  List depListCompile = customFileUtil.readIntoList( "$projectDir/dep-compile.txt" )
List depListTest = customFileUtil.readIntoList( "$projectDir/dep-testArtifacts.txt" )
List depListWar = customFileUtil.readIntoList( "$projectDir/dep-war.txt" )
    // Define dependencies
dependencies {
   // Compilation
   compile
depListCompile
     // Unit Tests
   testCompile depListTest
     // Integration tests
   // Everything from compile and testCompile targets
   integrationTestCompile configurations.compile
   integrationTestCompile configurations.testCompile
     // Output of compiling "main" files
   integrationTestCompile sourceSets.main.output
     // Additional dependencies from war and others
   integrationTestCompile depListTest, depListWar
     // All configuration files
   integrationTestRuntime files( 'conf' )
  }
  task deployTomcat( type: Copy, dependsOn: [ jar, compileIntegrationTestJava, warService ] ) {
   from "$buildDir/customWar/${project.name}.war"
   into "$projectDir/tomcat/webapps"
}
  build {
  dependsOn deployTomcat
}
    task integrationTest( type: Test, dependsOn: cleanTest ) {
   jacoco {
      //destinationFile = file("$buildDir/jacoco/jacocoTest.exec")
      destinationFile = file("$buildDir/jacoco/integrationTest.exec")
      //classDumpFile = file("$buildDir/jacoco/classpathdumps")
      classDumpFile = file("$buildDir/classes/integrationTest")
   }
   testClassesDir = sourceSets.integrationTest.output.classesDir
   classpath = sourceSets.integrationTest.runtimeClasspath
}
  apply plugin: 'eclipse'
  eclipse.classpath {
   // Define output directory so Eclipse does not accidentally clobber /bin
   defaultOutputDir = file( 'out/classes' )
     // Add integration test
   plusConfigurations += configurations.integrationTestCompile
     // Remove unnecessary files
   file.whenMerged { classpath ->
      classpath.entries.removeAll { entry -> ( entry.path.indexOf( '/build/classes/main' ) > 0 ) }
      classpath.entries.removeAll { entry -> ( entry.path.indexOf( '/build/resources/main' ) > 0 ) }
   }
}

My questions: 1) Why “gradle clean build integrationTest” – which is working successfully, is overwriting test results in build/reports/tests and build/test-results folders.

  1. It doesn’t matter what name I give for .exec file under common gradle file for test and in build.gradle for integrationTest task for jacoco, it always create test.exec and integrationTest.exec file but the resultant build/jacocoHtml folder index.html file doesn’t show coverage / files related to both Unit / Integration tests. To prove this, if I run “gradle clean build integrationTest jacocoTestReport sonarRunner”, I see the workspace for the job, now contains, “.sonar” folder and build/reports/sonar folder which contains another file called “overall-xxx.exec” some file, but that file size is not close to the “sum” of Unit test.exec and IT integrationTest.exec file size. Though its bigger than test.exec file size by few bytes.

  2. What configuration can I set to have overall coverage for both Unit and IT tests i.e. overall…exec file gets good size (after running sonarRunner task). During sonarRunner task, I do see SonarRunner task’s “jacocoSensor step” does see both UT and IT .exec files and the overall .exec file as well automatically (a good feature from Sonar).


(Peter Niederwieser) #2

ad 1. You have to configure the test results dir and test report dir for the ‘integrationTest’ task (otherwise they default to the same value as for the ‘test’ task). Perhaps 2. and 3. are consecutive errors.