JUnit Test Filtering and UP-TO-DATE

Help/Discuss
1

I have two different categories of tests in my build, these are Unit Tests and Integration Tests. As part of my build lifecylce, I intend to have my CI server invoke the two different types of tests in separate stages of a pipeline build job.

The first stage will invoke gradle test -Ptests=Unit, which will trigger logic in my build file to use the following style of configuration to exclude the integration tests:

test {
    useJUnit {
        excludeCategories 'com.acme.IntegrationTests'
    }
}

The second stage in the pipeline will run to invoke the integration tests using gradle test -Ptests=Integration which will trigger configuration akin to this:

test {
    useJUnit {
        includeCategories 'com.acme.IntegrationTests'
    }
}

However, when the second stage is executed the test task is considered to be UP-TO-DATE, presumably because none of the input files (the .class files) have been modified since the last time the test task was invoked.

I cannot afford to run the cleanTest task between the first and seconds stages, as I want the test results from both stages to be combined to provide an overall summary of the tests run, and which cases passed / failed.

What is the advised approach to this kind of set-up?

2

While this might also hint at a bug in the up-to-date checking of the test task, I’d like to propose a better approach in general:

Create a second task for your integration tests instead of reconfiguring the test task based on some switch. Switches are hard to discover for users and thus make interacting with the build less intuitive. Tasks on the other hand can easily be discovered by running gradle tasks

test {
  useJUnit {
    excludeCategories 'com.acme.IntegrationTests'
  }
}

task integrationTest(type: Test) {
  testClassesDir = sourceSets.test.output.classesDir
  classpath = sourceSets.test.runtimeClasspath + sourceSets.test.output
  useJunit {
    includeCategories 'com.acme.IntegrationTest'
  }
}

check.dependsOn(integrationTest)
3

You could pass the flag through to the tests as a taskInput

tasks.withType(Test).each { Task task ->
   task.inputs.property('testType', project.property['tests'])
   ...
}

Or perhaps use the categories as inputs

tasks.withType(Test).each { Task task ->
   task.inputs.property('includeCategories', test.useJUnit.includeCategories)
   task.inputs.property('excludeCategories', test.useJUnit.excludeCategories)
   ...
}

@see TaskInputs#property

4

Stefan - Thanks for the advice, certainly seems like a sensible approach.

If I were to pull-out the integration tests into a different task, how would the UP-TO-DATE checking work? It seems like there could be an additional benefit that multiple runs of integrationTest would then only execute if there were changes to the runtime or test classes?

5

Up-to-date checking is entirely based on the inputs/outputs of the task. If nothing changes, the task will not be re-run.