Add support for integration tests

Old Forum
21

@joachim, good suggestions. I’ve added the first 2 use cases to the spec. I’m not sure exactly what you meant by the 3rd use case. Can you explain a bit?

22

@adam Sure.

In our product, we have a huge number of integration tests. We also have a policy to include integration tests in the CI feedback loop. So in order to decrease the execution time, we would like to have some kind of testcase distribution mechanism towards a large number of machines. Kind of the same pattern as I believe you use for unit test execution. A number of executors and a distribute or fetch mechanism. The only difference is that for integration tests, it may be distributed to physical machines.

Does this make my suggestion clearer?

23

It does, thanks. You might be interested to know that there is some work happening to allow tests to be distributed over a number of Jenkins slaves.

24

Regarding the test slaves or machines on which the integration tests run on. In our current implementation we use a plugin and an extension to dynamically book and release the machines via a web service. So Gradle is actually providing the resource name to the deploy and test framework in our environment. And the implementation is a web service plugin. But I was thinking of using some other mechanism like Redis (http://redis.io/) so the booking can be done in a standard way.

Perhaps this is something to consider for the integration test plugin? A ‘standard’ mechanism to name resources and some other (redis plugin) to provide the implementation of booking and releasing?

I’ll be happy to discuss details of my idea further put maybe off this list.

25

Is anyone working on this?

26

it might be better to model integration tests as a separate source

This is the approach we are currently taking with ATG DUST which we are using for container integration tests separate from our standard unit tests. Something like this:

allprojects {
    sourceSets {
    dustTest {
      java {
        srcDir 'src/dust/java'
      }
      resources {
        srcDir 'src/dust/resources'
      }
      groovy {
        srcDir 'src/dust/groovy'
      }
    }
...
  configurations{
    dustTestCompile{
      extendsFrom testCompile
    }
    dustTestRuntime{
      extendsFrom dustTestCompile
    }
  }
...
  apply {
    task dustTest(type: Test) {
      description="Runs all ATG DUST tests"
      tasks.dustTest.setGroup('Verification')
      testClassesDir = sourceSets.dustTest.output.classesDir
      classpath = sourceSets.dustTest.runtimeClasspath
}
  }

So as dustTest (our integTests) are not build dependent, devs running a build only run unit tests. before checkin they run the dustTest task separately, unless actually creating these integration tests themselves. CI then runs dustTest as a task with a dependency on build …

27

You might want to take a look at my pull request which is linked to in one of my previous comments. This does pretty much the same thing I guess, but is based on naming conventions and it is possible to create multiple test types and so on. It is also possible to define dependencies, i.e. unit tests are required to run integTest, but you can run unit tests only etc. It eventually went nowhere as the requirements defined by Adam seem to be completely different and defined only after my initial pull request (not complaining, just sayin’), and since then I haven’t had the interest and time to do anything about it.