How do I execute different tests in the same project with different settings?


(Bair) #1

I have a single Project with multiple test directories:

src/test/java

src/test/resources

src/test/testsWithStubToolkit

src/test/resourcesWithStubToolkit

Essentially the tests in src/test/java need to run normally, and the tests in src/test/testsWithStubToolkit need to run with a command line flag.

I tried defining a new Test task for the second set, but I can’t get the source directories right:

test {
        enableAssertions = true
        testLogging.exceptionFormat = "full"
        forkEvery = 1
        dependsOn "stubToolkitTests"
    }
      task stubToolkitTests(type: Test) {
        testSrcDirs = [file("src/test/testsWithStubToolkit"), file("src/test/resourcesWithStubToolkit")]
        systemProperty "javafx.toolkit", "com.sun.javafx.pgstub.StubToolkit"
        enableAssertions = true
        testLogging.exceptionFormat = "full"
        forkEvery = 1
    }

I looked in the sources, and it looks like testSrcDirs on the Test task is not even used (other than by TestNG). What is the appropriate way to do this?


(Peter Niederwieser) #2

As you already noticed, ‘testSrcDirs’ is just for TestNG. What’s necessary in any case is to pass the compiled test code via ‘classesDir’ (this is how Gradle finds the tests to be executed), and the compiled test code plus its dependencies via ‘classpath’ (this is the class path for test execution). One way to achieve this is to add the directories to the ‘test’ source set and then work with includes/excludes on the test tasks. Another way is to set up a separate source set, which will automatically give you a separate compile/processResources/classes task, and separate compile/runtime configurations.


(Bair) #3

Hi Peter,

I have tried several variations here and wanted to report back. I have a Java project called “graphics”, which is setup as follows:

project(":graphics") {
    sourceSets {
        main {
            java.srcDirs = ["src/main/java", "src/main/dt", "src/main/jsl-generator"]
            resources.srcDirs = ["src/main/resources"]
        }
        test {
            java.srcDirs = ["src/test/java"]
            resources.srcDirs = ["src/test/resources"]
        }
        stub {
            java.srcDirs = ["src/test/testsWithStubToolkit"]
            resources.srcDirs = ["src/test/resourcesWithStubToolkit"]
        }
    }
      dependencies {
        compile project(":base"), project(":build-tools")
        compile ":$swtFileName:"
        compile files(rootDir.toString() + "/../crap/plugin_exports.jar")
        stubCompile "junit:junit:4.8.2"
    }
      tasks.replace("test")
    task testWithoutStub(type: Test) {
        jvmArgs '-Djava.ext.dirs='
        classpath = sourceSets.test.runtimeClasspath + rootProject.files(JFXRT)
        enableAssertions = true
        testLogging.exceptionFormat = "full"
        scanForTestClasses = false
        include '**/*Test.*'
        exclude '**/*Abstract*.*'
        forkEvery = 1
    }
    task testWithStub(type: Test) {
        jvmArgs '-Djava.ext.dirs='
        classpath = sourceSets.stub.runtimeClasspath + sourceSets.test.runtimeClasspath + rootProject.files(JFXRT)
        enableAssertions = true
        testLogging.exceptionFormat = "full"
        scanForTestClasses = false
        include '**/*Test.*'
        exclude '**/*Abstract*.*'
        exclude { fileTreeElement -> fileTreeElement.toString().contains("build/classes/test") }
        jvmArgs "-Djavafx.toolkit=com.sun.javafx.pgstub.StubToolkit"
        testResultsDir = file("build/stub-test-results")
        testReportDir = file("build/reports/stubTests")
        forkEvery = 1
    }
    test.dependsOn testWithoutStub, testWithStub
}

Basically the difference between the two is that one sets the -Djavafx.toolkit flag, and one doesn’t. The tests that require the flag have been physically separated from the other tests. It is less than ideal because there’s a bunch of logic here. Also both have forkEvery = 1, so in theory if I could hook in and change the VM settings for each forked VM based on the test that was about to run, I’d be golden. I tried that (by changing the VM args or system properties – tried both approaches – in the beforeSuite handler, but this looks like it happens after the VM is forked, so I was hosed).

I’m still playing around with this to try to get the right magic formula that will work. For example, in testWithStub it has a closure to omit certain classes. However this is probably only needed because both of these two test tasks are by default using the “test” source set. I wanted to be able to tell the testWithStub to use the “stub” source set, but I don’t see any direct way to do that.

I’m mostly just flailing around at the moment trying everything to see if I can get a better understanding of the mental model here and so far failing miserably :-). I’ll report back in once I’ve figured it out.


(Bair) #4

I got this worked out and it looks reasonable (the below is simplified to get rid of some of the noise). I created a new source set called stub.

project(":graphics") {
    sourceSets {
        main.java.srcDirs = ["src/main/java", "src/main/dt", "src/main/jsl-generator"]
        test
        stub
    }
      dependencies {
        compile project(":base"), project(":build-tools")
        stubCompile "junit:junit:4.8.2"
    }
      tasks.replace("test")
    task testWithoutStub(type: Test, dependsOn: ":graphics:compileJava") {
        jvmArgs "-Djava.ext.dirs="
        classpath = sourceSets.test.runtimeClasspath + rootProject.files(JFXRT)
        enableAssertions = true
        testLogging.exceptionFormat = "full"
        scanForTestClasses = false
        include "**/*Test.*"
        exclude "**/*Abstract*.*"
        forkEvery = 1
    }
    task testWithStub(type: Test, dependsOn: [":graphics:compileJava"]) {
        jvmArgs "-Djava.ext.dirs=", "-Djavafx.toolkit=com.sun.javafx.pgstub.StubToolkit", "-DCSS_META_DATA_TEST_DIR=${file('build/classes/main/javafx')}"
        classpath = sourceSets.stub.runtimeClasspath + sourceSets.main.runtimeClasspath + rootProject.files(JFXRT)
        testClassesDir = file("build/classes/stub")
        testResultsDir = file("build/stub-results")
        testReportDir = file("build/reports/stub")
        enableAssertions = true
        testLogging.exceptionFormat = "full"
        scanForTestClasses = false
        include "**/*Test.*"
        exclude "**/*Abstract*.*"
        forkEvery = 1
    }
    test.dependsOn testWithoutStub, testWithStub
}