intTest tasks are (wrongly) skipped

I have a little plugin that creates intTest and accTest in a loop. In both cases, we first need to start a java process in the background and stop it after we’re done.

It’s based on ‘best practices’ from what I could find on stackoverflow and other sources (http://www.slideshare.net/SpringCentral/cd-pipeline-gradlejenkins). The intTest and accTest tasks, however, were always skipped. When gradle was run with ‘-i’ it would report:

:my-server:compileIntTestJava (Thread[main,5,main]) started.
:my-server:compileIntTestJava
Executing task ':my-server:compileIntTestJava' (up-to-date check took 0.061 secs) due to:
  Output file /devel/my-project/src/my-server/build/dependency-cache has changed.
  Output file /devel/my-project/src/my-server/build/classes/intTest has changed.
  Output file /devel/my-project/src/my-server/build/classes/intTest/com/myserver/inttest/SomeIntTest.class has been removed.
All input files are considered out-of-date for incremental task ':my-server:compileIntTestJava'.
Compiling with JDK Java compiler API.
:my-server:compileIntTestJava (Thread[main,5,main]) completed. Took 0.144 secs.
:my-server:processIntTestResources (Thread[main,5,main]) started.
:my-server:processIntTestResources
Skipping task ':my-server:processIntTestResources' as it has no source files.
:my-server:processIntTestResources UP-TO-DATE
:my-server:processIntTestResources (Thread[main,5,main]) completed. Took 0.002 secs.
:my-server:intTestClasses (Thread[main,5,main]) started.
:my-server:intTestClasses
Skipping task ':my-server:intTestClasses' as it has no actions.
:my-server:intTestClasses (Thread[main,5,main]) completed. Took 0.001 secs.
:my-server:intTest (Thread[main,5,main]) started.
:my-server:intTest
file or directory '/devel/my-project/src/my-server/build/classes/test', not found
Skipping task ':my-server:intTest' as it has no source files.
:my-server:intTest UP-TO-DATE
:my-server:intTest (Thread[main,5,main]) completed. Took 0.002 secs.

After changing the Test task, everything is working as expected - But IMHO, both variations should be completely equivalent and it shouldn’t make a difference. https://github.com/stackmagic/gradle-dropwizard/commit/9bbf7a555e3cf0f2f470d4671fe2142de409a597?diff=unified#diff-2

Maybe there are too many changes in the diff, the key is this (dependency on testClasses vs. own classes didn’t seem to make a difference): broken:

task(taskName, type: Test, dependsOn: 'testClasses') {
... config
}

working:

task(taskName, dependsOn: "${taskName}Classes") {
    test {
    ... config
    }
}

The way I see it, gradle is looking in the wrong place for test classes (in the default/primary test output dir) even though I set a different one. Is this a bug? And gradle won’t find anything in ‘test’ because that one actually is empty. But the intTest folder right next to it would contain the test code I want it to run.

I’d be grateful for an explanation because I’ve spent quite a bit of time ‘fixing’ this and I still don’t understand it.

There is a difference between declaring your own ‘Test’ task and reconfiguring the ‘test’ task declared and preconfigured by the ‘java’ plugin. However, it doesn’t make sense to nest ‘test { … }’ inside another task declaration.

The variable taskName would be intTest and accTest or whatever someone would configure. So this

task(taskName, type: Test....

shouldn’t interfere with the original Test task, right?

No, but your second snippet nests configuration of the Java plugin’s ‘test’ task inside a task declaration, which isn’t necessary or useful. As it stands, the “outer” (i.e. newly declared) task will have no behavior whatsoever.

Ah of course, the ‘test’ closure configures the java plugin’s own test task.

I’m still tinkering with this. I’m back to the proper way of creating the new task like this:

task(taskName, type: Test, dependsOn: "${taskName}Classes") { ... }

And in the configure closure I print out the ‘testClassesDir’ property after it has been set and it points to ‘build/classes/intTest’ as it should. The classpath looks fine as well.

But later in the info output, intTest is skipped:

:my-server:intTest
file or directory '/devel/my-project/src/my-server/build/classes/test', not found
Skipping task ':my-server:intTest' as it has no source files.

Somewhere there still has to be crosstalk between test and intTest or is that message normal?

There are no src/test/java files yet, there’s only code in src/main/java and src/intTest/java but I expected that the absence of src/test/java really shouldn’t make a difference here.

Check out ‘samples/java/withIntegrationTests’ in the ‘gradle-all’ distribution. Hopefully this will help to find the cause of your problem.

The sample looks fairly familiar and switching from setting the compile/runtime classpaths of the source sets to using the dependency declarations like in the sample did nothing. Plus, I only need the main.output for the intTests. The test.output I don’t need for the intTests.

intTest is skipped because there’s no build/classes/test (although intTest uses build/classes/intTest, and compilation of a dummy intTest class ends up at the correct place). The intTest is executed as soon as I add a test to src/test/java. I could add a dummy test everywhere but I would really like so solve and understand this.

I’ve now checked every classpath/srcDirs and any properties I could think of in the sourceSets/tasks/configurations and it’s all fine. I can’t find a mix between test and the dynamically added extra test tasks. ‘test’ isn’t referenced from anywhere (except of course test itself).

What I noticed is that, when I call Task.getCandidateClassFiles (whose output is correct too) I get the dreaded ‘file or directory ‘/devel/my-project/src/my-server/build/classes/test’, not found’… this is the case on all 3 test tasks.

Does that ring any bells? Somehow build/classes/test must be tangled up with the extra test tasks. A few greps over the gradle code didn’t turn up any obvious problems. I also looked over the Test and TestNGFramework and DirectoryFileTree (the dreaded message is logged by the last of these classes) but couldn’t really find anything.

Sounds like your ‘intTest’ task isn’t configured correctly. The ‘withIntegrationTests’ sample should have everything needed. As for ‘getCandidateClassFiles()’, perhaps you were calling it at the wrong time (configuration phase instead of execution phase)?

I have it all: extra sourceSet, configuration, dependencies… It’s all by the book as far as I can tell. The only ‘special’ aspects are:

  • I use TestNG * The configuration must be declared in the project using the plugin so the user can add their own dependencies * SourceSet and Task are added by the plugin * SourceSet and Task are created in a loop

  • Most of the Plugin Config happens during afterEvaluate

And like I said I’ve verified everything, I’ve println’d myself to death and checked every configured path etc

If I call ‘project.tasks.intTest.getCandidateClassFiles()’ 5 time I get the evil message 5 times.

I even tried hooking into DirectoryFileTree to get a call stack so I can trace the problem back but the groovy metaClass magic doesn’t work with classes from a jar it seems.

Correction… I get the error when I manually call getCandidateClassFiles() right after configuration on test and accTest (both source sets have no tests) but not for intTest (has 1 dummy test).

But during execution both accTest and intTest print the message.

‘getCandidateClassFiles()’ will only return the correct result when called after (integ) tests have been compiled. It shouldn’t ever be called in the configuration phase. For TestNG, make sure to also configure ‘Test#testSrcDirs’.

Alright, the testSrcDirs were definitely wrong, this is now correct (I hope), at least the printlns confirm it:

testSrcDirs
  = sourceSets."${taskName}".java.srcDirs as List

But still the same symptoms. I’m going to look into this again at the weekend. Disable other plugins that are at play etc…

Thanks for the help so far - I’ll be back :wink:

So I’ve been at it again. The 2 plugins are merged back into 1 in preparation to later put it on plugins.gradle.org and I made a vanilla test project - which suffers from the same problem, that the integration/acceptance tests are only run if both points below are true

  • I either run test before intTest or the intTest task depends on the test task * There’s at least 1 test class in ‘src/test/java’

setting ‘options.testResources = testSrcDirs’ doesn’t change anything.

I don’t think the reports have an influence on it, everything looks ok, even moreso now that I’ve refactored it all a bit…

If you have time to glance over the code this would be great, maybe you’ll spot something odd?

The plugin: https://github.com/stackmagic/gradle-dropwizard The sample project: https://github.com/stackmagic/gradle-dropwizard-sample

In the current state it all works (unit test present and intTest depends on test), so you can run ‘gradle clean intTest’ in the sample project but once you remove the unit test it breaks: ‘rm -rf src/test/java && gradle clean intTest’

This is all on gradle 2.1 (2.2 breaks the shadow plugin (your guys already track this)).