Test task with forkEvery 1 and includeCategories performs poorly

Help/Discuss
1

I have a Test target that includes only one JUnit test category:

task acceptanceTest(type:Test) {
  useJUnit {
    includeCategories 'org.apache.geode.test.junit.categories.AcceptanceTest'
  }
  forkEvery 1
}

I only have a single Test class that is annotated with this Category. Yet, it looks like there are hundreds of Gradle Executor Threads being spun up needlessly:

Starting process 'Gradle Test Executor 1'. Working directory: /myProjectDir/build/acceptanceTest Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java /myProjectDir/-Dgemfire.disallowMcastDefaults=true -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djline.terminal=jline.UnsupportedTerminal -XX:+HeapDumpOnOutOfMemoryError -Xmx768m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /Users/jstewart/.gradle/caches/2.14.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 1'
Successfully started process 'Gradle Test Executor 1'
Gradle Test Executor 1 started executing tests.
Gradle Test Executor 1 finished executing tests.
Starting process 'Gradle Test Executor 2'. Working directory: /myProjectDir/build/acceptanceTest Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java /myProjectDir/-Dgemfire.disallowMcastDefaults=true -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djline.terminal=jline.UnsupportedTerminal -XX:+HeapDumpOnOutOfMemoryError -Xmx768m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /Users/jstewart/.gradle/caches/2.14.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 2'
Successfully started process 'Gradle Test Executor 2'
Gradle Test Executor 2 started executing tests.
Gradle Test Executor 2 finished executing tests.
Starting process 'Gradle Test Executor 3'. Working directory: /myProjectDir/build/acceptanceTest Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java /myProjectDir/-Dgemfire.disallowMcastDefaults=true -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djline.terminal=jline.UnsupportedTerminal -XX:+HeapDumpOnOutOfMemoryError -Xmx768m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /Users/jstewart/.gradle/caches/2.14.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 3'
Successfully started process 'Gradle Test Executor 3'
Gradle Test Executor 3 started executing tests.
Gradle Test Executor 3 finished executing tests.
Starting process 'Gradle Test Executor 4'. Working directory: /myProjectDir/build/acceptanceTest Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java /myProjectDir/-Dgemfire.disallowMcastDefaults=true -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djline.terminal=jline.UnsupportedTerminal -XX:+HeapDumpOnOutOfMemoryError -Xmx768m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /Users/jstewart/.gradle/caches/2.14.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 4'
Successfully started process 'Gradle Test Executor 4'
Gradle Test Executor 4 started executing tests.
Gradle Test Executor 4 finished executing tests.
Starting process 'Gradle Test Executor 5'. Working directory: /myProjectDir/build/acceptanceTest Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java /myProjectDir/-Dgemfire.disallowMcastDefaults=true -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djline.terminal=jline.UnsupportedTerminal -XX:+HeapDumpOnOutOfMemoryError -Xmx768m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /Users/jstewart/.gradle/caches/2.14.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 5'
Successfully started process 'Gradle Test Executor 5'
Gradle Test Executor 5 started executing tests.
Gradle Test Executor 5 finished executing tests.
Starting process 'Gradle Test Executor 6'. Working directory: /myProjectDir/build/acceptanceTest Command: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java /myProjectDir/-Dgemfire.disallowMcastDefaults=true -Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager -Djline.terminal=jline.UnsupportedTerminal -XX:+HeapDumpOnOutOfMemoryError -Xmx768m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /Users/jstewart/.gradle/caches/2.14.1/workerMain/gradle-worker.jar worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test Executor 6'
Successfully started process 'Gradle Test Executor 6'
Gradle Test Executor 6 started executing tests.
Gradle Test Executor 6 finished executing tests.

My guess is that perhaps Gradle is spinning up an Executor for each test class before it has applied the filter of the AcceptanceTest JUnit category? I would appreciate any help.

2

Do you get this behavior even without forkEvery 1? The assumption here is that we are preemptively starting up a number of test workers in accordance with the --max-workers value but if so this would be independent of a forkEvery value.

3

Yes that is expected, category filtering is done on the test worker side. Why are you setting forkEvery to 1?

4

We have a large number of legacy tests, many of which do ill-advised things like dynamically changing the value of the “user.dir” system property. These tests can leave behind state that interacts or interferes with subsequent test classes. So, we set forkEvery 1 to ensure test isolation.

5

It’s just that this will add significant overhead to every single test class, so I recommend to only do this in extreme cases. Maybe start isolating “well behaved” tests into a separate task that keeps the worker around for longer.