20 minutes longer build with useJUnitPlatform: hundreds of bogus Test Executors

Help/Discuss
1

I am trying to update our ui tests to geb 5 and spock 2 where a major change is that they now use JUnit instead of TestNG.
Our geb tests have run fine with setForkEvery(1) so far (we do not have that many of them and running each test takes a long time anyway, so we figured it’s easier to start each test with a fresh db/app state).

However after switching to useJUnitPlatform() hundreds of gradle Test Executors are run unnecessarily.
The build now takes 20 minutes longer.
With INFO level logging there is just a pause of 20 minutes.
With debug logging you can see that gradle runs lots of Test Executor that just test nothing at all.

For testing I deleted all except 2 test classes, but that did not change anything.
If I set a test filter to a single class it works fine and fast. If I have no test filter or a test filter with a wildcard the delay happens.

This is my test configuration:

tasks.test {
	maxHeapSize = "768m"
	useJUnitPlatform()
	setForkEvery(1)
}
2

Ok, finally figured it out - kind of.

It seems that the JUnit-Platform-Runner (as opposed to the TestNG-Runner) forks for every class in the module’s classpath regardless of whether it actually is a test class. Because there are a lot of geb page objects, geb modules and (I guess) anonymous classes, there were a few hundred bogus forks of test runners.

Also, for some reason it did not help to add a test filter for “*GebSpec”. What helped though was to move all actual tests into a “specs” package and add a test filter for that package (“com.acme.foo.specs.*”).

I’d say that’s a bug - at the very least a documentation bug, because the API docs clearly state

"The maximum number of test classes to execute in a forked test process. The forked test process will be restarted when this limit is reached. "