Test run slow when running with `--test com.example.FooTest`

Old Forum Archive Bugs
,
1

When running tests using --tests, gradle seems to initialize other tests (although it does not run them) this significantly adds to test run times. This makes it unbearable when running with forkEvery option, because gradle forks for each test, and does nothing :frowning: I’ve even tried using filter.includeTestsMatching to see if that affects the test run time.

I’ve setup a sample project to demonstrate this over on my github account (https://github.com/ketan/gradle-bug-report)

Here’s a snippet of the output from my actual build to demonstrate how severe the problem is, with thousands of tests, each test taking an average of 100ms seconds to do nothing, it adds up to several minutes to skip them.

Running Gradle Test Executor 1
Running com.thoughtworks.go.config.BackgroundEmailSenderTest
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.051 sec
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.058 sec
Running Gradle Test Executor 2
Running com.thoughtworks.go.config.CachedGoConfigIntegrationTest
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.228 sec
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.234 sec
Running Gradle Test Executor 3
Running com.thoughtworks.go.config.CachedGoConfigTest
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.1 sec
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.106 sec
Running Gradle Test Executor 4
Running com.thoughtworks.go.config.CachedGoPartialsTest
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.071 sec
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.078 sec
Running Gradle Test Executor 5
Running com.thoughtworks.go.config.ConfigCipherUpdaterTest
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.176 sec
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.184 sec
Running Gradle Test Executor 6
Running com.thoughtworks.go.config.ConfigConverterTest
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.193 sec
Tests run: 0, Failures: 0, Skipped: 0, Time elapsed: 0.202 sec
Running Gradle Test Executor 7
1 Like
2

Hi Ketan,

Thank you very much for reporting this. Looking quickly at the code it seems that you are correct - i.e. for each class a new JVM is forked regardless of the filter configuration - i.e. if the test class is included at all. This would require a change in the current logic, probably we would need to filter test classes before we hand them to the executor. Currently, this does not have a very high priority. It would be great if you could put together a pull request to fix the problem. If you need support in doing this please contact us and we try to guide you.

Best regards,
Stefan

3

A partial workaround is to only fork when not filtering:

test {
  if (filter.includePatterns) {
    forkEvery 1
  }
}
4

@Stefan_Wolf —

Could you point me to some places in code where I can look at to address this issue? Happy to submit a PR.

5

@ketan

Thanks for looking into this. The basic parts are:

  • org.gradle.api.internal.tasks.testing.junit.JUnitDetector: Responsible for discovering test classes
  • org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor and org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter: Responsible for running the test class
  • org.gradle.api.internal.tasks.testing.detection.DefaultTestExecuter: Responsible for starting the workers and bringing the test detector and the processor together

The current problem is that the JUnitTestClassExectutor is actually doing the filtering when the JVM is already forked.

Hope this makes some sense to you.

Regards,
Stefan