Hi All,
I have what I consider to be a bug, relating to the invocation of JUnit tests.
I am attempting to use the JVM @Catgegory
annotation to filter the tests that should be executed by my build. This allows me to annotate a test class with @Category(IntegrationTest.class)
and separate those classes into a different build phase from the vanilla JUnit tests.
I have this successfully working with the following syntax:
test {
useJUnit {
excludeCategories "com.acme.IntegrationTest"
}
}
So far so good , I am able to filter the tests that I would like to be able to run. However, it’s about to go very wrong very quickly
I have discovered through remotely debugging the gradle process that this filtering is taking place in the external process that is forked by the ForkingTestClassProcessor
. This is the first major problem that I have. I am in a really unfortunate situation where I need a new JVM for each test case, and thus have set forkEvery 1
.
From what I have observed, this means that gradle ends up forking a new JVM for every test class (even if it should be filtered out by the excludeCategories
setting). Given there is a significant cost of starting up a new JVM, this is really unfortunate. It means that if I have 100 integration tests, 100 JVMs are forked that subsequently exit when they realize there are no tests to be run due to the filter
This is having a DRAMATIC impact on my build times. Many of my integration tests are relatively quick to run, and the startup time for a new JVM far outweighs the amount of time it’d actual test code. As a result my unit-only build takes almost as much time as my unit+integration build.
There’s worse still. I’ve also found that the externally launched JVM is also instantiating the JUnit runner (as specified with the @RunWith
annotation) for each test before applying the category filter.
This means that any test runners – including those that unfortunately have expensive logic in their constructors – have their constructors invoked, despite the test subsequently being excluded by the excludeCategories
filter.
As a result, my “quick” complile+unit build is actually remarkably slow
I’d be interested to hear if other people agree that this is a bug? I know it’s a pretty niche setup, but it’s causing me real day-to-day problems.