I have a few tests that are named similar, something like this: FooResource.java, FoosResource.java, FooVersionResource.java, all in the same package. When gradle runs the test suite, it intermingles the test methods for these similar-named tests before finishing up each test class. This causes problems since the test class assumes it will start and finish all in one piece (it starts up an embedded grizzly server to do some jersey rest tests).
This seems like wrong behavior to me. It seems like each test class is considered by gradle to be a suite, so I’m not sure why class B starts before class A finishes (though it does return back to class A later). I do not have any parallel execution specified in my test options.
Looked into this a bit more (in ‘TestNGTestClassProcessor’) and it doesn’t look like this is gradle’s fault actually as it’s just passing an array of test class names to testng. I’m not sure what surefire was doing differently, or why testng is not running these tests sequentially and mixing the test classes… but I don’t think I can blame this on gradle. Sorry for the clutter!