Any way to not "Rerun failed tests first" (added in gradle 4.7)

(Sam Sieber) #1

Is there any way to not “Rerun failed tests first”? Gradle 4.7 added it, but changing test order on a subsequent run due to failures on previous runs has really caused us headaches. Is there a configuration switch somewhere?

Specifically, we have several groups of tests that happen to run in the right order. When one fails, even if fixed, the next run the tests run in a different order and stomp on each other, badly, causing lots of failures and false negatives. Worse yet, enabling the preserveOrder option on the test options doesn’t run the tests in the same way they are run when all are passing without the perserveOrder.

Basically, I’m stuck on 4.6 until I can find a work around. Is there any cache or file I can remove to get rid of Gradle’s test history?

(Sam Sieber) #2

Well, I need to eat my words. Apparently the preserveOrder does work for my use case (I had tried it a while ago and it had caused problems - guess that’s different now). Leaving this here in case anybody else has issues.

(Stefan Oehme) #3

Tests should never depend on ordering, I consider “preserveOrder” a code smell. I recommend investing some time into fixing those tests.

(Sam Sieber) #4

Yes, that goes without saying. It’ll get fixed in time, along with all the other technical debt in codebase I’m working on. I’m glad the perserverOrder feature is there to support non-perfect codebases.