Ordering Test Class Execution

Help/Discuss
1

Hello, I am trying to figure out how to modify the order on how tests are executed. Basically what I want to achieve is next thing.
Suppose I have 3 tests: ATest.java, BTest.java and CTest.java. A prior I hae no clue about which one is going to be the first one, the second one and so on.

I would like to control and for example first time it is build you run A, B and C and next time C. B andA. I have checked javadoc and it seems that there is no property to set this, so if yhis is true, the best way to achieve this would be creating a custom plugin?

Thanks.

2

AFAIK Gradle does not randomise test classes, it just executes them in othe order discovered on this. Thus I cannot guarantee that this will work, but try the following

test {
  scanForTestClasses = false

  include '/path/to/ATest.class'
  include '/path/to/BTest.class'
  include '/path/to/CTest.class'
}
3

Sadly it does not work, I have tried next:

test {
  scanForTestClasses = false
  includes = ['**/LLLibraryTest.class', '**/LLibraryTest.class', '**/LibraryTest.class']

  afterTest { desc, result ->
        println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
  }
}

And the output is:

Executing test testSomeLibraryMethod [LibraryTest] with result: SUCCESS
Executing test testSomeLibraryMethod [LLibraryTest] with result: SUCCESS
Executing test testSomeLibraryMethod [LLLibraryTest] with result: SUCCESS
4

This was asked in the past but no answer http://gradle.1045684.n5.nabble.com/Running-test-classes-in-a-custom-order-td1431425.html

5

This is hitting us as well.

What it’s doing now seems to be searching the directories in a platform-dependent order and recording the tests as they’re found.

If it sorted them into a reproducible order (e.g. sorting by filename) then we’d at least get consistent test results from machine to machine.

Or even better, it could deliberately shuffle them and print out the seed it used to determine the ordering, like what RSpec does.