Parallel test execution support for Gradle Test Logger Plugin

Help/Discuss
1

I am the author of the Gradle Test Logger Plugin and I am seeking help / suggestions for supporting Gradle’s parallel test execution in my plugin.

The plugin can be used to print beautiful logs on the console while tests are being executed. Typically the console output would look something like (colours stripped for simplicity):

com.example.test.FirstSpec

    Test abc PASSED
    Test def SKIPPED
    Test ghi PASSED

com.example.test.SecondSpec

    Test jkl PASSED
    Test mno SKIPPED
    Test pqr PASSED

As you can see, the tests are grouped by suites. This is all fine and dandy as long as maxParallelForks is set to 1 (Gradle default). But when more than one fork is executing the tests, grouping isn’t deterministic anymore as the tests are executed in parallel and the output can become mixed up. So we may see FirstSpec executing tests abc and def followed by SecondSpec executing jkl and mno followed by FirstSpec executing tests ghi followed by SecondSpec executing pqr. Classic problem with parallel processing and grouping.

com.example.test.FirstSpec

    Test abc PASSED
    Test def SKIPPED

com.example.test.SecondSpec

    Test jkl PASSED
    Test mno SKIPPED

com.example.test.FirstSpec

    Test ghi PASSED

com.example.test.SecondSpec

    Test pqr PASSED

I find that an output of the above form makes the grouping useless. To resolve this problem, I can think of two options but both have their limitations.

  1. Trap all output in a collection (a map keyed by suite name) and render after executing the last test. While this would resolve the grouping issue, visually it’s not very elegant as no output would appear on the console until the last test is executed and suddenly a large volume of output may be produced. Looks like this approach is used by mocha-parallel-tests.

  2. Use a different theme (the plugin has a concept of themes which changes the style of the output) that doesn’t involve grouping when maxParallelForks > 1. For instance the new theme could render the output as follows:

    com.example.test.FirstSpec: Test abc PASSED
com.example.test.FirstSpec: Test def SKIPPED
com.example.test.SecondSpec: Test jkl PASSED
com.example.test.SecondSpec: Test mno SKIPPED
com.example.test.FirstSpec: Test ghi PASSED
com.example.test.SecondSpec: Test pqr PASSED

    The downsides of this approach are that there may be too much repetition of the suite name thereby reducing the readability of the output and that the plugin will have to have separate themes for parallel and sequential modes.

I would like to get an opinion of the Gradle community regarding this problem and would also welcome any other suggestions. For example, if there is a way to get more information during the execution of forked tests in the Gradle API, that might prove invaluable too.

Thank you and appreciate your time.