Declaring JvmTestSuite targets with different engines

Help/Discuss
1

I have a code base where tests needs to be run with separate runners, but the code base remains within test.

E.g. something like

testing {
    suites {
        withType<JvmTestSuite>().configureEach {
            // other stuff
        }

        val test by getting(JvmTestSuite::class) {
            targets {
                create("junit5testTarget") {
                    testTask.configure {
                        doFirst{
                            println("Junit 5 target")
                        }
                        useJUnitPlatform {
                            includeEngines("junit-jupiter")
                            excludeEngines("junit-vintage")
                        }
                    }
                }
                create("junit4testTarget") {
                    testTask.configure {
                        doFirst{
                            println("Junit 4 target")
                        }
                        useJUnitPlatform {
                            includeEngines("junit-vintage")
                            excludeEngines("junit-jupiter")
                        }
                    }
                }
            }
        }
    }
}

But it doesn’t seem to be working as expected, i.e. the targets seems to be ignored.

2

Tried this as well, but it then runs nothing :face_with_monocle:

val test by getting(JvmTestSuite::class) {
    targets {
        all {
            testTask.configure {
                useJUnitPlatform() {
                    excludeEngines(
                        "junit-vintage",
                        "junit-jupiter"
                    )
                }
            }
        }
    }
}
val junit4test by registering(JvmTestSuite::class) {
    sources {
        kotlin {
            srcDir(sourceSets[test.name].kotlin.srcDirs)
        }
    }
    testType = "junit4-unit-test"
    targets {
        all {
            testTask.configure {
                useJUnitPlatform {
                    includeEngines("junit-vintage")
                    excludeEngines("junit-jupiter")
                }
                doLast {
                    println("Test task: " + testTask.get().name)
                }
                shouldRunAfter(test)
            }
        }
    }
}
val junit5test by registering(JvmTestSuite::class) {
    sources {
        kotlin {
            srcDir(sourceSets[test.name].kotlin.srcDirs)
        }
    }
    testType = "junit5-unit-test"
    targets {
        all {
            testTask.configure {
                useJUnitPlatform {
                    includeEngines("junit-jupiter")
                    excludeEngines("junit-vintage")
                }
                doLast {
                    println("Test task: " + testTask.get().name)
                }
                shouldRunAfter(test)
            }
        }
    }
}
3

Multiple targets are unfortunately not yet supported.
Currently it is a 1:1:1 relationship (test-suite / target / task).

May I ask why you need to separate Vintage and Jupiter tests?
If you really need it, from a cursory look I’d say your second try should work.
You could actually leave out the excludeEngines, just having includeEngines should be enough.

4

For the curiosity, the tests need to run with IntelliJ platform support. Unfortunately there’s an issue when JUnit 5 tests are run before a JUnit 4. There’s some static shared state I believe that makes all JUnit 4 tests to fail. And having a single unit test code base seemed desirable.
Hence the idea of segregating runs.

My second tentative try didn’t work as expected. And since my team decided to abandon JUnit 5 for the time being I haven’t gave it another shot. That said I believe this ought to be tried for the sake of knowledge.

It’s a bit sad no targets can be created I can well imagine running a code base on a different JDK, or else, in some of my other projects.

5

There’s some static shared state I believe that makes all JUnit 4 tests to fail.

Oh, that’s very unfortunate of course, maybe you should fix that? :smiley:

And having a single unit test code base seemed desirable. Hence the idea of segregating runs.

Yeah, makes definitely sense.

That said I believe this ought to be tried for the sake of knowledge.

Definitely. If you don’t mind to knit an MCVE with your second try, I might be able to have a quick look why it doesn’t work as expected.

It’s a bit sad no targets can be created I can well imagine running a code base on a different JDK, or else, in some of my other projects.

Definitely. But as it is designed to have multiple targets, let’s hope we can add multiple some day.

6

Hi,

Sorry for the delay, I had some mishaps and busy work days. So here’s a simple project with the ideas exposed above (second example). I’ve added you as a contributor, so you won’t have to fork.

I believe there’s 2 issues

  1. Defining the task graph, ideally I would like to have the test task depending on both junit4Test and junit5Test. In the current approach with shouldRunAfter, the junit4Test and junit5Test are not triggered.
  2. The junit4Test and junit5Test do not inherit the configuration performed on the test task, this is performed by the gradle-intellij-plugin. I believe this configuration happens in project.afterEvaluate.

Oh, that’s very unfortunate of course, maybe you should fix that? :smiley:

It’s not an easy task as it’s a platform issue : IDEA-333958.

7

ideally I would like to have the test task depending on both junit4Test and junit5Test.

What hinders you to do so?
Besides that it would usually be cleaner make check depend on junit4Test and junit5Test instead of test.

In the current approach with shouldRunAfter, the junit4Test and junit5Test are not triggered.

Of course not, why should they?
shouldRunAfter just means that if both tasks are going to run, the one should run after the other, unless they can be run in parallel.
mustRunAfter the same, but always.

The junit4Test and junit5Test do not inherit the configuration performed on the test task, this is performed by the gradle-intellij-plugin. I believe this configuration happens in project.afterEvaluate.

Yeah, why should they inherit it?
If it is configuration done on Properties, wire them together if you want.
If not, and the plugin really follows the bad practice to use afterEvaluate, you might need to try to delay an action even further and then copy the eager values that were set there.

But as this is about IntelliJ plugin, you might not have luck, as that even does worse and changes configuration at execution time: https://github.com/JetBrains/gradle-intellij-plugin/blob/a10fd139467dd34c8e292296b563edfed14fea9d/src/main/kotlin/org/jetbrains/intellij/IntelliJPlugin.kt#L1289-L1317

But with version 2 you might have more luck, as Jakub is actively working on v2 where many things are improved and changed and afaik also that part.

8

What hinders you to do so?

Convenience I think, but you’re right check is probably a better candidate to do that.

Of course not, why should they?
shouldRunAfter just means that if both tasks are going to run, the one should run after the other, unless they can be run in parallel.

:man_facepalming: Of course, was looking at the official documentation of JVM test suites and forgot about this. Almost never using shouldRunAfter is not an excuse for not looking at the javadoc.

Yeah, why should they inherit it?
as that even does worse and changes configuration at execution time: https://github.com/JetBrains/gradle-intellij-plugin/blob/a10fd139467dd34c8e292296b563edfed14fea9d/src/main/kotlin/org/jetbrains/intellij/IntelliJPlugin.kt#L1289-L1317

Yeah that’s precisely the issue (in the case of IntelliJ plugin). Indeed I’m following through Jakub’s changes. Recently he posted updates on test tasks, and we suggested soe JvmTestSuite support.

1 Like