How can I run a test inside each subproject?

Help/Discuss
1

I have a Gradle monolith project with around 50 subprojects. I’d like to ensure that the code in each subproject conforms to a specific rule, for example that each class named *Foo is annoted with Bar.

I already wrote a test that successfully scans the classpath and asserts the property I’m interested in.

Where do I put this code so that it is executed for the whole project, either individually for each subproject, or once for the whole codebase?

I know about test fixtures, so I could easily provide the logic in a class that is accessible by all subprojects. However, I’d still need to write a test class that runs the shared code, which I’d have to do for each of my subprojects.

How can I avoid this duplication? I’m looking for something like this:

subprojects {
    additionalTestsFrom project(':architecture-tests')
}
2

I think this can be made working by introducing a new Configuration into your Subprojects.
Declare the architecture-tests project as a dependency to this new configuration, and then create a test task that uses this configuration (adding other dependencies to it, if needed).

All of this could happen inside a subprojects configuration block.

The point of declaring a new configuration would be that the tests and their dependencies don’t conflict with existing configurations, and do not leak into any artefacts.

3

You could do similar to this where you copy a test case into each subproject under $buildDir/generated then add the generated directory as a test source directory in each subproject

4

@thokari this sounds very reasonable, but sadly I’m unable to follow along. I’m rather new to the concepts of (custom) configurations and tasks.

I am able to create a source set for my architecture-tests subproject, and define dependencies. I guess it is not necessary to have a dedicated source set if I also have a dedicated subproject.

I also managed to create a new configuration and let it depend on the architecture-tests subproject:

configurations {
    architectureTestConfiguration
}

dependencies {
    architectureTestConfiguration project(":architecture-tests")
}

However, I don’t know how to consume this configuration, especially how to wire the tests into the consuming subproject.

I was able to create a new test task (that depends on the “architecture-tests” subproject), but I don’t know how to configure it:

task architectureTestTask(type: Test) {
    group = 'verification'
    dependencies {
        testImplementation project(":architecture-tests")
    }
}

Note that I did not use the configuration.

Could you please elaborate a bit, or point at helpful documentation?

5

This works, but I’m not using a configuration - and I have a fair bit of duplication.

task architectureTestTask(type: Test) {
    group = 'verification'

    // refer to source set that contains shared tests
    testClassesDirs = project(':architecture-tests').sourceSets.test.output.classesDirs
    classpath += project(':architecture-tests').sourceSets.test.runtimeClasspath

    // optional, required for shared test to see subproject tests
    classpath += sourceSets.test.runtimeClasspath
    classpath += sourceSets.integrationTest.runtimeClasspath
}
check.dependsOn architectureTestTask
6

I’m guessing this won’t be reported correctly in JUnit reporting since the exact same test class/package is being run multiple times. Is there a single entry in your JUnit reports? (possibly last one wins) Or is the same package/class reported multiple times? If multiple and one fails, how do you know which project it came from?

With my copy based approach, you could use a template java file and change the package for each project. You can also run the test per project in your IDE (by right clicking on the test) which I believe you can’t do with your approach

7

I’m using TestNG. I get a report for each subproject. Furthermore, the test error message gives me all the information I need. I’m not interested in running the tests using the IDE. The tests I’m writing are not meant to interfere with regular dev work, and if they do it’s OK to jump through some hoops. Besides, I think you can just call the test/check/… task from your IDE, which includes the test you can’t see in the subproject source set.

8

Ah, ok… I prefer a single test report so don’t have a report per project. For my use case I think I’d need each test to be in a separate package.

I’m not interested in running the tests using the IDE.

Perhaps some of your teammates are

9

Due to a mistake on my side I’m unable to provide the answer here, so here’s a link to StackOverflow instead: