For a JVM application we’re using Spring REST Docs which basically record HTTP communication during test runs into files.
Further we use springdoc-openapi to generate the OpenAPI specification from our code at run-time. We are using the recorded communication to serve as examples in the specs. For this, we need the files of the recorded communication to be part of the resources at run-time.
I managed to do this with some simple ProcessResources tasks, and specialized Test tasks, which only execute the tests needed to record the communication, however, I need to call them in separate gradle executions.
From my understanding the problem is that the sourceset cannot be modified during the execution, but that’s what we would need to do.
As I said, the problem is “solved”, but I’d like to better understand what the proper gradle-way would be.
val snippetsDir by extra {
file("build/generated-snippets")
}
val test by testing.suites.existing(JvmTestSuite::class)
val apiTests = tasks.register<Test>("apiTests") {
outputs.dir(snippetsDir)
filter {
includeTestsMatching("*ControllerApiTests")
}
}
val processRestDocResources = tasks.register<ProcessResources>("processRestDocResources") {
dependsOn(apiTests)
from(snippetsDir)
into(layout.buildDirectory.dir("resources/main/http-snippets"))
}
val openApiTests = tasks.register<Test>("openApiTests") {
// this doesn't work help here
dependsOn(processRestDocResources)
inputs.dir(layout.buildDirectory.dir("resources/main/http-snippets"))
filter {
includeTestsMatching("…")
}
}
Let’s start with extra, practically any usage of ext or extra is a code-smell for doing something not properly but as work-around. In your case snippetsDir should probably just be a local variable.
Having a separate Test task that generates those files sounds like the proper idea generally. One note though if you care, you could do val apiTests by tasks.registering(Test::class) { ... } instead of duplicating the name, same for the other tasks.
Practically any explicit dependsOn (except if there is a lifecycle task on the left-hand side) is a code-smell too and usually mean that you do not properly wire task outputs and task inputs together like you should do. This is also here exactly the case, as instead of wiring tasks together you manually configure a path (from(snippetsDir)) and add a manual task dependency instead of properly wiring in- / outputs together which brings you necessary task dependencies automatically. So in your case I’d do from(apiTests) with an include so that only the files you want to have are synced by that task.
Your processRestDocResources as-is is extremely bad practice and highly discouraged though, for one reason. It has overlapping outputs with processResources. Tasks should never have overlapping outputs. This brings flaky build results, sometimes silently, sometimes even failing, wrong up-to-date check results, bad task output cache entries and so on. Each task should have dedicated outputs that do not overlap with any other task. In your case for example the build/resources/main/http-snippets dir will be output of processRestDocResources and also processResources. Gradle 7 will also warn you about this … sometimes … and Gradle 8 will fail because of this … sometimes …, giving you actually bad advice to add task dependencies which usually is just another work-around and not the proper solution.
Instead of having the own processRestDocResources task, you could just configure the processResources task to also include those files.
But most often also this is not the right thing to do, as then only tasks that use the processResources task result will get those files, not everyone consuming resources of the main source set.
Usually the proper way to include generated sources or resources is to add the task generating those files as srcDir to the respective source directory set. So in your case you should most probably:
change the processRestDocResources to be a Sync task
change its destination to something that is not overlapping with any other task like layout.buildDirectory.dir("rest-doc-resources")' and an into that makes sure the files land in http-snippets subdirectory (not the http-snippets as destination directly)
do something like sourceSets { main { resources { srcDir(processRestDocResources) } } }, then the outputs of that task are considered resources by all tasks needing them and you automatically get the necessary task dependencies where appropriate
Thank you so much, that already looks much cleaner.
However, this leads to a circular dependency, or am I overlooking something? The tests all depend on the main sourceSet, which depends on processRestDocResources, which depends on apiTests then.
But I don’t think I understand your 2nd bullet point
change its destination to something that is not overlapping … and an into that makes sure the files land in http-snippets subdirectory (not the http-snippets as destination directly)
A top-level into is like setting the destinationDir, yes, afair doesn’t really matter which you use.
But you can also use into in nested copy specs to define subdirectories like
If you have the “http-snippets” part of the top-level into, the “http-snippets” directory will be the root and so you miss it in the result when you use the task outputs as resources.
However, this leads to a circular dependency, or am I overlooking something? The tests all depend on the main sourceSet, which depends on processRestDocResources, which depends on apiTests then.
Well, I don’t know, you only shared a very small part of your configuration and it wasn’t clear how you configure your custom Test tasks.
If the Test tasks - especially the one generating the snippets - use the main source set, that’s of course then a problem because of circular dependency.
Maybe you can refactor things, so that the tests generating those snippets do not need the main source set?
If not, then maybe you really need to do it dirty and not declare the generation task as resources source directory.
Also configuring the processResources task will then likely not help, as the same problem would arise as processResources also contributes to the main source set output your tests need.
So in that case you probably need to configure the jar task and maybe the sourcesJar task if you use one and potentially other tasks that need resources to include the result of the processRestDocResources task.
Sorry if my explanation was too incomplete, and thanks for your help again!
springdoc-openapi looks at the Spring MVC controllers at runtime, I want to use this mechanism during a test run, so I don’t thinks it’s possible to not use the main source set (as that’s where the Spring MVC controllers reside).
Your remarks did point me in a direction that looks feasible to me: a second source set, which follows the official guide for an integration test source set.
This new source set then includes the outputs of the controller tests, which reside in the default test source set.
In case anyone else comes across this thread, here’s what I came up with:
val test by testing.suites.existing(JvmTestSuite::class)
// specialized test task (default test source set)
// only runs specific tests, registers Spring REST Docs snippets as output
val controllerApiTests by tasks.registering(Test::class) {
group = "verification"
outputs.dir(layout.buildDirectory.dir("generated-snippets"))
testClassesDirs = files(test.map { it.sources.output.classesDirs })
classpath = files(test.map { it.sources.runtimeClasspath })
filter {
includeTestsMatching("*ControllerApiTests")
}
}
val openApiTestSourceSet = sourceSets.create("openApiTest") {
// this is effectively an integration test source set as described here
// https://docs.gradle.org/8.10.2/userguide/java_testing.html#sec:configuring_java_integration_tests
// once we move our integration tests here, we could use sourceSets.main instead
compileClasspath += sourceSets.test.get().output
runtimeClasspath += sourceSets.test.get().output
resources.srcDirs(sourceSets.test.get().resources)
configurations[this.implementationConfigurationName]
.extendsFrom(configurations.testImplementation.get())
configurations[this.runtimeOnlyConfigurationName]
.extendsFrom(configurations.runtimeOnly.get())
}
val processOpenApiTestResources by tasks.getting(ProcessResources::class) {
// include the output of controllerApiTests as resources in openApiTestSourceSet
from(controllerApiTests) {
into("http-snippets")
}
}
val openApiTests by tasks.registering(Test::class) {
group = "verification"
outputs.dir(layout.buildDirectory.dir("openapi"))
testClassesDirs = openApiTestSourceSet.output.classesDirs
classpath = openApiTestSourceSet.runtimeClasspath
}
tasks.bootJar.configure {
// include controllerApiTests output (the Spring REST Docs snippets) in the classpath,
// so they are available as resources at runtime
// this also affects bootBuildImage
classpath += controllerApiTests.get().outputs.files
}
