I use Spock and the @Unroll option to parameterize my integration tests. This allows me to reuse the same Specification for multiple sets of integration tests. I put different files into place (which drives the data-driven tests), and then basically rerun the same test class.
To execute the “separate” tests, I create multiple Test tasks with different task names, inside each of those tasks are calls to put the desired files into place, and then I use an “include” to specify which Test class to call, which is the same class for several different tests.
This all works great, except the JUnit XML and HTML reports that get generated are always named based on the Test class instead… so they get overridden with each subsequent run.
I’ve inspected the “reports” object in the Test task… only the destination folder seems to be configurable. Is there some way to resolve this, using perhaps the task name instead? Currently, I have a custom Test task which does a copy/rename after execution… but that doesn’t work for the HTML reports that get generated… and generally feels like an uncomfortable hack. I’m currently investigating Spock to see if it has options, but this very much seems like a Gradle limitation. Ant has this functionality, but it doesn’t seem to exist in Gradle.
You can rename the reports themselves and create and individual report for each test task, which is what I think you want. You can take a look at how we configure the integration tests in the Gradle build for an example on how to name the report after the task name.
I just tested using the $name parameter, and it still creates just another folder, but doesn’t actually change the name of the XML file that JUnit produces. I’m not sure if I misread your post, but thanks for answering.
Ah yes, what we configure is the directory name for the reports, not the XML file itself. I suppose if you wanted you could just add a task to rename the file.
What’s the problem with the reports having the same name if they’re in different directories? Does it have something to do with something that aggregates it all?
How are you aggregating the results? If you simply running the same test, but in separate processes on different inputs then I’m not sure how we would create a uniquely identifying report.
So aggregation is no longer a concern… I tried the BuildDashboard plugin and it just plain works. Added to the end of a build, it knows of all tests that ran (JUnit and HTML) and aggregates them.
Remember that I have a data-driven Spock spec (using pipes and @Unroll) that I’m reusing for different tests with different test task names. I wanted the aggregated information to say the task name instead of the class name. A rename was easy with the JUnit XML results, but not so much with the HTML results.
In the end, I simply duplicated the Spock spec to have several different names. I abstracted some of the data pile logic into a separate non-test class, so the duplication isn’t as awful as it sounds. Still doesn’t feel exactly right… but it feels more right.
Thanks every one for responding. Though I have a workaround, I’m still very interested in opinions if folks still have some.