Please, consider the following scenario: I have integration tests in separate projects. These integration tests require the services of various servers built from the same multi-project build. To make matters worse, the servers themselves have dependencies amongst them.
As an example, assume that there are two servers: “Server1” and “Server2” where “Server2” needs “Server1” already running. Also, there are two integration tests: “IntTest1” and “IntTest2” where “IntTest1” needs “Server1” and “IntTest2” needs “Server2”.
Given this example, I want to able to do the following:
Execute “:IntTest1:test” alone, which would work this way:
Build, then start “Server1” in the background.
Execute the test code of “IntTest1”.
After tests complete, stop “Server1”. - Execute both tests:
Build, then start “Server1” in the background.
Build, then start “Server2” in the background.
Execute the test code of both integration tests.
Stop “Server2”.
Stop “Server1”.
To complicate matters, stopping server requires code built from the same multi-project build. What is harder to me is stopping the servers (in reverse order as they were started). I have tried to manually execute tasks in ‘gradle.buildFinished’ (i.e.: execute the actions of ‘JavaExec.getActions()’) but this seems to fail with various exceptions (usually, complains about not having a dependency locked in the artifact cache).
Do you have any idea how this issue might be best solved?
If you want to start/stop the servers as part of the build (e.g. in ‘test.doFirst’), then the code that does this must be built into a Gradle plugin and added to the buildscript’s classpath - this would be the cleanest solution IMHO. One can add such code in the ‘/buildSrc’ directory as well - but I think that having a custom plugin is a good thing - sooner or later somebody else will benefit from it (e.g. another product that wants to start/stop your server during their integration tests).
I have developed a similar plugin, this is how it is setup:
we have a standalone Gradle plugin project which builds and uploads the ‘Server plugin’ to our repository - the plugin does not have a dependency on the Server that it installs (though it makes some assumptions about the structure and configuration files of the Server)
the build of the ‘Server’ adds a ‘buildscript’ dependency to the latest stable version of the ‘Server plugin’, the build then builds and uploads the ‘Server’ to a local filesystem repository
the integration tests are then using the ‘Server plugin’ to install the latest version of the ‘Server’ - in this case this would be the one just uploaded locally, then deploy the integration tests and execute them
if tests succeed, the ‘Server’ is uploaded to the remote repository and available for other products to consume it
Thank you for your answer. Having the code on the classpath of the script is inconvenient because if it the way the server needs to shutdown changes, there will be no release build available for the shutdown code. So it needs manual tweaking, uploading this subproject to the Maven repo relying on current snapshot build (obviously, the version doesn’t have to be actually “-SNAPSHOT”).
To be more explicit, the server is shutdown through RMI, so if for example, I happen to rename the rmi interface, the old latest release of the shutdown code will be unusable.
If you want to test the logic that starts and stops the server, then this needs to happen as part of your tests instead of the build itself. Why don’t you create some base integration test class that does all this?
It is not about testing if start/stop works or not. It is just that I cannot have a “release” version of the stop server code because they share a common project dependency with the server (i.e.: the interface for the RMI call) and the dependency is released with the server. So if the interface changes, so need the stop code as well, making the older “release” version unusable.
Then this means that all the setup of the Servers needs to happen in your tests and not in the build - the build should use only stable code (if it is a release build).
The problem is that it is the build script which knows how to setup a server and not the test code. I want the test code to be independent of how the server was setup. So, if I want to manually test the client side against an already running server, I will be able to do so.
Actually this setup only happens during test and so I don’t feel that it must be a “release” version; and if I test start/shutdown separetly first, there is no reason to believe that start/stop is actually broken. Despite this, the shutdown code cannot be considered a “release” version.
The setup of the Server need not be in the test code, it can be built in a separate module and just referenced by the test code (as a dependency), I do not feel that it is a good idea to add this code to the build script of the project. You might have a base server bootstrap integration test class that starts/stops the server - the client tests might then be configured via a system property whether to use the embedded server or an external server.
If there was only a single server to set up, it would be more simple but since one server might depend on another, I thought, I can leverage the fact, that Gradle allows me declare dependencies between tasks. I would prefer to declare these dependencies in the script. For example, I can declare that “Server2:startServer” depends on “Server1:startServer” but if I start the server in the script, then it is natural to stop it there. Also, if I start/stop the server in the test code, then I have to do this for each test which might add a few minutes to the overal test execution time, although this is something I can live with.
Regardless, if what I want is not possible without writing too complex code in the build script, I will have to workaround it. Next week, I will decide one way or another and I’ll post what I chose. Encoding everything needed to start the servers in system properties might be awkward but if I can’t come up with a better solution in the mean time, probably it will be something like that.