Execute tests in remotely Jenkins


(Stefan Wolf) #1

I would like to implement/have the following feature: From the workspace on my computer I would like to execute tests remotely on Jenkins. This should happen without having to check in. My usecase is that we have a lot of GUI-Tests. If you run them locally, you cannot do anything with your computer while they are running. But we have enough jenkins windows slaves where we could execute these tests. So why not use them?

My current idea for an implementation of this feature would be the following: * Configure the Test-Task to use Jenkins to do the tests (configure Jenkins URL, …) * When running ‘gradle test’ then a Job on Jenkins is started which gets scheduled to one of the windows executors. This job then connects to the local gradle process via a hudson.remoting.Channel. * The Test-Task waits for the connection on the Channel and then uses it to send over the Test-Execution to the Jenkins-Slave * The Slave executes the Tests * The result of the Test-Run is sent back to the local gradle instance * The Test-Task finishes and displays the result.

It would also be awesome to use the current forking options on top of running on Jenkins. For example one could then use gradle from Jenkins to distribute Tests to slaves, to make them run concurrently.

I already poked around in the Gradle sources to try to find out where to integrate this feature. I could probably try to replace the DefaultTestExecutor in the Test Task. Another option would be to wrap on ObjectConnection over the Channel. What would you suggest?

For the Jenkins side I would have to implement a plugin which starts a vm and opens the Channel to the local gradle plugin. But this is not too difficult.


(Adam Murdoch) #2

I think this is an excellent idea and would love to see this implemented. Are you planning to implement this in Gradle core? You would be very welcome to do so.

Implementation-wise, I wouldn’t replace ‘DefaultTestExecuter’, as we still want most of the stuff in there. Instead, I’d think about adding another ‘Factory<TestClassProcessor>’ implementation that is responsible for starting remote test processes. This will allow us, for example, to mix both local and remote execution.

If you do want to proceed with the implementation, it would be nice if you could put together a quick design spec for what you intend to do (see ‘/design-docs’ in the Gradle source tree). It doesn’t have to be very detailed, just a quick sketch is fine.


(Stefan Wolf) #3

So I added a pull request for the design docs some while ago: https://github.com/gradle/gradle/pull/163

I also implemented a prototype which seems to work: https://github.com/TNG/remote-executor-gradle-plugin The interesting code there is in the buildSrc directory.

I still have some open questions: 1) Right now I am using a quite hacky solution (package private setter) to get my TestProcessor into the Test-Task. Would it be possible to open up some API in Jenkins core to make this possible? Currently the only difference of my implementation with ForkingTestClassProcessor is that I am using a different TestWorker (JenkinsTestWorker). My implementation hands in a TestWorkerFactory which is then called by the JenkinsTestClassProcessor. So would it be possible to add some API method on the Test-Task to configure such a Factory? 2) Should this plugin be integrated into Gradle-Core? I am not so sure about it, since it uses a dependency on Jenkins and you probably would not want that in the core. 3) To configure the plugin I need to select which Test-Tasks to execute on Jenkins with possible JVM-Options of the forked JVM on Jenkins. It would be great if I could do this in the context of the Test-Task. How would I go forward to do so?


(Jon Austen) #4

This kind of thing sounds like it would work pretty good for general GUI tests but obviously, if you are browser testing with Selenium, a grid+hub would work much better.

Also, I am curious, since I know that Jenkins runnning as a service cannot access the logged in users desktop, I would guess you must run your Jenkins slaves as foreground processes in order for it to work.


(Stefan Wolf) #5

You are right, for browser testing you probably want to use Selenium Grid. In order to do GUI tests at all, you need to configure the Jenkins slave correctly. I think we use JNLP to start the slaves, and this works.


(Sonal Bhide) #6

Hi Stefan,

I have already started slave using JNLP.

However, unable to execute two different GUI suits simultaneously on a single slave session, as one browser interferes another.

Could you please suggest a way for this? Is there any setting in Jenkins for parallel GUI execution on same slave session?

Thanks!