Gradle is attempting to execute my Spock tests twice

Help/Discuss
1

Background

I have a Selenium Grid project set up to execute my spec in two different browsers, Chrome and Firefox. I’m using Gradle to execute my specs. The spec will successfully execute twice, once in Chrome and once in Firefox, as expected, and then a third instance will execute in the default browser and fail.

Expected Outcome

  1. A Chrome instance will open, the spec will run, and it will pass.
  2. A Firefox instance (using geckodriver) will open, the spec will run, and it will pass.
  3. The Gradle task will finish successfully.

Actual Outcome

  1. A Chrome instance will open, the spec will run, and it will pass.
  2. A Firefox instance (using geckodriver) will open, the spec will run, and it will pass.
  3. A new Firefox instance (using the firefoxdriver) will open, the spec will not run, and it will fail.
  4. The Gradle task will fail as the last test execution failed.

My Thoughts

  • I’ve run into issues with Gradle executing Spock tests twice before. to fix this, I had to add the following code:

      test {
      actions = []
  }

  • I’ve also noticed that when my Selenium test gets executed again, it will open it up using the default firefox driver, not the gecko or marionette driver.

    • The firefox driver is old and doesn’t support the latest version of Firefox, but it’s Selenium’s “default” browser when you don’t specify a browser to execute your test in.

  • I have two Selenium Grid nodes set up, so I’m wondering if Gradle is executing a third version of my test that doesn’t match up with one of the nodes, but I’m only telling it to run two tests.

Code

I created a sample project that reproduces this issue on Bitbucket. Instructions on how to run the sample test is included in the readme.

As a snippet, this is the sample spec I have:

class W3SchoolsFormExampleSpec extends Specification {

    def 'Test form submission is successful on W3Schools'() {
        when: 'Name info is submitted into the form'
            open('https://www.w3schools.com/html/html_forms.asp')

            $(byName('firstname')).setValue('Clark')
            $(byName('lastname')).setValue('Kent')

            $x('//*[@id="main"]/div[3]/div/form/input[3]').click()

        and: 'Switch to newly opened tab'
            switchTo().window(1)

        then: 'New page should display the passed-in request params'
            $x('/html/body/div[1]').shouldHave(text('firstname=Clark&lastname=Kent'))
    }
}

and this is a snippet of my build.gradle file:

import org.openqa.grid.selenium.GridLauncherV3

...

test {
    // Prevent Gradle from strangely executing Spock tests twice
    actions = []
}

task testW3SchoolsForm(type: Test) {
    outputs.upToDateWhen { false }

    doFirst {
        // Check to see that the Selenium drivers are installed
        if (!file("C:/Selenium/chromedriver.exe").exists()) {
            throw new GradleException(
                    'ERROR: Please install the web drivers in the correct location.'
            )
        }

        // Register the hub
        GridLauncherV3.main('-role', 'hub')

        // Register the Chrome node
        GridLauncherV3.main('-role', 'node',
                '-browser', 'broswerName=chrome,platform=WINDOWS',
                '-hub', 'http://localhost:4444/grid/register',
                '-port', '4446'
        )

        // Register the Firefox node
        GridLauncherV3.main('-role', 'node',
                '-browser', 'broswerName=firefox,platform=WINDOWS',
                '-hub', 'http://localhost:4444/grid/register',
                '-port', '4446'
        )
    }
}

enum BrowserType {
    CHROME('chrome'),
    FIREFOX('gecko'),

    def browserString

    BrowserType(browserString) {
        this.browserString = browserString
    }
}

BrowserType.values().each { browserType ->
    tasks.create("testW3SchoolsForm${browserType}", Test) {
        // Force the tests to run every time
        outputs.upToDateWhen { false }

        def drivers = [
                (BrowserType.CHROME): 'chromedriver.exe',
                (BrowserType.FIREFOX): 'geckodriver.exe',
        ]

        def browserProperty = browserType.browserString
        def webdriverPath = file("C:/Selenium/${drivers[browserType]}")

        // Set the respective system properties for each browser
        systemProperties["webdriver.${browserProperty}.driver" as String] = webdriverPath
        systemProperties['selenide.browser'] = browserType.browserString

        filter {
            include 'com/example/dummy/W3SchoolsFormExampleSpec.class'
        }

        testLogging {
            events 'PASSED', 'FAILED', 'STARTED', 'SKIPPED'
        }

        testW3SchoolsForm.dependsOn "testW3SchoolsForm${browserType}"
    }
}

Any ideas as to why Gradle is executing a third instance of my test?

2

You have 4 different Test tasks in your example. There are a few odd things I see.

When you apply the java plugin, you automatically get a Test task. This would try to execute the tests. You’re preventing that when you’re removing all the actions from the task with test { actions = [] }.

You define a testW3SchoolsForm${browserType} per $browserType as a Test task. This will try to execute the tests with slightly different system properties.

You define a testW3SchoolsForm that’s also a Test task. That will also try to execute the tests. I think this is where the 3rd execution is coming from.

I would do a few different things…

  1. (Optional) Decide that test will run the “default” browser tests (e.g., Chrome) or the “quick” tests or something like that and keep it. Or decide that test is unnecessary and disable it (test.enabled = false)
  2. Create a testW3SchoolsForm${browserType} as you are doing now. Make sure these are wired to check so they run when you run gradle check.
  3. Create a separate “Grid Launcher” task for each browser type, so you only start one or the other when you run the particular browser’s testW3SchoolsForm$browserType task.
  4. The “Grid Launcher” task should use a JavaExec task so you don’t need to add selenium to the classpath of your buildscript (what you’re doing in buildSrc).

You would use it something like this (I haven’t tested this, so pardon any typos):

configurations {
    selenium
}

dependencies {
    selenium 'org.seleniumhq.selenium:selenium-server:3.6.0'
}

task launchHub(type: JavaExec) {
    classpath = configurations.selenium
    mainClassName = "org.openqa.grid.selenium.GridLauncherV3"
    args '-role', 'hub'
}
task launchChrome(type: JavaExec) {
    classpath = configurations.selenium
    mainClassName = "org.openqa.grid.selenium.GridLauncherV3"
    args '-role', 'node', 
        '-browser', 'broswerName=chrome,platform=WINDOWS',
        '-hub', 'http://localhost:4444/grid/register',
        '-port', '4446'
    dependsOn launchHub
}
task launchFirefox(type: JavaExec) {
    classpath = configurations.selenium
    mainClassName = "org.openqa.grid.selenium.GridLauncherV3"
    args '-role', 'node', 
        '-browser', 'broswerName=firefox,platform=WINDOWS',
        '-hub', 'http://localhost:4444/grid/register',
        '-port', '4446'
    dependsOn launchHub
}

HTH

3

You were sincerely helpful - none of my other co-workers know Gradle, and as someone who’s been using Gradle for the past two years, I’m trying to learn more of it and convince my co-workers to switch from Maven.

The JavaExec task is really cool, as well as the configurations block. I took your suggestions and implemented the code. I did have to apply the application plugin in order to use the mainClassName property.

I also made check and launchHub dependent upon the testW3SchoolsForm task, so doing a gradle check successfully runs the spec in each browser.

Thank you very much! I truly appreciate it.
– JJ

4

No problem.

I was wrong about mainClassName. For JavaExec, it’s actually just called main. You shouldn’t need to use the application plugin for that.

5

Ah, thank you for the clarification.