How to manually parse command line arguments?

Help/Discuss
1

There’s been a great deal of discussion over the years about passing command line arguments to tests with Gradle (IE: -Dfoo=bar). The solutions generally boil down to two approaches:

  1. Adding a check for every supported parameter
test {
    systemProperty "myPropOne", System.getProperty("myPropOne")
    systemProperty "myPropTwo", System.getProperty("myPropTwo")
    systemProperty "myPropThree", System.getProperty("myPropThree")
    // etc...
}

For projects with hundreds of possible parameters this quickly turns into a maintenance nightmare. Every time a parameter is added, removed, or changed in the source code a duplicate effort must be made in the build script(s).

  1. Passing all system properties onwards
test {
    systemProperties = System.getProperties()
}

This takes all of the current system properties and passes them on to each forked JVM in which the tests will run. Since this includes the properties which were passed via command line it eliminates the headache of having to manually support them (see approach #1). Unfortunately this can bring in all sorts of unwanted properties that can lead to instability when running tests. For instance, the following error only appears for me when I take approach #2:

Exception in thread "main" java.lang.ClassNotFoundException: org.jacoco.agent.rt.internal_932a715.PreMain
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:300)
	at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:397)
FATAL ERROR in native method: processing of -javaagent failed

At this point I’m not happy with either solution #1 or solution #2. Instead I’d like to manually parse the command line arguments passed to Gradle for system properties and allow only those arguments to be passed to my tests via the systemProperties map. Here’s the code which I’d like to use for this:

import java.util.regex.Matcher;
import java.util.regex.Pattern;

test {
    String commandLineString = getFullCommandUsed()

    // This is basically just Java code.  Is there a more 'Groovy' way?
    Pattern argPattern = Pattern.compile('-D(.*?)=(.*?)(\\s|$)')
    Matcher argMatcher = argPattern.matcher(commandLineString)
    while (argMatcher.find()) {
        String argKey = argMatcher.group(1)
        String argValue = argMatcher.group(2)
        systemProperties.put(argKey, argValue)
    }
}

I’ve confirmed that this would work by hard-coding the string in place of getFullCommandUsed(). Unfortunately the getFullCommandUsed function doesn’t exist. If it did it would return the command used to run Gradle along with any command line arguments which were passed to it (IE: “./gradlew test -Dfoo=pretty -Dbar=please”).

I’m wondering if there’s some other function or hidden feature in Gradle that will allow me to do this? If not, is there any other way to retrieve only the system properties that were set via command line arguments, or are they merged with the other system properties and thrown away immediately after Gradle starts?

2

We don’t expose the command-line. We just parse it and throw it away.

You could do something like (using the task name as a prefix for system properties):

test {
   systemProperties System.properties.findAll({ k,v -> k.startsWith(name) }).collectEntries({ k,v -> [ k - "${name}.", v ] })
}

Then you could do gradle test -Dtest.foo=bar and foo=bar would get passed along to the test.

1 Like
3

I was thinking about trying something like this. This does make the assumption that there won’t be any pre-existing system properties with the same prefix on whatever machines run the test task, but that’s a relatively safe assumption to make. I’ll go ahead and take it. Thanks!

4

yeah, it’s not perfect. You should probably check for startsWith("${name}.") instead. and you could always add another prefix:

gradle -DpleasePassThisTo.test.foo=bar test

What sort of things are you having to pass to your tests? Are they something like per-user database connection info or environment settings? Or are they different “variants” of tests? e.g., test against Firefox, then Chrome, then Edge, etc.

5

Pretty much anything. Our framework merges several configuration mechanisms into a hierarchy like this:

  1. The code
  2. System properties
  3. Environment variables (eek)
  4. TestNG suite XML parameters
  5. Property files (last loaded file has priority if there’s a conflict between keys)

Most teams store ~95% of their configurations in property files, but there’s always the possibility that some Jenkins job somewhere is overriding some particular value on the command line (system properties) for some particular reason. Here are some common cases:

  • Setting the environment in which to run the tests
  • Configuring an internal proxy server for the tests
  • Changing configurations for our mocking tool
  • Changing the TestNG suite to use for the tests
  • Changing the e-mail address to send reports to
  • Setting the tests to run via Selenium Grid or not
  • Changing timeout values for particular services
  • Changing login information for specific test cases

I just talked to a co-worker and I guess we’ll be debating whether to support only a smaller pre-defined set of such parameters (IE: solution #1) or to keep this open-ended. I guess I can see both sides of the argument. Any thoughts?

6

I’d say you should try to capture all of the variations in the build, where that makes sense. That keeps the information about how to build and test the application in the same place, let’s developers run the same kinds of builds as CI and can help modeling the pipeline in Jenkins a little easier (you don’t have to tweak a bunch of properties for each job).

I don’t know the exact specific, so take this as general guidance:

  • If there are different suites of tests, you could have a test task for each suite. This makes it easier for a developer to run the exact suite (gradle slowTests) vs running a “generic” task and parameterizing it with system properties ( gradle test -Dtest.suite=SlowTests).
  • If there are tests for running against a particular service, that could be done with separate tasks too. If those only make sense with additional configuration (e.g., it’s only going to work because you’re going through Jenkins), you could make it so those tasks only get created for CI.
  • If there are environmental related differences, like the developer should supply their credentials, but the CI has a separate set of credentials, you could push that out to the user home gradle.properties file. CI would be setup with one set of credentials and developers would supply their own. You can fail if the properties aren’t setup properly and you’re trying to run tests.
  • If there are even more combinations of properties that control different things, you might consider enumerating them and trying to make things less open ended (like above, you could have separate test tasks for each configuration).

As you can see, if you have a build (tests) that can be deeply impacted by tweaking system properties, you sort of want to limit that or at least give it a name. We have something like that in Gradle, where we want to run our integration tests with Gradle configured in different ways (with --parallel turned on, from the same API as the IDE, from the command line, with and without the daemon, etc). We create a separate integration test for each of those configurations. The “regular” integration test task runs the integration tests with the fastest Gradle executer, but you can also run specific ones (parallel mode) or override the default. But once you get to CI, we only run the tasks that exist and not a generic test task that has any number of system properties configured specifically for it. That way, when something fails, we can just say “run the parallel mode integration tests” and everyone knows what that means.

1 Like