Gradle application plugin and CMD.EXE line-size limitations

Old Forum Archive
1

Currently the Gradle Application plugin generates start file for windows that sets the classpath like this:

set CLASSPATH=%APP_HOME%\lib\myApp.jar;%APP_HOME%\lib\activation.jar;...

For many applications, this line can exceed the maximum line size of CMD.EXE (8,190 characters in case of Windows 7) and cause the following error:

c:\myApp\build\install>myApp\bin\myApp.bat
The input line is too long.
The syntax of the command is incorrect.

The maximum size for an environment variable is around 32K characters (not bytes), so it would be better if the script was changed to:

set CLASSPATH=
set CLASSPATH=%CLASSPATH%;%APP_HOME%\lib\myApp.jar
set CLASSPATH=%CLASSPATH%;;%APP_HOME%\lib\activation.jar
set CLASSPATH=%CLASSPATH%;...

Even then, it would be a nice touch if the Gradle script generator issues a warning when the total classpath exceeds 32,767 characters.

http://msdn.microsoft.com/en-us/library/windows/desktop/ms682653(v=vs.85).aspx http://support.microsoft.com/kb/830473/en-us http://blogs.msdn.com/b/oldnewthing/archive/2003/12/10/56028.aspx

2

An ugly workaround for this problem and the other one I posted today is to add the following to your build script:

applicationDistribution.eachFile {
FileCopyDetails f ->
        if (!f.name.endsWith('.bat')) return
        f.filter { line ->
            if (!line.startsWith('set CLASSPATH=')) return line
            def originalCp = (line - 'set CLASSPATH=').split(';')
            def newCp = ['']
+ originalCp.collect { cpe -> "%CLASSPATH%;$cpe"}.unique()
            return newCp.collect { cpe -> "set CLASSPATH=$cpe" }.join('\r\n')
        }
    }
3

I don’t think this actually fixes the problem, as the value has to be used eventually as the -cp arg when starting the actual app.

4

In fact no - if the environment variable is named CLASSPATH, the JVM would pick it up without specifying it on the command line. Alternative way is to use java -jar and make sure the correct class path is specified in the main jars manifest.

5

We currently do set this on the command line: https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/windowsStartScript.txt#L76

Relying on the JVM picking up CLASSPATH is probably not a bad solution to this problem. Will raise as an issue.

6

Raised as GRADLE-2992.

7

Would you be interested in committing such a fix to the codebase Dimitar?

8

Is there a reason why the script can’t use lib/*.jar for classpath for Java 6 and above?

9

How would that help?

10

The command line is too long because every dependency is listed individually on the classpath. The application plugin puts all of the dependencies in the same directory. So specifying that directory with a wildcard eliminates the problem of the long classpath being passed on command line.

11

That might not work as it might be the shell (it is from memory) that expands the wildcard, which means the problem still exists.

12

I’m not sure I understand what is meant by “it is from memory”.

I have a project right now with the same problem. If I edit the classpath variable in the batch file and change it to:

set CLASSPATH=%APP_HOME%\lib*

my program runs.

13

“it is from memory”

From my memory. That’s how I remember it working on window, which I was wrong about.

14

Sorry - it didn’t paste right and display right - should be slash lib slash star

15

The wildcard is a blunt tool - it is convenient, but it doesn’t make classpath ordering explicit, which can make troubleshooting more difficult than it needs to be.

It is way too easy that your application establishes a dependency on an artifact that embeds a library, rather than depending on it, and many months later you upgrade another artifact that starts depending on incompatible version of that same library and all of a sudden your application stops working.

Last time it happened to me while I was using Jalopy and Groovy - turned out that Jalopy’s Antlr is older than Groovys and if Groovy is at the front, all works fine, if Jalopy is first - it breaks.

That said, I think it is a good good option to have.

Another option is to enforce app working directory and use relative paths.

Yet another is to embed classpath in the manifest and use java -jar launcher (again relies on fixed work directory)

I am going to put a more coherent proposal of all the options in this topic and implement it when I get the green light from the Gradleware guys.

16

Before I start working on this issue, I’d like to summarize my thoughts so far.

What started as a simple issue turned into a bunch of related suggestions for improvement. I am dumping them below in no particular order:

  1. Deduplicate the classpath - each jar should appear exactly once

PROS: shorten classpath, make ordering clear

CONS: none

PROPOSING: change he default behavior without further customization options

  1. Build the CLASSPATH variable incrementally, change the Java invocation not to pass the classmate explicitly

PROS: Can handle up to 32kb classpaths - up from the current 7kb

CONS: Classpath not visible in the process description (i.e. when looking by PS). The 32k boundary can be exceeded when using deeply nested paths.

PROPOSING: Classpath should be built incrementally in all cases, add an option determining whether it should be passed on the actual cmd line.

  1. Add the notion of working directory and specify the classpath relative to it (rather than absolute paths)

PROS: shorter classpaths, generally useful for some types of apps

CONS: some types of apps benefit from being able to run from anywhere and not tied to a specific working dir

PROPOSING: add an option to specify an app working dir in the distribution - if specified use it, otherwise default to the current behavior

  1. In relation with the previous suggestion (not directly related to CP), add a ‘resources/config’ directory, containing resources available on the classpath.

PROS: useful for location independent apps for locating configs, etc.

CONS: often an overkill

PROPOSING: add an option to specify an unpacked resources dir in the distribution - if specified use it, otherwise default to the current behavior

  1. Enumerate only the jars that need explicit ordering and specify the rest using a wildcard classmate. Will require a new property to select the explicit part of the classpath.

PROS: makes the classpaths MUCH shorter, savior when nothing else works, ideally we wouldn’t need explicit ordering at all

CONS: the actual classpath is hidden and the ordering is platform-dependent, can be problematic when troubleshooting classloading problems

PROPOSING: add an option to enable wildcard, and optionally a list of dependencies that should be enumerated explicitly

  1. Specify the classpath in the manifest relative to the main application JAR and use java -jar to run the app

PROS: eliminates the problem altogether

CONS: classpath is hidden from ps (3), it is difficult to do ad-hoc tweaks to classpath (i.e. add another jar, change ordering, etc.)

PROPOSING: the other approaches may be sufficient, but we could implement this if we consider wildcards without prefix path

So, the proposal so far is to:

  • deduplicate the classpath + always build the CLASSPATH variable in incremental fashion + add an option whether to pass -cp $CLASSPATH on the command line + add an option to specify an ‘applicationWorkingDir’ in the distribution - if done so, CP entries are relative to that; otherwise use full paths + add an option to specify an ‘applicationResourcesDir’ in the distribution - if done so, it is added to classpath + add an option to use wildcard classpath + add an option to specify an ordered prefix to wildcard classpaths (perhaps an overkill)

  • add an option to use java -jar launcher and embedded classpath

Please provide feedback, especially if you think that any of these don’t make sense. This is a goldplated proposal and we need to carefully weigh the utility vs. complexity of each feature. I believe the first 3 items are the bare minimum we need.

17

Hi Dimitry,

We haven’t forgotten about this. The person who needs to comment on this is absolutely swamped at the moment so it might take a little bit before he can take a look.

Sorry for the delay, but we are very interested in addressing this.

18

I have implemented the first 2 proposals from this post and submitted a pull request.

I am willing to work on the rest of the scope outlined above, but I’d like to get some feedback first.

19

Thanks Dimitry.

We are about to invest some time into pull requests so you should get some feedback very soon.

Thanks for taking the time to contribute.