Filename too long in Windows


(Sathyakumar Seshachalam) #1

I am beginning to move my maven based project to a gradle based java application that runs on Windows 7. I ran in to an error java.io.IOException: CreateProcess error=206, The filename or extension is too long because of a really long classpath dependencies. I worked around it by shortening the path to my code and java executable location, While this was Ok for sometime, I hit that limit again today when I added few command line argument.

My questions are -

  1. What’s the prescribed way to run applications that have a huge dependency graph. ( I think what I’m working on is relatively small considering other big java apps I have seen).
  2. Is there a way to shorten some paths that gradle adds. For e.g I could see that classpath is pointing to a cached folder org.apache.pdfbox\fontbox\1.8.1`32879bb6bb87b15c6d53bc358e83ede40fc729ae`\fontbox-1.8.1.jar

Ideally I would love to use Gradle to just run my app gradle run. But at the moment, I feel that its not viable in its current state as even adding few extra command line arguments is going to break this in Windows.


JavaExec fails for long classpaths on Windows
(Mark Vieira) #2

This is a known limitation. There are some possible workarounds. See this response to a similar question for info.


(Jason Hatton) #3

I have encountered a similar issue with the generated start up scripts from the Application plugin and in particular the Spring Boot plugin where the classpath created results in an “Input too long” error. This link is a reference to the issue in the old forums.

Here is a Gradle script modification that uses relative paths to fix the issue. This would help avoid any out of order classpath problems using the asterisks workaround. Just depends on the use case which one is most valuable, neither is more right.

startScripts {
    // Support closures to add an additional element to
    // CLASSPATH definition in the start script files.
    def configureClasspathVar = { findClasspath, pathSeparator, line ->

        // Looking for the line that starts with either CLASSPATH=
        // or set CLASSPATH=, defined by the findClasspath closure argument.
        line = line.replaceAll(~/^${findClasspath}=.*$/) { original ->

            // Get original line and append it
            // with the configuration directory.
            // Use specified path separator, which is different
            // for Windows or Unix systems.
            original = original.replaceAll '%APP_HOME%\\\\', ''    
        }

    }

    def configureUnixClasspath = configureClasspathVar.curry('CLASSPATH', ':')
    def configureWindowsClasspath = configureClasspathVar.curry('set CLASSPATH', ';')

    // The default script content is generated and
    // with the doLast method we can still alter
    // the contents before the complete task ends.
    doLast {

        // Alter the start script for Unix systems.
        unixScript.text =
                unixScript
                        .readLines()
                        .collect(configureUnixClasspath)
                        .join('\n')

        // Alter the start script for Windows systems.
        windowsScript.text =
                windowsScript
                        .readLines()
                        .collect(configureWindowsClasspath)
                        .join('\r\n')

    }
}

Maybe someone can take this and work on a pull request that would introduce a relative path concept to the script generation, or another even better solution.


(Jaroslaw Nowosad) #7

How did you use “long path tool” to solve this? Not see anything in common… Please share.
At the moment looking into manifest-jar solution, but is not so straight forward :frowning:


(Stefan Oehme) #8

Please ignore the long path tool spam. Unfortunately some of them slip through spam detection.


(Trejkaz (pen name)) #9

Is there a ticket for this to get fixed properly in Gradle core?

It seems like a bit of a joke to me that a self-proclaimed “declarative” build system is forcing us to write all this procedural code just to get something basic like JavaExec to work at all. And not even a particularly funny joke.


(Trejkaz (pen name)) #10

Another half a year has passed - does JavaExec work in Gradle core yet?


(Malcolm McRoberts) #14

I think I understand how to add the classpath to the manifest. How does that reduce the command length? Don’t I also have to remove this path somewhere else? I’m trying to build a project someone else built, but I’m the only one that seems to be having this problem.


(Trejkaz (pen name)) #15

Well, I know I’m performing a bit of thread necromancy here, but I got reminded about it because of an alert, so to answer the “how to actually do this” portion, we ended up creating these alternate subclasses of JavaExec and Test which live in our buildSrc dir.

The general gist is to replace the task classpath during the execution. We also put it back after execution, in case not doing so breaks something else. This is because we couldn’t find a better place to intercept the attempt to execute the Java process.

You can apply a similar trick to anything which takes a classpath.

import java.io.File;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.file.Files;

import org.gradle.api.file.FileCollection;
import org.gradle.api.tasks.testing.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * <p>Workaround to make {@code type:Test} tasks actually work.</p>
 *
 * <p>Adapted from {@link WorkingJavaExec}.</p>
 */
public class WorkingTest extends Test {
    /**
     * The logger.
     */
    private final Logger logger = LoggerFactory.getLogger(getClass());

    @Override
    public void executeTests() {
        FileCollection oldClasspath = getClasspath();
        File jarFile = null;
        try {
            if (!oldClasspath.isEmpty()) {
                try {
                    jarFile = WorkingJavaExec.toJarWithClasspath(oldClasspath.getFiles());
                    setClasspath(getProject().files(jarFile));
                } catch (IOException e) {
                    throw new UncheckedIOException(e);
                }
            }

            super.executeTests();
        } finally {
            setClasspath(oldClasspath);

            if (jarFile != null) {
                try {
                    Files.delete(jarFile.toPath());
                } catch (Exception e) {
                    logger.warn("Couldn't delete: " + jarFile, e);
                }
            }
        }
    }
}

And

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.io.UncheckedIOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.Arrays;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.gradle.api.file.FileCollection;
import org.gradle.api.tasks.JavaExec;


/**
 * <p>Workaround to make {@code type:JavaExec} tasks actually work.</p>
 *
 * <p>Adapted from an example which does it by customising {@code DefaultJavaExecAction} instead,
 *    because they didn't actually document how to integrate it.</p>
 */
public class WorkingJavaExec extends JavaExec {
    /**
     * Pattern to match chunks of 70 characters. TODO This code is as originally written but probably should be precompiled.
     */
    private static final String MATCH_CHUNKS_OF_70_CHARACTERS = "(?<=\\G.{70})";

    /**
     * The logger.
     */
    private final Logger logger = LoggerFactory.getLogger(getClass());

    @Override
    public void exec() {
        FileCollection oldClasspath = getClasspath();
        File jarFile = null;
        try {
            if (!oldClasspath.isEmpty()) {
                try {
                    jarFile = toJarWithClasspath(oldClasspath.getFiles());
                    setClasspath(getProject().files(jarFile));
                } catch (IOException e) {
                    throw new UncheckedIOException(e);
                }
            }

            super.exec();
        } finally {
            setClasspath(oldClasspath);

            if (jarFile != null) {
                try {
                    Files.delete(jarFile.toPath());
                } catch (Exception e) {
                    logger.warn("Couldn't delete: " + jarFile, e);
                }
            }
        }
    }

    /**
     * Creates a jar file with the given classpath and returns the path to that jar.
     *
     * @param files the set of files making up the classpath.
     * @return the path to the jar.
     * @throws IOException if an I/O error occurs.
     */
    public static File toJarWithClasspath(Set<File> files) throws IOException {
        File jarFile = File.createTempFile("long-classpath", ".jar");
        try (ZipOutputStream zip = new ZipOutputStream(new FileOutputStream(jarFile))) {
            zip.putNextEntry(new ZipEntry("META-INF/MANIFEST.MF"));
            try (PrintWriter writer = new PrintWriter(new OutputStreamWriter(zip, StandardCharsets.UTF_8))) {
                writer.println("Manifest-Version: 1.0");
                String classPath = files.stream().map(file -> file.toURI().toString()).collect(Collectors.joining(" "));
                String classPathEntry = "Class-Path: " + classPath;
                writer.println(Arrays.stream(classPathEntry.split(MATCH_CHUNKS_OF_70_CHARACTERS))
                                     .collect(Collectors.joining("\n ")));
            }
        }
        return jarFile;
    }
}

To use it, you stomp the affected tasks:

  task test(overwrite: true, type: WorkingTest) {
    // we have stuff in here which I'm omitting, but I don't feel like testing
    // whether you can remove the braces so try it and see
  }

Also, anywhere where you would define a new test task, you obviously have to replace that too.

It’s only a matter of time until someone puts tasks like these into a plugin, considering that you hit this problem in pretty much anything larger than a toy project. I would have already done so, but I don’t really know how to write Gradle plugins, so we have them sitting in buildSrc still.

Ideally I would like it such that when the normal Test task is run, it actually performs this workaround. We were never able to find a way to do it from the available information, but surely it’s something that would be very easy to do inside Gradle itself, so I’m not really sure what all the stalling is about.