Could we run JavaExec in the same JVM without forking a new process?

yann_vo · January 9, 2015, 9:51am

Hello,

I am pretty new to Gradle and I am converting the build of our medium sized application from Maven to Gradle.

Although I searched the documentation, this forum and googled around for a while, I could not find a way to run a JavaExec without forking a new process. Both Ant and Maven support this out of the box, so I am pretty surprised… and also surprised not to see any requests for this in the forum or the bug database…

Did I miss something?

My reasons for not wanting to spawn a new Java process are:

Speed (obviously): we generate about 10 WSDL files using org.apache.cxf.tools.java2ws.JavaToWS and every time it takes about 1-2 sec when it was almost instantaneous using Maven exec-maven-plugin
This would solve the long classpath issue on Windows (“java.io.IOException: CreateProcess error=206, The filename or extension is too long”)

The only workaround I found was to use ant.java but it does not really fit into the nice picture and the stdout is not visible on the flow (only at the end).

task justDoIt(dependsOn:[classes]) << {
 def myclasspath = sourceSets.main.output
 myclasspath += configurations.compile
 def concatCp = myclasspath.collect{ it.getAbsolutePath() }.join(';')
 ant.java(classname: 'com.foo.bar.JustDoIt',
       classpath: concatCp, outputproperty: 'op', fork:false)
 println ant.properties.op
}

Many thanks in advance for your inputs. Best regards

Lance · January 9, 2015, 12:13pm

You could do something like:

task justDoIt(dependsOn:[classes]) << {
    def sysOut = System.out
    def classloader = new URLClassloader(gradle.class.classLoader)
    configurations.compile.each { classloader.addURL(it.toURI().toURL()) }
    sourceSets.main.output.each { classloader.addURL(it.toURI().toURL()) }
    def justDoIt = classloader.loadClass('com.foo.bar.JustDoIt').newInstance()
    def stream = new ByteArrayOutputStream()
    try {
       System.out = new PrintStream(stream)
       justDoIt.main([])
       logger.info(new String(stream.toByteArray())
    } finally {
       System.out = sysOut
    }
}

yann_vo · January 12, 2015, 7:31am

HI Lance, thanks for your reply and your proposed implementation!

Just like the workaround I found, I suppose that your implementation would only print the output at the end… (and BTW, since main() is a static method, we should not be calling newInstance on the class but rather make a static method invocation, correct? but that’s another story).

Actually my question, once a workaround has been found, would rather be: is there a reason (other than “we just did not code it”) why the JavaExec task does not allow for non-forked executions? If not, should I raise an enhancement request?

Lance · January 12, 2015, 1:22pm

I suppose that your implementation would only print the output at the end

Correct, but you could provide a “smarter” PrintStream which passes through to the logger instead of building up a ByteArrayOutputStream

since main() is a static method, we should not be calling newInstance on the class but rather make a static method invocation

My bad, you are correct

is there a reason (other than “we just did not code it”) why the JavaExec task does not allow for non-forked executions?

My guess is that the normal way to execute a java method in the current JVM is by a simple groovy call. You have a bit of a chicken or egg problem here in that the buildscript classpath is determined before the class is compiled. You could split the JustDoIt class into a separate project to the task. Then the task could reference the JustDoIt project using

buildscript {
    dependencies {
        classpath project(":justDoItProject")
    }
}
task justDoIt << {
   com.foo.bar.JustDoIt.main([])
}

yann_vo · January 12, 2015, 4:46pm

Hi Lance! Thanks for this renewed answer.

I have been playing with your proposed solution for a while and it is not that simple: for instance I do not want the initial class path of Gradle to be visible… (because of a conflict on org.apache.log4j.ConsoleAppender that JustDoIt is using in a different version than Gradle does).

Therefore, I end up calling something like

gradle.class.classLoader.parent.parent

… but this also needs to become the “current class loader” etc… Actually, looking at ant source code so see how what I called successfully in the workaround above is working, I realized that this is made of quite a few steps:

// from org.apache.tools.ant.taskdefs.ExecuteJava
                    loader = project.createClassLoader(classpath);
                    loader.setParent(project.getCoreLoader());
                    loader.setParentFirst(false);
                    loader.addJavaLibraries();
                    loader.setIsolated(true);
                    loader.setThreadContextLoader();
                    loader.forceLoadClass(classname);
                    target = Class.forName(classname, true, loader);

AntClassLoader has this notion of “isolated” mode… all of which I do not want to code inside my task. I could write a plugin, but remember I am pretty new to Gradle… and this seems to me like a very basic task… I wish it could be part of JavaExec since I believe that most people who need this in the future are going to search for the feature… exactly there.

PS: I did not understand the chicken and egg issue: the class path is fixed once and for all and does not depend on the JustDoIt class… (but unless I missed something critical here, let’s not dig into that in order to let the topic focused on the real question, OK?)

Lance · January 12, 2015, 5:17pm

The chicken or egg problem occurs when you want to execute main() using a simple groovy command. eg:

task justDoIt << {
   com.foo.bar.JustDoIt.main([])
}

These groovy commands use the buildscript classpath (the chicken) 2. The buildscript classpath is determined first 3. I assume JustDoIt.java is in the current project 4. JustDoIt.class does not exist until it is compiled (the egg)

I do not want the initial class path of Gradle to be visible

Ok, so don’t seet the parent

def classloader = new URLClassloader()

yann_vo · January 13, 2015, 7:32am

OK, I understand the chicken and egg issue, thanks. I guess I do not have the issue because my task depends on ‘classes’:

task justDoIt(dependsOn:[classes]) {
...

Playing around with the class loader is definitely the way to go, but it is not as simple as a simple one liner… I ended up having “no class def found” errors for java.io.IOException although rt.jar was in the class path for my class loader… After half a day of attempts and debugging, I just gave up because my ant workaround was equivalent and already functional.

Having something reliable probably needs to be thought about carefully and would deserve being in the standard JavaExec plugin IMHO

Lance · January 13, 2015, 9:08am

The classpath of the buildscript is determined when the following closure is executed

buildscript {
   dependencies { ... }
}

This occurs BEFORE JustDoIt.class exists (assumig JustDoIt.java is in the same project).

This is why I suggested splitting JustDoIt.java into a separate project from task JustDoIt thus removing the chicken or egg problem.

yann_vo · January 13, 2015, 12:42pm

I came up with following custom task based on my original workaround. It shares the same issue (no easy logging of the progress) but at least is solves the two issues I had (speed + Windows classpath):

class JavaExecNoFork extends DefaultTask {
    private String mainClass;
    private FileCollection classpath;
    private final List<Object> applicationArgs = new ArrayList<Object>();
    private final org.gradle.api.internal.file.FileResolver fileResolver;
          JavaExecNoFork() {
        fileResolver = getServices().get(org.gradle.api.internal.file.FileResolver.class);
        classpath = new org.gradle.api.internal.file.collections.DefaultConfigurableFileCollection(fileResolver, null);
    }
    public JavaExecNoFork setMain(String mainClassName) {
        mainClass = mainClassName;
        return this;
    }
          public JavaExecNoFork classpath(Object... paths) {
        classpath = classpath.plus(fileResolver.resolveFiles(paths));
        return this;
    }
    public JavaExecNoFork args(Object... zargs) {
        args(Arrays.asList(zargs));
        return this;
    }
      public JavaExecNoFork args(Iterable<?> zargs) {
        org.gradle.util.GUtil.addToCollection(applicationArgs, true, zargs);
        return this;
    }
      @TaskAction
    void start() {
        logger.info("Running ${this.mainClass}")
        classpath.each {
            logger.debug("Classpath entry ${it.name}")
        }
        applicationArgs.each {
            logger.debug("Args entry ${it}")
        }
        def javaArgs = applicationArgs
        ant.java(classname: this.mainClass,
            classpath: this.classpath.asPath,
            resultproperty: 'exitCode') {
            javaArgs.each {
                arg(value: it)
            }
        }
          if (ant.properties.exitCode != '0') {
            throw new TaskExecutionException(this, new Exception("JavaExecNoFork failure") )
        }
    }
}

(disclaimer: I am very new to Gradle, so this should not be taken as an example of good Gradle programming practices!)

This is then used similarly to the JavaExec:

task justDoIt(type:JavaExecNoFork) {
    main 'com.foo.bar.JustDoIt'
    classpath sourceSets.main.output
    classpath configurations.compile
    args 'foo'
    args 'bar'
}

I just wish this behavior was included in Gradle standard distribution or that someone tells me the reason it isn’t.