Why are gradle jars on compileJava Compiler arguments -classpath?

Old Forum Archive
1

I’m doing something that’s causing gradle to prepend my javaCompile classpath with the gradle dependencies (reformatted for readability): 08:54:53.569 [DEBUG] [org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler] Compiler arguments:

-d /Users/bedge/svn/foo/buildSrc/build/classes/main

-g -Xlint:deprecation

-classpath /usr/local/Cellar/gradle/1.8/libexec/lib/gradle-core-1.8.jar :/usr/local/Cellar/gradle/1.8/libexec/lib/groovy-all-1.8.6.jar :/usr/local/Cellar/gradle/1.8/libexec/lib/asm-all-4.0.jar :/usr/local/Cellar/gradle/1.8/libexec/lib/ant-1.9.2.jar :/usr/local/Cellar/gradle/1.8/libexec/lib/commons-collections-3.2.1.jar :/usr/local/Cellar/gradle/1.8/libexec/lib/commons-io-1.4.jar /usr/local/Cellar/gradle/1.8/libexec/lib/commons-lang-2.6.jar …my dependencies

This is putting some older jar versions ahead of the ones I need.

I can’t reproduce this with a small test case, but I also can’t get it to stop happening in my real project.

What would one to do to cause this to happen? IOW, how would I add all the gradle dependencies to the compileJava classpath, or, conversely stop it from happening?

2

After some divide & conquer debugging I found one thing that sounds like a bug.

If I cd into my buildSrc folder, and build, I get a different result based on whether an empty settings.gradle exists in the parent folder. I confirmed this with a minimal test case.

If …/settings.gradle exists, then the

Compiler arguments: … -classpath

is prepended by the gradle dependencies as above.

If there is no …/settings.gradle, then the same “gradle build” command does not add any gradle dependencies to the compiler classpath.

Here’s a shell session that illustrates the problem. “gradle clean” fails the first run, but after deleting the empty settings.gradle it succeeds:

~/src/test/gradle/commons-codec/buildSrc %> gradle clean :buildSrc:compileJava /Users/bedge/src/test/gradle/commons-codec/buildSrc/src/main/java/com/nim/Sha1.java:24: error: cannot find symbol

sha1 = DigestUtils.sha1Hex(new FileInputStream(sourceFile));

^

symbol:

method sha1Hex(FileInputStream)

location: class DigestUtils 1 error :buildSrc:compileJava FAILED

FAILURE: Build failed with an exception.

  • What went wrong: Execution failed for task ‘:compileJava’. > Compilation failed; see the compiler error output for details.

  • Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 1.134 secs ~/src/test/gradle/commons-codec/buildSrc %> rm …/settings.gradle ~/src/test/gradle/commons-codec/buildSrc %> gradle clean :clean

BUILD SUCCESSFUL

Total time: 0.779 secs

This specific failure is because gradle’s prepended dependencies put commons-codec 1.6 on the classpath and the sha1Hex in the error above is only in 1.8.

Here is the buildSrc/build.gradle:

apply plugin: 'groovy'
apply plugin: 'eclipse'
  tasks.withType(Compile) {
  options.compilerArgs << "-Xlint:deprecation"
}
  dependencies {
       compile(group: 'com.google.code.gson', name: 'gson', version: '2.2.4')
    compile(group: 'org.slf4j', name: 'slf4j-api', version: '1.7.5')
    compile(group: 'commons-lang', name: 'commons-lang', version: '2.6')
    compile(group: 'commons-io', name: 'commons-io', version: '2.4')
    compile(group: 'commons-collections', name: 'commons-collections', version: '3.2.1')
    compile(group: 'commons-codec', name: 'commons-codec', version: '1.8')
    compile(group: 'org.apache.pdfbox', name: 'pdfbox', version: '1.8.2')
    compile(group: 'com.nativelibs4java', name: 'bridj', version: '0.6.2')
    compile(group: 'com.typesafe', name: 'config', version: '1.0.2')
      compile group: 'org.slf4j', name:'slf4j-api', version: '1.6.4'
    def logbackVersion = '1.0.13'
    testRuntime group: 'ch.qos.logback', name:'logback-classic', version: logbackVersion
    testRuntime group: 'ch.qos.logback', name:'logback-core', version: logbackVersion
        testCompile 'junit:junit:4.8.1'
     compile("org.codehaus.groovy:groovy-all:2.1.7")
    testCompile "org.spockframework:spock-core:1.0-groovy-2.0-SNAPSHOT"
    testCompile "org.hamcrest:hamcrest-core:1.3"
    testRuntime "cglib:cglib-nodep:2.2.2"
    testRuntime "org.objenesis:objenesis:1.2"
  }
  repositories {
    mavenCentral()
    /* No local maven repo, use content-parser-v4/lib instead, this works for both
     installed /opt/nim/parserV4 as well as source tree invokation
    mavenLocal() */
    maven { url "http://oss.sonatype.org/content/repositories/snapshots/" }
}

This is only part of the problem as it still happens when I build from the parent folder. That is, I still get the prepended gradle deps when building in the parent folder.

3

‘buildSrc’ is treated specially in that Gradle dependencies are automatically added to the ‘compile’ configuration. If you don’t have a parent ‘settings.gradle’ and build ‘buildSrc’ directly, then it won’t be recognized as such and will be treated like any other build. To be on the safe side, you can add an explicit ‘compile gradleApi()’ dependency.

4

Gradle searches up the directory hierarchy looking for a settings.gradle file. So adding an empty settings.gradle will effectively hide one in a parent directory.

5

Is there anything I can do to prevent the gradle dependencies from being added to the buildSrc compile classpath? In this case it’s causing versioning issues as it puts an older version of commons-codec ahead of the one in my dependencies.

6

Apologies for the long-winded explanation but I’m quite stuck here.

I need the opposite of a

compile gradleApi()

dependency. I need to not have gradle deps on the classpath as they prevent using any version of commons-codec other than 1.6, which gradle needs. I need 1.8.

If it is not possible to remove the gradle deps from the buildSrc classpath, I suppose an option would be to rename buildSrc to a subproject and add a settings.gradle to include that project:

include "parser"

such that it’s built as a subproject rather than as buildSrc so that the gradle deps are not prepended to the compile classpath when building as a subproject. The questions then are: 1) How to force the subproject compileJava build before evaluating the parent build.gradle? 2) How to add the subproject’s built jars to the classpath before building the parent project?

I did a quick test and am missing both items above, as I cannot use a java method defined in the subproject in the top level build.gradle and nor is the subproject built before evaluating the top script.

├── build.gradle ├── settings.gradle ├── parser │ ├── build.gradle │ └── src │

└── main │

└── java │

└── com │

└── nim │

└── Sha1.java

Top build.gradle:

apply plugin: 'java'
import com.nim.Sha1
  // unable to resolve class
   evaluationDependsOnChildren()
  dependencies {
 compile project(':parser')
}
  task foo() {
 doLast {
  println getSha1() // defined in Sha1.java
 }
}

settings.gradle:

include "parser"

parser project build.gradle:

apply plugin: 'java'
  dependencies {
    compile(group: 'commons-codec', name: 'commons-codec', version: '1.8')
}
  repositories {
    mavenCentral()
}

and a Sha1.java that defines the Sha1 class needed in the top build.gradle.

Gradle evaluates the parser subproject, but does not compile it:

%> gradle -i foo Starting Build Starting file lock listener thread. Settings evaluated using settings file ‘/Users/bedge/src/test/gradle/commons-codec/settings.gradle’. Projects loaded. Root project using build file ‘/Users/bedge/src/test/gradle/commons-codec/build.gradle’. Included projects: [root project ‘commons-codec’, project ‘:parser’] Evaluating root project ‘commons-codec’ using build file ‘/Users/bedge/src/test/gradle/commons-codec/build.gradle’. Compiling build file ‘/Users/bedge/src/test/gradle/commons-codec/build.gradle’ using BuildScriptTransformer.

FAILURE: Build failed with an exception.

  • Where: Build file ‘/Users/bedge/src/test/gradle/commons-codec/build.gradle’ line: 2

  • What went wrong: Could not compile build file ‘/Users/bedge/src/test/gradle/commons-codec/build.gradle’. > startup failed:

build file ‘/Users/bedge/src/test/gradle/commons-codec/build.gradle’: 2: unable to resolve class com.nim.Sha1

@ line 2, column 1.

import com.nim.Sha1

Nor does it add it’s built jar to the classpath as I can build the parser project manually and it is not added to the classpath despite the evaluationDependsOnChildren.

7

This is turning out to be a huge problem to work around. We have a large body of code in buildSrc that’s needed to build the output and relocating it into another project and refactoring the dependencies is a major effort. I have to ask again as this just seems like too much effort for a relatively simple issue. How do I stop prepending gradle’s dependencies to the buildSrc compileJava classpath?

Is there no way to do this?

If so, is all buildSrc code locked into using only the specific jar versions that gradle uses for it’s dependencies?

8

One more dead end. Added resolutionStrategy to both top and buildSrc build.gradle

configurations.all {
 resolutionStrategy {
  failOnVersionConflict()
  force 'commons-codec:commons-codec:1.8'
 }
}

Still pulls in commons-codec 1.6

Note also that it doesn’t even mention the 1.6 vs 1.8 issue even with failOnVersionConflict.

Another misguided attempt. Tried to read compile configuration and filter classpath, but it’s read-only so it can’t be written back to the config:

project.properties.configurations['compile']['files'] = new LinkedHashSet( project.properties.configurations['compile']['files'].findAll { ! it.getPath().contains("/gradle/") }.collect{it} )
9

Apparently, ‘gradleApi’ is always added when ‘buildSrc’ is run implicitly as part of the main build, but not when ‘buildSrc’ is built directly. I don’t think you can use ‘force’ etc. for ‘gradleApi’ dependencies. At least for some of them (e.g. Groovy) it doesn’t make sense to change the version, because at runtime you’ll always get Gradle’s version anyway. That said, you can try something like:

configurations {
    pluginCompile
}
dependencies {
    pluginCompile ... // your dependencies first
    pluginCompile gradleApi()
}
sourceSets.main.compileClasspath = configurations.pluginCompile
10

Thanks for the tip Peter, I’m ready to try anything at this point.

I tried adding this alternately to my top level build.gradle and then to my buildSrc/build.gradle:

configurations {
    pluginCompile
}
dependencies {
    pluginCompile group: 'commons-codec', name: 'commons-codec', version: '1.8'
    pluginCompile gradleApi()
}
sourceSets.main.compileClasspath = configurations.pluginCompile

with no change in the compiler arg ordering or content:

[DEBUG] [org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler] Compiler arguments:

-d /Users/bedge/svn/nim/trunk/source/content/content-parser-v4/buildSrc/build/classes/main

-Xlint:deprecation

-g

-classpath /usr/local/Cellar/gradle/1.8/libexec/lib/gradle-core-1.8.jar :/usr/local/Cellar/gradle/1.8/libexec/lib/groovy-all-1.8.6.jar … /usr/local/Cellar/gradle/1.8/libexec/lib/plugins/commons-codec-1.6.jar: …my deps didn’t show up util later in the list: /Users/bedge/.gradle/caches/artifacts-26/filestore/commons-codec/commons-codec/1.8/jar/af3be3f74d25fc5163b54f56a0d394b462dafafd/commons-codec-1.8.jar

One thing I did notice, is that I have this to enable deprecation warnings:

tasks.withType(Compile) {
  options.compilerArgs << "-Xlint:deprecation"
}

and the “-Xlint:deprecation” shows up before -classpath. Is there something I could use to append my dependencies to the compilerArgs?

11

Argh, there’s a validation on the compilerArgs that breaks it.

As a test I tried:

tasks.withType(Compile) {
  options.compilerArgs << "-Xlint:deprecation"
 options.compilerArgs << "-classpath /Users/bedge/.gradle/caches/artifacts-26/filestore/commons-codec/commons-codec/1.8/jar/af3be3f74d25fc5163b54f56a0d394b462dafafd/commons-codec-1.8.jar"
  }
  • What went wrong: Execution failed for task ‘:compileJava’. > invalid flag: -classpath /Users/bedge/.gradle/caches/artifacts-26/filestore/commons-codec/commons-codec/1.8/jar/af3be3f74d25fc5163b54f56a0d394b462dafafd/commons-codec-1.8.jar
12

I cant reproduce this. The solution I proposed works fine for me.

13

OK, figured out the right syntax, but alas it appears that the second -classpath on the command line completely overrides the first:

tasks.withType(Compile) {
  options.compilerArgs << '-classpath'
 options.compilerArgs << '/Users/bedge/.gradle/caches/artifacts-26/filestore/commons-codec/commons-codec/1.8/jar/af3be3f74d25fc5163b54f56a0d394b462dafafd/commons-codec-1.8.jar'
}

which yields:

javac -d /Users/bedge/svn/nim/trunk/source/content/content-parser-v4/buildSrc/build/classes/main -g -Xlint:deprecation

-classpath

/Users/bedge/.gradle/caches/artifacts-26/filestore/commons-codec/commons-codec/1.8/jar/af3be3f74d25fc5163b54f56a0d394b462dafafd/commons-codec-1.8.jar

-classpath

/usr/local/Cellar/gradle/1.8/libexec/lib/gradle-core-3.4.jar … :/usr/local/Cellar/gradle/1.8/libexec/lib/plugins/commons-codec-1.6.jar:

and so the codec 1.8 methods are still not found

/Users/bedge/svn/nim/trunk/source/content/content-parser-v4/buildSrc/src/main/java/com/nim/tools/dataset/DefaultBuildDataset.java:122: error: cannot find symbol

sha1 = DigestUtils.sha1Hex(new FileInputStream(sourceFile));

14

I posted a minimal test case here: http://pastebin.com/h5g3JH54

Could you post an example where this is working?

15

My co-worker found a fix for this. It’s a variation of Peter’s suggestion.

in buildSrc/build.gradle:

configurations {
 nim
}
dependencies {
 nim(group: 'commons-codec', name: 'commons-codec', version: '1.8')
 compile configurations.nim
 runtime configurations.nim
}
tasks.withType(Compile) {
 classpath = configurations.nim
}
16

Can you (or your co-worker) explain what you think that’s doing and why it works?