Classpath issue moving to Gradle 2

Old Forum Archive
1

I have a project which works fine in Gradle 1.12, but the test fails with a classpath issue in Gradle 2.0 and 2.1. I’ve replicated the issue in a small example project: GitHub - keeganwitt/gradle2ClasspathIssue: Example project that works in Gradle 1, but not Gradle 2. The classpath exception I get under Gradle 2 is below. I’m totally stumped on this. Any suggestions?

java.lang.NoSuchMethodError: org.apache.hadoop.mapred.LocalJobRunner.(Lorg/apache/hadoop/conf/Configuration;)V

at org.apache.hadoop.mapred.LocalClientProtocolProvider.create(LocalClientProtocolProvider.java:42)

at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)

at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82)

at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:75)

at Tests.theTest(Tests.java:10)

2

I think I’ve figured this out (finally). The HBase dependency appears to bring in a Hadoop transitive dependency. And Gradle apparently changed the classspath ordering in Gradle 2. Adding this exclusion to the HBase dependency does the trick.

exclude(group: “org.apache.hadoop”)

I assumed the order should be alphabetical (in which case you’d think hadoop would win over hbase), but I suppose there’s no specification of the “correct” classpath order. If there is, or at least documentation of what changed in Gradle 2, I’d appreciate a link.

What’s more, I don’t understand why this transitive dependency didn’t show up when I ran gradle dependencies: https://github.com/keeganwitt/gradle2ClasspathIssue/blob/master/dependencies.txt

3

With 2.0, gradle started recognizing default pom profiles - see 2.0 release notes . Hbase-client has a pom whose default profile pulls in org.apache.hadoop:hadoop-core which has the additional implementation of LocalJobRunner.

Point is, this issue has been there all along, but Gradle didn’t honor the default pom profile before 2.0, so it was never an issue.

4

Ah, I see. I appreciate your explanation. What about the dependency tree the dependencies command gives? Shouldn’t it show the hadoop-core dependency it tries to pull in from the default profile?

5

Hadoop-core does show up for me if I run it with 2.1 (I assume it also shows with 2.0, but I didn’t try). It’s only not there if I use a 1.x version of gradle.

|
  |
  \--- org.apache.hadoop:hadoop-core:2.3.0-mr1-cdh5.0.2
|
  |
       +--- commons-cli:commons-cli:1.2
|
  |
       +--- xmlenc:xmlenc:0.52
|
  |
       +--- commons-httpclient:commons-httpclient:3.1 (*)
|
  |
       +--- commons-codec:commons-codec:1.4 -> 1.7
|
  |
       +--- commons-net:commons-net:3.1
|
  |
       +--- org.mortbay.jetty:jetty:6.1.26.cloudera.2 -> 6.1.26 (*)
|
  |
       +--- org.mortbay.jetty:jetty-util:6.1.26.cloudera.2 -> 6.1.26
|
  |
       +--- tomcat:jasper-runtime:5.5.23 (*)
|
  |
       +--- tomcat:jasper-compiler:5.5.23
|
  |
       +--- org.codehaus.jackson:jackson-core-asl:1.5.2 -> 1.9.13
|
  |
       +--- org.codehaus.jackson:jackson-mapper-asl:1.5.2 -> 1.9.13 (*)
|
  |
       +--- javax.servlet.jsp:jsp-api:2.1
|
  |
       +--- commons-el:commons-el:1.0 (*)
|
  |
       +--- net.java.dev.jets3t:jets3t:0.6.1 -> 0.9.0 (*)
|
  |
       +--- javax.servlet:servlet-api:2.5
|
  |
       +--- hsqldb:hsqldb:1.8.0.10
|
  |
       +--- org.eclipse.jdt:core:3.1.1
|
  |
       \--- org.apache.zookeeper:zookeeper:3.4.5-cdh5.0.2 (*)
6

My output doesn’t have hadoop-core in it (I tried with both 2.0 and 2.1). Could there be some global configuration I have that is altering Gradle’s behavior on my machine? I’ve linked my output in my first comment.

You know, I think I misinterpreted the tree representation. I got a tree that looked kinda like this for most scopes

±-- org.apache.hbase:hbase-client:0.96.1.1-cdh5.0.2

|

±-- …

— org.apache.hadoop:hadoop-client:2.3.0-cdh5.0.2

Except testCompile and testRuntime, which looked like this

±-- org.apache.hbase:hbase-client:0.96.1.1-cdh5.0.2

|

±-- …

±-- org.apache.hadoop:hadoop-client:2.3.0-cdh5.0.2

Could you also explain the semantic difference between “” and “+”?

7

Argh…Sorry, I’m not awake yet this morning. It didn’t show up in my tree because I forgot to comment back out my fix. I would still like to hear what the backslashes mean though.

8

Looking at this some more, I think backslashes are just the character used to terminate a (sub)tree. Thanks again for your help.