Eclipse plugin not downloading source if JAR found in mavenLocal()

Old Forum Archive
1

My environment consists of a mixture of Maven and Gradle projects which seems to cause issues with the downloading of source by the Gradle eclipse plug-in.

For example, I have 2 projects (A and B) both of which reference the same dependency (dropwizard in this case). Project A is built using Maven and Project B is built using Gradle. In ‘build.gradle’ for Project B I have the following:

repositories {
       mavenLocal()
       mavenCentral()
   }

This is necessary because I need to reference SNAPSHOT resources that I have built and installed locally.

I start by removing both the ‘~/.m2’ directory and the ‘~/.gradle’ directory. I am then comparing the results of running ‘./gradlew cleanEclipse eclipse’ in Project B. If I do that first, before building project A then all of the entries in the resulting ‘.classpath’ file will refer to resources in the Gradle cache and they will all have source attachments if possible. So it works as expected.

If I first build Project A, then the local maven repository will be populated with the shared jar files. Doing the build in maven does not result in the source jars being downloaded (that will only happen on demand as you reference them in Eclipse). Then when I build Project B Gradle finds the dependencies in mavenLocal() (again as expected). However, it does not download the source jars and so the entries in the .classpath have no source attachments.

Once this happens there is no way to fix the problem. Eclipse doesn’t allow the source attachments to be updated, nor does it trigger a download once it notices that there are none. Re-running the ‘cleanEclipse’ and ‘eclipse’ tasks just results in exactly the same issue, unless you happen to have downloaded the source jars in the meantime. Changing the order of the repositories so that mavenLocal() is second will work around the issue, but that’s not ideal since in general you want the local repository to override the remote one.

It seems that the correct behavior here is for the eclipse plug-in to notice that the source jars are missing and attempt a download of them (as it does when using the gradle cache).

2

I’d say that it works correctly. If one of your repositories (maveLocal() in this case) has a binary artifact and no source artifact there is no guarantee that the source artifact from the other repository with the same coordinates will be consistent with its binary counterpart taken from elsewhere.

As for the repository ordering: why do you need mavenLocal() there in the first place?

3

I need mavenLocal in order to reference build artifacts from maven projects that have not yet been deployed to any central repository. This is a fairly standard practice in multi-project maven builds.

The reason I would disagree with you on correctness is that I see no difference between the role of mavenLocal() and Gradle’s own cache. The ‘~/.m2’ directory can be thought of as a local cache by the Maven user. That’s why the Maven tools have no issues whatsoever in updating it to contain source when necessary. AFAIK if you are connected to multiple remote repos it won’t download the binary from one and the source from another, but to be honest I’ve never looked at that in detail (mainly because it always just worked). Gradle performs the same kind of management for its own cache, but it treats mavenLocal() as a full fledged repository. IMO that is a mistake.

Regardless of whether the behavior can be viewed as technically correct (a point on which we may just have to agree to disagree), from a practical standpoint it makes Gradle very hard to use. To illustrate, here is a simple example:

Start with 2 projects – Project A uses gradle for its build and references ‘commons-io’. Project B uses maven for its build and also references ‘commons-io’. Assume that for each of the alternatives below you start in the same state in which ‘commons-io’ is present in neither the Gradle cache or in the local maven repository.

Perform the following steps:

  • Run ‘mvn verify’ on Project B. * Run the ‘cleanEclipse’ and ‘eclipse’ tasks for Project A.

Here is the resulting ‘.classpath’ file:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
    <classpathentry kind="output" path="bin"/>
    <classpathentry kind="src" path="src/main/java"/>
    <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER" exported="true"/>
    <classpathentry kind="lib" path="<my_root>/.m2/repository/commons-io/commons-io/2.4/common
s-io-2.4.jar" exported="true"/>
</classpath>

Now instead do the following:

  • Run ‘mvn verify’ on Project B. * Import Project B into Eclipse and load a class (any class) from ‘commons-io’. Assuming you have M2E installed this will trigger a download of the source jar. * Run the ‘cleanEclipse’ and ‘eclipse’ tasks for Project A.

Here is the resulting ‘.classpath’ file:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
    <classpathentry kind="output" path="bin"/>
    <classpathentry kind="src" path="src/main/java"/>
    <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER" exported="true"/>
    <classpathentry sourcepath="<myroot>/.m2/repository/commons-io/commons-io/2.4/commons-io-
2.4-sources.jar" kind="lib" path="<myroot>/.m2/repository/commons-io/commons-io/2.4/commons-i
o-2.4.jar" exported="true"/>
</classpath>

Lastly do this:

  • Run the ‘cleanEclipse’ and ‘eclipse’ tasks for Project A.

Here is the resulting ‘.classpath’ file:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
    <classpathentry kind="output" path="bin"/>
    <classpathentry kind="src" path="src/main/java"/>
    <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER" exported="true"/>
    <classpathentry sourcepath="<myroot>/.gradle/caches/modules-2/files-2.1/commons-io/common
s-io/2.4/f2d8698c46d1167ff24b06a840a87d91a02db891/commons-io-2.4-sources.jar" kind="lib" path="<myroot>/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/2.4/b1b6ea3b7e4aa4f492509a495202
9cd8e48019ad/commons-io-2.4.jar" exported="true"/>
</classpath>

I’m not sure I understand how this can be viewed as correct. This isn’t an issue with where the source jar comes from, just whether or not we have access to it. The fact that this is influenced by seemingly unrelated activities (such as whether or not you view a project in Eclipse) doesn’t seem good to me.

BTW – if it would be helpful I can post the projects that illustrate this somewhere.

4

I understand that you are not happy with this situation. Yet I think the problem stems from the way how you use your mavenLocal() repository: at the same time you want to use it as a fully functional repository because you install your locally build artifacts there and you want it to be just a cache (transparent component) that caches only subset of artifact data (binary JAR without associated source). I don’t think Gradle will handle this case anytime soon unless someone contributes this as a patch with reasonable test coverage showing what problem it solves and how.

You can avoid the problem if you tell Maven/M2E to automatically download the sources/javadoc. There are enough tips how to do this (e.g. http://stackoverflow.com/questions/5780758/maven-always-download-sources-and-javadocs ).

Let me rephrase the question: why the local repository should override maveCentral and need to precede it?

Re the last question: I’d post a sample code to GitHub/BitBucket/PasteBin/…

5

Actually, I don’t want to use mavenLocal() as a fully functional repository. What I’d like is for Gradle to use it exactly the same way that Maven does. In fact, I’d be perfectly happy if it were completely invisible. The maven tools recognize that if only some artifacts have been cached in the local repo then it should fetch the others when needed. I was surprised when Gradle treated things differently.

I guess that’s the crux of the issue. I was assuming that Gradle’s support for the Maven repository (local and remote) would be in all ways identical to its use by Maven. The fact that it’s not and that I have no way to make it work that way is unfortunate. Could I change the way I (and everyone in my development organization) use Maven? Perhaps. But since Gradle is the new kid on the block (at least for us), I don’t see that being a fruitful direction.

Thanks at least for the honest response (even if I’m disappointed and surprised you don’t see a problem here).

1 Like
6

Gradle does treat Maven Local specially in a few places. For example, it can handle the situation where Maven Local only contains a module’s POM, but not its Jar. I’m sure there are still cases where Gradle treats Maven Local like any other repository although it shouldn’t, and it would be good to get them fixed. This doesn’t have the highest priority for us, but we’d be glad to work with someone interested in contributing improvements in this area.

7

I just encounter the same problem. But it takes 3 hours for me to found that problem were with maven repository with downloaded jars but without downloaded sources.

My main problem were that if I mouseover some classes in eclipse then I see their javadocs but for others don’t! It was very strange.

I type ‘./gradlew cleanEclipse eclipse’ and expect that gradle will download any dependencies AND their sources for me and generate correct ‘.classpath’. But in my ‘.classpath’ there are some entries with sources attached and some without. Now I go to ‘~.m2\repository\org\springframework\spring-context\4.0.7.RELEASE’ and there is ‘spring-context-4.0.7.RELEASE.jar’ but no corresponding ‘-sources.jar’. But in .gradle cache I found both ‘.jar’ and ‘-sources.jar’. But why gradle don’t use them?? It is because in ‘build.gradle’ file ‘mavenLocal()’ repository is upper. So gradle tries to find a dependency in it and found it! But there are no sources in it. So gradle nvm and go to the next dependency.

It is absolutely logical behaviour from gradle perspective but… from eclise user’s perspective (me) it first looks as strange bug (some entries with sources, some not - WHY?).

It will be cool if gradle can resolve this situation by downloading dependency sources to the local maven repo if the dependency already in it (ideal approach) or by using sources from gradle cache even if dependency without sources lives in local maven repo. Of course in theory second approach may lead to some inconsistency between the ‘.jar’ and ‘-sources.jar’ but in practice it is unlikely.

I repeat. The main problem is from the user experience perspective there is strange behaviour that some libs have javadocs and some not.

Of course there are simple solution: just remove ‘mavenLocal()’ or place it down in the list. But it hard to realize that I don’t see some javadocs in eclipse just because of presence of ‘mavenLocal()’. Maybe add it to gradle FAQ at least.

Edit I just realized that placing ‘mavenLocal()’ to bottom of the repositories list makes little sense because in this case gradle will almost never use it. Gradle will use it only for really local dependencies which not exist in any outer repository.