Gradle v1.12 fails to import ivy source artifacts into IntelliJ 13


(Dieter Stüken) #1

I have a local (file based) ivy repository containing additional source and javadoc artifacts.

Unfortunately the gradle tooling api imports all artifacts as jar file (=classes) regardless of the artifact type (i.e. source).

I traced it down to: DefaultIdeDependencyResolver:getIdeRepoFileDependencies() [121]

Set<ResolvedArtifact> artifacts = getExternalArtifacts(configuration);
  List<IdeExtendedRepoFileDependency> externalDependencies = new ArrayList<IdeExtendedRepoFileDependency>();
  for (ResolvedArtifact artifact : artifacts) {
    if (mappedResolvedDependencies.contains(artifact.getModuleVersion().getId())) {
        IdeExtendedRepoFileDependency ideRepoFileDependency = new IdeExtendedRepoFileDependency(configuration, artifact.getFile());
        ideRepoFileDependency.setId(artifact.getModuleVersion().getId());
        externalDependencies.add(ideRepoFileDependency);
    }
}

While the list of ResolvedArtifact still provides valid getType() infos, those never gets transfered into IdeExtendedRepoFileDependency, even if it has some extra fields for sourceFile and javadocFile.

The problem gets even more complicated, since different entries of externalDependencies have to be mapped into a single IdeExtendedRepoFileDependency …

Dieter.


(Radim Kubacki) #2

Can you make a self-contained example of this problem? It sounds like something what we can fix. Is it just about filtering out source and jar artifacts?


(Dieter Stüken) #3

I took the sample from samples/java/base and replaced mavenCentryl() by:

ivy {
   url "ivyrepo"
   layout 'pattern', {
       artifact "[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"
       ivy "[organisation]/[module]/ivys/ivy-[revision].xml"
   }
}

which refers to a local ivy repository containing commons-collections and junit artifacts taken from: https://code.google.com/p/ivyroundup/

Since I’m not sure how to append an zip file I put it on:

http://euve11644.vserver.de/~stueken/ivy/IvySample.zip

Importing this into Intellij (13.2.1) ends up with strange library content.

Dieter.


(Dieter Stüken) #4

I forked the gradle repository and prepared a solution which works fine for me:

https://github.com/dieterstueken/gradle

How to proceed? commit a pull request? post to gradle-dev?

become a contributor: https://github.com/gradle/gradle/blob/master/CONTRIBUTING.md#gradle-project--developer-guidelines ?


(Radim Kubacki) #5

The page about contributing is valid and describes currnet process.

Looking at your patch my first comment is that we would really like to see an integration test. There are several Idea*IntegrationTest Groovy classes where you can get an idea how to add it.


(Dieter Stüken) #6

OK, I found IdeaSourcesAndJavadocJarsIntegrationTest. It runs fine with the internal ivy test repository. My problem seems to be a wrong or unusual ivy repository structure. I used an other layout pattern, but the main problem seems to be related to the ivy s. The integration tests differ jar/source/javadoc artifacts by separate configurations which are addressed explicitly. My own repository uses ‘default’ for all and thus the javadoc and source jars end up as separate bogus java libs.

I still have some problems to set up a proper debug environment (using IDEA) to get a deeper understanding of my problem. Building gradle-2.0-rc-1 with gradlew works fine, but my IDEA complains about groovy errors (just started with groovy…).

Is it OK to compile gradle using java7 (or even java8?).


(Dieter Stüken) #7

After I was able to build gradle and analyzed the integration tests I realized that my problem was not related to ide integration in general nor Intellij in particular. Instead it simply depends on my ivy repository constitution since it does not discriminate different artifact types by different configurations.

Most of the ‘AbstractSourcesAndJavadocJarsIntegrationTest’ work well, since ‘addCompleteConfigurations()’ applies individual conf attributes. However the ‘“sources and javadoc jars stored with maven scheme in ivy repositories are resolved and attached”’ reproduces my problem nicely: the generated root.iml file contains two additional bogus libraries. Unfortunately this is not recognized.

I simply added some assertion to demonstrate the problem.

https://github.com/dieterstueken/gradle/commit/84225a22cd314fae1634b779c2b38d73803182be

(This should definitely go to a separate function to be called explicitly.

Currently it’s OK until all tests involve a single jar artifact only.)

The solution is also much simpler: ‘DefaultIdeDependencyResolver.getIdeRepoFileDependencies()’ should collect jar type artifacts only:

https://github.com/dieterstueken/gradle/commit/063787df578cf9cebff0d0f78ae00204339a4ec6

source and javadoc artifacts are handled properly later on by ‘IdeDependenciesExtractor.downloadAuxiliaryArtifacts()’.

Dieter.


#8

Thanks for your detailed analysis. I was able to replicate the issue you’re seeing by adding the extra assertion to ‘IdeaSourcesAndJavadocJarsIntegrationTest’. However, on further inspection the failure was caused by a bug in our test code: the line in the test

module.undeclaredArtifact(classifier: "sources", ext: "jar")

was actually adding the artifact to the generated ivy.xml file, unintentionally.

The intended behaviour is to handle sources and javadoc artifacts that are not declared in ivy.xml, but simply stored using the maven scheme of ‘{name}-sources.jar’ and ‘{name}-javadoc.jar’.

So for ivy files you have a few options: - Use whatever classifier scheme you like, and declare the artifacts in ivy.xml using the ‘sources’ and ‘javadoc’ configurations - Use the maven classifier scheme, and don’t declare the artifacts in ivy.xml

Hope that helps.


(Dieter Stüken) #9

Yep, I did not realize that undeclaredArtifact() adds artifacts without any type attribute (which is “jar” by default). It was probably a bad idea anyway to hitchhike some other test. Next try is a separate test I added to AbstractSourcesAndJavadocJarsIntegrationTest:

https://github.com/dieterstueken/gradle/compare/ivy_source_and_javadoc_test

I still think it is an error to get additional libs if any source or javadoc artifacts are explicitly declared (and properly tagged) in the ivy file.

I run into this problem when using lots of ivy.xml files downloaded from http://ivyroundup.googlecode.com/svn/trunk/repo/modules.xml and I can’t believe they are all wrong and have to be adjusted manually to fit.


(Dieter Stüken) #10

After it pointed out that my trouble was caused by an ill organized Ivy repository this thread seems dead now, since it did not show up any relevant bugs of Gadle or the tooling api.

If I still think to see a problem treating Ivy Artifact.getType() attributes right, does it help to open a new topic with a better subject to discuss it? May be it will be relevant for the upcoming Ivy publishing support, too.


(Dieter Stüken) #11

Finally I found this is the source of my problem: [http://forums.gradle.org/gradle/topics/why_are_sources_jars_on_my_compile_classpath#reply_8521485]

and it will not be resolved: [http://forums.gradle.org//gradle/topics/is_there_a_difference_between_ivy_resolving_when_using_gradle_and_when_using_ivy_directly#reply_9959297]