Why are sources jars on my compile classpath?

Why are sources jars being added to my compile classpath?

I’m running an internal Ivy repository. My simplified ivy.xml for org.apache.commons:commons-dbcp:1.2.2 (a random pick - this happens with every dependency) is:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
 <info organisation='org.apache.commons' module='commons-dbcp' revision='1.2.2' status='release' />
 <configurations>
  <conf name='default' />
  <conf name='sources' />
 </configurations>
 <publications>
  <artifact name="commons-dbcp" type="jar" ext="jar"/>
  <artifact name="commons-dbcp" type="jar" ext="jar" m:classifier="sources"/>
 </publications>
</ivy-module>

If I include this as a dependency, I end up with both the binary (commons-dbcp-1.2.2.jar) and source (comomns-dbcp-1.2.2-sources.jar) jars on the classpath.

According to the debug log,

15:37:49.821 [DEBUG] [org.gradle.api.internal.tasks.compile.AntJavaCompiler] Add /tmp/jars/thirdparty/org/apache/commons/commons-dbcp/1.2.2/commons-dbcp-1.2.2.jar to Ant classpath!
15:37:49.822 [DEBUG] [org.gradle.api.internal.tasks.compile.AntJavaCompiler] Add /tmp/jars/thirdparty/org/apache/commons/commons-dbcp/1.2.2/commons-dbcp-1.2.2-sources.jar to Ant classpath!

What’s going on here? How do I get the sources off of my classpath?

It appears as you’re not specifying the conf for each artifact, which I believe mean it defaults to “master”. You might want to change your second artifact to:

<artifact name="commons-dbcp" type="jar" ext="jar" conf="sources" m:classifier="sources"/>

I have the same problem. Is it indeed the intended behavior of gradle to include all artifacts of the matching configuration(s) to the classpath, even though type as well as classifier indicate it is a source artifact?

I’d appreciate if a gradle developer could confirm that source artifacts need to be in a different configuration to avoid having them on the IDE class path.

At the moment, this is the case.

This is compliant with the Ivy model as classifiers are meaningless to Ivy and is only used for Maven interop.

I can see a reasonable case for assuming this though as it’s a well entrenched convention.

In my case, also the Ivy type attribute is set to “source”. Is/should that be considered by gradle?

And what is the situation with maven repository artifacts? Are the classifiers considered there?

I would think so. Can you post an example of your ivy descriptor please.

Sure, thanks for looking into that:

ivy.xml:

<ivy-module xmlns:m="http://ant.apache.org/ivy/maven" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0" xsi:noNamespaceSchemaLocation="../../../../xsd/ivy.xsd">
    <info organisation="javax.servlet" module="servletapi" revision="2.5" status="release" publication="20060502130300">
        <license name="Sun Microsystems, Inc. Binary Code License Agreement" url="http://www.sun.com/"/>
    </info>
    <publications>
        <artifact/>
        <artifact name="servletapi-source" type="source" ext="jar" m:classifier="sources"/>
    </publications>
</ivy-module>

.classpath

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
 <classpathentry kind="output" path="bin"/>
 <classpathentry kind="src" path="src/main/java"/>
 <classpathentry kind="src" path="src/main/resources"/>
 <classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER" exported="true"/>
 <classpathentry kind="src" path="/share" exported="true"/>
 <classpathentry kind="lib" path="/Users/rfy/Developer/ivyrepo/artifact-repo/modules/javax.servlet/servletapi/2.5/jars/servletapi-2.5.jar" exported="true"/>
 <classpathentry kind="lib" path="/Users/rfy/Developer/ivyrepo/artifact-repo/modules/javax.servlet/servletapi/2.5/sources/servletapi-source-2.5.jar" exported="true"/>
</classpath>

build.gradle (excerpt)

repositories {
 ivy {
  url repoUrl
  layout 'pattern', {
   artifact "[organisation]/[module]/[revision]/[type]s/[artifact]-[revision].[ext]"
   ivy "[organisation]/[module]/[revision]/ivy.xml"
  }
 }
}
dependencies {
 compile "javax.servlet:servletapi:2.5"
 compile project(":share")
}

Any more input on this? Should ivy artifact type “source” prevent the artifact to be on the class path or not?

As Luke said, this works as designed. Gradle doesn’t currently try to interpret source or classifier values. For Maven in particular, this is in compliance with how Maven works. If you add a source Jar as a dependency in a POM (which is not normally done), it will end up on the corresponding class path.

Rainer: just to clarify, we believe that Gradle is doing the right thing here. Just declaring the “type” as source is not enough, you need to move it to a different configuration.

Thanks for clarifying, good to know that this behavior is intended, and I’ll try to design my repository according to it for the time being.

As for that being “the right thing”, I’m a bit confused why this is desirable. Maybe I’m missing something, as I never used maven, and don’t have any prior practical experience with ivy as well.

Peter said: > If you add a source Jar as a dependency in a POM (which is not normally done), it will end up on the corresponding class path.

I wonder when I would ever want to have a source jar in the compile classpath, or any other dependency artifact that is not a binary jar.

In ivy I don’t declare dependencies on an artifact, but on a module, in a certain configuration (usually).

So I would expect “the right thing” to be: add all artifacts of the matching module and configuration to the compile classpath, that are meant to be used as compile classpath.

And use other types of artifacts, also limited to the declared configuration(s), for other, suitable, purposes within the build, e.g. as IDE source references.

I’m using a private ivy repository only, no maven repository, and I’m still not sure what I should do regarding source jars. If I put the source jar artifacts into the same configuration as the actual binary jars, the sources are put on the compile classpath. If I put them in a separate configuration on which I must not depend in my gradle dependency declaration, they are never even fetched at all.

The IDE plugins’ dependency extractor uses a mechanism to try and fetch a single source artifact with the name of the complete ivy module and a classifier of “sources” (which, as Peter or Luke mentioned has usually no meaning in Ivy), and does not check at all what artifacts are declared for the module. This does not work at all for Ivy modules with multiple artifacts in different configurations (as I have several in my repository).

To work around this, I was hoping to be able to look into the full list of the artifacts of my gradle dependency configurations, and map (e.g. by repository-specific convention), source type artifacts to binary artifacts myself. Obviously this doesn’t work, because either the sources are in a module configuration that I depend on, and thus on the classpath, or they are not, and thus not referenced by the gradle dependency configurations at all.

I’d appreciate two things: 1. any advise how I could, with current gradle, achieve a clean classpath without sources, and still somehow get a hold on source artifacts for setting up IDEs with Ivy 2. maybe it could be considered for future versions of gradle to distinguish between artifact types or kinds, and only use a suitable artifact subset of the dependencies for the Java classpath, but still provide the other artifacts for other build purposes.

In Maven, you typically don’t add a source Jar as an explicit dependency. Instead, it is found automatically (given the “main” Jar) and treated correctly by Gradle’s IDE plugins. If you add a source Jar explicitly, it will end up on the class path you specified (compile, runtime, etc.).

Ok, but how is the source jar treated in the repository? If I understand maven correctly, there is only one artifact per POM. Does a source jar have it’s own POM then, and in addition to having a sources classifier, does it have a different artifactID then?

Please forgive these beginner’s questions, but I really have no experience using maven at all.

Still, I’m not interested in using maven repositories (at least regarding this question), I use Ivy, and I’m in the process of building the ivy repository along with the gradle build that uses it, so I try to learn best practices for both sides.

In Maven, a main artifact can have secondary artifacts, in particular a sources Jar and Javadoc Jar. These Jars don’t have their own POM; they are looked up relative to the main Jar according to naming conventions.

OK, I understand … so this is where the lookup mechanism in the IDE dependency mechanism comes from. Seems that ivy and maven are conceptually quite different in that regard - not easy to handle transparently in s tool like gradle.

For a few projects, we have got an Ivy descriptor as follows:

<?xml version="1.0" encoding="UTF-8"?>
  <ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
    <info organisation="my.foobar.qux" module="trox" branch="5.0.0">
       <extends organisation="my.foobar.qux" module="robotx" revision="latest.integration"
         location="../foobar-qux-robotx-jar/ivy.xml"/>
    </info>
          <publications defaultconf="default">
        <artifact name="${foobar.qux.trox.artifact}" type="bin" ext="jar"/>
        <artifact name="${foobar.qux.trox.artifact}" type="test-bin" ext="jar" conf="testing" e:classifier="tests"/>
        <artifact name="${foobar.qux.trox.artifact}" type="pom" ext="pom"/>
        <artifact name="${foobar.qux.trox.artifact}" type="doc" e:classifier="javadoc" ext="zip"/>
        <artifact name="${foobar.qux.trox.artifact}" type="src" e:classifier="sources" ext="zip"/>
    </publications>
      <dependencies defaultconf="default" defaultconfmapping="*->default">
        <dependency ....
    </dependencies>
  </ivy-module>

and out Ivy repo is layed out as follows:

ivy {
                name = repo
                url = "$repoRoot/$repo"
                layout ('pattern') {
                    ivy "[organisation]/[module]/[revision]/ivy.xml"
                    artifact "[organisation]/[module]/[revision]/[module]-[revision](-[classifier]).[ext]"
                    artifact "[organisation]/[module]/[revision]/[artifact]-[revision]-[type].[ext]"
                    m2compatible = true
                }
            }

Is there any way to specify which type/classifier gets to which configuration?

This post is quiet old but I am currently facing the same problem:

  • I have ivy as repository type - I want source jars to be resolved

  • source jars have type and m:classifier set to ‘sources’ - Gradle puts the source jars into the classpath

  • If I move source jars to another configuration the ide tasks will not be able to resolve the source

Did you find a solution to the problem?