Gradle does not report missing/failed artifact downloads


#1

If Gradle is unable to locate a declared artifact for a module, this artifact is simply omitted from the dependencies. There is no way to tell which artifacts are missing from the declared artifacts for a module.

This bad behaviour has been reported with a number of causes: * Artifact not present in repository * HTTP 409 errors: http://forums.gradle.org/gradle/topics/gradle_m5_dependency_downloader_swallowing_http_409s_without_telling_us * Pom Relocation Elements: GRADLE-1961

We’re actively working on a fix for this problem.


(Erick Dovale) #2

I am also seeing gradle failing to resolve a dependency in 1 repo and not trying in any other of the defined repos.


#3

Which version? This is not a known issue, so if you can provide an example build or some more details it would help a lot.

If you think this is a problem, please raise it in a new forum post so we can track it separately.


(Erick Dovale) #4

M6. What I am seeing is a bit of an unusual situation. I had poms but not jars in a local maven repo for httpcore and httpclient (apache httpcomponents version 4.1.2) Gradle would try to resolve it form the local repo and give up on it and not try to download th jars from other repos defined in my build.gradle.

here is my repos definition:

repositories {
 mavenLocal()
 maven{
  credentials {
   username repoUsername
   password repoPassword
  }
  url repoUrl
  name 'central'
 }
 mavenCentral()
}

and here are my dependencies:

dependencies {
   groovy group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.8.4', force: true
    compile group: 'commons-cli', name: 'commons-cli', version: '1.2'
 compile group: 'org.apache.ant', name: 'ant', version: '1.8.2'
 compile group: 'org.apache.ant', name: 'ant-jsch', version: '1.8.2'
 compile (group: 'com.syncapse.tools', name: 'aws-tools', version: '0.5'){
  exclude module: 'javax.mail:mail:1.4.4'
  exclude module: 'stax:stax-api:1.0.1'
 }
    compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.1.2'
 compile group: 'org.apache.httpcomponents', name: 'httpmime', version: '4.1.2'
    runtime group: 'com.h2database', name: 'h2', version: '1.3.158'
 runtime group: 'com.jcraft', name: 'jsch', version: '0.1.44-1'
   testCompile group: 'org.ccil.cowan.tagsoup', name: 'tagsoup', version: '1.2'
 testCompile group: 'org.spockframework', name:'spock-core', version: '0.5-groovy-1.8'
    testRuntime group: 'cglib', name:'cglib-nodep', version: '2.2.2'
 testRuntime group: 'org.objenesis', name:'objenesis', version: '1.2'
   }

I can send you the debug level logs grepped for both dependencies if you tell me how because I can’t find a way to attach files in this system.

Cheers.

Erick.


#5

Erick: Actually what you’re seeing is the expected behaviour. Repositories are considered to be self-contained: if a module is declared in a repository, then the artifacts will be resolved against the same repository.

If you want to add additional artifact locations to a maven repository use the ‘artifactUrls’ property. See http://gradle.org/current/docs/userguide/userguide_single.html#sec:repositories for more details.


(Erick Dovale) #6

Daz,

what do you mean by “a module is defined in a repository”?


#7

For a maven repository, this means that we can find the pom file for the module using the base url for the repository and the standard maven patterns.

For ivy, it means that we can find the ivy.xml using one of the defined ivyPatterns.

(There is a third possibility: ivy dependency resolvers also search for a jar file with the module name. In this case we assume a module with a single jar artifact and no dependencies. This sort of “assumed” module is only used if no real module descriptor (ivy.xml, pom.xml) can be found.)


(Erick Dovale) #8

Ok, but how does that explain that when the jar is not found it just gets ignored and not added to the classpath?


#9

There are 2 things here: 1) Gradle doesn’t complain if a module declares an artifact, and that artifact can’t be found in the same repository the module definition was found in. This will be fixed in Milestone7, so more builds will break rather that silently ignoring missing artifacts.

  1. Gradle doesn’t search for artifacts in repositories other than the one where the module definition was found. This is intended behaviour.

I think that your jars are missing because of 2), but you’re not seeing errors because of 1).

I assume that the problem you’re trying to solve is that the module definition for Apache HTTPComponents in Maven Central is not what you want. Often, a better way to solve this type of issue is to use a Client Module definition in your build. (see http://gradle.org/current/docs/userguide/dependency_management.html#client-modules)


(Erick Dovale) #10

Would it make sense for gradle to try to locate the artifact on the rest of the defined repositories? Why did you guys decided not to it this way?


(npace) #11

Was this ever fixed? 1.0m7 is out now, and we are experiencing this. I found this thread searching to see if there was some kind of flag we could enable to cause warning when artifacts are not found.


#12

Yes, stricter artifact resolution was implemented in Milestone 7.

But if you’re looking to upgrade, I’d recommend switching straight to Milestone 8. The release candidate is currently available at http://gradle.org/release-candidate.


(npace) #13

Hi,

So to get the build to fail we have to do something like ‘echo configurations.compile.files’ in our build file? (which works btw)

It would be nice if there was something available in the DSL like:

dependencies {

compile.resolution.failfast = true }

So that a failure would occur during a Gradle tools api operation. Such as the Eclipse STS Gradle -> Refresh Dependencies action.


(npace) #14

Oh, and I should rephrase, we already moved to m7, and experience this problem as I described. Thanks.


#15

Actually, the only action that won’t trigger a build failure is resolution of dependencies in the IDE. We really want the IDE project tasks to do their best to complete successfully in the face of missing dependencies. So we use the LenientConfiguration to get dependency artifacts for ide tasks. Otherwise, ‘gradle eclipse’ would never build your eclipse project, and you’d need to use vi to make changes until it worked :).

Any other task in gradle that uses dependency artifacts (eg compile) will fail on missing dependency artifacts.

That said, we think logging a warning about missing dependencies in the ide tasks would be useful. This is GRADLE-1945.


(npace) #16

Ah. Thanks for the clarification

I just voted for that issue.

Warnings would be fine for us. It’s the silent failures that are causing issues within our teams who are new to Gradle and/or Maven. I’ve seen that some team members have unreasonably blamed Gradle as “buggy” even though it was their typo in a dependency definition.

Thanks.


(Doug Lethin) #17

Using a nightly build of gradle 1.7 (gradle-1.7-20130613220038+0000), I’ve come across a bug which seems related to GRADLE-1945.

I’ve been working to convert a maven build to gradle and was trying to track down why some of the jars in my war build via maven were not in the war artifact when build by gradle. I whittled down the problem to a reproducible case which involved gradle not handling properly an artifact’s pom file pulled down from a maven repository that has a element in it.

In my case, the artifact in question is “xml-apis:xml-apis:2.0.2”

In the maven central repo that pom has a relocation element that looks like this:

<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>xml-apis</groupId>
  <artifactId>xml-apis</artifactId>
  <version>2.0.2</version>
  <distributionManagement>
    <relocation>
      <groupId>xml-apis</groupId>
      <artifactId>xml-apis</artifactId>
      <version>1.0.b2</version>
    </relocation>
  </distributionManagement>
</project>

My gradle build looks like this:

apply plugin: "war"
  repositories {
  mavenCentral()
}
  dependencies {
  compile "xml-apis:xml-apis:2.0.2" // pom redirects to "xml-apis:xml-apis:1.0.2b"
  compile "jdom:jdom:1.1"
 // pom redirects to new groupId 'org.jdom'
}
  task doIt << {
  for(file in configurations.compile) {
    println file.name
  }
}

When I run the ‘doIt’ task above, I get the following output:

[lighthouse:~/tmp/duh] dlethin% gradle doIt
:doIt
jdom-1.1.jar
  BUILD SUCCESSFUL

Note that the xml-api jar is not present. When I build a sample war using maven with the above dependency, xml-api-1.0.2b.jar does get packed in the war.

So gradle builds the war fine without fail, however the relocated jar is not packed into the war.

It should be noted that artifact relocation seems to partially work. GRADLE-1945 references jdom-1.1 which also has a relocation element (relocates the group ‘jdom’ to ‘org.jdom’. This seems to be processed fine by gradle as seen above, so it seem that particular use cases has been fixed.

However, with regards to xml-api’s relocation, it appears Gradle is not behaving as I would expect.

Perhaps another bug should be opened on this?

In the meantime, I’ll hunt through my transient dependencies and see where this is being picked up and I’ll see if I can get a workaround in place possibly to use the new dependency resolution rules.

Thanks.


(Doug Lethin) #18

Just to add a bit more to this, I added info level logging –

As you can see below, gradle appears to start to process the relocation since it downloads the xml-apis-1.0.b2.pom. But there is no warning about relocation like there is with the jdom jar.

:doIt
Executing task ':doIt' due to:
  Task has not declared any outputs.
Found locally available resource with matching checksum: [http://repo1.maven.org/maven2/xml-apis/xml-apis/2.0.2/xml-apis-2.0.2.pom, /Users/dlethin/.m2/repository/xml-apis/xml-apis/2.0.2/xml-apis-2.0.2.pom]
Found locally available resource with matching checksum: [http://repo1.maven.org/maven2/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.pom, /Users/dlethin/.m2/repository/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.pom]
Found locally available resource with matching checksum: [http://repo1.maven.org/maven2/jdom/jdom/1.1/jdom-1.1.pom, /Users/dlethin/.m2/repository/jdom/jdom/1.1/jdom-1.1.pom]
jdom#jdom;1.1 is relocated to org.jdom#jdom;1.1. Please update your dependencies.
Found locally available resource with matching checksum: [http://repo1.maven.org/maven2/org/jdom/jdom/1.1/jdom-1.1.pom, /Users/dlethin/.m2/repository/org/jdom/jdom/1.1/jdom-1.1.pom]
Found locally available resource with matching checksum: [http://repo1.maven.org/maven2/org/jdom/jdom/1.1/jdom-1.1.jar, /Users/dlethin/.m2/repository/org/jdom/jdom/1.1/jdom-1.1.jar]
jdom-1.1.jar
:doIt (Thread[Daemon,5,main]) - complete
Task worker [Thread[Daemon,5,main]] finished, busy: 3.871 secs, idle: 0.0020 secs, waited for cache: 0.0 secs
  BUILD SUCCESSFUL

#19

This turns out to be a combination of 2 things: 1. The adopted Ivy code for parsing POM files can’t handle POM relocation where only the version is changed: this is now GRADLE-2812. 2. Gradle isn’t correctly reporting Ivy errors and warnings (but info-level messages are logged): this is now

GRADLE-2811.

So it doesn’t work, but the error telling you about this is lost. Thanks for reporting with a nice reproducible sample.


(Mark Petrovic) #20

This relocation of xml-apis v2.0.2 is possibly what I’ve been seeing here:

http://forums.gradle.org/gradle/topics/oddity_in_the_output_of_dependencyinsight_task?utm_content=reply_link&utm_medium=email&utm_source=reply_notification#reply_13104153

wherein xml-apis v2.0.2 is selected in conflict resolution, but is not put onto the classpath of the application calling for the resolution of transitive dependencies in the first place.