Latest status resolution not working properly since Gradle 1.1


(detelinyordanov) #1

Hello everyone,

While migrating to Gradle 1.1, we noticed an issue with dynamic version resolution. Our setup is using a local fs and a remote Ivy repository defined (in this order) using RepositoryHandler’s ivy DSL. After we publish a milestone of our plugins (e.g. ‘1.0-m1’), we are normally also publishing an integration version (e.g. ‘1.0-m2-SNAPSHOT’) to the remote Ivy repository (Artifactory).

Now if a project depends on our plugin modules using ‘latest.milestone’, Gradle successfully downloads the ‘1.0-m1’ version. However, if a new ‘1.0-m2-SNAPSHOT’ is uploaded to the local fs repository, this somehow breaks the resolution and Gradle always selects this version, although it’s not having a milestone status. Furthermore, it does not load its Ivy descriptor, but uses a default one (see ‘org.apache.ivy.plugins.resolver.BasicResolver#getDependency’).

The issue is reproducible with Gradle 1.1 and 1.2, could it be due to the fix for GRADLE-2279 :

https://github.com/gradle/gradle/commit/461c509978d5e828da5fd77c94a968ac6cf7297a

I can provide you with a test case if needed, meanwhile here is a debug log done with Gradle 1.1 using a buildscript dependency to e.g. ‘org.foo:my-plugin:latest.milestone’:

Visiting configuration #GradleMilestoneTest;unspecified(classpath).

Visiting dependency #GradleMilestoneTest;unspecified(classpath) - > org.foo#my-plugin;latest.milestone([default])

Attempting to resolve module ‘org.foo#my-plugin;latest.milestone’ using repositories ‘[local-fs, remote-http]’

local-fs: no namespace defined: using system

tried D:\Users\detelin.gradle\local-fs/org.foo/my-plugin/[revision]/ivy-[revision].xml

Listing all in D:\Users\detelin.gradle\local-fs/org.foo/my-plugin/[revision]/ivy-[revision].xml

using local-fs to list all in D:/Users/detelin/.gradle/local-fs/org.foo/my-plugin/

found 1 resources

local-fs: no latest strategy defined: using default

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Closing cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Constructing external resource: D:\Users\detelin.gradle\local-fs/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml

local-fs: found md file for org.foo#my-plugin;latest.milestone

= > org.gradle.api.internal.externalresource.LocallyAvailableExternalResource@f93ee4 (1.0.0-m16-SNAPSHOT)

parser = ivy parser

impossible to get date for org.gradle.api.internal.externalresource.LocallyAvailableExternalResource@f93ee4: using ‘now’

post 1.3 ivy file: using exact as default matcher

local-fs: md rejected by version matcher: 1.0.0-m16-SNAPSHOT [org.gradle.api.internal.externalresource.LocallyAvailableExternalResource@f93ee4]

No resource found for org.foo#my-plugin;latest.milestone: pattern=D:\Users\detelin.gradle\local-fs/[organisation]/[module]/[revision]/ivy-[revision].xml

tried D:\Users\detelin.gradle\local-fs/org.foo/my-plugin/[revision]/my-plugin-[revision].jar

Listing all in D:\Users\detelin.gradle\local-fs/org.foo/my-plugin/[revision]/my-plugin-[revision].jar

using local-fs to list all in D:/Users/detelin/.gradle/local-fs/org.foo/my-plugin/

found 1 resources

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Closing cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Constructing external resource: D:\Users\detelin.gradle\local-fs/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/my-plugin-1.0.0-m16-SNAPSHOT.jar

local-fs: no ivy file found for org.foo#my-plugin;latest.milestone: using default data

[1.0.0-m16-SNAPSHOT] org.foo#my-plugin

checking org.foo#my-plugin;1.0.0-m16-SNAPSHOT[default] from local-fs against [none]

module revision kept as first found: org.foo#my-plugin;1.0.0-m16-SNAPSHOT[default] from local-fs

Performed resolved of module ‘org.foo#my-plugin;latest.milestone’ in repository ‘local-fs’: found

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache dynamic-revisions.bin (D:\Users\detelin.gradle\caches\artifacts-14\dynamic-revisions.bin)

Resolved revision in dynamic revision cache is expired: will perform fresh resolve of ‘{group: org.foo, module: my-plugin, version: latest.milestone}’ in ‘remote-http’

Opening cache module-metadata.bin (D:\Users\detelin.gradle\caches\artifacts-14\module-metadata.bin)

Closing cache module-metadata.bin (D:\Users\detelin.gradle\caches\artifacts-14\module-metadata.bin)

Closing cache dynamic-revisions.bin (D:\Users\detelin.gradle\caches\artifacts-14\dynamic-revisions.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

remote-http: no namespace defined: using system

tried http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/[revision]/ivy-[revision].xml

Listing all in http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/[revision]/ivy-[revision].xml

using remote-http to list all in http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/

Constructing external resource: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/

Performing HTTP GET: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/

Get connection for route {}- >http://my.ivy.repo:8082

Connecting to my.ivy.repo:8082

CookieSpec selected: best-match

Auth cache not set in the context

Target auth state: UNCHALLENGED

Proxy auth state: UNCHALLENGED

Attempt 1 to execute request

Attempting to download resource http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/.

Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@93b59

Connection can be kept alive indefinitely

found 18 resources

remote-http: no latest strategy defined: using default

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Closing cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Constructing external resource: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml

Constructing external resource metadata: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml

Performing HTTP HEAD: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml

Get connection for route {}- >http://my.ivy.repo:8082

Stale connection check

CookieSpec selected: best-match

Cookie [version: 0][name: JSESSIONID][value: dt9tnvma05jj][domain: my.ivy.repo][path: /artifactory][expiry: null] match [my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml]

Auth cache not set in the context

Target auth state: UNCHALLENGED

Proxy auth state: UNCHALLENGED

Attempt 1 to execute request

Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@1f2ae62

Connection can be kept alive indefinitely

Cached resource is up-to-date (lastModified: Mon Sep 17 04:37:22 EEST 2012). [HTTP: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml]

remote-http: found md file for org.foo#my-plugin;latest.milestone

= > CachedResource: D:\Users\detelin.gradle\caches\artifacts-14\filestore\org.foo\my-plugin.0.0-m16-SNAPSHOT\ivy\b74317304baa56f7ef5fb2b663500206ba6a57c7\ivy-1.0.0-m16-SNAPSHOT.xml for http://my.ivy.repo:8082/artifactory/

parser = ivy parser

downloading CachedResource: D:\Users\detelin.gradle\caches\artifacts-14\filestore\org.foo\my-plugin.0.0-m16-SNAPSHOT\ivy\b74317304baa56f7ef5fb2b663500206ba6a57c7\ivy-1.0.0-m16-SNAPSHOT.xml for http://my.ivy.repo:8082/artifactory/

Downloading http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m16-SNAPSHOT/ivy-1.0.0-m16-SNAPSHOT.xml to D:\Users\detelin.gradle\caches\artifacts-14\filestore\temp873332335132341332.part

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Closing cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

post 1.3 ivy file: using exact as default matcher

downloading: parsed downloaded md file for org.foo#my-plugin;1.0.0-m16-SNAPSHOT; parsed=org.foo#my-plugin;1.0.0-m16-SNAPSHOT

remote-http: md rejected by version matcher: 1.0.0-m16-SNAPSHOT [CachedResource: D:\Users\detelin.gradle\caches\artifacts-14\filestore\org.foo\my-plugin.0.0-m16-SNAPSHOT\ivy\b74317304baa56f7ef5fb2b663500206ba6a57c7\ivy-1.0.0-m1

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Closing cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Constructing external resource: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m15/ivy-1.0.0-m15.xml

Constructing external resource metadata: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m15/ivy-1.0.0-m15.xml

Performing HTTP HEAD: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m15/ivy-1.0.0-m15.xml

Get connection for route {}- >http://my.ivy.repo:8082

Stale connection check

CookieSpec selected: best-match

Cookie [version: 0][name: JSESSIONID][value: dt9tnvma05jj][domain: my.ivy.repo][path: /artifactory][expiry: null] match [my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m15/ivy-1.0.0-m15.xml]

Auth cache not set in the context

Target auth state: UNCHALLENGED

Proxy auth state: UNCHALLENGED

Attempt 1 to execute request

Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@10ee5b8

Connection can be kept alive indefinitely

Cached resource is up-to-date (lastModified: Thu Sep 13 00:58:25 EEST 2012). [HTTP: http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m15/ivy-1.0.0-m15.xml]

remote-http: found md file for org.foo#my-plugin;latest.milestone

= > CachedResource: D:\Users\detelin.gradle\caches\artifacts-14\filestore\org.foo\my-plugin.0.0-m15\ivy\a53996ebe38e506bb45edfb990b225d38769669d\ivy-1.0.0-m15.xml for http://my.ivy.repo:8082/artifactory/remote-http/com/softw

parser = ivy parser

downloading CachedResource: D:\Users\detelin.gradle\caches\artifacts-14\filestore\org.foo\my-plugin.0.0-m15\ivy\a53996ebe38e506bb45edfb990b225d38769669d\ivy-1.0.0-m15.xml for http://my.ivy.repo:8082/artifactory/remote-http/com/softwar

Downloading http://my.ivy.repo:8082/artifactory/remote-http/org.foo/my-plugin/1.0.0-m15/ivy-1.0.0-m15.xml to D:\Users\detelin.gradle\caches\artifacts-14\filestore\temp350452628783015526.part

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Closing cache artifact-at-url.bin (D:\Users\detelin.gradle\caches\artifacts-14\artifact-at-url.bin)

Releasing lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

post 1.3 ivy file: using exact as default matcher

downloading: parsed downloaded md file for org.foo#my-plugin;1.0.0-m15; parsed=org.foo#my-plugin;1.0.0-m15

[1.0.0-m15] org.foo#my-plugin

checking org.foo#my-plugin;1.0.0-m15 from remote-http against [none]

module revision kept as first found: org.foo#my-plugin;1.0.0-m15 from remote-http

Performed resolved of module ‘org.foo#my-plugin;latest.milestone’ in repository ‘remote-http’: found

Caching resolved revision in dynamic revision cache: Will use ‘org.foo#my-plugin;1.0.0-m15’ for ‘org.foo#my-plugin;latest.milestone’

Waiting to acquire exclusive lock on artifact cache (D:\Users\detelin.gradle\caches\artifacts-14).

Lock acquired.

Opening cache dynamic-revisions.bin (D:\Users\detelin.gradle\caches\artifacts-14\dynamic-revisions.bin)

Recording module descriptor in cache: org.foo#my-plugin;1.0.0-m15 [changing = false]

Opening cache module-metadata.bin (D:\Users\detelin.gradle\caches\artifacts-14\module-metadata.bin)

Using module ‘org.foo#my-plugin;1.0.0-m16-SNAPSHOT’ from repository ‘local-fs’

Selecting new module version org.foo#my-plugin;1.0.0-m16-SNAPSHOT

Visiting configuration org.foo#my-plugin;1.0.0-m16-SNAPSHOT(default).

Attaching #GradleMilestoneTest;unspecified(classpath) to its parents.

Attaching org.foo#my-plugin;1.0.0-m16-SNAPSHOT(default) to its parents.

In this case the local fs repository contains ‘1.0.0-m16-SNAPSHOT’ version, while the remote http repository contains ‘1.0.0-m15’ and ‘1.0.0-m16-SNAPSHOT’ (and a bunch of older versions). It is expected that the ‘1.0.0-m15’ version is selected, but Gradle chooses ‘1.0.0-m16-SNAPSHOT’ instead.

Regards,

Detelin


(Szczepan Faber) #2

Hello Detelin,

Furthermore, it does not load its Ivy descriptor, but uses a default one (see org.apache.ivy.plugins.resolver.BasicResolver#getDependency).

Hmmm, please check if the problem is not related to incorrect ivy metadata pattern (or incorrect location / name of the ivy.xml in the repository). Otherwise, it does not seem right that the default ivy.xml is generated.

In this case the local fs repository contains 1.0.0-m16-SNAPSHOT version, while the remote http repository contains 1.0.0-m15 and 1.0.0-m16-SNAPSHOT (and a bunch of older versions).

It is expected that the 1.0.0-m15 version is selected, but Gradle chooses 1.0.0-m16-SNAPSHOT instead.

Why do you expect the 1.0.0-m15 version selected? I might not fully understand your scenario. If Gradle finds the module in a first repository (in your case, the local fs repo) it does not search in the next repos in queue. So, provided that ‘1.0.0-m16-SNAPSHOT’ is the newest version in the local repo, this is the version that will be selected.

Hope that helps!


(thomas.glaeser) #3

Why do you expect the 1.0.0-m15 version selected?

Module ‘1.0.0-m15’ has ‘milestone’ status while ‘1.0.0-m16-SNAPSHOT’ has ‘integration’ status. Therefore when depending on ‘latest.milestone’ it is expected that ‘1.0.0-m15’ gets resolved.


(Szczepan Faber) #4

Ok, thanks for info. One more clarification - you mentioned that it worked as expected in Gradle 1.0 but no longer in 1.1+ ?


(thomas.glaeser) #5

Yes, we suspect that this is a regression of GRADLE-2279. In 1.0, when the issue was not yet addressed, only resolving via ‘latest.snapshot’ worked while ‘latest.milestone’ and ‘latest.release’ returned an error. But resolving from a resolver chain that has the local repository listed before the remote repository worked. Now, with 1.1, the first issue got fixed but the second one broken.

As mentioned before, we have a use case which we could share. It would require to change the dependency declarations though as they probably don’t match with your test repository.


(Phil Swenson) #6

When can we expect a fix on this?


(Szczepan Faber) #7

We planned to have it in 1.3 but the problem appeared more complicated and it requires more changes. So at the moment it is not decided it will be in 1.3.


(thomas.glaeser) #8

Not good, basically without the --refresh-dependencies switch Gradle is not working. Not exactly what I would expect from caching. Actually it would be good to have a fix for 1.1 as this is causing a lot of pain.


(Phil Swenson) #9

Agree with Thomas. If it’s too big to fix in 1.3, then I suggest a patch on 1.1. This is a serious problem IMO.