--refresh-dependencies should use cacheChangingModulesFor=0s


(Florian Schmaus) #1

Could we please have a simple switch that will invalidate the changing modules cache? Either make --refresh-dependencies use cacheChangingModulesFor=0s. Or some other switch. I find it really sad that i’ve to add

configurations.all {
    // check for updates every build
    resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
}

in every gradle project that uses changing modules to get the latest snapshot if I want it.

Related: http://forums.gradle.org/gradle/topics/how_to_get_gradle_to_download_newer_snapshots_to_gradle_cache_when_using_an_ivy_repository#reply_8011094


(Peter Niederwieser) #2

Can you double-check that this isn’t already the case? As far as I know, ‘–refresh-dependencies’ is semantically equivalent to deleting the cache.


(Luke Daley) #3

That’s definitely the case.

If this isn’t working then something else is going on.


(Florian Schmaus) #4

Hmm, sometimes I’ve even to “rm -rf ~/.gradle/caches/modules-2/files-2.1/group/artifact/4.1.0-alpha1-SNAPSHOT” in order to get the most recent deployed snapshot from sonatype… :frowning:


(Jörn Huxhorn) #5

I have a similar issue were ‘–refresh-dependencies’ is definitely not redownloading all SNAPSHOT dependencies from our nexus. We are using the daemon with Gradle 2.0 at the moment. Manually deleting the cache (obviously) causes the expected download.


(Jörn Huxhorn) #6

I can confirm that adding

configurations.all {
 // check for updates every build
 resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
}

as suggested by the OP is indeed fixing the issue. This is very strange…


(Luke Daley) #7

We’ve got pretty extensive coverage for this feature, so we are unlikely to find the cause here without being able to reproduce.


(Jörn Huxhorn) #8

The question is: what is the expected behavior?

I’ve done some tests and created two Maven projects: ‘foo’ ‘bar’, depends on ‘foo’

And I created a Gradle project ‘foobar’ that depends on ‘bar’.

If I just ‘mvn deploy’ the Maven projects then ‘gradle --refresh-dependencies --info build’ reports ‘Cached resource is up-to-date’ and ‘Found locally available resource with matching checksum:’ for those dependencies.

If I perform a ‘mvn clean install deploy’ then ‘gradle --refresh-dependencies build’ is downloading the new artifacts.

I would have expected that ‘gradle --refresh-dependencies build’ would always refresh/redownload any SNAPSHOT dependencies, regardless of anything that was already cached. (Yes, I’ve seen 50.9.2.2. Refresh in the user guide. It’s just unexpected.)

But even with

configurations.all {
    resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
    resolutionStrategy.cacheDynamicVersionsFor 0, 'seconds'
}

my current test (which is still part of a complex build) keeps reporting ‘Cached resource is up-to-date’ and ‘Found locally available resource with matching checksum:’.

Am I missing something?


#9

The expected behaviour is as described in the user guide: with ‘–refresh-dependencies’, Gradle will always hit the remote server to check for updated artifacts: however, Gradle will avoid downloading a file where the same file already exists in the cache.

First Gradle will make a HEAD request and check if the server reports the file as unchanged since last time (if the ‘Etag’ value in the header is the same or if the ‘content-length’ and ‘last-modified’ are unchanged). In this case you’ll get the message: "Cached resource is up-to-date (lastModified: {})."

Next Gradle will determine the remote checksum if possible (either from the HEAD request or by downloading a ‘.sha1’ file). If this checksum matches another file already downloaded (from any repository), then Gradle will simply copy the file in the cache, rather than re-downloading. In this case you’ll get the message: "“Found locally available resource with matching checksum: [{}, {}]”.

Hope that helps to explain what’s going on.


(Jörn Huxhorn) #10

But shouldn’t the above configuration override all this by setting the caching duration to zero?


#11

No. The ‘cacheChangingModulesFor’ and ‘cacheDynamicVersionsFor’ settings configure how often we check for updates.  Setting them to zero means “check for updates on every build”.

When using ‘–refresh-dependencies’ these settings have no impact, since we are forcing gradle to check for updates on every module (not just changing modules or dynamic versions).


(Luke Daley) #12

But it still appears there’s a bug here somewhere in that --refresh-depenencies is not causing some SNAPSHOT dependencies to be updated right?


(Jörn Huxhorn) #13

Yes, we definitely had issues with dependencies from a Maven build that had just been deployed to our Nexus which our Gradle build didn’t pick up with ‘–refresh-dependencies’, making it necessary to manually delete the cache. I know for sure since this caused compile errors because of an API change in our code. We had similar problems multiple times. The only difference is that I finally had the time to take a closer look instead of just wiping the cache and going on with the job at hand…

I thought that the above configuration fixed it - as I wrote 2 days ago - since after adding it, the dependencies were updated as expected. But I’m pretty sure this was only a temporary fix since setting and removing them didn’t change the behavior at all while I was trying to reproduce the issue.

This could also be a Maven problem since I’m not sure if we just did a ‘mvn deploy’ or a ‘mvn clean deploy’. It could also be a Nexus-related problem since fschmaus (the OP) is also using Nexus. Might be a coincidence…


(Chris Beams) #14

–refresh-dependencies works fine for me, but it’s too broad in many cases. What I really want is a --refresh-changing-dependencies switch, so that I can instruct Travis CI to always check for new snapshots, but not to bother checking other unchanging dependencies.