Filesystem dependencies are not being cached when using the ivy {} repository notation


(Kulbhushan Singhal) #1

As a part of upgrading to milestone-5, we changed the way repositories are defined.

The problem we are facing with the new ivy {} repository notation is that dependencies are not being cached in the gradle cache. Repositories are currently defined as follows:

repositories {

ivy {

artifactPattern “/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]”

ivyPattern “/[organisation]/[module]/ivy-[revision].xml”

}

}

On using FileSystemResolver dependency caching works fine.


Caching filesystem dependencies?
#2

When a dependency artifact is available on the local filesystem, it usually doesn’t make sense to copy this file into the cache. The purpose of the cache is to keep a temporary copy of any artifacts that would be expensive to re-fetch: local files usually don’t meet this criteria and keeping an additional copy would be wasteful.

However, there are times when the filesystem is slow to access (eg a network share), and caching would be useful. Currently there is no mechanism for telling Gradle to cache the artifacts in these cases.

A way to instruct Gradle to cache files from a local repository is something we’d like to add. (And perhaps a way to instruct Gradle NOT to cache files from a remote repository)


(Kulbhushan Singhal) #3

It would be great if this support is added soon as the network share is causing our builds to slow down drastically.

In the meantime, what workarounds do we have? Should we switch back to using FileSystemResolver?


#4

Yep, I’d recommend using FileSystemResolver to ensure caching of ‘local’ filesystem artifacts. The only alternative I can think of at the moment is to use an HTTP server instead of a network share to deliver your artifacts, but this may not be practical for other reasons.

Although we plan to provide official support for your scenario, I don’t see it happening in the next Milestone.


(Gradle Jira) #5

This is a major flaw in my opinion. We have a network share as corporate repository. There are multiple reasons why it is a bad idea not to cache those files.

  • The network may be slow sometimes or for some people that are connected via VPN always - The network may be down, e. g. if you travel with your laptop - The server is under more load than necessary - The network is under more load than necessary - …

Please add this possibility. My current workaround is to have a local copy of the whole repository on my disk and have the repository location configurable.


(Björn Kautler) #6

With rc1 this does not work anymore. FileSystemResolver is also not cached anymore. Is any new workaround available?


(Luke Daley) #7

vampire0. At the moment there is no workaround sorry. It will be addressed in 1.1.


(Björn Kautler) #8

Just for others if they have the same issue, I found a new workaround:

repositories {

add(new org.apache.ivy.plugins.resolver.ChainResolver()) {

add new org.apache.ivy.plugins.resolver.FileSystemResolver()

resolvers[0].identity {

name = ‘ivy’

addIvyPattern “$publicRepoDir/ivy/$GRADLE_IVY_PATTERN”

addArtifactPattern “$publicRepoDir/ivy/$GRADLE_ARTIFACT_PATTERN”

checkmodified = true

}

}

}


(Björn Kautler) #9

Spaces got stripped, so again with indentation:

…repositories { …add(new org.apache.ivy.plugins.resolver.ChainResolver()) { …add new org.apache.ivy.plugins.resolver.FileSystemResolver() …resolvers[0].identity { …name = ‘ivy’ …addIvyPattern “$publicRepoDir/ivy/$GRADLE_IVY_PATTERN” …addArtifactPattern “$publicRepoDir/ivy/$GRADLE_ARTIFACT_PATTERN” …checkmodified = true …} …} …}


(Baiyan Huang) #10

Is there any update on this issue? currently we are using a URL resolver, which is going to be deprecated: repositories {

add(new org.apache.ivy.plugins.resolver.URLResolver()) {

name = “customURLResolver”

addIvyPattern(“file:////network/path/[organisation]/[module]/[revision]/common/etc/ivys/ivy.xml”)

addArtifactPattern(“file:////network/path/[organisation]/[module]/[revision]/java/[artifact].[ext]”)

} }


#11

No. There has been no further development on this, and we’re currently busy working on a lot of other features. If you’re interested in helping out to produce a solution, then please contact us on the Gradle developers list (http://www.gradle.org/development).


(Björn Kautler) #12

Nope, not more than the workaround I posted on the linked JIRA issue and http://forums.gradle.org/gradle/topics/asdfasdf-1abqf0


#13

OK thanks for the feedback. This is certainly on our development roadmap: https://github.com/gradle/gradle/blob/master/design-docs/repository-extensibility.md.

Keep an eye on GRADLE-1964 for a fix.