Caching filesystem dependencies?

[bjakke (Bjarne Boström)]

I have been testing using gradle as alternative to using ant+ivy. I’m testing the gradle 2.4 version. The only problem so far is that it seems to be impossible to cache dependecies obtained from a filesystem ivy repo.

This has been discussed in:

Filesystem dependencies are not being cached when using the ivy {} repository notation

As a part of upgrading to milestone-5, we changed the way repositories are defined. The problem we are facing with the new ivy {} repository notation is that dependencies are not being cached in the gradle cache. Repositories are currently defined as follows: repositories { ivy { artifactPattern “/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]” ivyPattern “/[organisation]/[module]/ivy-[revision].xml” } } On using FileSystemResolver dependency caching works fine.

There is a bug tracker in:
https://issues.gradle.org/browse/GRADLE-1964

The repo in question is a shared network drive. And since I’m accessing it using the file:// uri, it’s not cache because gradle thinks it is a “local” repo. Is there any way I could get gradle to cache the dependencies? As I commented in the bugtracker, none of the workarounds works because ivy resolvers cannot be used manually (in 2.x if I understood correctly).

Currently I’m using the dependencies directly from the repo. This seems to cause 2 things. Firstly, the application starting time seems to be a bit slower (which is normal, since accessing the network share is slower than local ssd). Secondly, it seem that from time to time I seem to “lose” some dependencies because there is a small connection break (or something, I’m not exacly sure). Note that I use eclipse as main tool development/running and gradle only mainly as a build/dependencymanagement tool (the building mostly in CI, because its faster to use eclipse jdt incremental builder in development). Using the eclipse plugins “refresh dependencies/all” usually solves the problem (but building the workspace back takes some time). I think if the dependecies would be cached, I would not have this problem.

Based on some comments in that earler topic, it seems people think a local repo (a true local disk, not a shared drive) is not worth of caching. In my opinion it would be nice to also have that option, i.e. it would not hurt to have the option (default could be false to retain the current logic). That might be useful in some rare cases and would unify the logic that everything can be cache (e.g if you would have multiple caches for performance reasons, though you probably would need to have 100s or 1000s of concurrent builds for that to matter)

I guess one option would be to downgrade to use 1.x version to get the workarounds in the bug tracker to work… but for me it would seem crazy to not use 2.x since it’s under development.

[Schalk_Cronje (Schalk Cronjé)]

The way I understand it from the source code is that the url starts with ‘file:’, it is hard-cosed in org.gradle.internal.resource.transport.file.FileTransport to always return true for an isLocal() call.

Your best bet would be to create a Sync task to copy across files from the share to a local disk and then use the local disk as the first resolver in your chain of resolvers.

[bjakke (Bjarne Boström)]

Thanks for the idea.

I ended up rsyncing the network share to local disk. I still consider this as a workaroud/hack instead of a real solution, but I guess its good enough for now, at least with current size of the repo.