Is it possible to understand how gradle stores artifacts on disk?

Old Forum Archive
1

I need to understand how to programmatically crawl around the gradle artifact cache. It is a requirement of my projects to be able to: - download from artifactory using gradle - then crawl around the download cache so that I can find certain artifacts I am looking for.

How can I control where these are put? Or rather, how can I understand how to crawl through the cache area? For example, I can invoke gradle with the -g option and specify the top level gradle home.

The only thing that seems to control is the root of the downlaod cache. From there though it becomes interesting. I have no idea how I could programmatically crawl around the download cache to find the artifacts I need.

For example, I created a local folder called ‘patches4fa’ and specified the -g switch to point to that folder. Here is where it ended up putting the opatch I downloaded:

[p4fa.git]$ ll patches4fa/caches/artifacts-13/filestore/orc.patches4fa.wt/plugin_ps5rc4_13643211_linux64/ps5rup3/zip/b8737310e35b42da5149460a0d7edb9e8a3e14d7/
total 144
-rw-r--r-- 1 sraffane g900 143155 Jun 18 23:23 plugin_ps5rc4_13643211_linux64-ps5rup3-linux64.zip
[p4fa.git]$

As you can see the location is pretty wild.

And just in case this helps, here is how I declared the dependency on the above zip:

buildscript {
   repositories {
      maven {
         url 'http://artifactory-poc.mytest.com/artifactory/p4fa-virtual'
      }
    }
   dependencies {
      classpath (group: 'org.jfrog.buildinfo', name: 'build-info-extractor-gradle', version: '2.0.9')
      classpath (group: 'org.codehaus.groovy.modules.http-builder', name: 'http-builder', version: '0.5.1')
      classpath (group: 'orc.patches4fa.wt', name: 'plugin_ps5rc4_13643211_linux64', version: 'ps5rup3', classifier: 'linux64', ext: 'zip')
   }
}
2

The Gradle cache is not meant to be used directly. It’s a private file structure and the layout changes every once in a while (e.g. filestore-13, filestore-14, etc.). If you do decide to access it directly, you are on your own. I’m also worried that you might run into issues like you reading an artifact while Gradle writes to it.

3

Perhaps there is some other way to control where Gradle puts downloaded files or would this require a special plugin?

A simple use case would be:

  • I have a Java web project, that produces a WAR - Much of what ends up in the WAR is static (jpg, css, html etc), and is to be shared (via the repo) with other projects - The static content is packaged in a zip and put in the Maven repo - Now I want my WAR project to download this content, unzip it somewhere, perform some transformations, then add it to my WAR.

Now we may question the design of the web project above (use a content management system for example), but it illustrates the point I think.

Any ideas would be much appreciated.

4

This can be easily achieved without crawling the cache. Something like:

configurations {
  warResources
}
  dependencies {
  warResources "foo:bar:1.0"
}
  war {
  from(zipTree(configurations.warResources.singleFile)) {
    filter ...
  }
}

Unzipping and other forms of preprocessing the war resources can be achieved in a similar way. Basically, ‘configurations.warResources’ is an Iterable<File> that you can iterate over to get at the file(s). When you first do that, the files will get downloaded to the cache on the fly. (Since you have the ‘File’ objects, you don’t need to worry where exactly the files are located, or how to find them.) For more information, I recommend to study the Gradle user guide, specifically the chapters related to file handling and dependency management.

5

Damn - you answered my question too well :slight_smile:

How about doing something similar using from a non-java project (i.e. using just the base plugin)?

The reason I ask is because I suspect Scott is trying to do something non-java with that zip file. Right Scott?

6

Shouldn’t make a difference. Except for the ‘war’ task, I didn’t use anything Java-specific.

7

Very cool. Will give it a go and get back to you. Thanks Peter.

8

Thanks, this works quite well!