Req. Beginners guide to Remote Build Cache

I attended the inspiring talk by Justin Reock at Devoxx Belgium about the Remote Build Cache, and have set up a Docker hosting the build-cache-node.
Now a full build of our system completes with telling me that “185 actionable tasks: 151 executed, 34 from cache”.

Where can I find guidance in how to make more of the tasks cacheable?
And likewise, where can I find guidance in how to make a setup where tests are not executed if the code has not changed since last test-run?

Is this even possible to do in the free version, or would we need to purchase a Develocity-server to gain all the advantages?

Is this even possible to do in the free version, or would we need to purchase a Develocity-server to gain all the advantages?

No, you do not need Develocity.
Develocity you only need if you want to manage multiple remote build cache nodes centrally for example.
Actually if you for example have a Nexus Repository Manager running, you could as well just create a repository of type raw and use that as remote build cache, I do that and it works fine.

Where can I find guidance in how to make more of the tasks cacheable?

If you only use built-in tasks, you should not bother.
All tasks that are worth caching should be cacheable.
For example a task that simply copies around files, or packs or unpacks an archive like a jar is not worth caching, as the caching would not be faster and would just add overhead.
Because for the caching, the build cache key has to be built from the inputs, then the local cache checked for an entry, if none found the remote cache checked for an entry, if found downloaded and put to local cache. If no cache entry was found, it has to be built after the task was run and pushed to the local cache and if on an authorative machine, also to the remote cache. The entry value is an archive with the build outputs. So if you for example have a task that builds a jar, putting that to the cache would mean to pack the jar into an archive and put that to the caches and on reuse get the entry from the caches and unpack the jar from the archive. So it is not worth caching such a task.

Tasks that need long to execute like compiling sources, running tests, … are worth caching.

If you have custom tasks that you want to make cacheable, make sure that you properly define all inputs and outputs, this is now tremendously more important than just for the up-to-date checks where it is important already too, and mark the task cacheable per annotation. Also make sure you use the right annotations. I just today had a case where a 10 minutes functional test task was not taken from cache, because I marked an input that is used as a runtime classpath not as @Classpath but as @InputFiles and thus got always a cache miss as the file timestamps in the jar had changed. Also make sure that you use proper @PathSensitivity, because by default it is ABSOLUTE which would practically mean it cannot come from cache unless you build in the exact same directory.

For tasks like JavaExec or Exec for example they are by default also not cacheable, as Gradle cannot know which are the inputs, which are the outputs, and whether maybe other side-effects are done additionally. If you for example have such tasks and know, the called thing just processes input files to output files and does no other side-effects, you could properly declare the inputs and outputs (if you did not do already anyway for chance to be up-to-date) and mark it as cacheable, if the action is acutally worth caching like described above.

On Build Cache you should find all the infos you need I think.

Oh, and Develocity might be helpful, as you for example can compare builds and maybe easily find out why a task was not served from cache.

My incident today I just found out by running with -Dorg.gradle.caching.debug=true twice and comparing the outputs to see instantaneously where the difference is during cache key calculation.

Thank you for your reply, Björn.
I might have timed my question better, as I was away last week.

As I read your response, I understand that I am getting all cached that can be done without taking a deep-look into our flow of Gradle-sub-tasks, and look for optimizations. We will probably need to do that at some point, since the setup is quite old, and has been tinkered more that it has been really rewritten.

We use Artifactory and not Nexus. Would that really be just as good as a remote cache as the image given on Build Cache Node User Manual | Develocity Docs ? I was certain that there were some specific optimizations in that image.

If you connect multiple for geo-distribution with mirroring, or connect them to Develocity, sure.

When just having one node I don’t think there is much that could be optimized. A remote build cache with the default implementation on client side is just storing files sent via PUT and needs to deliver the files back when requested via GET. I don’t think the cache node can optimize much there over something like Nexus where exactly this is the core business. But your are welcome to proof me wrong. :slight_smile: