Concurrent build in Jenkins CI fail because of cache lock file

Help/Discuss
1

Hi,

I’m in a larger organization where the Jenkins CI is set up so that each Jenkins job runs a CI user, with the $HOME set to /home/ciuser/. That home dir has a lot of important files, like .rc files, ssh keys and other settings, in it. Keys to Gerrit and Artifactory, etc.

Trying to introduce a project built by Gradle into this infrastructure, I run into an issue where I can’t run two simultaneous builds, even though they execute on separate workspaces (and sometimes on separate executing hosts). The error message:

Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details
Starting process 'Gradle build daemon'. Working directory: /home/ciuser/.gradle/daemon/5.4.1 Command: /usr/jdk/1.8.0_152/bin/java -XX:MaxMetaspaceSize=256m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=ISO-8859-1 -Duser.country=US -Duser.language=en -Duser.variant -cp /home/ciuser/.gradle/wrapper/dists/gradle-5.4.1-all/du806mxq1b22dn20cjxjv3h3x/gradle-5.4.1/lib/gradle-launcher-5.4.1.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.4.1
Successfully started process 'Gradle build daemon'
An attempt to start the daemon took 2.331 secs.
[...]
FAILURE: Build failed with an exception.

* What went wrong:
Could not create service of type FileHasher using GradleUserHomeScopeServices.createCachingFileHasher().
> Timeout waiting to lock file hash cache (/home/ciuser/.gradle/caches/5.4.1/fileHashes). It is currently in use by another Gradle instance.
Owner PID: 12971
Our PID: 22994
Owner Operation: 
Our operation: 
Lock file: /home/ciuser/.gradle/caches/5.4.1/fileHashes/fileHashes.lock

Apparently this is because ciuser can’t run two daemons simultaneously, even on different hosts, because its home directory is shared between all jobs. This must be a rather common situation, and I expect someone must have solved it already?

1 Like
2

Hello,

I think that you are hitting that issue https://github.com/gradle/gradle/issues/851.
The easiest way to fix it is to use a gradle user home dir local for each project: gradle -g .gradle [options] [tasks].

This fix comes with a penalty though: each job has to download its dependencies and cannot capitalize on a common local file cache like Maven does. Sure you may have a corporate repository to cache artifacts, nonetheless the time required to download them from a remote repository is still far greater than a disk access.

In the past there were also concerns about the fact that having several dependencies caches would consume too much disk space, but Gradle 5+ should at least partially address these with its auto-cleanup of unused dependencies.

2 Likes
3

Looks like that does the trick indeed! I knew someone had done this before. =)