Hi,
we are trying to start more gradles at once to get N different outputs. Our gradle plugins print back what they are doing and that is in turn used by Jenkins - output is automatically printed out. We don’t want outputs from directories A, B and C (and more) mixed, so we basically do ./gradlew :A:test, ./gradlew :B:test and ./gradlew :C:test. So we execute them separately - so far so good.
Our builds seem to work reasonably well when started individually and our gradle plugins handle it well for small amount of parallelism (approx less than 10 simultaneous runs).
For more simultaneous runs (build root is in /home/build), we start getting errors like this:
Could not create service of type FileHasher using BuildSessionServices.createFileHasher().
Timeout waiting to lock file hash cache (/home/build/.gradle/6.4/fileHashes). It is currently in use by another Gradle instance.
Owner PID: 102
Our PID: 101
Owner Operation:
Our operation:
Lock file: /home/build/.gradle/6.4/fileHashes/fileHashes.lock
This is logical - other gradle builds are holding the lock for too long. Is there any way to avoid this issue?
My only idea so far was to create /home/build/A/gradlew, /home/build/B/gradlew, /home/build/C/gradlew and so on and change the gradle plugin to actually run from /home/build even if it’s actually running from /home/build/A. However, that seems to be clumsy as we’d have to have more copies of gradle somewhere.
Any other suggestions on what more to try to achieve my goals of having more separate outputs and parallel runs?