Gradle wrapper download/extract is not threadsafe

(Jeroen van Erp) #1

We use the gradle wrapper to distribute our “enterprise gradle” version to everyone, including our CI server. We however have multiple projects building with this gradle version.

When we update that to a new version, multiple builds on our buildserver kick in, and all of the builds try to download and extract the new version of the wrapper, resulting in a corrupted extracted wrapper, thus failing our CI builds.

(Luke Daley) #2

Thanks for the report, raised as GRADLE-2557.

(Luke Daley) #3

Which Gradle version are you using? Is this a new behaviour?

(Jeroen van Erp) #4

Hi Luke,

We’re using gradle 1.2, and only recently (ie. last 2 weeks) switched to using the wrapper and our own gradle bundle, so I cannot say whether this has been a problem with older gradle versions.

– Jeroen

(Luke Daley) #5

Do you have multiple build processes running as the one user? Or do you have a shared gradle user home directory?

(Peter Walker) #6

Can you post the evidence (stack trace, log file)?

(Jeroen van Erp) #7

We have one user “jenkins” who owns all the build processes, and his ‘.gradle’ directory is used by the different gradle processes.

(Jeroen van Erp) #8

@Peter: See this gist for the logs of one of the buildtasks that failed.

(Peter Walker) #9

Thanks. Is this something that happens “rarely” or “frequently”?

(Jeroen van Erp) #10

Good question. We’ll only notice it if we upgrade our gradle bundle again. Because we have 20+ builds triggering at virtually the same time then. (I have a script that updates the in each of the repositories).

So it isn’t really pressing, as I don’t intend to update that very frequently. But it is nasty that there is no locking there. The artifact repository locking nowadays is actually quite nice and works great. And we had some trouble with that back in 0.6/0.7 :wink:

(Justin Ryan) #11

I’d recommend running with the home directory of the jenkins user as read-only. There’s a whole category of problems which become really obvious once you do that. In our case, we added this blurb to our gradlew to isolate concurrent gradle processes:

if [ "x$GRADLE_USER_HOME" == "x" ]; then
    if [ "x$WORKSPACE" != "x" ]; then
        export GRADLE_USER_HOME="$WORKSPACE/.gradle"

The Gradle cache is excellent, but it does have to lock, and we wouldn’t want concurrent executors having locks on each other. This isolates them to prevent the kind of problem you’re seeing, but also should provide some additional speed.

(Jeroen van Erp) #12


I’ve updated the gist to include another log file that show that it can go wrong when having just downloaded the gradle zip. See:

(Emil Sit) #13

Do you worry about the wrapper task overwriting your changes?