Gradle wrapper download/extract is not threadsafe

Old Forum Archive
1

We use the gradle wrapper to distribute our “enterprise gradle” version to everyone, including our CI server. We however have multiple projects building with this gradle version.

When we update that to a new version, multiple builds on our buildserver kick in, and all of the builds try to download and extract the new version of the wrapper, resulting in a corrupted extracted wrapper, thus failing our CI builds.

2

Thanks for the report, raised as GRADLE-2557.

3

Which Gradle version are you using? Is this a new behaviour?

4

Hi Luke,

We’re using gradle 1.2, and only recently (ie. last 2 weeks) switched to using the wrapper and our own gradle bundle, so I cannot say whether this has been a problem with older gradle versions.

– Jeroen

5

Do you have multiple build processes running as the one user? Or do you have a shared gradle user home directory?

6

Can you post the evidence (stack trace, log file)?

7

We have one user “jenkins” who owns all the build processes, and his ‘.gradle’ directory is used by the different gradle processes.

8

@Peter: See this gist for the logs of one of the buildtasks that failed.

https://gist.github.com/4078514

9

Thanks. Is this something that happens “rarely” or “frequently”?

10

Good question. We’ll only notice it if we upgrade our gradle bundle again. Because we have 20+ builds triggering at virtually the same time then. (I have a script that updates the gradle-wrapper.properties in each of the repositories).

So it isn’t really pressing, as I don’t intend to update that very frequently. But it is nasty that there is no locking there. The artifact repository locking nowadays is actually quite nice and works great. And we had some trouble with that back in 0.6/0.7 :wink:

11

I’d recommend running with the home directory of the jenkins user as read-only. There’s a whole category of problems which become really obvious once you do that. In our case, we added this blurb to our gradlew to isolate concurrent gradle processes:

if [ "x$GRADLE_USER_HOME" == "x" ]; then
    if [ "x$WORKSPACE" != "x" ]; then
        export GRADLE_USER_HOME="$WORKSPACE/.gradle"
    fi
fi

The Gradle cache is excellent, but it does have to lock, and we wouldn’t want concurrent executors having locks on each other. This isolates them to prevent the kind of problem you’re seeing, but also should provide some additional speed.

12

@Peter/@Luke

I’ve updated the gist to include another log file that show that it can go wrong when having just downloaded the gradle zip. See: https://gist.github.com/4078514

13

Do you worry about the wrapper task overwriting your changes?