Gradle build is very slow depending on the order of repositories

Help/Discuss
1

Gradle Version: 3.0+ (tested with the lastest nightly, 3.3-20161023000015)
Operating System and JVM version: Ubuntu 12.04+, Oracle JDK8
Is this a regression? If yes, which version of Gradle do you know it last worked for? no

I discovered that the build is very slow for my project. I created a minimal example, which can reproduce the issue.

The project contains a single Dummy Java class, without any code (this is required for reproducing the error). By swapping the order of repositories, the build becomes much slower: the build time on Travis increases from ~2 minutes to ~10 minutes: https://travis-ci.org/szarnyasg/gradle-slow-build/builds

The build stops at the compileJava / compileClasspath goal for minutes. The problem occurs on my local machine as well, even if the dependencies were resolved earlier and are available in the local Gradle cache.

Scan for the fast build: https://scans.gradle.com/s/guyg46zdvxzfo
Scan for the slow build: https://scans.gradle.com/s/oac6opizyjvxu

Please let me know if I can provide more information.

2

eclipse.org has been having a lot of technical difficulties lately, so I’m not surprised you saw slow builds when using their repositories. It’s back down to 15s for me. Still not fast, but better than several minutes. I suggest using their repositories last.

3

Thanks for your answer. I still have issues building my larger projects so I did some experiments and made the following observation: the build is slow on the first try, but if I run the build with with --offline switch, it finishes very quickly and with success (as it already has the required dependencies).

Can I enforce Gradle to try the local cache first (even in online mode), instead of trying to connect to the servers? This should work for my project – there are no snapshot dependencies that should be checked every time the build runs.

I listed all the transitive dependencies of the project here: https://gist.github.com/szarnyasg/3bcb77b31f925c03a07eff184f7d05b5

4

Gradle should not recheck if there are no dynamic/changing dependencies. If it has metadata already it’ll use that.

Is what you describe reproducible with the project you linked earlier?

5

Yes, I can reproduce it - see this build: https://travis-ci.org/szarnyasg/gradle-slow-build/builds/171623212

The trick is that I added an unavailable repository to the Gradle configuration: https://github.com/szarnyasg/gradle-slow-build/blob/master/build.gradle#L8

The build results in the following execution times:

$ ./gradlew assemble 344.96s
$ ./gradlew assemble check 256.20s
$ ./gradlew assemble check --offline 2.17s
$ ./gradlew assemble check 256.57s

All exited successfully, with 0 exit value. Is there a way that could speed up subsequent builds (without --offline), even if there is an unavailable (but unused) repository?

6

Gradle will cache the result it gets from a repository, if it finds a particular module, or if it finds that the module is missing. The issue is that Gradle doesn’t cache any result for a repository when we fail to connect, if authentication fails, or if the repository returns a bad status.

With this build, Gradle will attempt to connect to http://www.systap.com/maven/releases on every execution, and this connection times out after about 1 minute. You can see this happening in the debug logs: https://gist.github.com/bigdaz/a27932aa5d74f5ca75468d4e2b461a0a.

To fix this, we’d need to record this “broken” or “timeout” status in our dependency cache, so that we wouldn’t attempt to connect on every build invocation. Naturally, this cache record should expire periodically, in case the broken status of the repository is transient.

7

Thank you for the detailed explanation. Fortunately, the artifacts that I used from the Systap repository are now available in the Maven Central (related ticket), so I can remove the repository from the build.