Scaling a Gradle project to house hundreds of projects?

Help/Discuss
1

As I start thinking about cross platform on Mobile, I always find myself approaching the conclusion that because the iOS and Android projects would be sharing code, they are then tightly coupled to each other, and there is a need to start thinking about the separate code bases as a single code base.

I’d love to use the xcode gradle plugin and start to pull in more and more subprojects, but I’ve noticed a problem, that the more sub-projects a gradle project has the longer the gradle configuration stage takes. This would be fine if it were a one time cost, but I seem to have to pay the configuration cost pretty regularly. The gradle daemons seem to not be re-usable a lot of the time, especially when I cancel a running gradle command. In my android project, I have 20 sub modules, and it ends up taking 10-15 seconds for initial configuration for a daemon.

This makes me a little bit hesitant to lean on gradle as the build system to use for cross platform. I’ve noticed that other build systems like bazel or buck seem to be consistently faster, and handle subprojects really well, without it affecting performance nearly as much. That being said, I’m currently reliant on many gradle plugins, and also prefer gradle to any of these other systems.

I thought I’d ask to see if maybe others are experiencing the same trouble with gradle performance that I am, and to see if maybe there were solutions that maybe I’m not taking advantage of currently.

I’m currently on Gradle 3.3

2

You mean when starting a new daemon? The cold daemon case should be very rare. The number for a warm daemon would be more interesting. You can use the Gradle Profiler to create build scans and low-level profiler snapshots to analyze your build’s performance.

Note that Android is a case with specific configuration time issues, which have partly been addressed in Android 2.3 and will be fully addressed in Android 2.5 (alpha version available). To get an idea of how fast Gradle is (and how fast Android will be once 2.5 is stable), have a look at our performance test for a 500-module Java project. Printing all tasks for that takes <2s. Just configuring would be even faster.

The Gradle daemon kills itself if it cannot stop the build after a grace period. That usually happens when calling a long-running tool. It would be interesting where this happens in your build.

3

I agree that a 500 module project being configured in 2 seconds is more than acceptable. It looks like I’m incorrectly attributing Android’s slow configuration to Gradle. Thanks for pointing out the fixes in 2.5!

4

If you see long configuration time problems in non-Android builds, please send us some data so we can make Gradle better. Also, if you have to restart the daemon a lot, it would be great to know why that is happening. Please feel free to open an issue for that.