How to explicitely define (and therefore skip) the initialization and configuration phase..?

[Dieter_K1 (Dieter K)]

Hi,

we have a (huge, deeply nested) project that needs to be build after a clean every time. No incremental builds! So far it was build using bash scripts/ANT/Maven. I got the task of rewriting everything in Gradle. The actual execution phase is on par with the previous (bash/ANT/Maven) solution. The problem is the initialization/configuration phase - which is much too slow.

I did a lot of performance tuning the last two weeks but I am pretty sure this is a dead end. I did improve configuration times quite a bit but it still takes much much too long. Even if I manage to cut it in half a couple of times (which I won’t), it is still too slow.

I hope the answers to my question will not try to convince me to speed up configuration phase

The previous bash/ANT/Maven solution basically has no configuration phase. It just executes compilers, copies files, etc., bare to the metal. It’s basically what you would do manually on a CLI, just automated.

The solution would be to do the same with Gradle and explicitly define the initialization and configuration phase and directly enter execution phase.

I did dig into the Gradle code (not just the API) and it seems pretty complex (although beautifully written). Still, due to the complexity I wonder what the best way would be to approach my goal.

Ideally I would import the entire Gradle codebase and have access to all internals. Instead of using build.gradle I would directly use Java to initialize and configure the Project objects before kicking off the execution phase. I would like to try this on a smaller mock project first and do extensive performance test - which I would report back.

Looking at the Gradle codebase it seems to be designed to keep people from doing this (why would you do that?). I am no Core Dev, so maybe I am mistaken.

I think the build.gradle is in itself a plugin that is applied to the correct object without calling apply by the bootstrap process. Maybe for my experiments you could suggest changes that allow me to directly write a plugin and convince the bootstrapper to use it directly without some ugly build.gradle script hack that calls my plugin.

Cheers,
Dieter

P.S.: Gradle is very flexible. That’s why I am really puzzled and confused to discover why you married build.gradle so tightly to the underlying project. For most users the build.gradle should offer a very nice and readable DSL which simplifies accessing the build-API for most standard builds - and the build.gralde and Groovy DSL (or Kotlin) does this very nicely. To force people to use build.gradle (at least in some minimal way) seems like a big mistake for many reasons.

[st_oehme (Stefan Oehme)]

Hi Dieter,

judging by the title of the post I feel like there is some wrong assumption about how Gradle works. The configuration phase is no magic “make things slow” thingy What happens is exactly what you are asking for: Gradle sets up its services, creates the relevant objects and then applies plugins to them. Your build scripts are just plugins themselves. They are compiled to bytecode and cached and they use exactly the same API that a Java based plugin would call. I hope this clarifies that there is nothing to “skip”, the configuration needs to be done, it’s simply wiring objects together. It is already highly optimized and most problems are to be found in user code, e.g. unnecessary IO at configuration time or some accidental O(n^2) algorithm. I’m sure we’ll get to the bottom of what is making your particular build slow.

More details below:

@Dieter_K1

that needs to be build after a clean every time. No incremental builds!

Why not? That’s giving away one of the main benefits of using Gradle.

@Dieter_K1

The problem is the initialization/configuration phase - which is much too slow.

Please use the Gradle profiler, I’m sure we can make it plenty fast. Instructions in this issue.

@Dieter_K1

Ideally I would import the entire Gradle codebase and have access to all internals. Instead of using build.gradle I would directly use Java to initialize and configure the Project objects before kicking off the execution phase.

If you want to write your build logic in Java, simply put it as a plugin into buildSrc. Alternatively you can use the Kotlin DSL to get rid of dynamic invocation overhead. But I can’t even say if this is actually a problem or if there is some other issue in your setup. Only profiling data can tell.

@Dieter_K1

Looking at the Gradle codebase it seems to be designed to keep people from doing this (why would you do that?)

Because Gradle is a build tool, not a library. It has a well-defined lifecycle that directly calling some internals would violate.

@Dieter_K1

Maybe for my experiments you could suggest changes that allow me to directly write a plugin and convince the bootstrapper to use it directly without some ugly build.gradle script hack that calls my plugin.

I don’t understand what’s ugly about a single-line build script applying your one-and-only Java-based plugin, if that’s what you would like to do.

apply plugin: 'my-plugin'

@Dieter_K1

To force people to use build.gradle (at least in some minimal way) seems like a big mistake for many reasons.

I disagree, it’s how every build tool works - You provide configuration, the tool controls the workflow. You don’t call internals of the tool. That’s no different for Make, Ant, Maven, Bazel, you name them. But let’s not discuss these philosophical points further. Let’s focus on profiling and making your build fast instead

[Schalk_Cronje (Schalk Cronjé)]

Without offering up any technical discussion at first, maybe you should consider some consulting time from the Gradle folks. I am sure the folks at LinkedIn can attest to how much cooperation with Gradle has reduced time for their hugely complex builds. (Disclaimer: I am not associated with Gradle Inc.)

My suggestion otherwise from having worked with really complex builds is that sometimes the conversion isn’t a like-for-like conversion. Maybe your build is slow because the original structure of the build lends it towards that, especially if it is a legacy build that has evolved over years. Sometimes these conversion opportunities are also the best time to restructure a build.

Having said that, do keep in mind that initial startup time. model building etc. are usually counter-acted at some stage to do the Gradle daemon. Build caching - both local and remote - can reduce build times quite dramatically under continuous use.

Of cose you have to ask yuorself why you really want to convert. If the build is working, just why?. If the answer is maintenance and complexity then you already asnwered the necessity of a simplified Gradle DSL.