Is it just me, or is the Extensions model fundamentally broken due to lazy properties?

No hate here. I just need to vent, and maybe be educated :grinning_face_with_smiling_eyes: I have been writing a suite of custom Gradle plugins for my company to help standardize and simplify our builds. We have literally hundreds of Java repositories that build with Gradle, so any standardization we can provide, or configuration we can simplify, really helps. Ensuring all projects are enforcing code coverage, Checkstyle, Sonar analysis, etc.. Setting up Spring Boot, Lombok, DataDog and the rest.

I think I’ve become pretty well versed in extending Gradle at this point. I’ve written over a dozen plugins, and overall, I have found it to be pretty straightforward to work with. But I have to ask..

Did Gradle miss the boat with the whole lazy properties thing? As a plugin developer, the complexity and race conditions it imposes upon even the simplest task configuration is so painful. And, to be honest, it kinda of smacks of premature optimization.

Maybe the problem is that not everyone uses Properties and Providers for their configuration, and for this pattern to work correctly, you really need universal support for deferred configuration across the board? One rogue plugin that doesn’t use Properties and Providers will basically break deferred configuration for the entire build.

This really comes into focus when trying to write and then use your own custom Extensions. You end up doing extra work to support deferred configuration, but then, in order for your build to actually use that configuration via build.gradle and DSL, you end up slapping the processing of it into a project.afterEvaluate() :face_vomiting: Without doing so, nothing that a user provides in build.gradle is actually available at configuration time. :melting_face:

Consider a simple example:

// build.gradle
plugins {
  id 'mycompany.java-application'
}

mycompany {
  jacoco {
    coverageRequirement = 0.75
  }
}

Here, the mycompany.java-application plugin is itself applying a bunch of Gradle and 3rd party plugins, such as Jacoco, and providing sane defaults to those plugins for convenience. The few levers that we want teams to have easy control over, like test coverage thresholds, are configured on our custom extension. But the Jacoco plugin does not support setting coverage requirement via lazy property – rather, it requires a BigDecimal. And so this will not work.

And there are countless other examples of this problem I’ve encountered. For a long time, I had an project.afterEvaluate block in our custom Extension that would iterate all of our custom Plugins and reconfigure them. And this mostly worked, but there were still challenges and edge cases (for example dynamically defining tasks based on configuration).

After battling this pattern for so long, I finally gave up. Instead, I’ve moved all user-defined configuration to gradle.properties. It’s not as friendly or obvious as the Groovy or Kotlin DSL, e.g.:

# gradle.properties
jacoco.coverageRequirement=0.75

It lacks any sort of IDE support / code completion. It’s not necessarily where a developer would look for these options. But honestly, it’s by far the sanest solution. My Extension can easily configure itself when it is instantiated by interrogating project.getExtensions().getExtraProperties(), and then do all of the right things without deferred configuration, race conditions or other headaches.

It’s really a shame. I would love to use the Gradle DSL for this kind of configuration. But the fact that it’s not available until after the build has been evaluated is just … ugh. And why? To save 1 second of processing at startup? Oof.

Okay. Flame suit on. Tell me why I’m wrong – please :folded_hands: If I’m a dummy and there’s an elegant way to do this, I would love to know.

You are right, that to fully benefit of Property/Provider/…, all parts should have adopted it, so that you can just wire things together and they are only evaluated at execution time where no further configuration happens.

You are right, that not all 3rd party plugins yet support it, but any that does not you should nudge to adopt it, as it is the future. :slight_smile:

You are unfortunately also right that not every built-in plugin has fully adopted the new approach yet. This is a huge amount of work and also lots of breaking changes to get it switched over so has to be done carefully.

Luckily the Gradle folks are working hard on migration all the built-in stuff to the new approach.
It was planned to do a full switch-over for Gradle 9.
Unfortunatley, it seems they underestimated the effort and don’t want to postpone the Gradle 9 release any longer to get other stuff out, so hopefully in Gradle 10 the built-in stuff will be fully enabled.
Here is the roadmap item for this transition: Provider API migration · Issue #28 · gradle/build-tool-roadmap · GitHub.

Just in case you are not aware, before the Propertys etc. it was way worse.
Everything was just primitives, so you always had to use things like afterEvaluate { ... } to give the user the chance to change the configuration.
But then for some reason the user also had to use afterEvaluate { ... } to do the configuration and so - as he registered his action after your plugin - the configuration was done later so your plugin again missed to have the updated value and so on and so forth.
The main earnings of using afterEvaluate are timing problems, ordering problems, and race conditions.

When sometime in the bright future, everything uses Property and friends properly, you can just wire things together, the interested partys only read the values at execution time and all will be fantastic. :slight_smile:

In the meantime with not all plugins and extensions having it adopted yet, you indeed have to take measures.

One option without afterEvaluate for example is to not have a Property for coverageRequirement in your extension, but a function, so that the user does

mycompany {
  jacoco {
    coverageRequirement(0.75)
  }
}

and the function then does the according configuration.
For something like this case where another call to the function can just reconfigure, this is enough, for other cases where the effect of calling the function cannot easily be undone, you could consider preventing the function to be called multiple times.

If you want to maintain the syntax, you can also use another trick, especially as you have the required levels of nesting already.
With your original snippet, you can make jacoco a function in the mycompany extension.
This function takes as argument an Action<MyJacocoExtension>.
The MyJacocoExtension then just has the Property like now and can be configured like now.
In the jacoco function you then let Gradle create an instance of MyJacocoExtension using objects.newInstance and feed it to the supplied Action, then you have the user-configured values.
And then in the end of that function you again do the configuration changes based on those supplied values and if necessary prevent multiple calls of the function as described above.

1 Like

Thanks for not biting my head off :grinning_face_with_smiling_eyes: And for the explanation, and for the suggestions! The implementation I had in place for the past year or two was quite close to what you described. It looked kind of like this:

@Data
public class MyExtension {

  @Data
  public class JacocoOptions {
    BigDecimal coverageRequirement = BigDecimal.ZERO;
    String coverageType = "LINE";
  }

  JacocoOptions jacoco = new JacocoOptions();

  public void jacoco(Action<JacocoOptions> action) {
    action.apply(jacoco);
  }
}

And then build.gradle:

myCompany {
  jacoco {
    coverageRequirement = 0.80
  }
}

So your suggestion would be quite similar, but instead of instantiating the nested class in MyExtension, you suggest using project.getObjects().newInstance(JacocoOptions.class). Is there a material difference?

Or is the main change you’re proposing to convert the fields in JacocoOptions into methods that actually do the configuration, rather than leaving the configuration up to my Plugin classes?

Well, you should imho always let Gradle construct objects,
because you can save some boilerplate (like having just abstract Property fields and Gradle implements them, or injecting Gradle services, and so on,
and also all objects you create through Gradle are made ExtensionAware whether you explicitly declare it or not (so I usually also declare it explicitly to not need to cast it) so you (or someone else) can create extensions on the objects or (even though usually a work-around for not doing something properly) setting “extra” properties on it, and so on.

The Property fields are of course not a topic in your quoted case, because you probably don’t want here that a user wires some other Provider in, as you have to then eagerly evaluate it anyway, but maybe the class also gets some other properties where the target they are used on are Propertys so those you could then wire properly instead of eagerly realizing, and then you can save that boilerplate.

Like e.g.

public abstract class JacocoOptions {
    BigDecimal coverageRequirement = BigDecimal.ZERO;
    String coverageType = "LINE";
    public abstract Property<String> getFoo();
}

and Gradle makes it work properly.

1 Like