Extra side-effects of Project.getTasksByName()?

Help/Discuss
1

I am trying to update/rewrite an old Gradle plugin of ours, but have stumbled across something I don’t understand. (I am using Gradle 5.6.4 and cannot upgrade to Gradle 6.x yet.) Our plugin is applied to the root project and invokes:

project.getTasksByName("doesNotExist", true);

as the first line of its afterEvaluate() handler. The handler then continues by invoking project.getTasks().findByPath(t) and performing Task discovery across the entire project tree. This all works, although it is ugly.

The documentation for Project.getTasksByName() says:

NOTE: This is an expensive operation since it requires all projects to be configured.

which I assumed was the reason why the plugin was invoking it in the first place. And so then I tried replacing it with:

project.evaluationDependsOnChildren();

because I thought this would make the plugin’s intention clearer. However, this did not work! More precisely, Gradle suddenly became unable to configure some sub-projects that were using the

org.springframework.boot

and

io.spring.dependency-management

plugins, saying (e.g.) that it hadn’t registered an id of org.springframework.boot and that therefore it was not correctly implemented.

Revert my plugin back to using Project.findTasksByName(name, true) and suddenly the Srping plugins can be applied successfully again.

Clearly Project.findTasksByName() is doing more than Project.evaluationDependsOnChildren() does. Can anyone explain what that might be, please? I have considered “evaluation” and “configuration” to be the same thing, as far as Gradle projects are concerned. Is this incorrect?

Thanks for any help here,
Cheers,
Chris

2

You could just change your logic to

allprojects {
   tasks.matching { it.name == 'foo' }.all {
      // logic goes here
   } 
}

Then you wouldn’t need an afterEvaluate{...} closure or evaluationDependsOnX. See here for related topic

3

Hi Lance, thanks for the reply.

The plugin is already working much as you describe. The problem is with how it creates its matching condition. Basically, starting from the Gradle command line:

Set<String> requestedTaskNames = new HashSet<>(project.getGradle().getStartParameter().getTaskNames());
Set<Task> requestedTasks = requestedTaskNames.stream()
    .map(it -> project.getTasks().findByPath(it))
    .collect(toSet());

which is then used as:

tasks.matching(requestedTasks::contains)

I’m not entirely clear on why the plugin is doing this. Perhaps to navigate some “task name” vs “task path” confusion from the command line arguments? I can try using the command line arguments directly, on the assumption that I can just add a “:” character to the start of any task name that doesn’t already have one.

However, my changes really need to be “evolutionary” rather than “revolutionary”, which is why I was seeking a deeper understanding of Gradle’s configuration model.

Thanks,
Chris

4

The plugin is already working much as you describe

No, it’s not. The fact that it’s using Set<String> and Set<Task> shows the problem.

Please read here for related topic

5

This Set<String> contains the task names read from Gradle’s command line parameters:

Set<String> requestedTaskNames = new HashSet<>(project.getGradle().getStartParameter().getTaskNames());

I am certain that this Set<String> is “static” and not “live”, and is therefore safe to use in a matching() lambda.

However, I do agree that then using findByPath() to convert the Set<String> to Set<Task> is a Bad Idea, and have removed this code.

BTW, would it be fair to say that Project.findTasksByName() will eagerly configure every single task in the entire project tree? (And is therefore toxic to any efforts to use task configuration avoidance?)

Cheers,
Chris

6

I am certain that this Set<String> is “static” and not “live”

I agree, this is fine

However, I do agree that then using findByPath() to convert the Set<String> to Set<Task> is a Bad Idea, and have removed this code.

Ok, great. I assume you have replaced with tasks.matching { ... }.all { ...}

would it be fair to say that Project.findTasksByName() will eagerly configure every single task in the entire project tree?

I’m assuming you are referring to Project.getTasksByName(...) and I wouldn’t jump to this conclusion. All of the register(...) methods require a task name. So I think it’s fair to assume Gradle would only configure the tasks with the provided name. But, as we’ve discussed, this method is not “live” so it’s best to avoid so that your plugin is agnostic to the order in which it is applied.

7

Here’s a few comments on your original post.

This note is a disclaimer about the impact it will have on the --configure-on-demand feature, which attempts to limit the number of projects configured to only those that are relevant to the build. If you’re not using --configure-on-demand, or your projects depend on each other, they will all be configured regardless, so this specific NOTE isn’t impactful.

It’s not necessarily more. Project.findTasksByName() and Project.evaluationDependsOnChildren() are doing completely different things. Project.evaluationDependsOnChildren() can absolutely break things horribly depending on your project.

Basically, Project.evaluationDependsOnChildren() pauses the execution of the code in the build.gradle where it is called and causes the build.gradle of the child projects to be evaluated before continuing. This might be necessary if you’re configuring something in a parent project that you need to reference, but it is created in the subproject’s build.gradle.

A contrived example could be that you want your subprojects to have:

plugins {
    id 'java'
}

but want to put this configuration in the root project (compileJava exists only after applying the Java plugin):

subprojects {
    compileJava {
        encoding = 'UTF-8'
    }
}

Yes, there’s much better ways to handle this, but it’s generally this kind of not quite following best practices that ends up introducing ordering dependencies anyway.

If you have the opposite (or your plugins do) where the subprojects are actually expecting something to be accessible from or inherited from the root, calling this will definitely break things. This really shouldn’t be used in a plugin because it makes little sense for the plugin to force a different order for what’s in the build.gradle files (i.e. not in the plugin).

8

Hi, thanks for the explanation about Project.evaluationDependsOnChildren().

This particular plugin can only be applied by the root project, and is invoking evaluationDependsOnChildren() as the first step in the root’s Project.afterEvaluate() handler. The intention is that whatever the handler does next can assume that absolutely everything else in the entire project tree has been evaluated, and so exists to be found. I can’t think of any other way of achieving this either, short of maybe applying this plugin on the very last line of the root project’s build.gradle file :face_vomiting:…

9

I consider afterEvaluate {...} and evaluationDependsOnX() as hacks that should be avoided. Think of the case where two plugins want to perform some logic in an afterEvaluate closure. You can often achieve the desired behaviour using the “live” APIs.

When using the “live” APIs, a plugin is (usually) agnostic to evaluation order

10

Oh, I agree. I wrote my first Gradle plugin back in the Gradle 3.x days, and was burnt by afterEvaluate() when I needed to coordinate with another (third party) plugin, but couldn’t because there is no way to control whose afterEvaluate() handler runs first. So naturally I’ve since ported our plugins to use lazy properties, and more recently to use “task configuration avoidance” too.

I am only using Project.afterEvaluate() in this particular case because I really cannot think of any other way of achieving what this plugin needs to do.

11

Please start another thread with your actual problem, NOT your current solution

12

Thanks, but my actual problem was solved by removing Project.getTasksByName(...) from the plugin. I am still using

project.evaluationDependsOnChildren()

in my root project’s afterEvaluate() handler, but I would only describe this as “vulgar and distasteful” :wink:.

13

As I said, afterEvaluate and evaluationDependsOnX are both hacks. Perhaps if you describe the problem they can be eliminated

14

Thanks, I may take you up on that offer at some point. However, for now, the plugin is working sufficiently well that I need to focus more on what it is supposed to be doing rather than how it is using Gradle. I may have had several dark and miserable experiences with Project.afterEvaluate() in the past, but it is still a documented and supported API that everyone learns about from Gradle’s introductory guide. Using both it and evaluationDependsOnChildren() is far better that invoking

project.getTasksByName("doesNotExist");

in order to configure the entire project tree by force, which is what it was doing before. Now at least its dirty little secret is exposed. The plugin has duly signed the “Gradle Offenders” register, and it will no longer be allowed to frighten small children at parties.

That will have to do for now.
Thanks again,
Chris

15

project.getTasksByName(“doesNotExist”)

Just so you know there’s

gradle.taskGraph.whenReady { ... }

Which is an event where you can inspect the TaskExecutionGraph and perform actions based on the tasks that are, or are not, in the task graph.

There’s also Task Rules where you can create dynamic/missing tasks based on name patterns.

1 Like
16

The problem with using TaskExecutionGraph is that once the graph has been built, it is by definition too late to create any more tasks or add any new task dependencies. I have looked at using ProjectEvaluationListener, but this just appears to be Project.afterEvaluate() again in a not very good disguise. However, it would allow me to coordinate afterEvaluate() handling across the entire project tree.

I haven’t considered using TaskRules before, and will need to think about this.

Cheers,
Chris

1 Like
17

We have had to remove the

project.afterEvaluate(Project::evaluationDependsOnChildren);

line from our plugin in the end because we have discovered that evaluationDependsOnChildren() only forces the evaluation of the immediate child projects and not of all sub-projects. We then toyed briefly with

project.subprojects { sub ->
    project.evaluationDependsOn(sub);
}

but I think that way would have lead to madness. We have finally settled on replacing afterEvaluate() with

project.getGradle().projectsEvaluated(new OurHandler(project));

class OurHandler implements Action<Gradle> {
    ...
}

which does what we had been trying to do all along, i.e. execute a block of code after every project has been evaluated but before the task graph has been finalized.

18

Just one last try at an event based solution, have you consider using the live methods on PluginContainer

Eg:

allprojects {
   plugins.withType(JavaPlugin) {
      // do stuff only if java plugin is applied 
      tasks.withType(JavaCompile) { ... } 
   } 
   plugins.withType(FooPlugin) {
      // you get the idea 
   } 
}
19

Thanks, but it no longer matters. The plugin already has an appointment scheduled with The Wicker Man, and there is no more time, money or appetite for anything other than this simple fix before we all hold hands and sing songs while it burns.

The plugin might have escaped this fate if I’d realised a couple of months ago that Project.evaluationDependsOnChildren() only affects the project’s immediate children and doesn’t traverse all sub-projects(*), but it’s too late for that now. I have at least checked the rest of my code to be sure that I’m not using it anywhere else (which thankfully I am not), because there are no circumstances where “immediate children” would be the behaviour that I would want.

Judging by your reaction, it sounds like the projectsEvaluated() handler it yet another of those “documented and un-deprecated but still should never be used under any circumstances” APIs that make Gradle’s programming model so ridiculously complicated. Duly noted, but it still does exactly what we were wanting all along.

At least I won’t have to update this plugin to support “configuration caching” now :relieved:.

Cheers,
Chris

(*) Possible, but not guaranteed - I was trying to salvage its “donkey” at the time even then.

20

The problem with Project.afterEvaluate and anything similar is that you want to run “after everything else”. What happens when another plugin comes along that also wants to run last? You’re just kicking the can down the road.

Unfortunately, you never actually described your problem. In reality, you probably don’t need to run “after everything else”. You probably actually need to run “after some specific type of interaction”. It’s much better to target the exact event rather the blanket “run after everything” approach.

This reminds me of web designers and z-index where everyone wants their element to “be on top of everything else on the page”. So the first developer sets z-index=999 on their element, then the next sets z-index=9999 on their element etc, etc, etc. Before you know it you’re up to 999999999