Allow tasks to be configured just before execution


(Attila Kelemen) #1

Sometimes you need to defer configuration to be done just before task execution because the information required for configuration is specified in other scripts (or by depenedent tasks). Consider the task when you want to create a javadoc from all the sources of every subprojects.

Currently the best solution I have found is this:

task mergedJavadoc(type: Javadoc, dependsOn: ‘mergedJavadocConfig’) {

}

task mergedJavadocConfig doLast {

project.subprojects {

mergedJavadoc.source(sourceSets.main.allSource);

}

mergedJavadoc.title = ‘All modules’;

mergedJavadoc.destinationDir = new File(project.buildDir, ‘merged-javadoc’);

}

I have tried the following but it does not work:

task mergedJavadoc(type: Javadoc) {

}

mergedJavadoc.doFirst {

project.subprojects {

mergedJavadoc.source(sourceSets.main.allSource);

}

mergedJavadoc.title = ‘All modules’;

mergedJavadoc.destinationDir = new File(project.buildDir, ‘merged-javadoc’);

}

The above code will not produce anything but attempting to execute ‘mergedJavadoc’ will simply report that it is “up to date”.

My prefered solution would be however if something like this would be possible:

task mergedJavadoc(type: Javadoc) configureLater {

project.subprojects {

mergedJavadoc.source(sourceSets.main.allSource);

}

mergedJavadoc.title = ‘All modules’;

mergedJavadoc.destinationDir = new File(project.buildDir, ‘merged-javadoc’);

}

Where ‘configureLater’ would simply defer the configuration until ‘mergedJavadoc’ is to be executed (and before it checks if it is up to date). Theoretically this could alse improve performance since tasks not executed does not need to be configured (although probably this performance gain is negligable).


(Adrian Abraham) #2

I’ve had to hack up something like this in my own build scripts. My packaging tasks actually have to scan a large directory structure to figure out what needs to go in the package, and I don’t want to take the hit if I’m not actually going to be packaging.

My tasks that require lazy configuration look like

task foo(type: Jar) {
  quick_assignment = true
      ext.lazyConfiguration = {
    do_something_hard()
  }
}

Then, I define the following hook:

gradle.taskGraph.beforeTask { Task task ->
  if (task.hasProperty('lazyConfiguration')) {
    if (task.lazyConfiguration instanceof List) {
      task.lazyConfiguration.each { task.configure it }
    } else {
      task.configure task.lazyConfiguration
    }
  }
}

Back in the M3 era, I was actually doing some meta-programming to implement a configureLater method to the Task class, which would append the block it was given to the lazyConfiguration property. At some point, Gradle improved its classloader separation, and my metaprogramming broke, so I had to go to straight variable assignment. This can probably be done in a plugin, but I haven’t taken the time to figure out how yet.

Of course, it would be nice if this were supported natively and I didn’t need to go through this.


(Luke Daley) #3

When dealing with files, you can defer evaluation by using closures:

task mergedJavadoc {
  source { subprojects*.sourceSets*.main*.allSource }
     …
 }

Our long term strategy is to support this kind of laziness for all properties.

However, there are times when it doesn’t work and something like you suggest would be useful.


(Attila Kelemen) #4

Thank you, for the answer, this was actually helpful. But I find this somewhat counter-intuitive because everywhere in a gradle script closures are executed immediately, rather than lazily. But thanks anyway.


(Luke Daley) #5

The majority might be, but there are plenty of cases where closures are not executed immediately.


(Attila Kelemen) #6

While using Gradle I came across many problems which I couldn’t easily solve without using afterEvaluate. However, I have found afterEvaluate to be a fragile solution. That is, in almost all cases, using afterEvaluate was a dirty hack marking a failure of design in the build scripts.

In almost all cases, I have found that if I used the lazy task configuration proposed in the post, I could easily avoid using afterEvaluate. As an added benefit, this would make build script evaluation faster due to skipping configuration code not actually in use.

Why does lazy configuration solves the majority of problems? Because in Gradle most of the time you just need to configure tasks and then let Gradle run them. Using afterEvaluate becomes necessary when you need a property configured somewhere outside of the control of the code needing the property. However, if a task needs to be configured, it can usually be expected that it does need properties only from tasks it depends on. This dependency can usually be easily enforced.

When cannot lazy configuration be used? When you need to affect the task graph. Obviously, it is too late to change the graph when the tasks are being executed. So this RFE does not intended to solve this case. I guess, there is no one tool fits all. (Well, maybe adding another graph - or many more - where tasks are just doing configuration, so configuration dependency can also be expressed the same way as task dependency)

Are there any plans for adding this kind of lazy task configuration in the forseeable future?


(Luke Daley) #7

Hi Attilla,

This is one of our biggest “issues” at the moment. We haven’t fully mapped out a direction, but are in the process of doing so. You can have a look at the lazy configuration spec for what we have currently distilled from our conversations.

https://github.com/gradle/gradle/blob/master/design-docs/lazy-configuration.md

What’s in there is not set in stone, and probably will change going forward.


(Attila Kelemen) #8

I have read through the doc you linked. As I understand you want domain objects to be lazily configured. This seems to be a good step forward to me. I would like to comment on them here:

  • You say, that “Lazy data structures cannot be shared between threads for parallel execution.” But this is not necessarily true. The only thing which must be ensured is to have the data structure initialized before accessing. This can be done by callind an “ensureInitialized” method before getting the structure. - I feel that lazy configuration of tasks fits well with the goals set in the design doc. Although, it does not solve every problem described but could be an easy to use building block for 3rd party plugins.

However, whatever you do, I would like to ask you to not do something like making the configuration block of publications lazy by default. That is, having some task eager configuration and others lazy is extremely confusing. So if you make it lazy, please make it something like:

publications lazy {

// do something.

}

Also if you allow lazy configuration for other things, this (or something like this) could be a convention, easily recognized even by users not intimately familiar with Gradle.


(Luke Daley) #9

The forums don’t really work well for this type of “discussion”. If you want to inject your opinions, the most effective way to do so is via the dev list.