Structured configuration of plugin task

Help/Discuss
1

We have a custom project plugin that wraps
GitHub - michel-kraemer/gradle-download-task: 📥 Adds a download task to Gradle that displays progress information with additional
functionality.

Each project configures the task using gradle.properties. As java properties are
raw text strings we are looking into other means of configuring tasks using a
configure-block (and possibly extensions), but cannot unfortunately figure out
how to do this, especially while also correctly wiring input and outputs between
the tasks.

Below is a simplified fictive working example based on gradle.properties. The
task producer produces a set of output files that are configured within each
project’s gradle.properties like so:

project gradle.properties

files = path/to/file1 path/to/file2 etc.

plugin build.gradle:

  task producer(type: DefaultTask) {
    outputs.files ((project.findProperty('files')?.split(' ') ?: []).collect { x -> "/tmp/producer/${x}" })
  
    doLast {
      outputs.files.each { x -> x.text = "${x.name} contents" }
    }
  }
  
  task consumer(type: Copy) {
    def producerTask = project.tasks.getByPath(':producer')
    def src = producerTask.outputs.files
  
    src.each { x -> from x }
    into '/tmp/consumer'
  }

given the files:

  /tmp/producer/
  /tmp/producer/etc.
  /tmp/producer/path/
  /tmp/producer/path/to/
  /tmp/producer/path/to/file1
  /tmp/producer/path/to/file2

this produces:

  /tmp/consumer/
  /tmp/consumer/file1
  /tmp/consumer/file2
  /tmp/consumer/etc.

It avoids running the tasks if up-to-date. Each project only needs to configure
gradle.properties. The output of producer is based on the configured value of
files and in this case transformed slightly (by prefixing with /tmp dir). The
value of files is supplied by each project. The output of producer is fed as
input to consumer by the plugin itself, without each project having to wire
this.

We would instead like to configure this using a task Property and/or extensions
instead of gradle.properties. This would allow us to use gradle structured data
instead of parsing text strings. Something like:

project build.gradle:
  consumer.configure {
    files = [ "path/to/file1", "path/to/file2", "etc." ]
  }

or a corresponding configuration using extensions or similar. The idea is to
only require each project to supply the value for files and keep everything else
within the plugin, especially the wiring of input/outputs between the tasks.

We are using Gradle 6.8.3. Any ideas on how to approach this? We have
experimented with using lazy properties but are struggling with figuring out how
to do this.

Thanks for any help,

2

If you want the consumer project to configure a bunch of files, expose an extension with a property of type ConfigurableFileCollection.

Btw. just in case this is not just due to your minified example.
tasks.getByPath is bad, especially in case you request tasks from another project, but even without, you break taks-configuration avoidance.
And iterating over the list of files at configuration time is also not the best idea, you probably would simply want from(tasks.named('producer')) or similar

Also you seldomly really want Copy or copy { ... }, you usually want Sync or sync { ... } to wipe out stale files.

Btw. you might consider switching to Kotlin DSL. By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and amazingly better IDE support if you use a good IDE like IntelliJ IDEA or Android Studio. :slight_smile:

1 Like
3

If you want the consumer project to configure a bunch of files, expose an extension with a property of type ConfigurableFileCollection.

OK, I see now that my example was kind of misleading. The consumer project really configures a list of strings, not a list of files. The plugin producer produces a list of files from these strings. The file names are not known until the consumer project have configured them. We have problems with getting the producer to correctly configure its list of output files from the names.

Btw. just in case this is not just due to your minified example.
tasks.getByPath is bad, especially in case you request tasks from another project, but even without, you break taks-configuration avoidance.
And iterating over the list of files at configuration time is also not the best idea, you probably would simply want from(tasks.named('producer')) or similar

Also you seldomly really want Copy or copy { ... }, you usually want Sync or sync { ... } to wipe out stale files.

Thanks! Got it :slight_smile:

Btw. you might consider switching to Kotlin DSL. By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and amazingly better IDE support if you use a good IDE like IntelliJ IDEA or Android Studio. :slight_smile:

OK, yes it seems like a good idea.

4

hmmm, maybe we could build upon something like this (maybe using an extension) with support for a collection of strings and outputfiles:

import org.gradle.api.DefaultTask
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.OutputFile
import org.gradle.api.tasks.TaskAction

class Producer extends DefaultTask {
  @Input
  final Property<String> fileName = project.objects.property(String)

  @OutputFile
  Provider<File> outFile = fileName.map { x -> new File("/tmp/$x") }

  @TaskAction
  void produce() {
    outFile.get().text = 'a string'
  }
}

def producer = tasks.register("producer", Producer)

task consumer(type: Copy) {
  from producer
  into '/tmp/consumer'
}

producer.configure {
  fileName = 'example-file'
}
5

The configuration of outFiles below is wrong. It is set to the single file “/tmp/[file1, file2]”. How do I map fileNames to outFiles? (The expected value of outFiles should be the two-element collection [ /tmp/file1, /tmp/file2])

import org.gradle.api.DefaultTask
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.OutputFile
import org.gradle.api.tasks.TaskAction

class Producer extends DefaultTask {
  @Input
  final ListProperty<String> fileNames = project.objects.listProperty(String)

  @OutputFiles
  final ConfigurableFileCollection outFiles = project.files fileNames.map { x -> "/tmp/$x" }

  @TaskAction
  void produce() {
    outFiles.each { x -> new File(x.name).text = 'some text...' }
  }
}

def producer = tasks.register("producer", Producer)

task consumer(type: Copy) {
  from producer
  into '/tmp/consumer'
}

producer.configure {
  fileNames = [ 'file1', 'file2' ]
}
6

Hmmm, this might actually solve our original problem:

import org.gradle.api.DefaultTask
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.OutputFile
import org.gradle.api.tasks.TaskAction

class Producer extends DefaultTask {
  @Input
  final ListProperty<String> fileNames = project.objects.listProperty(String)

  @OutputFiles
  final FileCollection outFiles = project.files fileNames.map { lst ->
    lst.collect { x ->
      "/tmp/$x"
    }
  }   

  @TaskAction
  void produce() {
    outFiles.each { x ->
      x.text = "some text...\n"
    }
  }
}

def producer = tasks.register("producer", Producer)

task consumer(type: Copy) {
  from producer
  into '/tmp/consumer'
}

producer.configure {
  fileNames = [ 'file1', 'file2' ]
}
7

The consumer project really configures a list of strings, not a list of files.

Well, then maybe a ListProperty<String>?
Or alternatively a function that you call with the String(s) as argument and the extension then handles or records them accordingly.

In Kotlin DSL the producer task could for example be something like

var fileNames = objects.listProperty<String>()
val producer by tasks.registering {
    outputs.files(fileNames.map { it.map { fileName -> "/tmp/producer/${fileName}" } })

    doLast {
        outputs.files.forEach { it.writeText("${it.name} contents") }
    }
}
fileNames.addAll("foo.txt", "bar.txt")

Btw. task producer(type: DefaultTask) { ... } is bad, it breaks task-configuration avoidance for that task, you should register tasks instead.
And when using DefaultTask you also do not need to specify any type at all.

hmmm, maybe we could build upon something like this

That’s only for one file though.

The configuration of outFiles below is wrong

x in your outFiles line is a List<String>, if you used Kotlin DSL it would have been more obvious. :smiley:

class Producer extends DefaultTask {
    @Input
    final ListProperty<String> fileNames = project.objects.listProperty(String)

is way more boilerplate than necessary. :slight_smile:
This is the same:

abstract class Producer extends DefaultTask {
   @Input
   abstract ListProperty<String> getFileNames()

But besides that, your last version already looks quite promising, yes. :slight_smile:

8

Well, then maybe a ListProperty<String>?
Or alternatively a function that you call with the String(s) as argument and the extension then handles or records them accordingly.

Are there any other positive effects from using an extension instead of configuring the task properties directly? aside from possibly having a nicer syntax?

Btw. task producer(type: DefaultTask) { ... } is bad, it breaks task-configuration avoidance for that task, you should register tasks instead.

OK, I did not know that. I see that it is indeed documented also :slight_smile:

And when using DefaultTask you also do not need to specify any type at all.

OK,

x in your outFiles line is a List<String>, if you used Kotlin DSL it would have been more obvious. :smiley:

yes, we should definitely consider this. We do not have the time to do it at the moment though. But it seems to make it easier to reason about the code. I guess we should be able to introduce Kotlin gradually by migrating plugins incrementally

9

Are there any other positive effects from using an extension instead of configuring the task properties directly?

More a conceptual issue.
If you for example register various tasks from your plugin you can have one extension that your consumer can configure and your plugin wires the extension properties to the task properties.
But you can of course also let the consumer configure the tasks directly.

1 Like