Multi-level Plugin DSL with Multiple Collection Properties

Help/Discuss
1

I want to create a plugin in Kotlin that has behavior to the maven-publish plugin (let’s call it foo).
The main feature is to produce a set of tasks which are the product of two collections:

foo { 
   targetDir = layout.projectDirectory.dir("output-dir")

   dim1 {
      create<Dim1>("G") {
          x = 1
          y = "dog"
      }
      create<Dim1>("H") {
          x = 2
          y = "cat"
      }
  }

   dim2 {
      create<Dim2>("X") {
          z = 1
      }
      create<Dim2>("Y") {
          z = 2
      }
  }
}

The goal is for this to cause the foo plugin to generate 4 tasks: taskGX, taskGY, taskHX, and task HY.

I wrote a plugin using the following code fragments:

Dim1.kt

abstract class Dim1 @Inject constructor(val name: String) {
    abstract val x: Property<Int>
    abstract val y: Property<String>
}

Dim2.kt

abstract class Dim2 @Inject constructor(val name: String) {
    abstract val z: Property<Int>
}

 override fun apply(project: Project): Unit = project.run {
        val extension = project.extensions.create<FooExtension>("foo")
        with (extension as ExtensionAware) {
            val dim1 = project.extensions.create<Dim1>("dim1")
            val dim2 = project.extensions.create<Dim2>("dim2")

            tasks {
                  register<FooTask>("task{dim1.name}{dim2.name}") {
                      ...
                  }
             }
       }
  }

This works but dim1 and dim2 are not containers.
I want to create two containers of data objects (one for Dim1 and one for Dim2)
as described in Implementing Binary Plugins
My problem (I think) is that the example shows a Java example and I want to use Kotlin.
I believe I need to somehow replace the project.extensions.create<Dim1>("dim1") with something like project.extensions().add("dim1", dim1Container).

 override fun apply(project: Project): Unit = project.run {
        val extension = project.extensions.create<FooExtension>("foo")
        with (extension as ExtensionAware) {
            val dim1Container: NamedDomainObjectContainer<Dim1> =
                project.container(Dim1::class)
            project.extensions.add("dim1", dim1Container)

            val dim2Container: NamedDomainObjectContainer<Dim2> =
                project.container(Dim2::class)
            project.extensions.add("dim2", dim2Container)

            tasks {
               // product over the two containers
                  register<FooTask>("task{dim1.name}{dim2.name}") {
                      ...
                  }
              }
           }
       }
  }

I get this error
No type arguments expected for fun create(name: String!, configureClosure: Closure<(raw) Any!>!): T!

That is where I get lost.
Can I get some advice?

2

Let’s start with your domain objects, you already let Gradle do some boilerplate, but you can even reduce it further like this:

interface Dim1: Named {
    val x: Property<Int>
    val y: Property<String>
}

The same you can do for the extension, assuming it looks similar.

You can also declare FooExtension to implement ExtensionAware explicitly.
Gralde will take care of the remaining boilerplate and it makes it ExtensionAware anyway, so you can also declare it, then you do not need to cast it but can directly use its methods.

This works but dim1 and dim2 are not containers.

Of course not, why should they?
They are exactly what you declared, extensions on the project with name dim1 and dim2.
So they are also not nested in your extension.
You could of course add extensions to your extension of the containertype you want to have, or you simply add properties to your extension of the intended type.
As usual, Gradle will take care of the remaining boilerplate automatically.

My problem (I think) is that the example shows a Java example and I want to use Kotlin.

The APIs you use are basically the same, just the syntax is a bit different, so I don’t think so.

I believe I need to somehow replace the project.extensions.create<Dim1>("dim1") with something like project.extensions().add("dim1", dim1Container) .

Both create an extension on project and not something within your extension.
You could use it inside your extension, but also outside, just like you can but java { ... } inside dependencies { ... } it’s just unnecessary visual clutter to do so.

As I said, easiest is you just give your extension accordingly typed properties and that’s it.

If you really want to add it dynamically, you probably want something like what you tried, just not on the project.

I get this error
No type arguments expected for fun create(name: String!, configureClosure: Closure<(raw) Any!>!): T!

That is where I get lost.
Can I get some advice?

If you applied the kotlin-dsl plugin or depended on kotlinDsl() as dependency, you just miss the import of org.gradle.kotlin.dsl.create, but your IDE should help you with that. If neither of the formers and you also don’t want to, then you do not have the Gradle Kotlin DSL helpers and have to use the not-so-Kotlin-y Java methods like extensions.create("foo", FooExtension::class.java).

3

Just to be clear, I was not expecting the first version to treat dim1 and dim2 as containers; I was trying to show two versions of apply, one with containers and one without.

The suggestion you made about implementing Named has helped.
I made a git project to track the changes and to have a complete project to discuss.

I am trying to generate the names for tasks based on the DSL in the plugin.
However, it appears that the values needed to construct the task names are not known at
the point in the configuration phase where I want to register the tasks.

Here is part of the log from running gradlew on the greeting task.

dim1 container size: 0
dim2 container size: 0
tasks context: container size dim1: 0,  dim2: 0
greeting context: container size dim1: 2,  dim2: 2
greeting context: container name dim1: G, dim2: X
greeting context: container name dim1: G, dim2: Y
greeting context: container name dim1: H, dim2: X
greeting context: container name dim1: H, dim2: Y

These all seem to be written during the configuration phase.
Are there multiple phases within the configuration phase?

You can see that from the tasks (taskScope) context the DSL properties are not yet populated.
However, within a task register context they are populated.
The problem is that I need to produce the task names from the tasks context (I think).

Usage of the DSL in a build.gradle.kts

4

No, there is only one configuration phase.
But still things are running in certain order of course.
Actually, I think you greatly misinterpret the logging you added.
The “contexts” how you call them (they are not, not in the way you might envision) where the size is 0 is, because when you apply the plugin, you create the extension and everywhere you get the 0 is, because you immediately read the contents of the containers, before any consumer had the chance to add contents to the containers.

The “greeting context” how you call it, simply is a deferred configuration according to task-configuration avoidance. That means this code is executed as soon as the task greeting is configured, but also only if the task greeting is going to be executed and thus configured. (Or if task-configuration avoidance is broken somewhere, which can happen pretty fast in a consumer build). At this point you get the elements added to those collections so far at the point the greeting task is configured, but not anything maybe added later.

But this actually is only valid for your size logging output, because then you use all on the container, which will handle all currently existing and also all future added elements. So in your case, you would indeed handle all dimension elements, but only as soon as the greeting task is configured, and only if it is configured at all.

So you should ignore your misleading size-logging and instead move your all calls up to line 34 for example.

The name value btw. is the only value you can safely access at that point in time though.
The properties of the dimension elements will not be set yet and you should only wire them to other properties or query them at execution time.

If you need those values at configuration time there are other patterns you could use. (Spoiler: afterEvaluate is not the solution, never, don’t use it even if you find it online)

5

Thanks for your help.
The root problem was that

dim1Container.all {
            val dim1 = this
}

was picking up the wrong all.
Specifically, not the all from DomainObjectCollection but the all from

public inline fun <T> Iterable<T>.all(predicate: (T) -> Boolean): Boolean {
    if (this is Collection && isEmpty()) return true
    for (element in this) if (!predicate(element)) return false
    return true
}

Somehow the following caused the proper all to be selected.
I changed all to configureEach which compiled successfully;
then, changed it back to all.

dim1Container.configureEach {
            val dim1 = this
}

Which now uses the proper implementation.
I am guessing my IDE did something but I sure do not see what it did.

6

Your IDE did nothing.
When you changed to configureEach you removed those ill true statements that made the lambda a predicate, and when you switched back to all you kept it absent which made it work.

7

You are right, that is exactly what happened.

That raises related questions.

In kotlin, is there a tidy way to do the following?:

dim1Container.configureEach {
     val dim1 = this
      ...do something with dim1...
}

The following gives an error:

dim1Container.configureEach {  dim1 ->
      ...do something with dim1...
}

Type mismatch: inferred type is (Dim1).(Any?) -> Unit but (Dim1).() -> Unit was expected

Does configureEach differ from all in an important way?
Should configureEach be preferred?

8

In kotlin, is there a tidy way to do the following?:

Sure, leave out that line and just use this explicitly or implicitly?
What should be un-tidy with that?
If you need it somewhere another this shadows it, you can either do like you showed, or use this@configureEach or configureEach foo@{ ... } together with this@foo inside the block.

The following gives an error:

It gives an error because you apply the kotlin-dsl plugin which among other things configures the sam-with-receiver plugin. Without that convenience-syntax plugin you would have to do dim1Container.configureEach { it.foo = "bar" } or dim1Container.configureEach { dim1 -> dim1.foo = "bar" }, with it you can just do dim1Container.configureEach { foo = "bar" }, it makes that sole argument the receiver instead, so this instead of it.

Does configureEach differ from all in an important way?

Yes, pretty much. configureEach is lazy and only executes for elements that are realized due to some other reason, but never in itself causes an element to be realized.
all realizes all elements immediately.
You can read more about it on Avoiding Unnecessary Task Configuration.
For the built-in containers such lazy handling where it is relevant is afaik currently only the tasks container.
For all other containers it is just more consistent and / or future-proof to use it.

In your case, using configureEach could have unexpected effect.
Because if your consumer also has the habit of using the lazy-safe methods and uses register, not create to add a dimension element, then noone ever causes the element to be realized and thus your configuration never be done.

Should configureEach be preferred?

As explained, heavily depends on the intended effect.

1 Like
9

Thanks, that is what I was looking for.

dim1Container.all dim1@{
      ...do something with this@dim1...
}
1 Like