Is there a way to hook support for new types into `NotationParser`'s conversion chain?

Help/Discuss
1

How we got here:

I had a convenience method for converting a Provider<PluginDependency> to a Provider<ExternalModuleDependency>, which I’m sure people have seen before:

/**
 * Some additional syntax sugar so you can more easily convert a `libs.plugins`
 * accessor for use in the `dependencies` block.
 */
fun Provider<PluginDependency>.asDependencyProvider(): Provider<ExternalModuleDependency> =
    this.map { pluginDependency ->
        // Exploiting standard Gradle conventions for locating the plugin's
        // artifact as there does not appear to be a public way to get this info.
        // Maybe there is, probably involving reading files directly from
        // the repo...
        val pluginId = pluginDependency.pluginId
        val id = DefaultModuleIdentifier.newId(pluginId, "$pluginId.gradle.plugin")
        // Crazy that this thing _demands_ the version constraint be mutable, but OK, fine.
        val versionConstraint = DefaultMutableVersionConstraint(pluginDependency.version)
        val configuration: String? = null
        DefaultExternalModuleDependency(id, versionConstraint, configuration)
    }

The usage looks like:

dependencies {
    implementation(libs.plugins.kotlin.multiplatform.asDependencyProvider())
    implementation(libs.plugins.compose.multiplatform.asDependencyProvider())
    implementation(libs.plugins.compose.compiler.asDependencyProvider())
    implementation(libs.plugins.dokka.asDependencyProvider())
    implementation(libs.plugins.kotest.asDependencyProvider())
    implementation(libs.plugins.ksp.asDependencyProvider())
}

I imagine some people just duplicate the entries in their libs.versions.toml file and go on with their day, especially when it’s for work, but I rather enjoy removing unnecessary duplication in my build. So, after some time, I decided I didn’t want to have so many of these .asDependencyProvider() things, so I started trying to golf down the API. Initially, I made some convenience overloads:

fun DependencyHandler.add(
    configurationName: String,
    dependencyNotation: Provider<PluginDependency>,
) {
    // This will always return `null`, which is why we didn't bother returning it
    add(configurationName, dependencyNotation.asDependencyProvider())
}

fun DependencyHandler.implementation(dependencyNotation: Provider<PluginDependency>) {
    // These parameter names are LOAD BEARING
    add(configurationName = "implementation", dependencyNotation = dependencyNotation)
}

So now you can drop the extra calls:

dependencies {
    implementation(libs.plugins.compose.multiplatform)
    implementation(libs.plugins.compose.compiler)
    implementation(libs.plugins.dokka)
    implementation(libs.plugins.kotest)
    implementation(libs.plugins.ksp)
}

And it works, but not without some quirks:

  • Hard-coding implementation is technically cheating because who knows whether a configuration with that name exists, really. (Well, I guess for a Gradle plugin build, it does, but it still feels bad to be applying that plugin when you don’t really need to…)
  • It’s super easy to fail to import the overloaded method, and you get no compiler warning when it happens, just a failed build the next build. IDEA, at least, is not the best at method overloads being defined in more than one place. (Or not the best at extension methods which happen to be named like the other methods, however you want to phrase that.)
    • I even shot myself in the foot with this while implementing it - turns out Kotlin prefers to call the Any overload on the actual interface, even if I’m trying to call a more specific overload defined in the same source file as the caller. I got around it by adding parameter names, but there is no way that will hold forever. If Gradle ever rewrites that in Kotlin, and they happen to choose the same parameter names? Bang.
  • I would still need to write a heck of a lot more to complete it.
    • Example: create would be nice sometimes

This led to thinking:

⇒ Why can’t I just override create somehow?

⇒ Hang on, Gradle has all sorts of different things it can convert into a Dependency, surely they didn’t do that in one big if-else statement

⇒ Oh, NotationConverter and NotationParser exist

⇒ Oh, they use some kind of pluggable service registration?!

Anyway, my excitement was stomped when I discovered that the actual NotationParser used at runtime seemed hard-coded with only half a dozen or so supported input types. But there are certainly a lot of layers of indirection in how those things are being created, so:

  1. Is there any way to plug into this, at any level? (I know it’s internal API, but sometimes the benefits do outweigh the drawbacks)
  2. Is there some other way to get the same sort of behaviour? (The best idea I was able to pull out of any LLMs was to use global init scripts, and yeah, with that, I can certainly apply plugins that might be able to get it to work, but there isn’t any clean way to have those per-project, is there?)
  3. Is Gradle’s lack of automatic conversion for this just an oversight, or was it done deliberately? I figure keeping the interfaces separate was deliberate, but it surely didn’t take long until one of Gradle’s own developers hit this. I mean, they make more plugins than almost anyone else… surely?

Maybe some plugin like kotlin-dsl or java-gradle-plugin would be a place for this sort of machinery, because it really seems to be something that you only ever want when writing Gradle plugins. But maybe it’s just something that was never implemented because the hook doesn’t exist yet.

2

The only way I’m aware of, is using a custom Gradle distribution where you modified that code to do what you want.

I actually wonder why in your first snippet you use that many internal classes when exactly 0 are necessary, so no, never seen such voice specifically.

That you cannot use the catalog entry as dependency indeed is a missing feature tracked at Accept plugin declarations from version catalog also as libraries · Issue #17963 · gradle/gradle · GitHub. So to have it like the built-in support most probably will be in the future, have a plugin(...) function that maps the given provider.

And just map it to a String, then you don’t need any internals. :slight_smile:

3

Yeah, when it was still living in the build script, I was just using a string, because there was nobody else who would see it other than the local dependencies block, but when functions move to an area where other people can use them, you start getting all sorts of inconvenient questions like “why is string the best option in a language with this much type safety?”

I did end up having a bit of a dig to figure out the “correct-correct” way of getting this mapping, which either uses 3-4 times more internal calls, or duplicates the same amount of internal API in my own project.

I also went and looked at a bunch of other projects, and it really does look like most are just manually inserting a library entry to match the plugin entry. At least version.ref mitigates the main risk.

4

why is string the best option in a language with this much type safety?

The answer is “this is about API, not language”, or “ask Gradle” :man_shrugging:

I did end up having a bit of a dig to figure out the “correct-correct” way of getting this mapping, which either uses 3-4 times more internal calls, or duplicates the same amount of internal API in my own project.

Any way using internal API is not “correct” imho. The “correct-correct” was is to use string.

it really does look like most are just manually inserting a library entry to match the plugin entry. At least version.ref mitigates the main risk.

I’ve seen both :man_shrugging:

5

There are supposedly cases where it isn’t just appending something to make the artifact name, so I am not even sure how you would programmatically build that string correctly without using any internal APIs. Projects can override the behaviour at all sorts of intercept points too.

That is, unless I’m missing some wild, new API that makes this easy.

6

Not sure what you mean.
It is always <plugin id>:<plugin id>.gradle.plugin:<plugin version>.
That’s also how Gradle itself translates a plugins block entry to a dependency.

7

This or similar is what I usually use:

fun DependencyHandlerScope.plugin(plugin: Provider<PluginDependency>) = _plugin(plugin)
fun GradleDependencies.plugin(plugin: Provider<PluginDependency>) = _plugin(plugin)

private fun _plugin(plugin: Provider<PluginDependency>) = plugin.map {
    val version = if (it.version.requiredVersion == "embeddedKotlinVersion") embeddedKotlinVersion else it.version.requiredVersion
    "${it.pluginId}:${it.pluginId}.gradle.plugin:$version"
}