Best approach for a build plugin to configure another non gradle plugin

What is the best / sound approach to make a build plugin, not a kotlin dsl one, (from buildSrc or included-build) type-safely configure an optional (projects may apply it or not) external (non gradle) plugin ?

Case at hand, I have a “binary” plugin implemented in buildSrc/ (it cannot be moved into included build yet for other reasons). It configures the JVM used by tests according to a command-line property. When the JMH plugin is also applied, I want the same selected JVM to be assigned to JmhParameters.jvm property.

The important constraints are:

  • JMH is only applied to some projects
  • My plugin must not apply JMH.
  • The consuming build apply JMH with an alias.
  • I would like to use JMH’s typed API rather than reflection (or groovy builder).

The Gradle documentation recommends reacting to optional plugins using pluginManager.withPlugin(id), describing this approach as order-independent and safe:

But the doc is not enough, as mentioned in Class loader hierarchy cause plugin reacting failed · Issue #33305 · gradle/gradle · GitHub

Minimal reproducer

The project has this structure:

  .
  ├── settings.gradle.kts
  ├── build.gradle.kts
  └── buildSrc
      ├── build.gradle.kts
      └── src/main/kotlin/com/example
          ├── TestJvmConstraintsPlugin.kt
          └── JmhTestJvmIntegration.kt

./settings.gradle.kts

pluginManagement {
  repositories {
    gradlePluginPortal()
    mavenCentral()
  }
}

rootProject.name = "optional-jmh-integration"

./build.gradle.kts

  plugins {
    id("me.champeau.jmh") version "0.7.3"
    id("com.example.test-jvm-constraints")
  }

buildSrc/build.gradle.kts

  plugins {
    `kotlin-dsl`
    `java-gradle-plugin`
  }

  repositories {
    gradlePluginPortal()
    mavenCentral()
  }

  dependencies {
    compileOnly("me.champeau.jmh:jmh-gradle-plugin:0.7.3")
  }

  gradlePlugin {
    plugins {
      create("testJvmConstraints") {
        id = "com.example.test-jvm-constraints"
        implementationClass = "com.example.TestJvmConstraintsPlugin"
      }
    }
  }

The main plugin contains no references to JMH types:

  package com.example

  import org.gradle.api.Plugin
  import org.gradle.api.Project

  class TestJvmConstraintsPlugin : Plugin<Project> {
    override fun apply(project: Project) {
      project.pluginManager.withPlugin("me.champeau.jmh") {
        JmhTestJvmIntegration.configure(project)
      }
    }
  }

The JMH-specific typed configuration is isolated in another class:

  package com.example

  import me.champeau.jmh.JmhParameters
  import org.gradle.api.Project
  import org.gradle.kotlin.dsl.configure

  internal object JmhTestJvmIntegration {
    fun configure(project: Project) {
      project.extensions.configure<JmhParameters> {
        jvm.set("/path/to/java")
      }
    }
  }

Running: ./gradlew help --stacktrace fails during configuration:

  java.lang.NoClassDefFoundError: me/champeau/jmh/JmhParameters
      at com.example.JmhTestJvmIntegration.configure(JmhTestJvmIntegration.kt:9)
      at com.example.TestJvmConstraintsPlugin$apply$1.execute(TestJvmConstraintsPlugin.kt:9)
      at com.example.TestJvmConstraintsPlugin$apply$1.execute(TestJvmConstraintsPlugin.kt:8)
      at org.gradle.api.internal.plugins.DefaultPluginManager$2.execute(DefaultPluginManager.java:261)
      ...
  Caused by: java.lang.ClassNotFoundException:
      me.champeau.jmh.JmhParameters

The callback has been compiled into the separate class TestJvmConstraintsPlugin$apply$1, and the JMH types have also been isolated in JmhTestJvmIntegration. However, both classes are still loaded by the buildSrc classloader.

My understanding is that the buildSrc classloader is a parent of the project plugin classloader. Consequently, it cannot see the JMH classes loaded by its child, even though withPlugin(...) only invokes the callback after JMH has been applied.

Changing the dependency to:

  dependencies {
    implementation("me.champeau.jmh:jmh-gradle-plugin:0.7.3")
  }

makes the JMH types visible to the buildSrc/ plugin, but the consuming build can no longer declare the JMH plugin version (Error resolving plugin [id: 'me.champeau.jmh', version: '0.7.3'], The request for this plugin could not be satisfied because the plugin is already on the classpath with an unknown version, so compatibility cannot be checked.).
Note this one might be a specific buildSrc/ gotcha.

The documentation for applying external plugins from precompiled script plugins also prescribes an implementation dependency and says the dependency declaration owns the plugin version:

However, this plugin does not want to apply JMH. It only wants to configure it when the consuming project has independently applied it.

Reflection workaround

The following works because it does not “link” against any JMH class:

  project.pluginManager.withPlugin("me.champeau.jmh") {
    val jmh = project.extensions.getByName("jmh")

    @Suppress("UNCHECKED_CAST")
    val jvm =
      jmh.javaClass.getMethod("getJvm").invoke(jmh) as Property<String>

    jvm.set(
      testJvmSpec.javaTestLauncher.map {
        it.executablePath.asFile.absolutePath
      }
    )
  }

It does, however, lose the compile-time safety that would normally be expected when both plugins expose public APIs. In my case the API surface is ok, and unlikely to change. But it raises question when the plugin surface is bigger or less stable.

Questions

  1. Is there a way to have type-safe way for a buildSrc/ binary plugin to configure an optional external plugin without also “owning” that plugin’s version?
  2. it seems to me pluginManager.withPlugin(id) only guarantees application timing, without providing any classloader visibility between the two plugins?
  3. Is separating the integration into JmhTestJvmIntegration expected to make no and clearly do not help with class loading, i.e. an intuitive code is still defined and loaded by the buildSrc classloader?
  4. Would moving the plugin from buildSrc/ into an included plugin build allow the two plugins to participate in normal plugin dependency resolution, or would consumers still need to arrange for both plugins to be loaded in the same classloader scope? (I need to try that in isolation)
  5. If reflection is the only way to preserve version ownership in this situation, I would stress again this limitation should be mentioned alongside the documentation describing withPlugin(...) as the safe way to integrate with an optional plugin. As mentioned in Class loader hierarchy cause plugin reacting failed · Issue #33305 · gradle/gradle · GitHub

By the way, I forgot to link it, but this discussion seems related Add configuration when plugin is applied, handling different plugin versions - #10 by maxnitze

App you said before the questions was analyzed correctly.

The proper way is to have implementation dependency and remove the version from where you apply it. The version is only needed to add the plugin to the classpath, but the having an implementation dependency in buildSrc you added it already by adding to a parent class loader as you mentioned, so the version is simply unnecessary anyway.

This problem is not exclusive for buildSrc, but also for other constellations. If you for example have

  • plugin A in the classpath of the root project build script
  • plugin B in the classpath of the sub project build script, but not in the classpath of the root project build script
  • plugin A reacts to plugin B being applied
  • A and B are applied to the sub project

Then you have the exact same problem.
A is in the class loader of the root project, B is in the classloader of the sub project which is a child of the other one, so A cannot see the classes of B. So in that case you would pull up B to the root project build script classpath when when not applying it to the root project, so that A can see B’s classes. And just the same you do by having it as implementation dependency in buildSrc.

The only difference is, that with the root project / sub project example, but could have a version as Gradle there knows the versions and same version is ok, but also there you could just leave out the version in the sub project. But if you got example use a version catalog, this is not easily possible.

Is there a way to have type-safe way for a buildSrc/ binary plugin to configure an optional external plugin without also “owning” that plugin’s version?

I don’t think so.
Well, yes there is it I have the classloader hierarchy in mind correctly.
You could add the plugin to the settings script classpath as that should be a parent of the buildSrc one, so the classes in buildSrc can see the plugin classes at time and compileOnly would be fine.

it seems to me pluginManager.withPlugin(id) only guarantees application timing, without providing any classloader visibility between the two plugins?

That’s correct.
Such “classloader visibility” would also not be possible technically I think.

Is separating the integration into JmhTestJvmIntegration expected to make no and clearly do not help with class loading, i.e. an intuitive code is still defined and loaded by the buildSrc classloader?

I’m not sure I got the question correctly, but I think the answer is “yes”. Well, if a parent class loader or the same class loader provides the class, then it doesn’t matter where that class is used.

Where it makes a difference is when the plugin is neither on the classpath nor applied. Because if it is not on the classpath the class with the withPlugin might have problems loading due to the missing classes if it did the configuring directly. If it does it in the separate class, then that class is not loaded as that plugin is not applied.

But either way, the class is in the same class loader. If it would somehow magically be loaded with the child classloader so that it can access the other plugin’s classes, then your class in the parent classloader could already not access that class and thus not instantiate / call it.

Would moving the plugin from buildSrc/ into an included plugin build allow the two plugins to participate in normal plugin dependency resolution, or would consumers still need to arrange for both plugins to be loaded in the same classloader scope? (I need to try that in isolation)

As described above with the root and sub project example with A and B, this is not exclusive to buildSrc and could happen the same. You (or actually your consumer) always have to make sure that plugin B is in the same or a parent classloader of plugin A if plugin A wants to use classes from plugin B. Especially when the consumer has to control the version of B. If the consumer does not have to control the version of B like in your example where everything is in one composite build anyway, then A can just depend on B even if not applying it.

By the I forgot, but this discussion seems related

Yes, that’s the exact same conversation :slight_smile:

Indeed. I think some better docs are overdue in this matter.