Selection and scope of plugin dependencies in a multi-project build

Help/Discuss
1

Hello,

I maintain a binary plugin written in Java. It allows to integrate the build of a Javascript application inside a Gradle multi-projects build. Most of dependencies used by my plugin are applied in the implementation configuration, and consequently, they have a runtime scope in the POM artifact published. That means there’s no guarantee the same version of these dependencies is selected in end-user projects, and there may be dependency conflicts. However, I noticed the most recent version selection strategy does not take into account the version of the dependency used by my plugin (see hereafter).

To prevent conflicts, I usually recommend to isolate projects in subprojects so as the root project does not apply any plugin and/or dependencies. But I cannot explain why to end-users, because I cannot find neither official guidance in Gradle docs about that, nor a detailed explanation of how plugin dependencies are taken into account during the dependency resolution process, and their scope within the whole project.

When I explore end-user project layouts, there are 3 typical setups:

LAYOUT 1
An empty root project with a Java subproject applying plugins and dependencies, and a subproject applying my plugin only:

root-project
|__ settings.gradle.kts // include("subproject1", "subproject2")
|__ subproject1
    |__ build.gradle.kts
        |__ java
        |__ plugin(<another-plugin-id>) version "<version>"
|__ subproject2
    |__ build.gradle.kts
        |__ plugin(<my-plugin-id>) version "<my-version>"

With this layout, my plugin is unlikely to suffer a conflict with one of its dependencies: each subproject does not seem to interfere with each other.

LAYOUT 2
A Java root project applying plugins and dependencies, and a subproject applying my plugin only:

root-project
|__ settings.gradle.kts // include("subproject")
|__ build.gradle.kts
    |__ java
    |__ plugin(<another-plugin-id>) version "<version>"
|__ subproject
    |__ build.gradle.kts
        |__ plugin(<my-plugin>) version "<my-version>"

With this layout, sometimes, end-users report a dependency conflict when my plugin executes one of its task. If I refactor the root project to get something like the first layout, the dependency conflict does not appear anymore.

LAYOUT 3
A root project applying all plugins and dependencies:

root-project
|__ build.gradle.kts
    |__ java
    |__ plugin(<another-plugin-id>) version "<version>"
    |__ plugin(<my-plugin>) version "<my-version>"

With this layout, there are sometimes dependency conflicts that I understand perfectly because plugins and dependencies share the same classpath.

Q1. What is the recommended project layout: 1, 2, 3?
Q2. How a plugin, its dependencies, and Java dependencies defined in a root project apply to subprojects?
Q3. In layout 2, though my plugin uses the most recent version of a dependency, it seems an older and incompatible version in the root project is selected and still applied in the subproject. Am I mistaken?
Q4. In layout 2, the dependencyInsight task does not help to identify the cause of a conflict occuring inside a plugin task execution because it focuses on configurations rather than on plugin dependencies. How to efficiently deal with such issue in this case?

Thanks a lot for your help!
BR

2

That means there’s no guarantee the same version of these dependencies is selected in end-user projects, and there may be dependency conflicts.

That is not really correct.
It is for plugins like for normal libraries.
If someone compiles against your plugin in an own plugin project like convention plugins, then only the compile dependencies are in the downstream compile classpath.
But at runtime, i.e. when your plugin is applied, also the runtime dependencies are part of the resolution process as usual.

how plugin dependencies are taken into account during the dependency resolution process, and their scope within the whole project.

Plugin dependencies and production dependencies are completely separate things, so either should not influence the other, if that was not clear.

LAYOUT 1
With this layout, my plugin is unlikely to suffer a conflict with one of its dependencies: each subproject does not seem to interfere with each other.

Exactly, the build script classpaths are resolved separately, and their result is put to a class loader which is used for that build script and is a parent of the classloader for the classloaders of child build scripts.

LAYOUT 2
With this layout, sometimes, end-users report a dependency conflict when my plugin executes one of its task.

The problem is exactly the opposite, that you do not get a dependency conflict.
If your plugin uses library-A v2 and the other plugin uses library-A v1, then in the classloader of the subproject build script there indeed will be library-A v2, in the classloader of the root project build script there will be library-A v1. But the root project build script classloader is a parent of the subproject build script classloader. And due to classloader delegation rules, your plugin will see the v1 classes from the parent classloader.

The proper fix / work-around / whatever for this would be to for example add plugin(<my-plugin>) version "<my-version>" apply false to the root project. This way your plugin is added to the root project build script’s classpath but it is not applied. But by doing so, you now do get the dependency conflict for library-A and it will resolve to v2, making both plugins see the v2 classes. In the subproject you could then also leave out the version "<my-version>" part, as the plugin is already part of the classpath and the version is irrelevant (or gives an error if different).

LAYOUT 3
With this layout, there are sometimes dependency conflicts that I understand perfectly because plugins and dependencies share the same classpath.

No, they don’t, as described above, those are totally different and separate things that do not influence each other. Only the buildscript dependencies and plugins and their dependencies form one classpath together that might get conflicts and resolve to a common version if the conflict could be resolved properly.

What is the recommended project layout: 1, 2, 3?

Yes. :slight_smile:
All three are just fine, depending on what you need.
Where you apply which plugins should in no way influence the project layout.
The project layout is more a structural question how you want to structure your project.
The plugins you apply just support that and should just be made so that it works properly on consumer side like with my example solution for 2.

How a plugin, its dependencies, and Java dependencies defined in a root project apply to subprojects?

Java dependencies don’t at all unless the subproject has a dependency on the root project and then it is like any other library dependency.
Regarding plugins and their dependencies and build script dependencies, like I just described above.

In layout 2, though my plugin uses the most recent version of a dependency, it seems an older and incompatible version in the root project is selected and still applied in the subproject. Am I mistaken?

See above. :slight_smile:

In layout 2, the dependencyInsight task does not help to identify the cause of a conflict occuring inside a plugin task execution because it focuses on configurations rather than on plugin dependencies. How to efficiently deal with such issue in this case?

dependencyInsight is only for project dependencies, plugins and their dependencies will never appear there. The same for dependencies. What you can use is buildEnvironment which shows the buildscript classpath. But this would also partly help in this case, as it will most probably still show the latest version for the dependency as in the classpath the correct version is present, just overridden by the classes in the parent classloader as described above. Build --scans are usually also very helpful in investigating dependency stuff and so on, but same here, the classloader delegation problematic would probably not really be nicely discoverable I think.

1 Like
3

Hi @Vampire,

Fantastic helpful answer! I really appreciate it.
Thanks for all!

Best regards

1 Like