Understanding Classloader issue

Help/Discuss
1

Hi, I’ve had an issue that I know how to solve, but I don’t understand why it’s solved.

A project applies a custom plugin for code generation. Inside a task from this plugin, I run into an issue where a class couldn’t be initialized because it used the wrong protobuf-java version. The project declares protobuf-java:3.23.2, but the classloader returned 3.17.2.

The following logs prints resolving a protobuf java class from each classloader, bottom-up. (I know that children delegate resolution to parents)

java.net.URLClassLoader@179cf3cb -> protobuf-java-3.17.2.jar
        
ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings[:]:buildSrc[:]:root-project[:]:project-app(export)} -> protobuf-java-3.17.2.jar
               
ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings[:]:buildSrc[:]:root-project[:](export)} -> protobuf-java-3.17.2.jar

ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings[:CodegenConventionPlugins]:buildSrc[:CodegenConventionPlugins](export)}) -> Not found

The project where the plugin is applied

// settings.gradle

pluginManagement {
    ...

    plugins {
        id 'com.android.library' version "7.4.2"
    }
}

// build.gradle

plugins {
    id 'com.android.library' apply false
}

// app/build.gradle

plugins {
    id 'com.android.library'
}

dependencies {
        implementation("com.google.protobuf:protobuf-java:3.23.2")
}

If I remove the plugins block from the root build.gradle, things work because protobuf-java:3.17.2 is not in the classloader.

java.net.URLClassLoader@8c350d -> protobuf-java-3.23.2.jar

ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings[:]:buildSrc[:]:root-project[:]:project-app(export)} -> protobuf-java-3.23.2.jar

ClassLoaderScopeIdentifier.Id{coreAndPlugins:settings[:CodegenConventionPlugins]:buildSrc[:CodegenConventionPlugins](export)}) -> Not found

There’s no root-project ClassLoader, but I don’t understand why.

I know that com.android.library has a dependency on protobuf-java:3.17.2, but I would like to understand at a deeper level how things work.

Also, are there other/better ways to enforce a transitive dependency version on plugin dependencies? This misuse of protobuf version went undetected for a long time because there wasn’t any breaking change until 3.22

2

Actually I wonder more why the implementation dependency on 3.23.2 at all influences the plugin classpath.
That dependency is a production dependency and should usually not interfere with the plugin class path.

That there is not “root-project” class loader anymore is probably simply because there is no need for it. You neither declare plugins in the plugins { ... } block, nor have any classpath dependencies in the buildscript { ... } block of the root project, so why waste time with an empty root-project class loader that would just forward to the parent anyway.

As you use buildSrc, you could declare the dependencies you want to enforce there and there use any normal means like version constraints, or a platform or whatever. As buildSrc is the nasty special beast it is, it is prepended to all build script classpaths by being in a parent class loader, so the versions resolved there should always win.

1 Like
3

Thanks for your answer, as always!

I’m using precompiled script plugins, which I imagine is a form of buildSrc

// settings.gradle

pluginManagement {
    includeBuild "<path>/CodegenConventionPlugins"

    includeBuild("<path>/CodegenPlugin")

    ...
}

I did try to add protobuf-java to CodegenConventionPlugins project, but it didn’t fix anything. I too was expecting that to fix the issues, once I discovered where the culprit was.

Also, CodegenPlugin, or at least some of its modules, already declares a protobuf-java dependency.

4

I’m using precompiled script plugins, which I imagine is a form of buildSrc

… yeah … no

“precompiled script plugin” just means you write plugins as ...gradle.kts files which then are precompiled into binary plugins.
Whether you have them in buildSrc or an included build has nothing to do with them being precompiled script plugins.

I really meant only buildSrc as it is always special, and as you had the buildSrc class loader in what you showed, I thought that is what you use.
In an included build it is different.

Also, CodegenPlugin , or at least some of its modules, already declares a protobuf-java dependency.

Ah, ok, that was missing from the picture.
So the implementation dependency indeed is irrelevant as it should be. :slight_smile:

But then things are pretty clear.
As you use an included build - which I also prefer over buildSrc for various reasons - the dependencies take part in normal conflict resolution and class loader hierarchy.

Where you apply your convention plugin, the protobuf dependency takes part in conflict resolution.
You didn’t show it, but I guess you apply it in app/build.gradle.
So for the app/build.gradle classpath you have conflict resolution between the protobuf coming from your convention plugin and the protobuf coming from com.android.library and your version wins as it is higher. So the intended protobuf version is on that class loader.

The problem is, that then the class loader hierarchy kicks in.
As the parent project class loader is a parent of the subproject class loader, the subproject class loader asks the parent where your protobuf version is not included.
So there the com.android.library one won the “conflict resolution” as there was no conflict and thus is the one in the class loader and thus wins when you get the class.

By removing com.android.library from the root project, protobuf is not in the parent class loader anymore, so the class loader of the subproject wins and you get the intended version.

Another mitigation option would be to also add your convention plugin to the root project classpath by also apply false it there. Then again you get conflict restolution and your version would win and be supplied.

Another mitigation option is to have the dependency as dependency in buildSrc, as the buildSrc dependencies and code is in a class loader that is the parent of the root project class loader and thus again would win over the root project class loader.
This does not mean you need to put your convention plugins into buildSrc.
You could still have your convention plugins in the included builds and just use an otherwise empty buildSrc to define the dependencies that then win over the root project classpath / class loader.

… yes, the class loader quirks in Gradle are sometimes really nasty. :frowning:

2 Likes