Dynamically loading depedencies from configuration into plugin classpath

Help/Discuss
1

The asciidoctor-gradle-plugin provides an extension to define the version of the AsciidoctorJ library to use (https://github.com/asciidoctor/asciidoctor-gradle-plugin#configuration).
This is key to allow users to use new Asciidoctor features without requiring a new plugin release.

Currently this is done in two steps:

  1. From a custom extension, the dependencies are initialized into a custom configuration (https://github.com/asciidoctor/asciidoctor-gradle-plugin/blob/development/src/main/groovy/org/asciidoctor/gradle/AsciidoctorPlugin.groovy#L55).
  2. Then, the jars in the configuration are used to initialize a new ClassLoader: https://github.com/asciidoctor/asciidoctor-gradle-plugin/blob/development/src/main/groovy/org/asciidoctor/gradle/AsciidoctorTask.groovy#L843.

Recently we have seen that this causes issues when executed in the Gradle Daemon (https://github.com/asciidoctor/asciidoctor-gradle-plugin/issues/203), because old class definitions are kept in the heap so static attributes are not guaranteed to belong to the same class.

Summing up, my question is: ‘does gradle provide a way to load libraries without playing with a classloader?’.
Reading this about defaultDependencies https://docs.gradle.org/2.5/release-notes#simpler-default-dependencies, it seems to me that the aim of that method is precisely to support what the plugin does, but no logic to load the jars is provided, or I am missing something.

2

Using an isolated classloader is correct. The task is not closing the classloader after usage, that’s why it stays in memory.

3

I tried closing the classloader at the end of the task but the problem remains.
Just to double check I even set the attribute to null and forced a GC, but nothing. Also removed the static qualifier of the classloader attribute just in case.

4

You’ll have to use a profiler to find out what is keeping a strong reference to that classloader.

5

I have tried to do use a profiler without success. Is there some official documentation on how-to?
The thing is that the Daemon process does not appear in VisuakVM and setting the arguments to enable a remote connection ’ -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.rmi.port=19988 -Dcom.sun.management.jmxremote.port=19988 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false’ did not work either. The only info I found is some outdated stackoverflow posts.

Also, 2 things: I have been able to write a code that replicates part of the logic of the plugin loading the libraries and it worked, and I noticed that the Daemon uses a few customized ClassLoaders. Afaik they should not interfere but I after all my tests I am not 100% sure it is a think with the plugin code.

6

After taking another look at the example the user gave, the problem is that he added a buildscript dependencies to asciidoctor. Thus your plugin is now loading asciidoctor from there instead of from the asciidoctor configuration (because the buildscript classloader is the parent).

You might wanna throw an exception when the asciidoctor classes are directly visible to your plugin or alternatively write a filtering classloader implementation, so the buildscript classpath doesn’t leak into your custom URL classloader.

7

Thanks a lot for the support. I just did some thorough testing and it works
Now that I see the problem and even better, I understand it, it all makes sense.

Regarding this post, for me is FIXED and finished.:+1:. On the other hand, we will keep the conversation in Github issue to see how to solve this in out plugin, we do need to control this situations. Feel free to join if you want to contribute but don’t feel obliged to.

1 Like