I am currently writing a gradle plugin. In this plugin, I need to use a native library for performance reasons. The library is written in Rust and provides an JNI interface. I am loading the native library via System.load("<lib-path>") and the library itself can be called from Kotlin programs without any problems.
However, there seems to be a clash with the gradle daemon:
I applied the plugin to a small app for testing. Building the app with
./gradlew assembleRelease
works like a charm at first. However, republishing the plugin to the local maven and then reapplying it leads to:
UnsatisfiedLinkError: Native <PATH-TO-LIB> already loaded in another classloader
Following the idea from this source, I could implement a check if the library was already loaded and skip the loading step.
Yet, while the plugin then starts correctly, once it tries to call the native library, it throws an error:
That’s a problem with the Gradle daemon being reused for multiple builds.
If the second build runs, it cannot load the native library as it was loaded in another classloader already.
And iirc you can only access that library from the classloader where you loaded it or probably child classloaders which is the reason for the UnsatisfiedLinkError you get second.
I guess the best solution would be to not load the library within the Gradle daemon, but instead use javaexec { ... } or exec { ... } to execute the code that needs to use this library in a separate process so that there is no problem with repeated builds.
That was also my suspicion, although I am rather new to developing Gradle plugins, so I was not sure.
But that kind of defeats the purpose of using a native library and adds an extra layer of complication that seems a bit unnecessary. It should be possible to simply use a native library in a Gradle plugin.
Thank you for your insight anyhow! This seems like an issue in Gradle itself, that is probably not easy to get around (as ‘unloading’ of libraries does not seem to be a thing).
That was also my suspicion, although I am rather new to developing Gradle plugins, so I was not sure.
That is not specific to developing Gradle plugins.
That is a general limiation of Java itself.
The Gradle-specific part is just, that your plugins are loaded in specific classloaders and that multiple build runs are run on the same Gradle daemon and thus the same JVM instance.
Actually your plugin could be applied to multiple projects or even multiple builds when using composite builds when doing one build run, so even without the Gradle daemon you could run into issues easily.
But that kind of defeats the purpose of using a native library and adds an extra layer of complication that seems a bit unnecessary. It should be possible to simply use a native library in a Gradle plugin.
Thank you for your insight anyhow! This seems like an issue in Gradle itself, that is probably not easy to get around (as ‘unloading’ of libraries does not seem to be a thing).
Again, this is not an issue or limitation of Gradle, but of Java itself.
Maybe Gradle could provide some facility to load a native library in a classloader that is the parent of all classloaders of all builds that could run on that agent.
But even then you could easily run into problems.
If you use a library that uses native libraries, you might not be able to make it not load the library itself. Or if different version of your plugin need different versions of the same native library and different builds using your plugin use different versions of your plugin you could again run into problems easily.
Feel free to open a feature request on the Gradle issue tracker to request some supporting Gradle functionality for this, but to be honest, I would expect it to be addressed, especially as it is rather unusual to use native libs from Java code and so other topics will most probably get more priority.