That was also my suspicion, although I am rather new to developing Gradle plugins, so I was not sure.
But that kind of defeats the purpose of using a native library and adds an extra layer of complication that seems a bit unnecessary. It should be possible to simply use a native library in a Gradle plugin.
Thank you for your insight anyhow! This seems like an issue in Gradle itself, that is probably not easy to get around (as ‘unloading’ of libraries does not seem to be a thing).