You’re correct, plugins aren’t isolated from each other. We’ve been moving towards this for some time, but it is unlikely to happen soon (there’s lots to do) and is going to be disruptive.
The way around this now is to use script plugins to isolate plugins. Each script can have its own buildscript {} block and has its own classloader. If you have conflicting plugins, you can use this as a boundary. As with any classloader graph system, this may require some fiddling to get things to cooperate if the dependency graphs are complex.