Specifying Gradle version and Java toolchain version isn't enough

I just encountered a curious issue… maybe not a bug, but it caught me off guard. I figured that using a build.gradle file with:

java {
    toolchain {
        languageVersion.set(JavaLanguageVersion.of(17))
    }
}

and the gradle wrapper to specify Gradle 7.2, that my build should work consistently on different machines. It does not.
It is still very much dependent on what version of the JVM is running the gradle daemon. Otherwise you can get errors like this:

FAILURE: Build failed with an exception.

* What went wrong:

java.lang.UnsupportedClassVersionError: org/openjfx/gradle/JavaFXPlugin has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

> org/openjfx/gradle/JavaFXPlugin has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

Is there a standard way of telling Gradle that the daemon itself must be running with a specific minimum JVM version? Something that can be put in the project build script so it doesn’t rely on setting up the machine just right before hand?

Since the error is triggered by applying a plugin, I have come up with this early check. Is there a better way? Is there a way to get Gradle to launch a daemon running on the correct JVM version so it all “just works” ?

build script {
  if (!JavaVersion.current().java11Compatible) {
    throw new GradleException("Gradle daemon must be running on Java 11 or greater.")
  }
}
plugins {
  id 'application'
  id 'org.openjfx.javafxplugin' version '0.0.8'
}