Gradle with Java11 build produces intermittent compilation errors

We are using gradle:6.9.1-jdk11-hotspot image to build one of our java jar components.

Gradle 6.9.1

Build time: 2021-08-20 11:15:18 UTC.
Revision: f0ddb54aaae0e44f0a7209c3c0274d506ea742a0
Kotlin: 1.4.20
Groovy: 2.5.12
Ant: Apache Ant™ version 1.10.9 compiled on September 27 2020
JVM: 11.0.11 (AdoptOpenJDK 11.0.11+9)
OS: Linux 5.10.47-linuxkit amd64.

The Gradle build file also use io.freefair.aspectj.compile-time-weaving:3.8.4 for compile time weaving.

We get the following compilation errors

The type javax.xml.stream.XMLEventReader is not accessible.
import javax.xml.stream.XMLEventReader;

The type javax.xml.stream.XMLInputFactory is not accessible
import javax.xml.stream.XMLInputFactory;

XMLEventReader cannot be resolved to a type
private XMLEventReader xmlEventReader;

XMLInputFactory cannot be resolved
XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();

This compilation error does not happen consistently and it happens intermittently. Retrying multiple times the build succeeds.

Can someone help figuring out the issue?

Also posted in