Gradle can't delete build folders

I have this problem with my Kotlin project. Gradle sometimes gives me an error when trying to build the project. It says it can’t delete some build folders. I can delete them manually, I also tried to use gradlew --stop command and TASKKILL /F /IM java.exe however none of them did anything. The error persists until I manually delete those folders. Also in my project, I have 1 additional module and a buildSrc module for the Gradle plugin (an error was present before this plugin). I have this error in the build folders of all 3 modules randomly.

The error:
Execution failed for task ‘:buildSrc:compileKotlin’.

java.io.IOException: Unable to delete directory ‘A:\kotlinProjects\ILIT-Library-Bot\buildSrc\build\kotlin\compileKotlin\cacheable’
Failed to delete some children. This might happen because a process has files open or has its working directory set in the target directory.
- A:\kotlinProjects\ILIT-Library-Bot\buildSrc\build\kotlin\compileKotlin\cacheable\caches-jvm\inputs
- A:\kotlinProjects\ILIT-Library-Bot\buildSrc\build\kotlin\compileKotlin\cacheable\caches-jvm\jvm\kotlin
- A:\kotlinProjects\ILIT-Library-Bot\buildSrc\build\kotlin\compileKotlin\cacheable\caches-jvm\jvm
- A:\kotlinProjects\ILIT-Library-Bot\buildSrc\build\kotlin\compileKotlin\cacheable\caches-jvm\lookups
- A:\kotlinProjects\ILIT-Library-Bot\buildSrc\build\kotlin\compileKotlin\cacheable\caches-jvm

The structure of my project:

I can also send the content of my gradle files and a full stack trace of the error.

I would appreciate any help with this.