[Gradle 5.3] Gradle does not update stale files in build folder

I’m constantly switching branches in my source folder. Now I reached a state, where the sources would not contain a specific resource, but the build output of Gradle (build/resources/main) still contains the file. When I run ./gradlew test --fail-fast, Gradle would not remove the stale resource. Instead it runs the tests using the stale files (which in turn causes tests to fail).

Any suggestions on how to diagnose the root cause (Gradle no detecting a removed resource), so I can create a proper bug report? Any modifications to other files in the sources folder will trigger a build output update.