Tests up-to-date although compileTestJava was not (1.12)


from time to time we got problems running gradle on our CI builds:

When a test class is removed, the compileTestJava is triggered. However, the test target still has this class in the classpath, so it is either up-to-date or it will run the deleted test under build/classes/test.

When deleting the whole src/test folder, the compileTestJava is up-to-date due to

Skipping task ':project....:compileTestJava' as it has no source files.

. In this case the test runner would run all the deleted tests.

The obvious workaround would be to do a clean first, but this would kill the incremental speed-up.