Gradle still runs junit test when last test source file is removed

Steps to reproduce 1. Create a failing junit test, in a project without any tests 2. run gradle test

  1. remove the test (i.e. remove the java source file) 4. run gradle test

Since there are no java source files to compile, compileTestJava does nothing (although it really should remove the .class file) Since gradle will find the old .class file it will still run the test, even though the java source file is removed. It does not help if I add --rerun-tasks.

This leads to problems in continuous integration environments like Jenkins