Test class written in groovy is still executed when running test task after the class has been removed

I’m using groovy plugin,I have a class in main source set, let’s call it A, and a spock specification for it in test source set, ASpec. I run test task and see the tests passing. Then I remove them both and run test again. I’m expecting to see no tests run, instead I see the tests from the removed specification executed and they throw ClassNotFoundException for A.

An interesting fact is that compileTestGroovy sees the change, is not up to date and gets executed.

Why would compileTestGroovy be any different to compileGroovy when it comes to removed source files?

Please ignore this. It seems like it only happened on our CI and I cannot reproduce it locally. Sorry for the noise.