Java 11 - Test Detection Logic - initializationError

Help/Discuss
1

I have a JDK 8 project that I am attempting to migrate to JDK 11, and I’m having a problem where Gradle’s test detection logic is behaving differently and is capturing static inner classes inside test classes that are not meant to be tested. This is resulting in test failures at build time, due to errors of either the form: “Test class should have exactly one public constructor”, “No runnable methods”, or “The class com.mycompany.MyTest$Helper is not public”. These errors are not happening in our JDK 8 branch.

There are lots of potentially relevant changes in this migration, but let me focus on what I hope are the most pertinent. Firstly, I started off using the same version of gradle for this new JDK 11 branch, 4.10.3 (we’re using a gradle wrapper in case that’s pertinent). Additionally, I’m using JUnit 4 though I’m not sure if that has any impact on how the gradle test plugin scans for test classes. I have tried updating gradle to 5.6.4, but that doesn’t seem to have had any impact. I have read the Test detection section of the documentation for version 5.6.4 which indicates that a class is considered to be a test if it “inherits from TestCase … is annotated with RunWith, or contains a method annotated with @Test or a super class does.” The test classes in question that have these static inner helper classes have not changed at all from our JDK 8 branch to JDK 11 branch, and they definitely have never met any of the aforementioned criteria from the Test Detection section.

As another data point, if I run gradle as follows:

./gradlew :subproject:test --tests com.mycompany.MyTest

I have no issues, and all the tests in MyTest run and pass without gradle attempting to execute anything in the static inner Helper class (which I know is expected, but I at least wanted to confirm that the tests do indeed work).

Additionally, if I refactor my code and move the static inner Helper class to be a top level class in the com.mycompany package and then run the following:

./gradlew :subproject:test --tests com.mycompany.*

I see the same failures, as gradle still attempts to test the Helper class. However, without making any other changes, if I switch back to the JDK 8 branch (with these unstaged/uncommitted refactoring changes), switch JAVA_HOME to JDK 8, and re-run the tests, the tests work, and Gradle does not attempt to run any tests in the Helper class.

(after switching to JDK 8 branch - the following works fine)
./gradlew clean :subproject:test --tests com.mycompany.*

Is anyone aware of any differences between JDK 8 and 11 that might be contributing to this issue, or does anyone have suggestions on how I might troubleshoot this issue further? As a workaround, I realize I could enumerate every single class that I want to test instead of relying on the automatic Test Detection, but that is going to be extremely onerous and error prone. Any guidance at all would be very much appreciated.

2

I’ve looked more closely at our failing tests (there are many), and I discovered something else interesting. In some cases, for organizational purposes, we may have test classes defined within an interface. For example, I might have a com.mycompany.MessageLoggerTest where MessageLoggerTest is an interface, and there are inner classes such as ConsoleLoggerTest etc. These instances are also generating the same initializationErrors with warnings about no public constructor and no runnable methods. Something seems very broken if the plugin doesn’t recognize that an interface doesn’t, by definition (notwithstanding default methods), have runnable methods or a constructor.

3

I was able to resolve the problem. Firstly, I verified in a simple HelloWorld project that the failures I noted in my original post happen with gradle 4.10.3 with sourceCompatibility set to 11 (but everything works with sourceCompability set to 1.8). I then verified that upgrading to 5.6.4 resolved the problems in the HelloWorld project. Finally, I went back to our larger project and found that the gradle wrapper hadn’t been upgraded to 5.6.4 as I thought I had attempted to do. The upshot is that there was a problem with even the latest versions of 4.x, but the problems have been resolved with 5.x.