Gradle test discovery using annotations rather than a regex against the filename

Old Forum Archive
1

I would really like to find tests using custom annotations rather than a regex on the class name.

For example some of our functional tests test a single system where all external dependencies are stubbed, others test a system where some but maybe not all of the external dependencies are real instances and not stubbed.

Some examples of annotations I might annotate my test classes with

@SingleServiceTest @MultiServiceTest

or

@RequiresSystemA @RequiresSystemB

2

Thanks for the suggestion. Definitely a nice feature to have.

3

I’m doing something similar to what you are doing that made me make a suggestion some time ago http://forums.gradle.org/gradle/topics/direct_communication_between_java_annotation_processors_and_gradle_scripts. Instead of marking classes for doing tests I’m marking classes for export to separate jar files.

My solution was to make a jar file with all the necessary annotations and processors and call into it from a Gradle script. The trick is to grab the classloader that is used to load classes that are imported into the Gradle script. Then pass that classloader to a Java compiler so it loads the same annotation processor class that your using in your Gradle script. You can tell that Java compiler to do annotation processing only, with “-proc:only”, so that it doesn’t create any class files. That way you could do in a Gradle script:

import com.example.MyAnnotationProcessor
// declare buildscript dependency on the jar that contains MyAnnotationProcessor
 task annotationProcessing << {
    def results = MyAnnotationProcessor.doProcessingOn (sourceSets.main.java, /* any extra data */)
    // do something with results
}

It would be cool though if something like this were built into Gradle.

I’m willing to go into more detail about my solution if anyone is interested.

4

Hi Alexander, that sounds great. How did you pass gradle’s ClassLoader into MyAnnotationProcessor?

5

All you have to do in MyAnnotationProcessor.java is:

ClassLoader cl = MyAnnotationProcessor.class.getClassLoader();

This is assuming that you’ve done an import of MyAnnotationProcessor somewhere in your Gradle script. That forces Gradle to load your class. Since all classes know what classloader loaded them, it’s easy to get it. The real trick is passing that classloader to an instance of a JavaCompiler.

Get an instance of a JavaCompiler (http://docs.oracle.com/javase/6/docs/api/javax/tools/JavaCompiler.html) using ToolProvider (http://docs.oracle.com/javase/6/docs/api/javax/tools/ToolProvider.html), and get an instance of a StandardJavaFileManager (http://docs.oracle.com/javase/6/docs/api/javax/tools/StandardJavaFileManager.html) from the JavaCompiler. Passing nulls to getStandardFileManager() works for me.

Create your own implementation of StandardJavaFileManager. Let’s call it MyFileManager, and delegate all of the methods (except getClassLoader(), this is the key) to the StandardJavaFileManager you have obtained. Return the MyAnnotationProcessor classloader from the getClassLoader() method of MyFileManager. You can pass the classloader however you wish to MyFileManager. You can even have your annotation processor implement StandardJavaFileManager since it’s an interface.

When you do a getTask() using the JavaCompiler pass along your instance of MyFileManager. Now when the JavaCompiler requests a classloader from MyFileManager it will get the classloader that has already loaded MyAnnotationProcessor. Now your Gradle script and JavaCompiler have the exact same MyAnnotationProcessor class.

Note that you’ll have to pass the classpaths of all of the compilation dependencies to your JavaCompiler or else it will fail to process your source files. From your Gradle script you can use

project.configurations.compile*.path

to get all of the classpaths necessary for compilation. Remember to pass “-proc:only” to the compiler or you’ll get a bunch of unwanted .class files. You will also have to manage any compilation errors that occur during processing.

If someone can think of a better solution that would be quite nifty. Or maybe the Gradle devs could make this a built-in feature of Gradle nudge nudge wink wink.

6

Code contributions are welcome. However it should probably be implemented as an enhancement to Gradle’s existing test class scanning.

7

That sounds a lot more feasible Peter, but I would really like to see a more general solution that any Gradle user could modify to fit their needs. A closure that is run every time a certain annotation is detected would be very flexible. I’ll see what I can do to implement this and contribute it to the Gradle community if I’m successful.

8

What use cases do you have in mind, other than including/excluding annotated test classes/methods?

9

There’s also what I’ve been doing and that’s marking packages and classes with the jars that they should be sent to.

Also, if I’m not mistaken, you could do annotation processing while ignoring package import errors (works on my machine, but no extensive testing) so that you could annotate classes and packages with the jars that they depend on. I often don’t know what jars provide what packages, but the developer who writes the code should know and they could mark their class with the jars that are necessary for doing proper compilation. They could add whatever packages they need without fearing that their build will fail and they will have to contact me to add the dependency.

This doesn’t even have to be limited to annotation processing. You are allowed to get the abstract syntax tree during compilation so you could add various checks on the source code. Such as checking that a custom resource is properly initialized and disposed. Some classes have a compilation warning you if you forget to call a certain method when you finish using them. Adding your own custom version would be useful.

I would hazard a guess that their are other use cases that someone more clever than I could think of.

10

I fail to see how this is related to the topic at hand. If you want to do something totally custom, just write an annotation processor. The ‘Test’ task, on the other hand, should provide a focused solution to filter tests based on annotations.

11

I understand your point.

What I’m trying to suggesting is that Gradle could provide a way to give annotation processors to the JavaCompiler it uses, or use one general processor provided by Gradle. This could cover the Test task annotations and anything else someone could think of. It’s currently not as easy as “import MyProcessor” in a Gradle script. Luke Daley mentioned to me that the classloaders are kept separate on purpose. Considering that, my solution to 0xC0FFEE above is a bit of a hack since I’m essentially bypassing Gradle’s classloader system. It could easily break if a major change occurs in Gradle.

But a focused solution is better than nothing.