Implementing a custom test framework

Help/Discuss
1

Hello,

I am creating a Gradle plugin which must, amongst other things, support the execution of tests on a specific environment. Basically, a developer will create a Gradle project, add our plugin and write tests with the JUnit API, then when the developer launches the tests, our plugin will do some manipulations on the test classes (we rely on JUnit API to write the tests but then they are executed on a specific environment without JUnit, on a custom JVM, with Ant scripts, …) and launch an Ant script to execute the tests.
My first intuition was to implement a custom test framework, so instead of using useJUnit() or useTestNG(), we could use useOurTestFramework(). But I did not find a way to plug a custom test framework, neither with the classic way to run tests, nor with the new JVM Test Suite plugin.
Do I miss something? Is there a way to plug a custom test framework?

For the moment I implemented it in a custom task, plugged to the “test” task. This works fine. My concern is that I reuse the test task options (especially includesPattern/excludesPattern) to allow the application developer to use a well-known and documented syntax. But this means using an object of a different task in my task, I am not sure it is a good practice:

Test testTask = (Test) getProject().getTasks().getByName("test");
DefaultTestFilter testFilter = (DefaultTestFilter) testTask.getFilter();
// then I use testFilter.getIncludePatterns() for example

Is there a better way to do this?

Regards

2

I don’t think you can implement an own test framework without using internal classes.
And no, I don’t think your approach with the test filter is a good idea, besides that it also uses an internal class.

Usually if you need an own way to run tests, I recommend writing a custom JUnit Platform Engine.
Like Spock for example, that is also implemented as own JUnit Platform Engine.
This also has the advantage, that immediately all tools supporting JUnit Platform like CI servers, IDEs, build tools, … support your custom tests nicely.

3

Thanks for the reply. I will have a look at implementing a custom JUnit Platform Engine, it looks interesting. Thanks!

4

What about the same question for non-JVM technologies? Using JUnit is not an option, but I still want to communicate test successes and failures to Gradle so they can appear in reports/in the IDE/etc…

5

Which non-JVM technologies?
Why can’t you use JUnit for them?
The JUnit engine can be a really thin layer translating your test results to a form Gradle can properly understand and handle.