Using GradleRunner to test a plugin with GradleBuild tasks


(David Byron) #1

I’m using GradleRunner to test a plugin. Initially (with gradle 2.13) I started with the createClasspathManifest / plugin-classpath.txt mechanism described in https://docs.gradle.org/2.14/userguide/test_kit.html#sub:test-kit-classpath-injection. Now with gradle 2.14 I’m using the java-gradle-plugin plugin. I see the same behavior here in both cases.

In general these work fine I construct build.gradle files to test with that apply my plugin under test like this:
plugins { id 'myplugin' }

However, I’ve run into a snag when the plugin I’m testing has GradleBuild tasks. Using the plugins block I get this error:

 * What went wrong:
Plugin [id: 'myplugin'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Gradle Central Plugin Repository (plugin dependency must include a version number for this source)

* Exception is:
org.gradle.api.plugins.UnknownPluginException: Plugin [id: 'myplugin'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Gradle Central Plugin Repository (plugin dependency must include a version number for this source)

If I switch to
apply plugin: 'myplugin'
I get

* What went wrong:
A problem occurred evaluating root project 'junit7373222146585466019'.
> Plugin with id 'myplugin' not found.

However, if I also include the

def classpathString = pluginClasspath
    .collect { it.absolutePath.replace('\\', '\\\\') } // escape backslashes in Windows paths
    .collect { "'$it'" }
    .join(", ")

buildFile << """
    buildscript {
        dependencies {
            classpath files($classpathString)
        }
"""

logic (from https://docs.gradle.org/2.14/userguide/test_kit.html#N14CFD) before
apply plugin: 'myplugin'

then the tests work fine.

So, now the question…is there some way to get tests for plugins that use GradleBuild tasks to work with gradle 2.13 or gradle 2.14 without having this extra logic in my test build.gradle files?

Thanks much.

-DB


(Benjamin Muschko) #2

The error message indicates that the plugin cannot be found. I don’t think it’s necessarily related to the fact that you are using a task of type GradleBuild. If you are using the Java plugin development plugin (requires Gradle >= 2.13) then you will not need to set up the classpath manifest manually anymore. There are two important steps you need to take in your test class to make it work:

  1. Apply your plugin with the plugin DSL: plugins { ... }
  2. Call the method .withPluginClasspath() on your GradleRunner instance to indicate that you want to automatically inject the classpath generated by the Java plugin development plugin.

You can also find a sample project using this feature in the Gradle distribution under samples/testKit/GradleRunner/automaticClasspathInjectionQuickstart.

Can you please verify this is working for you? If it isn’t can you please provide a sample project on GitHub that demonstrates the issue?


(David Byron) #3

Here’s a diff I made to that sample project (from the REL_2.14 tag):

diff --git a/subprojects/docs/src/samples/testKit/gradleRunner/automaticClasspathInjectionQuickstart/src/main/groovy/org/gradle/sample/HelloWorldPlugin.groovy b/subprojects/docs/src/samples/t
index 85f808e..2bde5a5 100644
--- a/subprojects/docs/src/samples/testKit/gradleRunner/automaticClasspathInjectionQuickstart/src/main/groovy/org/gradle/sample/HelloWorldPlugin.groovy
+++ b/subprojects/docs/src/samples/testKit/gradleRunner/automaticClasspathInjectionQuickstart/src/main/groovy/org/gradle/sample/HelloWorldPlugin.groovy
@@ -18,13 +18,17 @@ package org.gradle.sample
 
 import org.gradle.api.Plugin
 import org.gradle.api.Project
+import org.gradle.api.tasks.GradleBuild
 
 class HelloWorldPlugin implements Plugin<Project> {
     void apply(Project project) {
-        project.task('helloWorld') {
+        project.task('doWork') {
             doLast {
                 println 'Hello world!'
             }
         }
+        project.task('helloWorld', type: GradleBuild) {
+           tasks = ['doWork']
+        }
     }
 }

I then ran the build task using a gradle 2.14 wrapper and got this output:

$ /path/to/gradlew build
:buildSrc:compileJava UP-TO-DATE
:buildSrc:compileGroovy UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar UP-TO-DATE
:buildSrc:assemble UP-TO-DATE
:buildSrc:checkstyleMain UP-TO-DATE
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:compileTestGroovy UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:checkstyleTest UP-TO-DATE
:buildSrc:codenarcMain UP-TO-DATE
:buildSrc:codenarcTest UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build UP-TO-DATE
:compileJava UP-TO-DATE
:compileGroovy
:pluginDescriptors UP-TO-DATE
:processResources UP-TO-DATE
:classes
:jar
:assemble
:pluginUnderTestMetadata
:compileTestJava UP-TO-DATE
:compileTestGroovy
:processTestResources UP-TO-DATE
:testClasses
:test

org.gradle.sample.BuildLogicFunctionalTest > hello world task prints hello world FAILED
    org.gradle.testkit.runner.UnexpectedBuildFailure at BuildLogicFunctionalTest.groovy:45

1 test completed, 1 failed
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/david.byron/src/gradle/subprojects/docs/src/samples/testKit/gradleRunner/automaticClasspathInjectionQuickstart/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 5.507 secs

$ /path/to/gradlew --version

------------------------------------------------------------
Gradle 2.14
------------------------------------------------------------

Build time:   2016-06-14 07:16:37 UTC
Revision:     cba5fea19f1e0c6a00cc904828a6ec4e11739abc

Groovy:       2.4.4
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.8.0_66 (Oracle Corporation 25.66-b17)
OS:           Mac OS X 10.10.5 x86_64

The test report contains:

org.gradle.testkit.runner.UnexpectedBuildFailure: Unexpected build execution failure in /var/folders/dn/0rbsn8cs25l0yzczh63yyw4r0000gp/T/junit1173628617284125689 with arguments [helloWorld]

Output:
:helloWorld FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/private/var/folders/dn/0rbsn8cs25l0yzczh63yyw4r0000gp/T/junit6269190035044751096/build.gradle' line: 3

* What went wrong:
Plugin [id: 'org.gradle.sample.helloworld'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Gradle Central Plugin Repository (plugin dependency must include a version number for this source)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 1.881 secs

	at org.gradle.testkit.runner.internal.DefaultGradleRunner$1.execute(DefaultGradleRunner.java:222)
	at org.gradle.testkit.runner.internal.DefaultGradleRunner$1.execute(DefaultGradleRunner.java:219)
	at org.gradle.testkit.runner.internal.DefaultGradleRunner.run(DefaultGradleRunner.java:282)
	at org.gradle.testkit.runner.internal.DefaultGradleRunner.build(DefaultGradleRunner.java:219)
	at org.gradle.sample.BuildLogicFunctionalTest.hello world task prints hello world(BuildLogicFunctionalTest.groovy:45)

(David Byron) #4

Any word on this? Is the diff here a valid test?

-DB


(Benjamin Muschko) #5

Hi David,

I cannot reproduce the issue you are seeing. Please find attached a sample project that invokes a GradleBuild as part of the plugin under test. The sample project uses Gradle 2.14.

Cheers,

Ben


(David Byron) #6

That project passes for me too. I made a small tweak to make it fail: https://github.com/bmuschko/gradle-testkit-gradlebuild/pull/1.


(Benjamin Muschko) #7

I responded in the pull request. Let’s just use that as the source for further discussion.


(David Byron) #8

I wish you hadn’t closed the pull request so quickly. Whether GradleBuild is the right way to solve this particular problem or not, it still seems to be that there’s some interaction between GradleBuild and GradleRunner that’s less than ideal. It may only be about setting buildFile, but it does seem like there’s something there.

As for the solution you propose there, I think it assumes I have full control (i.e. I am the author of) all the tasks that run after changing the version. If I’m not, and there are tasks that read the version at configuration time, I’m not sure what to do.


(Benjamin Muschko) #9

Closing the pull request on my test project doesn’t really determine whether we can or cannot continue the discussion on your issue. The PR wasn’t really intended to be merged. We can definitely move forward with the discussion.

I believe what’s happening in your case is that Gradle executes a completely new build using your build.gradle file. However, GradleBuild has its own classloader and that classloader does not know about the plugin you define in the plugins block as the new build does not inject the classpath (which we do with GradleRunner through the Tooling API via withPluginClasspath()). The result is that the plugin cannot be resolved. In the stack trace you can also see that that we do not list “Gradle TestKit” as source. I wonder if you can work your way around it by manually injecting the plugin classpath as described in the user guide.

As for the solution you propose there, I think it assumes I have full control (i.e. I am the author of) all the tasks that run after changing the version. If I’m not, and there are tasks that read the version at configuration time, I’m not sure what to do.

As long as a task exposes the properties you want to change, then you can change them if though you might not have control over the source code.


(David Byron) #10

I am manually injecting the classpath. It feels like a combination of methods in the user guide though…I needed to do the extra stuff in the section marked ‘Working with Gradle versions prior to 2.8’ to get things to work.

I guess what I’m really asking is…what are the chances of teaching GradleRunner to do more so that this extra code isn’t required? Ideally I’d like to use the >= 2.13 ‘Automatic injection with the Java Gradle Plugin Development plugin’ method.

I’m happy to give it a shot myself, but would love to hear if it’s feasible / has a chance of being accepted, and even better some pointers on a good way to go about it.

Thanks much.

-DB


(Benjamin Muschko) #11

I am manually injecting the classpath. It feels like a combination of methods in the user guide though…I needed to do the extra stuff in the section marked ‘Working with Gradle versions prior to 2.8’ to get things to work.

I’d just write my own plugin that hides the complexity from the end user. It would also allow you reuse the functionality for other plugins you are developing. The approach would be similar to what we do with the Java Gradle Plugin Development plugin.

I guess what I’m really asking is…what are the chances of teaching GradleRunner to do more so that this extra code isn’t required? Ideally I’d like to use the >= 2.13 ‘Automatic injection with the Java Gradle Plugin Development plugin’ method.

What’s the reason for not upgrading to Gradle 2.13?


(David Byron) #12

I have upgraded to 2.13 (2.14.1 actually)…but without the extra code, I get the exception above.

I agree, if I need to keep this around, I’ll find a way to hide it or share it across plugins or something.

-DB


(Benjamin Muschko) #13

I have upgraded to 2.13 (2.14.1 actually)…but without the extra code, I get the exception above.

I laid out the reasons why it fails in the previous post. I don’t think it would very straight forward to get it right. So if you can make it work by manually injecting the classpath then I’d go for that.


(David Byron) #14

If there’s no hope for a change to the test kit (or somewhere else in gradle) to make this work, then that’s what I’ll do. Some mention that GradleBuild is special in the docs would be a big help.