Unable to catch stdout/stderr when using Tooling API i Gradle 2.x

After upgrade to Gradle 2.x (tested with 2.2.1, 2.3-rc-1, 2.4-20150131230028+0000) builds run using Tooling API doesn’t catch stdout and stderr. This it important when you try to analyze output generated by 3rd party plugins and other entries logged with println. There could be a workaround with redirecting stdout to logging level in ‘build.gradle’ with ‘logging.captureStandardOutput LogLevel.INFO’, but it doesn’t work with Tooling API as well…

As a result there is no way to write acceptance tests using stdout with nebula-test and any other application using Tooling API can reach a problem when stdout/stderr message is not visible.

It worked fine with 1.x (tested with 1.10 and 1.12). I can provide a test case to reproduce that problem if needed.

A test case would be highly appreciated. You can provide it in a form of a pull request as it might be easier to write your tests using the fixtures and infrastructure we have set up for testing Tooling API in core: https://github.com/gradle/gradle/tree/master/subprojects/tooling-api/src/integTest/groovy/org/gradle/integtests/tooling

Marcin, I have already test case written in nebula-test. Would it be enough?

Anything that I can easily clone and reproduce will do.

Sure: 1. git clone https://github.com/szpak/nebula-test/ -b issues/29-stdout 2. Unignore the last two tests in TestSpec.groovy 3. ./gradlew check - those two tests should fail

In addition to verify that a mentioned workaround (with stdout redirecting to logger level) doesn’t work as well the following test could be used (in the context of TestSpec.groovy):

def "stdout redirected to WARN included in standardOutput"() {
        GradleRunner runner = GradleRunnerFactory.createTooling(false)
        tmp.newFile("build.gradle") << """
            logging.captureStandardOutput LogLevel.WARN
            apply plugin: ${SomePlugin.name}
        ExecutionResult result = runner.run(tmp.root, ["print"])
        result.standardOutput.contains("Printed (stdout)")

To make thing even easier I created a new branch (the old one was related to nebula-test issue report), removed ‘@Ignore’ annotations and in addition I created two more tests to verify that a mentioned workaround (with stdout redirecting to logger level) doesn’t work. Tests not are failing by default.

‘git clone https://github.com/szpak/nebula-test/ -b issues/29-stdout-ext’

Btw, there is an issue with running nebula-test with Gradle 2.3 - see https://github.com/nebula-plugins/nebula-publishing-plugin/issues/28

Marcin, if you had problem with nebula-test and code in master let me know, I would make a test case with the Tooling API testing mechanism you mentioned.

There’s no problem, I just didn’t have time to look into it yet. Hopefully I’ll find some time over the weekend.

No problem, take your time. I just wanted to make your work as easy as possible :).

Work is never easy;). Taking a look now.

This is basically down to ‘nebula-test’ using Tooling API in embedded mode which is not part of the public API. This means that tests run in the same JVM as the build under test and ‘System.out’ interception doesn’t get a chance to be set up correctly. I have confirmed that if you pass ‘true’ to ‘GradleRunnerFactory.createTooling()’ in your tests then text printed using ‘println()’ calls appears in ‘ExecutionResult#getStandardOutput()’.

From what I’ve heard, the guys behind ‘nebula’ plugins use this mode because it’s easier to debug your builds under test this way - breakpoints will just work if you execute such tests in debug mode. But it’s not impossible to attach a debugger to a daemon started from Tooling API. Simply add ‘-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005’ as JVM argument for ‘BuildLauncher’ and then connect the debugger from your IDE to the selected debug port. Running your tests in a daemon means that you will run them in an environment closer to the real one compared to running them using Tooling API embedded mode.

1 Like

Forked mode did a trick. Big thanks Marcin for your work!

No worries, we’re here to keep our users happy :slight_smile: