After upgrade to Gradle 2.x (tested with 2.2.1, 2.3-rc-1, 2.4-20150131230028+0000) builds run using Tooling API doesn’t catch stdout and stderr. This it important when you try to analyze output generated by 3rd party plugins and other entries logged with println. There could be a workaround with redirecting stdout to logging level in ‘build.gradle’ with ‘logging.captureStandardOutput LogLevel.INFO’, but it doesn’t work with Tooling API as well…
As a result there is no way to write acceptance tests using stdout with nebula-test and any other application using Tooling API can reach a problem when stdout/stderr message is not visible.
It worked fine with 1.x (tested with 1.10 and 1.12). I can provide a test case to reproduce that problem if needed.
Sure: 1. git clone https://github.com/szpak/nebula-test/ -b issues/29-stdout 2. Unignore the last two tests in TestSpec.groovy 3. ./gradlew check - those two tests should fail
In addition to verify that a mentioned workaround (with stdout redirecting to logger level) doesn’t work as well the following test could be used (in the context of TestSpec.groovy):
def "stdout redirected to WARN included in standardOutput"() {
given:
GradleRunner runner = GradleRunnerFactory.createTooling(false)
tmp.newFile("build.gradle") << """
logging.captureStandardOutput LogLevel.WARN
apply plugin: ${SomePlugin.name}
"""
when:
ExecutionResult result = runner.run(tmp.root, ["print"])
then:
result.standardOutput.contains("Printed (stdout)")
}
To make thing even easier I created a new branch (the old one was related to nebula-test issue report), removed ‘@Ignore’ annotations and in addition I created two more tests to verify that a mentioned workaround (with stdout redirecting to logger level) doesn’t work. Tests not are failing by default.
Marcin, if you had problem with nebula-test and code in master let me know, I would make a test case with the Tooling API testing mechanism you mentioned.
This is basically down to ‘nebula-test’ using Tooling API in embedded mode which is not part of the public API. This means that tests run in the same JVM as the build under test and ‘System.out’ interception doesn’t get a chance to be set up correctly. I have confirmed that if you pass ‘true’ to ‘GradleRunnerFactory.createTooling()’ in your tests then text printed using ‘println()’ calls appears in ‘ExecutionResult#getStandardOutput()’.
From what I’ve heard, the guys behind ‘nebula’ plugins use this mode because it’s easier to debug your builds under test this way - breakpoints will just work if you execute such tests in debug mode. But it’s not impossible to attach a debugger to a daemon started from Tooling API. Simply add ‘-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005’ as JVM argument for ‘BuildLauncher’ and then connect the debugger from your IDE to the selected debug port. Running your tests in a daemon means that you will run them in an environment closer to the real one compared to running them using Tooling API embedded mode.
