Capturing output in unit test and verifying it


(Andrey Adamovich) #1

I’m creating a library that is wrapper around an old library that is printing a lot of stuff directly to ‘System.out’.

So, I’m trying to test it through capturing standard output stream by creating a tee output stream around ‘System.out’ and then asserting on collected data in the end (see example code below).

It all works great in IDE and through local (Windows) Gradle build. But on CI server (RedHat + Jenkins) it seems to loose connection to System.out after 2nd test (i.e. nothing is captured inside test, but JUnit report still shows full stdout correclty) and that’s why assertions fail.

I understand that Gradle does its own capturing of standard output and after some digging I have found that Gradle uses ‘LinePerThreadBufferingOutputStream’, but I’m not sure how to connect that to my problem.

Any ideas what can make the test below also capture the standard output on Linux in the 2nd test?

class LauncherTest extends BaseGeneratorTestCase {

static ByteArrayOutputStream stdOutData

static ByteArrayOutputStream stdErrData

@BeforeClass

static void captureOutput() {

stdOutData = new ByteArrayOutputStream()

stdErrData = new ByteArrayOutputStream()

System.out = new TeeStream(System.out, new PrintStream(stdOutData))

System.err = new TeeStream(System.err, new PrintStream(stdErrData))

}

@Test

// This works

void testExecution1() {

String output = captureOutput {

// execute test code

}

// assert on output

}

@Test

// This fails on Linux, but works on Windows

void testExecution2() {

String output = captureOutput {

// execute test code

}

// assert on output

}

static String captureOutput(Closure cl) {

cl()

System.out.flush()

System.err.flush()

stdOutData.flush()

stdErrData.flush()

new String(stdOutData.toByteArray()) + new String(stdErrData.toByteArray())

}

static class TeeStream extends PrintStream {

PrintStream out

TeeStream(PrintStream out1, PrintStream out2) {

super(out1)

this.out = out2

}

void write(byte[] buf, int off, int len) {

super.write(buf, off, len)

out.write(buf, off, len)

}

void write(int b) {

super.write(b)

out.write(b)

}

void close() {

super.close()

out.close()

}

void flush() {

super.flush()

out.flush()

}

}

}


(Peter Niederwieser) #2

Try to redirect output before every method (@Before).


(Andrey Adamovich) #3

It didn’t help. Still get empty capture in CI server, but full one in IDE/local build. That’s the error

java.lang.AssertionError:

Expected: a string containing “Validation successful!”

but: was “”

The applied changes are the following:

class LauncherTest extends BaseGeneratorTestCase {

ByteArrayOutputStream stdOutData

ByteArrayOutputStream stdErrData

PrintStream originalOut

PrintStream originalErr

@Before

void captureOutput() {

originalOut = System.out

originalErr = System.err

stdOutData = new ByteArrayOutputStream()

stdErrData = new ByteArrayOutputStream()

System.out = new TeeStream(System.out, new PrintStream(stdOutData))

System.err = new TeeStream(System.err, new PrintStream(stdErrData))

}

@After

void restoreOutput() {

System.out = originalOut

System.err = originalErr

}

As you can notice I even restore the original values after each test.


(Peter Niederwieser) #4

Not sure what the problem is then, and whether it’s related to Gradle.


(Andrey Adamovich) #5

I think it is Gradle related, and it’s probably just a corner case that is related to assumptions that my code and Gradle code makes about output capturing. And we just don’t match somewhere :).

If I do ‘println orginialOut.class’ in ‘@Before’, I get the following:

class org.gradle.util.LinePerThreadBufferingOutputStream

Also, I finally get this test to fail on Windows in local build (I probably executed the old code before, but now I’m sure that it fails also in command-line build on Windows and on Linux). It only works from IDE which does not involve Gradle when executing unit tests.


(Peter Niederwieser) #6

Between ‘@Before’ and ‘@After’, your code “owns” the streams, and I don’t see how Gradle could interfer with this.


(Andrey Adamovich) #7

After more digging, I realized that the code I’m trying to wrap/test is partially using SimpleLogger, which is initialized after Gradle, but before my unit test. And it appears that SimpleLogger caches original value for System.out/System.err. So, it does not matter if I reassign System.out later, SimpleLogger will still use the original value to print data, that’s why it wasn’t captured.

Sorry for assuming it was caused by Gradle.