java.io.IOException: Stream Closed in project.exec

Help/Discuss
1

I’m implementing a custom task that uses project.exec in its @TaskAction.
This task is part of a custom plugin.
Here is a simplified implementation of the @TaskAction

@TaskAction
fun run(){
	// OutputStream to where the scanner std and err outputs are written
	val consoleLogFile: File = project.layout.buildDirectory.file("logs/console.log").get().asFile
	consoleLogFile.parentFile.mkdirs()
	val fileOutStream: FileOutputStream = FileOutputStream(consoleLogFile)

	// Execute scanner
	val result = project.exec {
		// Set both std and err outputs to a single file
		it.setStandardOutput(fileOutStream)
		it.setErrorOutput(fileOutStream)
		it.commandLine(composeScannerCommand())
	}
}

The composeScannerCommand() creates an array with executable and parameters. Simple. It does detect check whether it’s running on Win or Unix, which translates either into cmd /c scanner.bat or just scanner. But I believe this info isn’t relevant because my tests fail after the tool’s execution…

As for the functional tests, I simply read that logs/console.log and do a content assertion (omitted).

val runner = GradleRunner.create()
runner.forwardOutput()
runner.withPluginClasspath()
runner.withArguments("runScanner", "--info")
runner.withProjectDir(projectDir)
val result = runner.build()

// Read build console.log file for the mocked scanner, which simply prints the parameters
val console = projectDir.resolve("build/logs/console.log").readText()

As a result, I’m getting the following Exception:

Could not read standard output of command '/tmp/junit9281083532723264258/scanner/bin/scanner'.
java.io.IOException: Stream Closed
   at java.base/java.io.FileOutputStream.writeBytes(Native Method)
   at java.base/java.io.FileOutputStream.write(FileOutputStream.java:354)
   at org.gradle.process.internal.streams.ExecOutputHandleRunner.writeBuffer(ExecOutputHandleRunner.java:97)
   at org.gradle.process.internal.streams.ExecOutputHandleRunner.lambda$forwardContent$0(ExecOutputHandleRunner.java:81)
   at org.gradle.internal.operations.CurrentBuildOperationRef.with(CurrentBuildOperationRef.java:70)
   at org.gradle.process.internal.streams.ExecOutputHandleRunner.forwardContent(ExecOutputHandleRunner.java:80)
   at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:64)
   at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
   at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:47)
   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   at java.base/java.lang.Thread.run(Thread.java:829)

What’s most interesting is that:

  • The Exception happens in Linux only! The same tests work fine in Windows.
  • If I run two tests with the test block above (each in its own temporary directory!), one will succeed, the other will throw the Exception. If I run the successful test alone, it then throws the Exception itself. That’s one test is influencing the other, although the console.log are different, in separate temp dir.

Info

------------------------------------------------------------
Gradle 8.3
------------------------------------------------------------

Build time:   2023-08-17 07:06:47 UTC
Revision:     8afbf24b469158b714b36e84c6f4d4976c86fcd5

Kotlin:       1.9.0
Groovy:       3.0.17
Ant:          Apache Ant(TM) version 1.10.13 compiled on January 4 2023
JVM:          11.0.20 (Ubuntu 11.0.20+8-post-Ubuntu-1ubuntu120.04)
OS:           Linux 5.10.16.3-microsoft-standard-WSL2 amd64

Any idea what could be causing the issue?

As an alternative, if I use a ByteArrayOutputStream instead of FileOutputStream and then write it into a file after project.exec (using again a FileOutputStream), then it works fine. But looks workaroundish.

2

While writing the question, it occurred to me that the issue might be related to using the same FileOutputStream for both standardOutput and errorOutput.

The ExecSpec is known to close the Streams automatically, which would explain the issue (to a certain extent, see below)
So I update my @TaskAction as follows:

@TaskAction
fun run(){
	// Create logs folder
	val consoleLogDir: File = project.layout.buildDirectory.file("logs/").get().asFile
	consoleLogDir.mkdirs()

	// Execute scanner
	val result = project.exec {
		it.setStandardOutput(FileOutputStream(consoleLogDir.resolve("console.log")))
        it.setErrorOutput(FileOutputStream(consoleLogDir.resolve("console.err")))
		it.commandLine(composeScannerCommand())
	}
}

It works, and it’s actually good to have the std and err logs separate.
I still wrote both the question and answer here because it might help other people.

STILL, I cannot explain why it worked 100% in Windows, while, in Linux, if I create two tests reading the console.log, one of them works, the other doesn’t.

If anybody wants to add anything to that, please feel free :slight_smile:

1 Like
3

Without investigating deeper, I’d say it is because of the different file handling in both.
If you for example have a file handle open in Windows, it is locked and you for example cannot delete it.
If you have a file handle open in Linux and try to delete it, the filesystem entry is happily deleted, while you can still have the file open. The underlying inode that you have open stays there until you close the open file handle, but the filesystem entry is already gone.

Besides that, you should not use project. at execution time, this is deprecated and will fail in the future. Better @Inject ExecOperations for exec and ProjectLayout for layout. :slight_smile:

1 Like
4

Oh ok. Let me see if I understood it…
I’ll add a little twist to my snippet: assume I also need to print the project name and version.

Would this be the correct?

@get:Inject
abstract val proj: Project

@TaskAction
fun run(){
	logger.info( "${proj.name}_${proj.version}" )

	// Create logs folder
	val consoleLogDir: File = proj.layout.buildDirectory.file("logs/").get().asFile
	consoleLogDir.mkdirs()

	// Execute scanner
	val result = proj.exec {
		it.setStandardOutput(FileOutputStream(consoleLogDir.resolve("console.log")))
		it.setErrorOutput(FileOutputStream(consoleLogDir.resolve("console.err")))
		it.commandLine(composeScannerCommand())
	}
}

Since I need name and version, I injected the Project proj.
Then, I used both layout and exec from proj, because… well… it’s already there.

I assume I don’t need to inject ExecOperations and ProjectLayout separately, right?
Or did I get it all wrong? :smiley:

5

I don’t think you can inject Project.
And even if you could, you must not use it at execution time.
So no, you didn’t get it right. :slight_smile:
Create two Property<String> that you set from project name and version, inject ExecOperations and ProjectLayout and use those.

1 Like
6

I see…
Injecting Project technically worked for me, but that just means I found another path to deprecation :smiley:
I misunderstood earlier that only the existing project was an issue, and assumed that an injected one would be ok. Now I see: execution time = no Project. Period.

Thanks for the help!!

EDIT: When is it getting deprecated? I get no warning at all about using project yet. Even with all warnings enabled. Using Gradle 8.3

7

Probably only if you enable configuration cache, or if you enable the STABLE_CONFIGURATION_CACHE feature currently