I have a run task created by the application plugin. This task is a JavaExec task, and that’s not negotiable: I don’t get to use a different task type or a new JavaExec subclass.
I’d like to log the standard output and error of the task’s subprocess to a file. That will involve a FileOutputStream. We should create that output stream just before subprocess execution and close it promptly once the subprocess is done. Seems easy enough:
val run by
tasks.existing(JavaExec::class) {
var logFile = layout.buildDirectory.file("$name.log")
outputs.file(logFile)
doFirst {
logFile.get().asFile.outputStream().let {
standardOutput = it
errorOutput = it
}
}
doLast { standardOutput.close() }
}
Unfortunately, this approach only closes the stream promptly if the subprocess succeeds. If the subprocess fails, then the doLast action will never run and the FileOutputStream will remain open until the garbage collector reclaims it.
Can we close the stream promptly by using .use { … } instead of .let { … }? No: that closes the stream too early, before the subprocess has run at all.
Can we close the stream promptly by using .use { … } instead of .let { … }, and by embedding the entire subprocess execution inside that code block using ExecOperations.javaexec { … }? No. That would work if we were creating our own generic DefaultTask from scratch. But we have to work with an existing JavaExec task created by a plugin.
Can we rely on some other task to close the stream, and configure that other task as our run task’s finalizer using finalizeBy? Not that I can figure out: the challenge is how to get access to the run task’s standardOutput inside the finalizer task, so that the latter can close() it. Everything I’ve tried is incompatible with the configuration cache.
Is there a way to get this done that (1) closes the FileOutputStream promptly; (2) closes it regardless of task success or failure; and (3) obeys all Gradle usage rules, including those for the configuration cache? For bonus points, can you craft a solution that (1) works for Exec tasks too, and (2) has just a single reusable implementation that multiple Exec and JavaExec tasks can share without lots of code cloning?
Changing task configuration at execution time is evil of course, but as it is neither an input nor output of the task it might not be that problematic in this case and probably the only way to follow your requirements.
Btw. without CC there would be a more hacky and slightly less reliable way (because someone else could add actions later) but this is not CC-safe:
tasks.run.configure {
val logFile = layout.buildDirectory.file("$name.log")
outputs.file(logFile)
val taskActions = buildList { addAll(actions) }
actions.clear()
doLast {
logFile.get().asFile.outputStream().use {
standardOutput = it
errorOutput = it
taskActions.forEach { it.execute(this) }
}
}
}
Hah! I’m delighted that you aimed high, Vampire. But I don’t follow how your proposed solution handles “multiple Exec and JavaExec tasks […] without lots of code cloning”. I see a single instance of a RunLogStreamHoler service, which in turn holds one nullable OutputStream. If I want to apply this technique to five more tasks, wouldn’t I need five copies of everything you’ve done here, including registering five more RunLogStreamHoler services? What am I missing?
For comparison, here’s the reusable implementation of my doLast-based approach:
import org.gradle.api.Task
import org.gradle.api.provider.Provider
import org.gradle.api.tasks.Exec
import org.gradle.api.tasks.JavaExec
import org.gradle.process.BaseExecSpec
/**
* Extension function to redirect standard output and error of an execution task (such as [Exec] or
* [JavaExec]) to a file under `build/`.
*
* @param baseName Base name (without extension) for the log file. The actual file will be created
* as `build/<baseName>.log`.
* @return a [provider][Provider] for the log file, which is also registered as a
* [task output][Task.outputs]. This can be used for wiring task dependencies or additional
* configuration if needed.
*/
fun <T> T.logToFile(baseName: String) where T : Task, T : BaseExecSpec =
project.layout.buildDirectory.file("$baseName.log").also { logFile ->
outputs.file(logFile)
doFirst {
logFile.get().asFile.outputStream().let {
standardOutput = it
errorOutput = it
}
}
// Try to close the `FileOutputStream` promptly when the task is finished. If the task fails,
// then `doLast` will not run, so the stream will remain open until the garbage collector
// reclaims it.
doLast { standardOutput.close() }
}
As the comment notes, it only works if the subprocess succeeds. But it is very easy to reuse in multiple tasks. You just add a logToFile(name) call to the configuration block for any Exec or JavaExec task.
Quite interesting! I was not aware of the Task.actions list. Not CC-safe, as you say, but it’s always good to know more. Thanks for teaching me something new.
Thank you as well for turning my question into a Gradle feature request. Task.doFinally would be the perfect mechanism here. I hope that your request is received favorably.
But I don’t follow how your proposed solution handles “multiple
Not multiple, that was not part of the requirements.
The requirements for the bonus points were that you can use the same solution for Exec tasks, optimally without much code reuse.
Nothing in the code is JavaExec specific, so you can easily make this into a solution that is reusable for both, like using the same build service class, but different instances and having a common implementation for the close class that you then just instantiate or similiar.
wouldn’t I need five copies of everything you’ve done here, including registering five more RunLogStreamHoler services?
You can easily make the service instead have a Map where you store some key and as value the stream. Or you can register multiple instances of that service, doesn’t matter much, as you anyway just use it as transport vehicle between the different tasks.
For comparison, here’s the reusable implementation of my doLast -based approach
And what hinders you to do the same with my approach?
For example
interface RunLogStreamHoler : BuildService<BuildServiceParameters.None> {
var runLogStream: OutputStream?
}
fun <T> NamedDomainObjectProvider<T>.logToFile(baseName: String) where T : Task, T : BaseExecSpec =
project.layout.buildDirectory.file("$baseName.log").also { logFile ->
val runLogStreamHoler = gradle.sharedServices.registerIfAbsent("${baseName}RunLogStreamHoler", RunLogStreamHoler::class)
val closeRunLog = tasks.register("${baseName}CloseRunLog") {
usesService(runLogStreamHoler)
doLast {
runLogStreamHoler.get().runLogStream?.close()
}
}
configure {
usesService(runLogStreamHoler)
outputs.file(logFile)
doFirst {
logFile.get().asFile.outputStream().let {
runLogStreamHoler.get().runLogStream = it
standardOutput = it
errorOutput = it
}
}
finalizedBy(closeRunLog)
}
}
tasks.run.logToFile("foo")
or
abstract class RunLogStreamHoler : BuildService<BuildServiceParameters.None> {
val runLogStreams: MutableMap<String, OutputStream> = mutableMapOf()
}
val runLogStreamHoler = gradle.sharedServices.registerIfAbsent("runLogStreamHoler", RunLogStreamHoler::class)
fun <T> NamedDomainObjectProvider<T>.logToFile(baseName: String) where T : Task, T : BaseExecSpec =
project.layout.buildDirectory.file("$baseName.log").also { logFile ->
val closeRunLog = tasks.register("${baseName}CloseRunLog") {
usesService(runLogStreamHoler)
val runLogStreamHoler = runLogStreamHoler
doLast {
runLogStreamHoler.get().runLogStreams.remove(baseName)?.close()
}
}
configure {
usesService(runLogStreamHoler)
val runLogStreamHoler = runLogStreamHoler
outputs.file(logFile)
doFirst {
logFile.get().asFile.outputStream().let {
runLogStreamHoler.get().runLogStreams[baseName] = it
standardOutput = it
errorOutput = it
}
}
finalizedBy(closeRunLog)
}
}
tasks.run.logToFile("foo")
The bonus requirements included "a single reusable implementation that multiple Exec and JavaExec tasks can share without lots of code cloning”. But I could have been more explicit about wanting to support multiple such tasks within a single Gradle project.
Aah, more interesting material to learn from. You used an extension method on NamedDomainObjectProvider, rather than an extension method on the task type itself. That lets you create the …CloseRunLog cleanup task outside of the configure logic for the task that needs cleanup. I like it!
Now that’s a solution that earns full bonus points. It closes “soon” after a task completes, regardless of success or failure, obeys all Gradle usage rules, works for Exec tasks too, and works for multiple tasks within a project without code duplication. You also earn an ad hoc extra gold star for submitting a Gradle feature request that may make this problem easier to solve in the future: . Well done, and thank you!
I would consider only two small tweaks to your solution:
For my purposes, it makes sense to consider logToFile(…) as just another part of configuring a task. So I plan to remove the configure { … } and make this an extension method on T rather than on NamedDomainObjectProvider<T>. That way I can call it inside a suitable task’s configuration block.
After onFinish sees the TaskFinishEvent for a given task, I plan to have onFinish remove that task’s key from runLogStreams. Otherwise, runLogStreams just keeps growing forever, keeping entries that will never have a future use. This doesn’t really matter for the number of tasks in my project, but it just feels tidier in principle.
Thanks again, @Vampire. I’ve learned much from you, as always.
For my purposes, it makes sense to consider logToFile(…) as just another part of configuring a task. So I plan to remove the configure { … } and make this an extension method on T rather than on NamedDomainObjectProvider<T> . That way I can call it inside a suitable task’s configuration block.
Yeah, for operation completion listener solution that should be fine.
For the first solution it would have been bad, because if you do not break task-configuration avoidance, the task would have usually been configured too late to register a new task, you could only have created a new task, also breaking task-configuration avoidance for that one.
But as no additional task is necessary in that solution, that should not be necessary anymore.
After onFinish sees the TaskFinishEvent for a given task, I plan to have onFinish remove that task’s key from runLogStreams .
But I don’t follow how your proposed solution handles “multiple 
It closes 
. Well done, and thank you!