Stale output being deleted despite being an output of the task

Help/Discuss
1

I’m trying to figure out why an output of my custom task gets deleted, and thus triggers a rebuild each time.

The file in question is being marked as an output file as follows:

@Override // there's a base class defining some common setup for this task)
@OutputFile File getSymbols() { calculateFileNameBasedOnSomeProperty() }

The property it uses to calculate the name is marked with @Input and is a string in the task base class. It’s all static, and there’s nothing to suggest that it changes in any way between executions.

Despite that, it gets deleted as a “stale output” and the task is then re-run. No inputs are listed as the cause of the rebuild, and two other outputs declared using @OutputDirectory are not deleted.
When I list all inputs and outputs from the task, all three outputs are there.

There’s no other task that writes to the same folder, but there are those using it for input.

The debug output doesn’t really show anything else useful from what I can tell, so is there anything else I can do to debug this?

2

Hi @axl,

could you provide the output of the info log when the stale output file is deleted? When do you change the input which influences calculateFileNameBasedOnSomeProperty()? Does calculateFileNameBasedOnSomeProperty() always return the same value in the course of your build?

Cheers,
Stefan

3 
:compileFooDevelopmentWin64Client (Thread[Task worker for ':',5,main]) started.

> Task :compileFooDevelopmentWin64Client
Putting task artifact state for task ':compileFooDevelopmentWin64Client' into context took 0.0 secs.
Deleting stale output file: Q:\directive\foo\Foo\Binaries\Win64\Foo.pdb
Up-to-date check for task ':compileFooDevelopmentWin64Client' took 3.661 secs. It is not up-to-date because:
 Output property 'symbols' file Q:\directive\foo\Foo\Binaries\Win64\Foo.pdb has been removed.

calculateFileNameBasedOnSomeProperty() should return the same every time, as long as the configuration step has completed.

Admittedly, the setup behind all of this is rather complex, but it was working fine until I bumped the gradle version up to 4.4.1 from 4.0. I’m trying to create a minimal repro case right now, but the initial naïve test didn’t show the same symptoms unfortunately.

4

Btw, I’m using gradle.taskGraph.beforeTask and gradle.taskGraph.afterTask to list task.inputs.files and task.outputs.files to determine what the inputs and outputs are considered to be, and they look ok.

5

Could you try if the same problem occurs with Gradle 4.3?

6

On first run 4.3 deleted both the file and the folders, which I assume is because the version changed. On subsequent runs the 4.4.1 behavior where only the file is deleted persists. I’m using the new Property/Provider system so I can’t back out to 4.0 and test.

I noticed one possible issue; the file name of the @OutputFile is generated as foo.pdb but the actual file generated is Foo.pdb. On Windows this is of course the same file, but perhaps some logic is getting confused by this? However, I can’t replicate this with a simpler test, so it’s probably not that.

7

There’s a chance that the case of the name is actually the problem. I log the name during configuration, then before the task executes, and after, and during configuration it’s Foo.pdb, but before and after execution it’s foo.pdb. The file generated is named Foo.pdb.

The reason for the difference is that I need to manually tell gradle what the output of an external tool is, and it’s based on a property that is setup during the configuration phase.

I still can’t reproduce the stale file in a smaller test, but at least I can reproduce the behavor of the shifting file name case. It also looks like the file property will use the actual case of the file, regardless of what’s passed in, on Windows. E.g. new File(‘foo’) will actually be ‘Foo’ if that’s what the name of the file is.

8

This is the current test, which shows as close as possible to the original plugin, what is being done:

abstract class TestBase extends DefaultTask {
    Object outName = null

    @Input
    String getOutName() {
        GUtil.elvis(outName, 'Foo').toString()
    }

    @OutputFile
    abstract File getTest()
}

class TestX extends TestBase {
    @Override
    @OutputFile
    File getTest() {
        project.file("Binaries/Win64/${getOutName()}.pdb")
    }

    @TaskAction
    void build() {
        new File("Binaries/Win64/Foo.pdb").withWriter {
            it << "bar"
        }
    }
}

tasks.withType(TestX) {
    logger.lifecycle "With: $it.outName $it.test"
}

tasks.create('xxx', TestX, new Action<TestX>() {
    @Override
    void execute(TestX it) {
        logger.lifecycle "Before: $it.outName $it.test"
        it.outName = 'foo'
        logger.lifecycle "After: $it.outName $it.test"
    }
})
9

By ensuring the case of the file was always the same in the task, the problem went away. I’m still not sure at what point the case must change to trigger the problem, since the above task does everything in the same order without showing the problem. At least I’m good to continue for now.

10

Looks like the behavior of project.file() to grab the case of actual files on disk has been removed in gradle 4.5 too.

11

@axl Thank you for reporting (and solving your own issue :wink: ). I think your investigation is correct:
The case matters for stale output detection - i.e. when you define that the output file is foo.pdb then that is the file registered as created output. Even if the task creates Foo.pdb. For a case sensitive file system this is perfectly fine behaviour, for a case insensitive file system like on Windows or macOS, this may cause problems as you described.

Do you think this deserves an issue in gradle/gradle? If yes, please open one.

12

Thanks for listening. :slight_smile: I’ll try to replicate this in gradle 4.5 where the project.file() case logic was changed, and if it still happens there, I’ll consider opening a case.