False positive in up-to-date check

Help/Discuss
1

Part of our build creates scanners and parsers using Byacc/J and JFlex. The build itself is a recent re-write of an old ant build, and so these are handled using ant tasks in gradle.

My compileJava task depends on a generated_code task which in turn depends on both lexers and parsers tasks.

The lexers and parsers tasks are luckily able to very cleanly define their inputs and outputs because for each *.l/.y file there is a corresponding java file in build/generated_src/lexers and build/generated_src/parsers, respectively. I have the tasks set up to use a fileTree to grab all the *.l/.y files and then iterate the results, map the filename to the expected output, and register an output file for each.

task lexers {
    def lexers = fileTree('src') {
        include '**/*.l'
    }
    inputs.files lexers.files
    // for every input file, there should be a java file in generated_src/lexers
    lexers.files.each {
        outputs.file file(it
                .absolutePath
                .replace('\\', '/')
                .replace('/src/', '/build/generated_src/lexers/')
                .replace('.l', '.java'))
    }
    doLast {
        ant.taskdef(name: 'flex',
                classname: 'FlexTask',
                classpath: configurations.flex.asPath)
        ant.flex(destdir: "$genSrcDir/lexers", intern: 'true') {
            lexers.addToAntBuilder(project.ant, 'fileset', FileCollection.AntType.FileSet)
        }
    }
}

If I run gradlew clean build I frequently end up with a problem where the lexers and parsers tasks are marked as up-to-date even though they’re not since the clean task removes the build dir.

Once I get into this state, I have no recourse but to run the build with --rerun-tasks because every build after this first incorrect one thinks that the lexers and parsers tasks are still up-to-date.

Is it not enough to register specific files in the output? Do I have to specify an upToDateWhen closure?

2

Although this is not a self-contained reproducible example, I was able to run something close to this with a couple of modifications and some dummy files. The only way I can reproduce the problem you’re experiencing is when the output files declared don’t accurately reflect the true path of the files created. If that’s the case, the task is caching that the output is a non-existent file, which clean doesn’t do anything that changes that state.

Your current outputs looks overcomplicated and error prone. I would recommend that you simplify the output declaration rather than trying to debug what might be going wrong with it. If you’re generating everything for the lexers task into build/generated_src/lexers, Gradle doesn’t need you to calculate and set each output file individually. Just specify the output as outputs.dir "$genSrcDir/lexers" to match what you’re passing to the Ant FlexTask and Gradle will take it from there.

3

Thanks for the response.

Yeah, I used to have this setup exactly that way (outputs.dir "$genSrcDir/lexers") but was running into this same issue. I tried switching to outputs.files "$genSrcDir/lexers" with no change, and then I switched to the more explicit method here thinking that specifying explicit files would somehow make this work, but I kept running into this.

I’m positive that the spec I’m using now accurately reflects the files that get generated, so your suggestion of this being an incorrect mapping isn’t the issue. I’m going to try to boil this down to a reproducible example I can share and I’ll update this thread.

4

And of course I can’t reproduce it in a smaller build. It was happening earlier today in my real build. I added the following to each of my lexers/parsers tasks to make sure the file mapping was correct:

doLast {
  outputs.files.files.each{println "${it.exists()}: ${it.absolutePath}"}
}

I got all trues. It also forced those tasks to re-run the next time because of the code change, and now I can’t get them back into the state where they think they’re up-to-date but they’re not. Fun stuff.

I’ll keep trying, but I don’t expect to find the setup anytime real soon.