Up-to-date check failes on Linux

Help/Discuss
1

I have a problem that I can’t reproduces on Windows. It can be reproduces on Linux but only sometimes. It is vary random.

The problem is that a task is considered up-to-date when it is not.

I have two project; ‘Convert’ witch depends on SQLSource’.

The problem is that sometimes the task ‘makeAntlrOutputDir’ is up-to-date even if the folder does not exist. And sometimes is it not up-to-date when the folder is missing. which is the behavior I like.
The result is that the task *generateSqlParserAntlr’ fails because some folders are missing.

The commands that are run are:
gradlew clean
gradlew build

When the build first fails it fails consistently there after.
If I then changes a build files and run the commands again, then everything is fin. But only for some time and it goes back to failing.

Am I doing something wrong or this a bug?
I’m worried about the inconsistency.

Convert/build.gradle

import java.text.SimpleDateFormat

apply plugin: 'java'

sourceCompatibility = 1.7
targetCompatibility = 1.7

javadoc.options.encoding = 'UTF-8'
tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

repositories {
    mavenCentral()
}

dependencies {
    compile project(':SQLSource')

    testCompile 'junit:junit:4.12'
}

sourceSets {
    main {
        java.srcDirs = ['src']
    }
}

SQLSource/build.gradle

apply plugin: 'java'

ext.antlr4 = [
    generatedDir             : 'target/generated-sources/antlr4',

    sqlParserSourceDir       : 'src/dk/xact/sqlparser',
    sqlParserDestDir         : 'target/generated-sources/antlr4/dk/xact/sqlparser',
    sqlParserGrammarPackage  : 'dk.xact.sqlparser'
]

sourceCompatibility = 1.7
targetCompatibility = 1.7

javadoc.options.encoding = 'UTF-8'
tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

repositories {
    mavenCentral()
}

configurations {
    antlr
}

dependencies {
    compile 'org.antlr:antlr4:4.5'

    testCompile 'junit:junit:4.12'

    antlr 'org.antlr:antlr4:4.5'
}

sourceSets {
    main {
        java.srcDirs = ['src', 'target/generated-sources/antlr4']
    }
}

task makeAntlrOutputDir() {
    description 'Create output dir for Antlr.'

    doFirst {
        file(antlr4.sqlParserDestDir).mkdirs()
    }

    outputs.dir antlr4.sqlParserDestDir
}

task generateSqlParserAntlr(type: JavaExec, dependsOn: makeAntlrOutputDir) {
    description 'Generates ANTLR parser from SQL grammar files.'

    main = 'org.antlr.v4.Tool'
    classpath = configurations.antlr
    def grammars = fileTree(dir: antlr4.sqlParserSourceDir, includes: ['**/*.g', '**/*.g4'])
    def target = file(antlr4.sqlParserDestDir)

    args = [
        '-o', target,                // specify output directory where all output is generated
        '-lib', target,              // specify location of grammars, tokens files
        '-encoding', 'UTF-8',        // specify grammar file encoding; e.g., euc-jp
        '-message-format', 'antlr',  // specify output style for messages in antlr, gnu, vs2005
        '-long-messages',            // show exception details when available for errors and warnings
        '-listener',                 // generate parse tree listener (default)
        '-visitor',                  // generate parse tree visitor
        '-package', antlr4.sqlParserGrammarPackage,   // specify a package/namespace for the generated code
        grammars.files
    ].flatten();

    // setup when this task is up to date.
    inputs.files grammars

    // TODO: Tweak the outputs collection so it is correct with combined grammars as well as separate Lexer and Parser grammars.
    outputs.dir antlr4.sqlParserDestDir
}

compileJava {
    dependsOn 'generateSqlParserAntlr'

    // add Antlr generated dir to source so it can be compiled.
    source antlr4.generatedDir
}

javadoc {
    source antlr4.generatedDir
    exclude '**/*.tokens'
}

task cleanAntlr() {
    description 'deletes the Antlr generated dir.'

    doLast {
        delete antlr4.generatedDir
    }
}

clean {
    dependsOn 'cleanAntlr'
}

Gradlew -v

Gradle 3.2

Build time: 2016-11-14 12:32:59 UTC
Revision: 5d11ba7bc3d79aa2fbe7c30a022766f4532bbe0f

Groovy: 2.4.7
Ant: Apache Ant™ version 1.9.6 compiled on June 29 2015
JVM: 1.7.0_80 (Oracle Corporation 24.80-b11)
OS: Windows 7 6.1 amd64

2

You don’t need a separate task to create the output dir, just move the doFirst into your generator task.

3

Sure that fixes my problem. But it does not explain why I experience inconsistent behavior.

4

A year into Gradle and this still throws me off.
I belive the problem is 2fold

  1. You have no inputs defined
  2. Your only output is a directory.

Reading here:
https://docs.gradle.org/current/userguide/more_about_tasks.html

Note that if a task has an output directory specified, any files added to that directory since the last time it was executed are ignored and will NOT cause the task to be out of date. This is so unrelated tasks may share an output directory without interfering with each other. If this is not the behaviour you want for some reason, consider using TaskOutputs.upToDateWhen(groovy.lang.Closure)

5

I need to read that chapter again.
I think you link is spot on.

But it still does not explain why I can’t replicate the problem on Windows and only sometimes on Linux.

6

Informed Guess:

If you start from 0 (clean the .gradle caches in the project and $HOME/.gradle/task-cache directory)
The very first time you run – the task should be not ‘UP TO DATE’ – and should run successfully.
Now if you do a ‘clean’ or otherwise delete the directory but change nothing else – not the caches or the build script or anything , since there are no inputs and due to the rule above, the task may be considered “UP TO DATE” and not run.
Once in this state it wont run again. Actually it wont run again either way until gradle decides its task state cache is out of wack.

First suggestion, change your clean to use the Delete task type

task cleanAntlr(type: Delete ) {
    description 'deletes the Antlr generated dir.'
    delete antlr4.generatedDir
}

As it stands, gradle doesn’t ‘know’ your task deleted the directory – so may cache its prior existance.
Whenever possible use a built-in grade Task type for all file creation/deletion/copy – otherwise you need to inform gradle of the effect to seemingly unrelated tasks via explicit inputs and outputs.

If that doesn’t work

Suggest adding one or more of

makeAntlrOutputDir.outputs.files.upToDateWhen {false  /* or a check for directory existing */}

generateSqlParserAntlr.mustRunAfter makeAntlrOutputDir

an explicit input dependency property or file that is guaranteed to change when you need it to.

makeAnglrOutputDir.inputs.property  "a.property" , somePropertyThatChanges
7

If you change the output of a task, it is re-run.

Gradle does know, it can see the file missing on the file system. This has nothing to do with Delete tasks.

If you delete the output directory and Gradle doesn’t pick that up, that would be a bug. A reproducible example project would be very welcome.