Gradle 2.4 - Cache unreliable

I’m currently trying to port a large project from maven to gradle (so the build.gradle files change frequently). Since this is a pretty complex build, we have a lot of custom tasks, each of which specifies its input files and the output directories to make sure it doesn’t run more often than necessary.

Every once in a while, gradle suddenly decides that one of our custom tasks is UP-TO-DATE, although the output directory doesn’t even exist. “gradle clean” doesn’t help for obvious reasons, so I have to delete my .gradle directory at least twice a day. This is becoming a huge issue for me, since I can never be sure whether there’s something wrong with my build script or gradle itself when the build fails after editing build.gradle.

Any idea how to debug this issue? Or are there known limitations/bugs that might cause this behavior? I only found GRADLE-750, which is pretty old.

This is likely due to a change in one of your custom task implementations that has some kind of UP-TO-DATE implications that Gradle doesn’t necessarily know about. It shouldn’t be necessary to delete the .gradle folder in these cases. If you know you should run tasks again regardless of what Gradle thinks is UP-TO-DATE you can simply use the --rerun-tasks command line option.

Good guess, but that can’t be it, those tasks haven’t changed in days and I encountered the bug again today when I moved other stuff around in my build scripts. The set of input/output/source files of every single task never changed during the last few weeks, either.

Also, this wouldn’t explain why gradle thinks a task is UP-TO-DATE if the output directory doesn’t even exist, would it?

Only if it believes the previous execution produced no output. Or if there are no inputs.

Are you explicitly deleting the task’s output (running clean, etc) running it again and encountering that it is UP-TO-DATE? If so I would love to have some kind of repeatable test case for this.

In that case, that would be the bug, because the script either produces output files or fails with a non-zero exit code (the tasks all extend Exec and execute an external script ATM).

Yes, that’s what I’m experiencing. I run gradle build, then change something in the build scripts, run gradle clean, run gradle build again and experience the bug. Unfortunately, I haven’t been able to reproduce it reliably, so I can’t tell you what exactly causes that behavior. Is there anything I can do to debug this the next time I experience this situation?

Usually using the --info option from the command line will give you some additional insight into the UP-TO-DATE checking that might be helpful. If you are indeed specifying an output directory, and that directory is being populated then removed and Gradle still thinks the task is UP-TO-DATE that is indeed a bug.

Ok, it just happened again, so I ran gradle clean and gradle --info generate in the affected subproject. The only output regarding the failing task (generate) was:

:subproject:generate (Thread[Task worker Thread 2,5,main]) started.
:subproject:generate
Skipping task ':subproject:generate' as it is up-to-date (took 0.001 secs).
:subproject:generate UP-TO-DATE
:subproject:generate (Thread[Task worker Thread 2,5,main]) completed. Took 0.001 secs.

For the other custom tasks, I’m getting info messages about the in-memory cache, but not for this one (!).

The task generate is defined in the root project as follows:

subprojects {
  afterEvaluate { project ->
    if (project.needsGenerate) {
      task generate(dependsOn: [...]) {
        inputs.dir [...]
        inputs.file [...]
        inputs.source [...]
        outputs.dir [...]
        doLast {
          // find some files in the input dir (inputs.dir, relative to subproject root) via fileTree and execute an external script on each of them
        }
      }

      project.compileJava.dependsOn generate
    }
  }
}

Any ideas?

Edit: The task was defined in the same place before and worked. What happened between the successful and the failing run (as far as I remember) was:

  • I added another, unrelated task definition afterEvaluate in a similar manner (that gets added to another subset of subprojects)
  • I built another subproject to test the new task definition

I’m not sure if this is affecting UP-TO-DATE checking but it shouldn’t be necessary to conditionally create the task, and in general this is somewhat of a bad practice. A better pattern is to use onlyIf { }.

task generate {
   onlyIf {
       project.needsGenerate
   }
}

Thanks, that’s good to know.

But I don’t think the way I’m adding the task to the subprojects has anything to do with the problem either, because different tasks that were defined in the build.gradle of some subproject directly have failed before.

Note, a task will be considered UP-TO-DATE if you have specified a path using inputs.source but no source files exist. I’m grasping at straws here at this point as there’s not a lot we can do without some self-contained reproducible example.

The source file’s still there and after deleting the .gradle folder, everything builds fine again.

I’m still trying to figure out the exact circumstances under which the problem occurs, so I have no idea how to build an example for you. Is there anything else I can do to dig deeper? I really want to find the source of this bug, but I’m completely lost right now.

The only other thing I could think of is if you are conditionally do other things (like defining the inputs or outputs of the task). My guess is something about your build script is dynamic and changing between runs. You can attempt to debug this but again, that’s not going to be of much help unless you can consistently reproduce the problem. Are you using features like --parallel or --configure-on-demand? If so, have you tried turning those off?

Also, did this just start happening with Gradle 2.4 or do you have the same issue with Gradle 2.3?

Yes, I am. So far, I have only tried turning them off after encountering the problem first. It did not help in that situation. I’ll turn them off permanently now and check whether the problem persists.

Gradle 2.4 is the first version I’ve ever used.

OK, turning those features off helped, but in another way than I expected. As it turns out, there was a race condition between :subprojectA:generate and :subprojectB:generate. There was a dependency between those tasks that I wasn’t aware of, so the build would only succeed if :subprojectA:generate ran before :subprojectB:generate. The order in which they were actually run was arbitrary, since subprojectB specified a compile-time dependency on subprojectA and compileJava depended on generate in both subprojects, but :subprojectB:generate didn’t explicitly depend on :subprojectA:generate.

I’m still not sure why gradle stored the result of :subprojectB:generate in the first place, though, since it should have failed with a non-zero exit code in case :subprojectA:generate had not run before it. If it had ever reported an error in :subprojectB:generate, I would have found the bug in my build script much faster. Anyway, consider this issue resolved and thanks for your help.

In general having cross-project task dependencies is bad practice. Can you elaborate one why the generate task for one project needs to run before another? This should be modeled explicitly using project dependencies.

:subprojectA:generate generates files in a directory that :subprojectB:generate reads from.

Is there another way of modeling project dependencies than specifying the following in subprojectB:

dependencies {
  compile project(':subprojectA')
}

Because I already did that.

Also, in another java subproject, I need to run a task which uses the binary generated by a C++ subproject before compileJava. How would I do that without cross-project task dependencies?

Then ‘:subprojectA’ should declare an artifact with those files that ‘:subprojectB’ consumes.

Yes, you can define a dependency on a particular project configuration, which would include the artifact containing the required files.

generatedFiles project(path: ':subprojectA', configuration: 'generatedFiles')

Take a look at the docs for ArtifactHandler for details on how to set this up.