Two cases where the incremental API is lacking

Help/Discuss
1

I have two Gradle plugins which use the Gradle incremental API, but have to do wonky workarounds. Perhaps these use-cases aren’t worth supporting, but I just wanted to throw them out as possibly helpful testcases in case you revisit the API.

A task that may modify its input

It’s generally a bad idea for a task to modify any of its inputs, but one notable counter-example is a code formatting task. Example:

  1. Change MyClass.java, and its formatting is now bad.
  2. Run spotlessApply, which uses Gradle’s incremental to run a formatter on only that one changed file.
  3. If the user immediately runs spotlessApply again it ought to be up-to-date (in an ideal world), but it’s not because the file was changed by spotlessApply.
  4. The third run of spotlessApply will now be up-to-date.

Ideally, there would be something like IncrementalTaskInputs::setUpToDate(File, boolean) or InputFileDetails::setUpToDate(boolean) that we could call after modifying a file so that the task would be up-to-date in step 3 instead of step 4.

That minor performance hit is not a huge deal, but there’s actually a bug that I missed for a long time hiding there:

  1. Change MyClass.java, and its formatting is now bad.
  2. Run spotlessApply, which uses Gradle’s incremental to run a formatter on only that file.
  3. The user manually changes MyClass.java back to its pre-formatted content.
  4. Run spotlessApply, which ought to run in an ideal world because MyClass.java has bad formatting. But because step 2 was successful, and the content of MyClass.java is exactly the same as it was at the beginning of step 2, the task is marked as up-to-date, and the badly formatted file lives on.

We handle this by creating a synthetic extra file in the build directory which we treat as an incremental input. Into this file we store a timestamp and the file names that were changed in the previous round. That way we are able to detect and handle the obscure use-case above. This would be unnecessary if we had something like InputFileDetails::setUpToDate(boolean).

code change, bug, full PR

A task where a file’s output files might depend on its content

There are some cases where, for each input file, the name and number of its output files might depend on its content. But with the current API, when a file is removed or changed, there is no way to get information on its previous content.

We handle this by storing this information ourselves in a special file in the build directory.

Ideally, there would be something like InputFileDetails::setMetadata(Object serializableMetadata) which would be available in future runs for changed and removed files
as InputFileDetails::getMetadata().

code example, test case where this is useful

Probs not your top priority

Right now, the incremental API assumes:

  1. None of the inputs will change (probably a pretty good assumption)
  2. Responding to a changed or removed file will not require any information about the previous content (an okay assumption for most Java, not great for other kinds of tasks)

If you’re looking for simple but useful tasks which don’t meet these two assumptions, see the links above.

Thanks again for Gradle! Can’t believe the consistent improvement, release after release.

2 Likes
2

Did you get any reply outside this thread, @Ned_Twigg? We’re currently facing problem #1 as well.

A proper solution to the first problem would be nice, since that use case is not too uncommon.

3

Perhaps you could have two tasks

  1. A Formatter task which formats files in src/main/java but writes changes to $buildDir/formatted/main/java
  2. A Copy task which copies from $buildDir/formatted/main/java to src/main/java

Each of these two tasks would have separate input and output directories so you can benefit from up-to-date checking

4

Nope. @uklance’s suggestion or the code change links above are all I’ve got.