which contains incremental changes to the local sources that are uploaded and compiled remotely.
The workflow itself is working great, however I’m having problems when a file fails to compile and I have to change another file which compiled fine before to resolve this issue.
I might have a misunderstanding on how incremental tasks work at this point, so any suggestions are appreciated
Currently it runs like this:
First build is not incremental, so all e.g. 1000 files (in the ConfigurableFileCollection) get uploaded and compiled
One file has an error
Building again is not an incremental build, because the previous build, which wasn’t incremental, has failed (maybe here is my problem, can I flag files as “failed” and others as “passed”?)
To resolve this, I have a @LocalState file which contains the paths to files that have an error so I can pick them up again
Next build is running because (I think?) the @LocalState file changed. I extract the file from the state file and compile it again
Now if step 5 doesn’t succeed because it still can’t be compiled and I have to do changes in a different file, then the next build still only takes the original failed file from the state file, because InputChanges::isIncremental returns false and would want to build all the files again.
Is there a way I accomplish what I’m trying to do or are any assumptions I make wrong? Do I need to structure the task differently?
@LocalState is meant for when you have something with built-in incrementality handling like the Kotlin compiler with local state that must not be reused when the results are coming from the task output cache.
In those cases, the local state is annotated like that, so that it gets deleted when the task result is coming from the task output cache.
In the same situation, but with relocatable local state, it could be declared as output of the task, so that it is saved in the task output cache too and can be reused even if the results are coming from the task output cache.
When using InputChanges you usually do not use @LocalState, but you just rely on Gradle telling you which input files changed and need re-processing.
How can I tell Gradle know what files in InputChanges need re-processing in the next run?
The task takes all sources files as input, so I have e.g. 1000 files that need processing.
If one file fails to be processed and I fail the build, the next build is another full build.
Can I tell Gradle in the first full build which files need re-processing, so that the next build is incremental and not a full build again?
if the task failed last time => all files are processed
if any output file was removed or modified => all files are processed
if the last time the task was successful and only input files were changed, added, or removed => only the changed files are processed
Not you tell Gradle what needs to be done, Gradle is telling your task what needs to be done.
If you need a different logic, you should not use InputChanges, but then you have to do the full incremental logic yourself, including recognizing when outputs and inputs were modified or removed and so on. The state for this you can either make relocatable, so that you can declare it as output files for your task so that it is put to task output cache too if your task is cacheable and so that when the task result comes from the cache it can immediately work with the incremental state, or if it is non-relocatable, then you annotate it with @LocalState, so that it gets deleted when the task output is restored from task output cache.
If your task is not cacheable, you neither need to declare the incremental state as output files, nor as @LocalState, as it is irrelevant then.
For example, if I have 10 Java sources and 9 of them successfully create class files, there is then 1 class file missing.
Invoking compileJava again, does it process all files or just the one which didn’t create a class file?
I’m trying to figure out if I can still leverage the Gradle cache in my case, because it is so god damn fast to recognize incremental changes
One hacky way I can think about is making my build task always successful and write a non-output and non-localstate “state” file with failures. A followup task runs after the build task and fails if the “state” file is not empty.
With getOutputs().upToDateWhen() in the build task I could then see that I have to do something again and get both incremental changes and take the stuff from the “state” file and add it to the compilation.
