Incremental Task - false Up to Date result -

Help/Discuss
1

HI

While implementing a task which calls “npm install” we encountered a problem with the up-to-date check:

The goal was to take advantage of gradle’s incremental task feature – so we defined “package.json” as a task input and npm’s output folder (“node_modules”) as a single @OutputDir.

The problem started when on one environment the user already ran “npm install” manually before running the relevant gradle task – meaning the node_modules folder already existed with everything in it.
When he ran the gradle task for the first time – it ran “npm install” but npm is “smart” in the way that it checks if there’s anything to install – in our case everything in node_modules was already up-to-date so it did nothing.
When the same user deleted one of the modules (a folder) under the node_modules folder – and ran the task for the 2nd time - gradle did not recognize any change in the outputs and the task was considered up-to-date.

After thoroughly testing this scenario we came to the conclusion that gradle is only cashing output files that were created or modified during the task execution - is that correct?

We looked for an official documentation to support that, but we only found this:
“Note that if a task has an output directory specified, any files added to that directory since the last time it was executed are ignored and will NOT cause the task to be out of date”
This only hints about this behavior in the sense of extra files added AFTER the task has been run – but it does not specify that only files created/modified by the task are cached and pre-existing files are not.
If that is really the “expected behavior” I would love to see it stated more clearly in the documentation.

We tried to work around this with cashing the outputs as input property as suggested here:


but that won’t work in case the outputs are recreated and the timestamp is taken into consideration when caching.

We tried to use upToDateWhen but since we are talking about a large amount of files it can be expensive, in other words it will be like implementing gradle’s cache mechanism in the upToDateWhen…

Any other solution or a way to force gradle consider the @OutPutFiles to be cached even if they are not created/modified by the task ?

Thanks
Jonathan

2

This is basically gradle/gradle #1168. The issue is considered fixed, but only if you’re willing to define the folder as something that will be deleted in the clean task. In this case, you would have to be willing to set node_modules as something that will be removed by the clean task.

We worked around this before there was any fix and will probably continue to do so as our developers are used to the clean task only removing things that are built, not installed. There’s an early version of what we used linked in the issue, but basically we just touch all the files in the output directory as an additional task action of our npmInstall task. This causes the snapshotting to work properly.

1 Like
3

Thanks for the quick answer,
Still i am surprise there is no even a small clue on the official documentation regarding this behavior.
I will try both solution to see which one suite most. thanks again