Workaround for any change to buildSrc making all tasks become out of date


(Ned Twigg) #1

As was mentioned here, if you make any change in the buildSrc directory, then every task based on buildSrc will be out-of-date.

One place where Gradle really shines is heavy-duty computation and aggregation, with buildSrc building the piping. That use case is totally shot in 3.0. For now, the only workaround I can see is to stick with 2.14.1.

It would be great if there was someway for a task to opt-out of changed-classpath-out-of-dating, e.g. @ClasspathInsensitive. A more general mechanism might be a method boolean overrideOutOfDate(Reason outOfDateReason).

Are there any other workarounds out there?


(Matt) #2

I have a similar issue. I have some incremental tasks defined in my build.gradle and some of those tasks contain a doLast{}

Any task with a doLast{} (or a TaskAction) defined in the build.gradle will become out of date if any change is made to the build.gradle file. If I run with --info I’ll get a message like
Task ‘:MyProject:SomeTask’ additional action class path has changed from 80a8004a5462187adfc3318bac1201ba to dbc7630445531ccf11544ecb920a728d.

There should be a way to disable this behaviour e.g. a property on TaskInputs.


(Stefan Oehme) #3

Making this configurable is just a recipe for people to shoot themselves in the foot. If an action or one of the classes it uses is changed, then it needs to re-run, as it might do something completely different now.

What you really want is a more fine grained detection of the classpath that a task or action was actually using. We could record all loadClass calls and then only check whether those classes are still up-to-date //cc @lptr


(Ned Twigg) #4

Fine-grained detection of the classpath would not solve my problem. Here is my usecase:

  1. Download and extract ~ a gig of data, into folder 01_someStep
  2. Transform and filter that data into 02_nextStep
  3. Transform and filter that into 03_nextStep
  4. Several other steps, some of which depend on multiple previous steps.

In each case, a standard up-to-date check based on the files takes literally minutes. So, instead I do up-to-date checks based on a single “token” file in the root of the directory, which has the input metadata to the tasks that I expect to change.

Since 3.x, it is entirely impossible to do development on steps 2, 3, 4 in this process, because step 1 will always rerun. Absolutely it is possible to shoot ones’ self in the foot during this process, and we have done so, but dangerous tools are occasionally useful :slight_smile:

Before we adopted gradle, these tasks were run by a public void main(), with clumsy commenting to control which steps to run. With gradle 2.x, we have a great tool for building a task graph with fine control over its execution.

The new default behavior in 3.x (change whenever the classpath changes) is absolutely the right default. But if it’s not possible to override this, then there are some workflows that will have to stick with 2.x indefinitely.


(Stefan Oehme) #5

Of course it would solve your problem. We’d detect that the classes used by step 1 didn’t change and thus not rerun it.


(Ned Twigg) #6

It’s quite likely that the classes used by Step 1 will change while I’m working on Step N. “Then it’s important for Step 1 to run anyway!” To which I’d say “No, Step N uses different methods of the same class. And even if I do something that changes how Step 1 works, I can’t get anything done on Step N unless I can at least hold some fixed input around for a little bit while I tinker.”

In gradle 2.x, the user had the option to take complete control over what defined a task as up-to-date. With 3.x, the user still has all that power except for one magic parameter which is completely outside of user control, with no workaround. I agree that the feature is an improvement for most users, but it’s a showstopper for some power user workflows. I really hope some workaround can be introduced.


(Ben Keller) #7

Ned,
Have you gotten anywhere with this? I’m wondering if making the subdirectories into subprojects with their own build scripts will resolve the issue.


(Ned Twigg) #8

I still use gradle 2.14.1 for these projects. It’s painful to not have all the great improvements gradle has made in 3.x, but it’s the only workaround I’m aware of for this usecase.