How does Gradle decide a task has changed?

Help/Discuss
1

I have a custom task that I’m implementing in Groovy.

task mytask(type: MyTask) {
  arg 'one'
  arg 'two'
  arg 'three'
}

I have a method arg(String name) on MyTask that gets called for each arg.

Fine.

Now I edit the build.gradle file and remove arg 'three', but when I run Gradle again, even if I delete the build directory entirely, the task executed has one, two, and three. So clearly the daemon doesn’t think that the task has changed. Or something.

There’s a subtler case as well when I have a global configuration:

mytask.configure {
   arg 'global'
}

where MyTask uses the global configuration in addition to the local config. If I change the global config, that should invalidate mytask as well.

I don’t really understand what the daemon is doing to decide if the task changed. I’d have thought editing the build.gradle file would be enough, but apparently not.

Is there an API I should be calling to provide a hash for the daemon or something?

2

the task executed has one, two, and three

Does it really execute with 1, 2, and 3?
Or is it considered up-to-date, or taken from build cache?
The former would be really strange and would need an MCVE to see what is going on.
If one of the latter, then you probably didn’t declare the inputs and outputs of MyTask properly.

3

I’m deleting build between runs so that it’s never up-to-date. I’ll try to assemble a test case to demonstrate it.

4

It could still come from the build cache if your task is cacheable and the cache is enabled.

5

How can I tell if it’s coming from the cache and what can I do to specify what constitutes valid in the cache?

I can’t now reproduce the “simple” case, though I swear I saw it this morning. In can still reproduce this one:

saxon.configure {
  debug true
  arg '-t'
}

task xform(type: SaxonXsltTask) {
  stylesheet 'xsl/html5.xsl'
  input 'xml/input-2.xml'
  output 'build/out1.xml'
  arg '-now:2023-04-17T12:34:44Z'
}

Now the transform runs with -t and -now because the extension knows to combine the global configuration with the task configuration.

Run it. Edit build.gradle:

saxon.configure {
  debug true
}

task xform(type: SaxonXsltTask) {
  stylesheet 'xsl/html5.xsl'
  input 'xml/input-2.xml'
  output 'build/out1.xml'
  arg '-now:2023-04-17T12:34:44Z'
}

and it still runs with -t and -now. It isn’t up-to-date. It runs the task.

In the trace below 1.gradle contains the former, 2.gradle the latter:

$ ./gradlew --stop
Stopping Daemon(s)
1 Daemon stopped

$ cp 1.gradle build.gradle; rm -rf build

$ ./gradlew --console plain xform
Starting a Gradle Daemon, 6 stopped Daemons could not be reused, use --status for details
> Task :saxon-gradle2:compileJava NO-SOURCE
> Task :saxon-gradle2:compileGroovy UP-TO-DATE
> Task :saxon-gradle2:pluginDescriptors UP-TO-DATE
> Task :saxon-gradle2:processResources UP-TO-DATE
> Task :saxon-gradle2:classes UP-TO-DATE
> Task :saxon-gradle2:jar UP-TO-DATE

> Configure project :
Configuring default SaxonXslt plugin
Plugin option: debug=true
Arg: -t
Configuring 'null' SaxonXslt plugin
Sty: xsl/html5.xsl
Inp: xml/input-2.xml
Out: build/out1.xml
Arg: -now:2023-04-17T12:34:44Z

> Task :xform
com.nwalsh.gradle.saxon.SaxonPluginConfiguration@5384996c
[-t, -now:2023-04-17T12:34:44Z]
Transform -t -now:2023-04-17T12:34:44Z -s:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xml/input-2.xml -xsl:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xsl/html5.xsl -o:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/build/out1.xml title=Purchase Order padding=0.625rem
SaxonJ-HE 12.0 from Saxonica
Java version 11.0.13
Stylesheet compilation time: 558.337879ms
Processing file:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xml/input-2.xml
Using parser com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser
Building tree for file:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xml/input-2.xml using class net.sf.saxon.tree.tiny.TinyBuilder
Tree built in 3.921007ms
Tree size: 28 nodes, 187 characters, 6 attributes
Execution time: 97.631967ms
Memory used: 99Mb

BUILD SUCCESSFUL in 8s
5 actionable tasks: 1 executed, 4 up-to-date

$ cp 2.gradle build.gradle; rm -rf build

$ ./gradlew --console plain xform
> Task :saxon-gradle2:compileJava NO-SOURCE
> Task :saxon-gradle2:compileGroovy UP-TO-DATE
> Task :saxon-gradle2:pluginDescriptors UP-TO-DATE
> Task :saxon-gradle2:processResources UP-TO-DATE
> Task :saxon-gradle2:classes UP-TO-DATE
> Task :saxon-gradle2:jar UP-TO-DATE

> Configure project :
Plugin option: debug=true
Configuring 'null' SaxonXslt plugin
Sty: xsl/html5.xsl
Inp: xml/input-2.xml
Out: build/out1.xml
Arg: -now:2023-04-17T12:34:44Z

> Task :xform
com.nwalsh.gradle.saxon.SaxonPluginConfiguration@21466e18
[-t, -now:2023-04-17T12:34:44Z]
Transform -t -now:2023-04-17T12:34:44Z -s:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xml/input-2.xml -xsl:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xsl/html5.xsl -o:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/build/out1.xml title=Purchase Order padding=0.625rem
SaxonJ-HE 12.0 from Saxonica
Java version 11.0.13
Stylesheet compilation time: 47.300734ms
Processing file:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xml/input-2.xml
Using parser com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser
Building tree for file:/Volumes/Projects/gradle/saxon-gradle2/examples/simple/xml/input-2.xml using class net.sf.saxon.tree.tiny.TinyBuilder
Tree built in 1.455715ms
Tree size: 28 nodes, 187 characters, 6 attributes
Execution time: 10.440671ms
Memory used: 128Mb

BUILD SUCCESSFUL in 1s
5 actionable tasks: 1 executed, 4 up-to-date

You can see in the second run that the -t argument isn’t in the configuration and yet that’s what runs. I suppose this is because the task is cached (by default in Gradle 8.0?). But it’s really probematic for the user unless my use of a global configuration is just a bad ideal

6

How can I tell if it’s coming from the cache

When executing with -i or with --console=verbose in a build scan (--scan), the task should show as FROM-CACHE if it comes from the cache.

what can I do to specify what constitutes valid in the cache?

As I said, a task needs to define its inputs and output properly for up-to-date checks and build cache to work properly: Incremental build

In can still reproduce this one:

From what I see, I would guess you use static state, for example in SaxonPluginConfiguration which is a really bad idea. Because static state leaks into other builds run on the same daemon, even when building completely different projects. If you need a “data holder” that is scoped to the execution of one build, you could for example use a shared build service which has exactly that lifecycle. static is evil in that context.

7

Okay. I can accept that my attempt was a bad idea :slight_smile: It might be easiest to abandon it except that it is convenient to be able to collect a bunch of common settings together for reuse. Can you point me to any kind of example of what a “shared build service” with that lifecycle woud look like?

Maybe I should just stick them in a variable or something that I reuse, perhaps in a(nother) closure.

8

Shared Build Services documents the shared build services. They are things that have one build execution as scope. It can be used for services that need to be started once per build and reused by several tasks and then shut down, or it can be used to restrict access to shared resources to a maximum amount of concurrent usage, or they can be used to share state during one build execution.

9

But actually, in your situation it might be more appropriate to have an extensions where you define the global settings that are then used by multiple tasks.

10

Thanks. I’ll try to remember about shared build services in the future, but that seems much too heavyweight for what I need here. One of my goals in (re)writing this extension is to strip it back to the bare essentials. I think I can probaby get by with:

def sharedargs = ['-t', '-x:y', '-etc']

task myTask(type: MyType) {
  args sharedargs
  arg '-somethingForJustThisTask'
}

And that seems to work. Editing sharedargs causes the task to be re-executed with the correct arguments.

But I’m curious about what you mean by defining an extension that’s used by multiple tasks. That’s what I was trying to do with my naive use of static fields. What woud it look like if I was “doing it right”?

11

Here you see simple examples of how to add extensions that are then configured in the build script and wired to tasks: Developing Custom Gradle Plugins

If you just want it in your specific build and not for all plugin users, you can of course simply use such a variable, or configure all tasks of that type like

tasks.withType(MyType).configureEach {
    args '-t', '-x:y', '-etc'
}

it if is for all tasks of that type anyway.

12

Right. That plugin documentation is basically what I’m doing. The tasks.withType trick is worth remembering, thanks!

I’m mostly thinking of the case where there are maybe a dozen or more tasks of type MyTask and I want many of them to share a common set of options. Doing that with a variable seems easy enough for now.

1 Like