Incremental / Caching from Settings Plugins

Help/Discuss
1

How are settings plugins supposed to take advantage of caching or incremental build features?

The @Input* annotations appear to only be related to tasks, which aren’t applicable to Settings plugins.

In addition, if I hand roll my own caching mechanism and write files to the the build directory, it conflicts with the clean task. For example, If I run ./gradlew clean assemble It seems like the order of operations is settings evaluation → clean → assemble/build, so the clean task wipes out my newly created cache files and they won’t get saved.

Is there any best practice out there for implementing caching / incremental builds with settings plugins?

2

What exactly do you do in the settings plugin that warrants caching?
Maybe it is more an error to do such heavy computation in a settings plugin.
Maybe what you are after to speed up your builds is the configuration cache, which can skip all things and jump directly to task execution.

3

In my version catalog plugin I am saving the generated catalog to a TOML file. If the catalog exists I want to just load the file instead of regenerating it from scratch

4

If you really need to cache there, you should probably do it somewhere in gradle.projectCacheDir (which usually is .gradle in the root project directory, but can be changed to be a different location by the user, so better not hard-code it)

5

I’m not seeing an accessor to projectCacheDir anywhere, do you know where to find it? Also would the clean task properly remove my files from there?

6

No it doesn’t, that’s why I said it.
Sorry, my fault, not gradle.projectCacheDir, but gradle.startParameter.projectCacheDir.

7

I can use buildFinished but looks like that’s being deprecated

 try {
  config.settings.gradle.buildFinished {
    cachedPath.parent.createDirectories()
    Files.write(
      cachedPath, // user supplied path, but works with build directory
      myToml.toToml().toByteArray(Charset.defaultCharset()),
    )
  }
} catch (e: IOException) {
  logger.warn("error creating cached library file {}", cachedPath, e)
}

I feel like the available functionality for settings plugins is very limited compared to project plugins

8

Actually, I think anything you can do from a project plugin should also be doable from a settings plugin and more.
But yeah, project plugins are the main extension point you should use.
You can also have a project plugin and a settings plugin and apply the project plugin from the settings plugin if that really is what you want to do.

buildFinished should indeed not be used. You can instead either create a shared build service that is also an operation completion listener and an autocloseable and register it as task completion listener, then the close method will be executed between the last task finishing to run and the end of the whole build. Or you can use the new and incubating Dataflow Actions.

9

The build service and Dataflow actions look interesting, thanks for the info. Do you have any examples of applying plugins to projects from a setting plugin? Usually when I try to access projects from settings plugins I get an exception because the projects haven’t been initialized yet.

10 
gradle.projectsEvaluated {
    gradle.allprojects {
        apply<MyBasePlugin>()
    }
}
11

thanks, this is helpful. I did some experimenting and I think I can use this approach and just add a custom task. The below seems to be working well and handling how I want it to.

EDIT: After I wrote this I realized this will only work for saving. I don’t have any way of getting the the layout.buildDirectory value before the project is evaluated, but I need to create / generate the version catalogs before that step

settings.gradle.projectsEvaluated {
  registerSaveTask(rootProject, fileName, contents)
}

fun registerSaveTask(project: Project, fileName: String, contents: String) {
        with(project) {
            val task =
                tasks.register<SaveTask>("save${fileName}") {
                    destinationDir.set(layout.buildDirectory.dir("version-catalogs").get())
                    destinationFile.set(project.file(fileName))
                    contents.set(container.toToml())
                }
            project.tasks["assemble"].finalizedBy(task)
        }
    }

abstract class SaveTask : DefaultTask() {
    @get:Input abstract val contents: Property<String>
    @get:OutputDirectory abstract val destinationDir: DirectoryProperty
    @get:OutputFile abstract val destinationFile: RegularFileProperty

    @TaskAction
    fun save() {
        with(destinationDir.get()) {
            asFile.mkdirs()
            file(destinationFile.get().asFile.name).asFile.writeText(contents.get())
        }
    }
}
12

I thought you want to not store it in the build directory which is deleted by clean.

13

Apologies for the confusion, I do want to store in build and I do want clean to remove the files.

The issue I was encountering was based on the order of operations:

  1. settings plugin run
  2. clean

So if you run ./gradlew clean assemble, my plugin code stores the artifacts in the build folder but then the clean task removes them right after.

However, if instead you do

  1. ./gradlew clean
  2. ./gradlew assemble

Everything works, but that is not great experience for the user.

14

Ah, I see. Yeah, then registering a task that stores the file might be what you want. :slight_smile: