Using jgit with Gradle configuration cache

TL;DR at the bottom

I’m working on writing a Gradle plugin task that should be able to calculate a new version for a Gradle project. The version computation utilizes jgit as a means to determine your current version and what the next version should be. When updating the Gradle plugin to Gradle 8.1, the stable configuration cache now complains that an external process has been started during configuration time.

    2 problems were found storing the configuration cache.
    - Plugin 'x.y.z.plugin': external process started '/usr/local/bin/git --version'
      See https://docs.gradle.org/8.1/userguide/configuration_cache.html#config_cache:requirements:external_processes
    - Plugin 'x.y.z.plugin': external process started '/usr/local/bin/git config --system --show-origin --list -z'
      See https://docs.gradle.org/8.1/userguide/configuration_cache.html#config_cache:requirements:external_processes

    See the complete report at file:///path/to/configuration-cache-report.html
    > Starting an external process '/usr/local/bin/git --version' during configuration time is unsupported.
    > Starting an external process '/usr/local/bin/git config --system --show-origin --list -z' during configuration time is unsupported.

I followed the links provided to see if there was anything I could do. I don’t think any of the suggested links work from above since I can’t simply wrap jgit commands in an ExecOperations process and expect things to work (I tried it and it still complained). Example below:

abstract class CurrentSemverTask @Inject constructor(
    private val execOperations: ExecOperations
) : DefaultTask() {
    @get:Input
    abstract val version: Property<String>

    @get:Input
    abstract val versionTagName: Property<String>

    @TaskAction
    fun currentSemver() {
        execOperations.exec {
            logger.lifecycle("version: ${version.get()}")
            logger.lifecycle("versionTagName: ${versionTagName.get()}")
        }
    }
}

The task registration looks like this (please note that the extension fields version and versionTagName are lazy { } computed fields)

val semver = project.extensions.create<SemverExtension>("semver")

project.tasks.register<CurrentSemverTask>("currentSemver") {
    version.set(semver.version)
     versionTagName.set(semver.versionTagName)
}

A few questions I have from this are:

  1. How would I execute git commands via jgit in a Gradle plugin to be compliant with the configuration cache?
  2. How do I properly defer version calculation to task execution time instead of configuration time? Maybe these fields need to be Gradle properties with conventions instead?
  3. Or… how do I completely disable configuration cache tracking for this task so that the configuration cache is never used for it even when a project using this has the configuration cache enabled?

TL;DR: How to you invoke an external process via a library like jgit that is compatible with the configuration cache?

Do the external system calls in a ValueSource and make sure what you do in there finishes fast.
It is executed always, wether configuration cache is to be reused or not.
Actually, if the output of the ValueSource changes, this invalidates the configuration cache entry.
And in this case, the ValueSource is also evaluated again.
So really make sure what you do in there is executed fast.


If you care about a personal opinion, don’t do it at all.
I personally dislike any such auto-versioning-from-VCS things, as you for example then cannot reproducibly build from exports where no VCS is available.

And as JGit does still not yet support Git worktrees, you could not even build in a worktree.

Thanks @Vampire!

Just to clarify, are you suggesting to make the external system calls that I need as a complete alternative to using JGit, executing the calls that JGit will make in a ValueSource prior to making the JGit calls, or are you suggesting wrapping the JGit calls in a ValueSource?

The latter. Doing the calls inside and outside the value source wouldn’t change anything.

Besides that you don’t know which other calls it might make. If you for example look at the sources it might also make some bash calls. And it might even make different calls on different platforms.

Excellent! I will give that a try when I have time. Hopefully that will yield some better results. Thanks again for the help!

1 Like

Hi again @Vampire :wave:

I tried implementing some different variations of ValueSources. None of them seemed to work, erroring with the same message shown at the bottom.

As a test to validate my understanding of ValueSource used in conjunction with JGit, I tried the following:

GitStatusIsCleanValueSource

import org.eclipse.jgit.api.Git
import org.gradle.api.provider.Property
import org.gradle.api.provider.ValueSource
import org.gradle.api.provider.ValueSourceParameters

abstract class GitStatusIsCleanValueSource : ValueSource<Boolean, GitStatusIsCleanTest.Params> {
    interface Params : ValueSourceParameters {
        val git: Property<Git>
    }

    override fun obtain(): Boolean {
        return parameters.git.get().status().call().isClean
    }
}

IsStatusCleanTask

abstract class IsStatusCleanTask : DefaultTask() {
    @get:Input
    abstract val status: Property<Boolean>

    @TaskAction
    fun printStatus() {
        logger.lifecycle("Is status clean: ${status.get()}")
    }
}

Get ValueSource result

fun checkIfCleanStatus(): Boolean {
    val git = // Setup `Git` object
    val statusProvider = providerFactory.of(GitStatusIsCleanValueSource::class.java) { spec ->
        spec.parameters { params ->
            params.git.set(git)
        }
    }
    return statusProvider.get()
}

Register IsStatusCleanTask

project.tasks.register<IsStatusCleanTask>("isClean") {
    status.set(checkIfCleanStatus())
}

This all seems to error with:

1: Task failed with an exception.
-----------
* What went wrong:
A problem occurred configuring root project 'config-cache-tester'.
> Could not create task ':isClean'.
   > Could not isolate value x.y.z.internal.valuesources.GitStatusIsCleanValueSource$Params_Decorated@11ad3d84 of type GitStatusIsCleanValueSource.Params
      > Could not serialize value of type Git

My assumption is that the ValueSource prefers simple data classes as parameters as opposed to any of the complex classes that JGit provides. Maybe I should be wrapping my JGit calls in execOperations.exec { }? But then I’m not really sure how to get the output from the exec call.

Any thoughts?

If Git were serializable it could maybe work, but without that it cannot work, as Gradle needs to be able to isolate the value source parameters.

But even if it would work, are you sure the external process calls are not already done during “// Setup Git object” part anyway, so you would win nothing. JGit is a pure Java implementation of a Git client. The native calls to git.exe or bash (at least the ones it is complaining here about, not sure whether there are others) is just to determine the path to the global Git configuration file so that JGit uses the same configuration as the commandline Git client would use.

But anyway, just do the “// Setup Git object” within your value source and it should probably work.

Thanks for the suggestion! I just recalled that I did that before in another task I created in order to avoid attempting to serialize/store the Git object in the configuration cache. After creating the Git object in the ValueSource the task seemed to work without issues.

Now I guess I’ll have to try to clean all of this up and remove some redundancies since I won’t just be making one or two git calls :sweat_smile:.

Thanks again for your help!

For anyone reading in the future, this how I changed my ValueSource definition:

import org.eclipse.jgit.api.Git
import org.eclipse.jgit.storage.file.FileRepositoryBuilder
import org.gradle.api.provider.Property
import org.gradle.api.provider.ValueSource
import org.gradle.api.provider.ValueSourceParameters
import java.io.File

abstract class GitStatusIsCleanValueSource : ValueSource<Boolean, GitStatusIsCleanValueSource.Params> {
    interface Params : ValueSourceParameters {
        val gitDir: Property<File>
    }

    override fun obtain(): Boolean {
        // Clearly, creating a git object for every value source could become expensive. It may be worth
        // exploring if this can be put into some global `lazy` field that gets fetched from.
        val git = Git(
            FileRepositoryBuilder()
                .setGitDir(parameters.gitDir.get())
                .readEnvironment()
                .findGitDir()
                .build()
        )

        return git.status().call().isClean
    }
}
1 Like

Maybe you should then instead have a shared build service that does the various Git calls and use that. But it might not yet work.
I remember there were some quirks with wedding a build service with a value source that might not yet be resolved.

Very interesting! Thank you for sharing about BuildServices. I can see how this could be very useful. I haven’t utilized them yet, but I will have to watch this issue and hopefully utilize a build service in a value source in the future.

1 Like