Understanding what we can do at configuration time while staying configuration cache compatible

Help/Discuss
1

I’ve been reading Configuration Cache Requirements for your Build Logic a lot, and I’m having a hard time following some parts of it. Here is what I mean:

  1. Clear part: Running External Processes

Here you say, *.execute() shouldn’t be used in configuration time, but if I must, then I should use a compatible API like providers.exec { commandLine("git", "--version") }.standardOutput.asText.get(). Nice and clear.

  1. Slightly confusing part: Reading System Properties and Environment Variables

I had to read this a few times to understand what you mean. The docs say:

In general, you should avoid reading the value of system properties and environment variables at configuration time to avoid cache misses when these values change. Instead, you can connect the Provider returned by providers.systemProperty() or providers.environmentVariable() to task properties.

I get the part that env vars can cause the configuration cache to be invalidated. That’s okay, if we use it responsibly, right? Because I think there is a confusion about whether we are allowed to use System.getenv('MY_VAR') or we should use something else like providers.environmentVariable('MY_VAR').get() in case we need the value in configuration time.

And then, for example you warn that System.getenv().findAll { ... } is a problematic pattern, and if we need to use something like that, at least we should use providers.environmentVariablesPrefixedBy("JDK_"). However, that’s not the same code, because now you got a provider. You no longer .get() it like in the case of running git --version. I understand that if possible, we should connect such providers to the task input, but what if we can’t, and we need to read the value in configuration time? Right now I can’t come up with a specific example though, and therefore I would like to keep this conversation in theory level.

  1. More confusing part: Undeclared Reading of Files

The docs say:

Plugins and build scripts should not read files directly using the Java, Groovy or Kotlin APIs at configuration time. Instead, declare files as potential build configuration inputs using the value supplier APIs.

But once again, the examples are misleading, because providers.fileContents(layout.projectDirectory.file('some.conf')).asText is not the same as def config = file('some.conf').text. The problem is the same as in the previous point, but now with even less information on “what if I need the value in configuration time?” So this time, I can’t even tell: is it safe to use the original API, or am I supposed to use the provider-based API, but then is it okay to .get() it in configuration time?

It would mean a lot to me if you could clarify these questions for me.

Thanks in advance!

2

Because I think there is a confusion about whether we are allowed to use System.getenv('MY_VAR') or we should use something else like providers.environmentVariable('MY_VAR').get() in case we need the value in configuration time.

If you need the value at configuration time, there should be no difference between the two.
The difference comes in to play when you have an extension or task that has a property - let’s call it foo - of type Property<String> that is only read at execution time.
You can set it with foo = System.getenv('MY_VAR'), but then the CC entry cannot be reused if the env var has a different value.
Whereas if you use foo = providers.environmentVariable('MY_VAR'), then the value of the env var is not part of the CC entry key so changing the value will still allow to reuse the CC entry.

And then, for example you warn that System.getenv().findAll { ... } is a problematic pattern, and if we need to use something like that, at least we should use providers.environmentVariablesPrefixedBy("JDK_") . However, that’s not the same code, because now you got a provider. You no longer .get()

The problem with System.getenv().findAll { ... } (if you do it at configuration time) is, that you read each and every environment variable. So if any environment variable changes, the CC entry cannot be reused.

If you for example need to read all environment variables that start with “JDK_” at configuration time, and use the findAll pattern, you likely never reuse the CC entry. If you instead use providers.environmentVariablesPrefixedBy("JDK_").get(), only those that start wtih JDK_ invalidate the CC entry if they change. Noone said you cannot get() the provider if you really need the value at configuration time.

Another alternative would be to use a ValueSource, in which you can do any retrieval and action that you need to obtain some value and the CC is invalidate only if the final result changes.

But once again, the examples are misleading, because

That depends on the use-case.
Gradle Folks there too probably semantically target cases where the value is actually not needed at confguration time but only used to configure some extension or task properties, and for that it is better to use the Provider-returning method without getting it, as then the contents of the file are not a configuration cache input.

Just the same as for the environment variables example, if you actually do need the value at configuration time for some value, either method is fine, either the traditional way that is captured by the buildscript classpath instrumentation or using the new CC-safe API and then getting the value immediately.

Well, there is one tiny point that might make it better to use the explicitly CC-safe API. Theoretically, the buildscript classpath instrumentation that recognizes the traditional ways to get to those values could be disabled and then those CC inputs would be missed. But that’s just a small edge case because hopefully noone does that.

With running processes it is different, there using the “traditional” ways is explicitly unsupported if you enable CC and the build will fail if you use them at configuration time.

3

Thanks, that’s exactly what I wanted to know: the difference whether in certain cases the traditional way causes issues, or whether the provider way is just best practice, or whether there are nuances like you said. And this is what was not always clear for me just by reading the documentation.

4

You might consider opening a doc-improvement issue so that the Gradle folks can fix it if they agree that it is not clear enough. :slight_smile: