Resolving a configuration while configuring task

Hello
I tried to provide a file from one project to another project using configuration and it took me a few hours to figure out that resolving a configuration in the configuration block of a task can cause a failure in gradle.
If you use configuration on demand and have a producer (project :list) like this:

plugins {
    id("buildlogic.java-library-conventions")
    //id ("ivy-publish") since ivy-publish uses afterEvaluate, same explosion
}
//if this wouldnt be here it work...but cant easily avoid it, see  used by publishing for example
project.afterEvaluate{}

val myFile by configurations.registering{
    isCanBeResolved=false
    isCanBeConsumed=true
}
dependencies{
    //as seen on https://docs.gradle.org/current/userguide/declaring_dependencies.html#ex-declaring-multiple-file-dependencies
    myFile(project.files("Hello"))
}

And a consumer (project :utilities) like this:

plugins {
    id("buildlogic.java-library-conventions")
}
val getTheFile by configurations.registering

dependencies {
    api(project(":list"))
    getTheFile(project(":list","myFile"))
}
// if the next line would be uncommented it would work
// but i guess not recommended?
//val singleFile = getTheFile.get().resolve()
tasks.register("explode"){
    //the line here causes  afterEvaluate in :list to explode
    val singleFile = getTheFile.get().singleFile
    doFirst {
        println("we dont get here... " + singleFile)
    }
}

and execute gradle :utilities:explode we get:

* What went wrong:
Project#afterEvaluate(Action) on project ':list' cannot be executed in the current context.

reproducer repo can be found here GitHub - TheGoesen/gradle-reproducer

So what i would get from that is that resolving a configuration directly in the configuration block of a task is bad (or at least when you use configure on demand) But There isnt really anything about this in the docs, or am I missing something?

From a quick look I’d say there are at least two quirks.

  1. You define “Hello” as dependency. If this is to share the file with another project you should instead add it as artifact. See Sharing outputs between projects for more information.

  2. The getTheFile configuration is an input to your task, so you should declare it as input like inputs.files(getTheFile). This will probably also solve your actual problem.

Hi, thanks for your interest.

  1. In this simplified case yes since i am only sharing one file the artifact block seems like a reasonable alternative. However the original problem I need to share a filecollection with another project… Something similar to val collection = fileTree("res").filter { it.name.endsWith(".xml") }. I cant do that using artifacts I think?
  2. Inputs where ommited for brevity :wink: declaring as an input alone doesnt fix the issue, converting everything into providers seems to. With the configuration cache placing limitations, Add ability to get named Task file inputs and outputs · Issue #21456 · gradle/gradle · GitHub not solved and the behaviour described here it takes a bit of trial and error to figure out something that does work…
//but is this the recommended way?
tasks.register("doesNotExplode"){
    val works = getTheFile.map { it.singleFile }
    inputs.file(works)
    outputs.file(project.layout.buildDirectory.file("blub"))
    doFirst {
        // in this case we could also directly resolve inputs
       // but in production there are often multiple different inputs...
        println("this works" + works)
        outputs.files.singleFile.writeText("we did it")
    }

}

I now cloned your reproducer and run gw explode:

[...]
> Task :utilities:explode
we dont get here... D:\Sourcecode\other\others\gradle-reproducer\list\Hello

BUILD SUCCESSFUL in 4s

So how would the issue be reproduced with your reproducer?
Or do you maybe have some init script that is breaking things?

cant do that using artifacts I think

Why not?
You are not restricted to share only one file.

Furthermore, if you would not use a legacy configuration setup, you would also see that it is not idiomatic what you do.
If you for example use the new configurations.consumable helper, you would instantly get a complaint, that you cannot declare dependencies on that, as for that you should use configurations.dependencyScope, already hinting at what you do is slightly off-track. :slight_smile:

Hello

Sorry for the late reply, I must have missed the notification.
I gave the repo another spin and your are right:
gw explode does not explode :wink:
but gw :utilities:explode does. Also if you execute explode in the utilities directory. But from top level it seems to work fine…

Ok I have to admit I dont understand what you mean…

val asArtifact by configurations.registering{
isCanBeResolved = false
}

artifacts{
    add("asArtifact",files("1","2")) // gradle complains you cant have multiple files as artifacts
    add("asArtifact",file("blub")) // you can also not add another file..
}

like maybe something if I dynamically create a configuration per file, and then create another configuration that extends all other configurations and that configuration can be consumed? Is that a reasonable approach?

Is there any documentation available on what you are talking about? I went by
https://docs.gradle.org/current/userguide/cross_project_publications.html
and i cant find any mentions of configurations.dependencyScope, or configurations.consumable
If I do some guesswork using the release notes I can rewrite my buildfiles like this, if thats what you mean…

configurations.consumable("asArtifact")

artifacts{
    add("asArtifact",file("Hello"))
}

But that does not improve anything? (Also plz enable val config by configurations.registerConsumable or similar)
I guess i can rewrite my consumer like this if its more recommended (for me its just slightly longer…, also does not change anything about the behaviour…)

plugins {
    id("buildlogic.java-library-conventions")
}
val getTheFileAsArtifact = configurations.dependencyScope("deps")
val resolveable = configurations.resolvable("resolve"){
    extendsFrom(getTheFileAsArtifact.get())
}

dependencies {
    api(project(":list"))
    getTheFileAsArtifact(project(":list","asArtifact"))
}

tasks.register("explode"){
    inputs.files(resolveable)
    //the line here explodes afterEvaluate in :list
    val singleFile = resolveable.get().singleFile
    doFirst {
        println("we dont get here... " + singleFile)
    }
}

Are you talking about what is in the documentation referred to as Variant Selection?
I am using configurations over variant selection, because variant selection requires more steps and in my case these artifacts are really not a “variant” of the main jar, its just a bunch of completely unrelated files which have nothing to do with the main jar, but they need to be exported from this project for historical reasons…

Best regards,

but gw :utilities:explode does

Ah ok, I see.
And it makes sense.
You properly leverage task-configuration avoidance, so the code in register { ... } is evaluated pretty late.
At the time it is evaluated, you resolve the getTheFile configuration which actually is a pretty bad idea.
This then causes the list build to be evaluated at a time where afterEvaluate must not be called anymore.

I’m aware of why you most probably do

val singleFile = getTheFile.get().singleFile
doFirst {
    println("we dont get here... " + singleFile)
}

You most probably had

doFirst {
    println("we dont get here... " + getTheFile.get().singleFile)
}

and then got CC problems.
The better fix for that is to do

val getTheFile = getTheFile.map { it as FileCollection }
// or alternatively and the `get()` below removed
//val getTheFile = getTheFile.get() as FileCollection
// or alternatively and the `get()` below removed
//val getTheFile: FileCollection = getTheFile.get()
doFirst {
    println("we dont get here... " + getTheFile.get().singleFile)
}

which then also fixes your exploding issue. (val getTheFile = getTheFile or val getTheFile = getTheFile.get() do not work as CC cannot persist Configuration, but it can persist FileCollection and Configuration is-a FileCollection, you just have to be explicit about that you only need that.

add(“asArtifact”,files(“1”,“2”)) // gradle complains you cant have multiple files as artifacts

Ah, right.
Well, it does not complain you cannot add multiple files, because you can.
It complains that you cannot add a FileCollection.
But you can without problem do files("1","2").forEach { add("asArtifact", it) }.
The problem is just, that the files have to be clear at configuration time already.
If it is only known which files will be there mid-execution, then this does not work as it must be known up-front.

If you do not know up-front which files will be in the collection, you could still have a Sync task that syncs those files to a directory known at configuration time and then configure that directory as artifact, then you would have all the dynamically generated files as artifacts properly.

add(“asArtifact”,file(“blub”)) // you can also not add another file…

Sure you can. You can call that as often as you want and need.

Is that a reasonable approach?

No, because if configuration A extendsFrom configuration B, that means that all dependencies you declare on B are automatically also decleared on A. The artifacts are not “inherited”. But as I just explained this should also not be necessary. As you suggested to create one configuration per file, I assume the files indeed are known at configuration time already and not dynamic, so just add all the files to the one configuration, just one-by-one, not as FileCollection.

Is there any documentation available on what you are talking about? I went by
https://docs.gradle.org/current/userguide/cross_project_publications.html
and i cant find any mentions of configurations.dependencyScope , or configurations.consumable

They are pretty new and still incubating (but I don’t expect much changes to them).
So they might not be documented properly yet, and not used in all places where appropriate.
But basically
isCanBeConsumed = false && isCanBeResolved = false => dependency scope
isCanBeConsumed = true && isCanBeResolved = false => consumable
isCanBeConsumed = false && isCanBeResolved = true => resolvable
isCanBeConsumed = true && isCanBeResolved = true => default for backwards compatibility, but legacy and discouraged, so don’t use it, also has no convenience helper

Also plz enable val config by configurations.registerConsumable or similar)

I’m just a user like you.
But I agree: Kotlin DSL property delegates for fixed-type configuration factory methods · Issue #27204 · gradle/gradle · GitHub
Unfortunately Gradle folks not. :frowning:

also does not change anything about the behaviour

I did not say or imply it would change anything on the behavior.
It just shows that - even while shorter - what you do is not idiomatic and not recommended,
as those new convenience methods create configurations with a narrow role like how they are meant to be used while you previously used mixed-role configurations which is not recommended.

Are you talking about what is in the documentation referred to as Variant Selection?

It is 20 days ago, so I don’t know what I said.
Which part do you mean?
I don’t think I would have suggested anything with variant selection here.
Using the configurations directly is perfectly fine if it is just within the same build that those files are shared.

1 Like

Ok Thank you…
The magic ingrediant is indeed the Upcast from Configuration as FileCollection. fairly awkward to have to do an upcast of all things fix my code but o well.

Ah yes you are right about the artifacts. I must have always had some different errors in my artifact configuration, so thats good to know.

Sad about the missing by … support.
Best regards

1 Like