Appending to properties file

snorbi · February 21, 2023, 9:43am

I try to generate a build-info.properties file using dynamic data not available in the configuration phase.
The file does not exist initially.

This is my best try so far (using Kotlin everywhere):

abstract class WriteBuildInfoFile : WriteProperties() {

    companion object {

        private val buildInfoFilePath = "resources/main/build-info.properties"
    }

    @InputFile
    @Optional
    val inputBuildInfoFile = project.objects.fileProperty()

    init {
        val buildInfoFile = project.layout.buildDirectory.file(buildInfoFilePath)

        if (buildInfoFile.get().asFile.exists()) {
            inputBuildInfoFile.set(buildInfoFile)
        }

        setOutputFile(buildInfoFile)
    }

    @TaskAction
    override fun writeProperties() {
        val existingProperties = Properties()
        inputBuildInfoFile.orNull?.asFile?.also {
            if (it.exists()) {
                it.inputStream().use { existingProperties.load(it) }
            }
        }

        fun <K, V> Map<K, V>.mergeEntries() {
            forEach {
                val key = it.key.toString()
                if (!properties.containsKey(key)) {
                    property(key, it.value.toString())
                }
            }
        }

        existingProperties.mergeEntries() // Merge properties from the optionally existing file
        extra.properties.mergeEntries() // Merge properties added by tasks

        super.writeProperties()
    }
}

And the usage:

val generateBuildInfo: TaskProvider<WriteBuildInfoFile> by tasks.registering(WriteBuildInfoFile::class) {
    dependsOn(someTask)

    // The following line works if uncommented
    // extraProperties["propertyOnlyForTesting"] = "testValue"
}

val someTask by tasks.registering {
    doLast {
        ...

        // The following line causes the exception
        generateBuildInfo.extraProperties["someProperty"] = someDynamicValue
    }
}

tasks.named<BootJar>("bootJar") {
    dependsOn(generateBuildInfo)
    ...
}

The exception I get:

class org.gradle.api.internal.tasks.DefaultTaskContainer$TaskCreatingProvider_Decorated cannot be cast to class org.gradle.api.plugins.ExtensionAware (org.gradle.api.internal.tasks.DefaultTaskContainer$TaskCreatingProvider_Decorated and org.gradle.api.plugins.ExtensionAware are in unnamed module of loader org.gradle.internal.classloader.VisitableURLClassLoader @5e265ba4)

Do you have any idea how can I pass dynamic properties to a task if not by using its extra properties?

Thanks.

Vampire · February 21, 2023, 10:06am

Besides that using extra properties and explicit dependsOn (where on the left-hand side is not a lifecycle task) are both code smells that are usually just a work-around for not doing it properly, the error says exactly what the problem is.

Additionally, configuring one task from the execution phase of another task is also bad practice by now, especially because it is incompatible with the upcoming configuration cache.

Actually, I assume you do not show the actual code, because ti will not compile for two reasons.

you use someTask before defining it
you for sure have an explicit cast to ExtensionAware of generateBuildInfo somewhere in there.

The second point also is the source of your problem.
While all instances created through Gradle means are decorated to be ExtensionAware just do not have it in their signature, this is true for the actual WriteBuildInfoFile task created, but not for the provider where you try to set the value. You would want to do generateBuildInfo.configure { extraProperties... } or generateBuildInfo.get().extraProperties....

But still, you should not use extra properties for that, but for example a MapProperty.
Actually, the WriteProperties task already has such a property that you should configure instead.

Actually, you do configure it, but even worse than the one task configures the other, you change the configuration of the task itself during the execution phase, which is also very bad, also not supported with configuration cache and more importantly your task would be up-to-date when it shouldn’t because you do not properly declare your inputs and the task even is cacheable which makes it even worse.

snorbi · February 21, 2023, 10:51am

Sorry, I’m new to writing Gradle tasks

you use someTask before defining it

It compiles without problems, the only additional thing needed was to explicitly declare the type of generateBuildInfo. (Otherwise there is a compilation error: Type checking has run into a recursive problem. Easiest workaround: specify types of your declarations explicitly)

you for sure have an explicit cast to ExtensionAware of generateBuildInfo somewhere in there.

No, it works because the extraProperties extension property is unfortunately almost non-type-safe:

inline val Any.extraProperties: ExtraPropertiesExtension
    get() = (this as ExtensionAware).extensions.extraProperties

generateBuildInfo.configure { extraProperties... } or generateBuildInfo.get().extraProperties

Both work but based on your comment these are incorrect usage of the API.

you do not properly declare your inputs

Aren’t

    @InputFile
    @Optional
    val inputBuildInfoFile = project.objects.fileProperty()

and the init block appropriate for declaring the file as input?

you should not use extra properties for that, but for example a MapProperty

Wow, I thought that task properties are not allowed to be modified during the execution of the tasks!

Based on your recommendations, the latest version:

import java.util.Properties

abstract class WriteBuildInfoFile : WriteProperties() {

    companion object {

        private val buildInfoFileDefaultPath = "resources/main/build-info.properties"
    }

    @InputFile
    @Optional
    val inputBuildInfoFile = project.objects.fileProperty()

    init {
        val buildInfoFile = project.layout.buildDirectory.file(buildInfoFileDefaultPath)

        if (buildInfoFile.get().asFile.exists()) {
            inputBuildInfoFile.set(buildInfoFile)
        }

        setOutputFile(buildInfoFile)
    }

    @TaskAction
    override fun writeProperties() {
        val propertiesFromFile = Properties()
        inputBuildInfoFile.orNull?.asFile?.also {
            if (it.exists()) {
                it.inputStream().use { propertiesFromFile.load(it) }
            }
        }

        val configuredProperties = properties
        propertiesFromFile.forEach {
            val key = it.key.toString()
            if (!configuredProperties.containsKey(key)) {
                property(key, it.value.toString())
            }
        }

        super.writeProperties()
    }
}

And its usage:

val generateBuildInfo: TaskProvider<WriteBuildInfoFile> by tasks.registering(WriteBuildInfoFile::class) {
    dependsOn(someTask)
}

val someTask by tasks.registering {
    doLast {
        ...

        generateBuildInfo.configure {
            property("someProperty", someDynamicValue)
        }
    }
}

tasks.named<BootJar>("bootJar") {
    dependsOn(generateBuildInfo)
    ...
}

And it seems to work even after dynamicValue is changing (because of a change in the source codes) and then rebuilding.

Maybe do you spot some problems with this version? I would be very greatful to be able to learn more

Vampire · February 21, 2023, 3:22pm

Sorry, I’m new to writing Gradle tasks

Don’t worry, we all started at some point.

It compiles without problems, the only additional thing needed was to explicitly declare the type of generateBuildInfo. (Otherwise there is a compilation error: Type checking has run into a recursive problem. Easiest workaround: specify types of your declarations explicitly)

Ah, ok, forgot forward references work in those scripts.

No, it works because the extraProperties extension property is unfortunately almost non-type-safe:
inline val Any.extraProperties: ExtraPropertiesExtension
    get() = (this as ExtensionAware).extensions.extraProperties

Yeah, there you have your explicit cast.
At least I do not find this accessor anywhere with Gradle 8.0.1.

generateBuildInfo.configure { extraProperties... } or generateBuildInfo.get().extraProperties

Both work but based on your comment these are incorrect usage of the API.

you do not properly declare your inputs

Aren’t
and the init block appropriate for declaring the file as input?

It was more about the extra properties you had set.

you should not use extra properties for that, but for example a MapProperty

Wow, I thought that task properties are not allowed to be modified during the execution of the tasks!

Which is exactly what you were doing.
You did set extra properties during the execution of someTask and then you set those to the input MapProperty during the execution phase of generateBuildInfo.
Both is forbidden with configuration cache.
The latter is super bad, as you change the inputs during task execution time which makes up-to-date checks and task output caching behave wrongly.

Based on your recommendations, the latest version:
Maybe do you spot some problems with this version? I would be very greatful to be able to learn more

Setting the inputs and outputs in the task constructor (init block) can be done, it is just not fully idiomatic. Usually you try to have tasks and extensions dumb and without opinion and use a plugin to add opinion and wire things together, but that’s probably ok for your use-case.

Be aware that your latest version is still not configuration cache compatible, as you still configure one task from the execution phase of another task.
To fix that, you could for example instead make someTask produce a file and then wire the someTask task as input for your merge task and merge that file in too.
Then you also do not need an explicit dependsOn.

And the explicit dependsOn for bootJar also is still a code smell, you probably in the ... part configure the file to be included in the jar, instead use the tasks output and remove the explicit dependency.

snorbi · February 21, 2023, 5:20pm

Thanks for the additional suggestions.

To fix that, you could for example instead make someTask produce a file and then wire the someTask task as input for your merge task and merge that file in too.

The only way in Gradle to pass some dynamic value from one task to another is using files?

Vampire · February 21, 2023, 5:38pm

No, you could use a shared build service to hold shared state in the scope of a build run.

guai · February 27, 2023, 7:11pm

isn’t boot plugin’s buildInfo work for you? it seems you can add an Action to hook into properties generation process. didn’t check it myself though