Version catalogs vs convention plugins confusion

I’m very excited to see that Gradle provides and promotes new, better and cleaner ways of sharing build configuration and logic between modules. That means version catalogs and convention plugins (buildSrc). However, it confuses me as these two features, which seem to be two complementary parts of the same functionality, are actually incompatible with each other.

We can import libs.versions.toml to buildSrc, but catalog will be only available in build.gradle.kts and not in conventions plugins where we really need it. I read Cédric Champeau’s posts here and here. There is a way to access version catalog in plugins, but it seems more like a workaround. We don’t get type-safe accessors which kills the idea and makes Libs.kt/Deps.kt approach just much more convenient. Also, Cédric says that one of advantages of version catalogs over Libs.kt is that we don’t need to recompile everything on each version change. But if we plan to import version catalogs into buildSrc then I guess we have exactly the same problem.

So my question are:

  • What is the current/future recommended approach for sharing both build logic and dependencies at the same time?
  • Do version catalogs apply only to use cases when we need to share dependencies, but don’t need to share anything else?
  • Are there plans to make both features closer, more compatible with each other? I’m aware version catalogs is incubating feature, so maybe it just needs time

Thank you!

3 Likes

In regards to your question about libs.versions.toml not being available to plugins, I’ve got a fairly simple work around.
I have a custom plug that reads the libs.version.toml and converts each entry section into a Properties file that I make globally available. Here’s some code showing the version section of the lib.version.toml file being added to a Properties class. I put version, libs, plugins and bundles each in there own properties file.

enum class PropertyType {
    versions:{
        override fun writeProperties(writer: WriteProperties, catalog: VersionCatalog) {
            catalog.versionAliases.forEach {
                //println("key: " + it + " value: " + catalog.findVersion(it).get().toString())
                if (it.isNotEmpty()) {
                    //val versionConstraint = catalog.findVersion(it)
                    writer.property(it, catalog.findVersion(it).get().toString())
                }
            }
            writer.writeProperties()
        }
    }
   ... repeat for each libs section you want to capture, adjusting accordingly:

}

fun createPropertiesFile(project: Project, type: PropertyType, properties: Properties) {

    project.tasks.create(type.name,WriteProperties::class.javaObjectType) {
        val projDirPath:String = project.rootProject.path
        val tomlFile = projDirPath + "/gradle/libs.versions.toml"
        inputs.file(tomlFile)
        val propertiesFile = project.rootProject.projectDir.resolve("src/main/resources/" + type.name.toLowerCase() + ".properties")

        outputFile = propertiesFile
        //println("output file - " + writer.outputFile.absolutePath)
        if (!outputFile.exists()) {
            outputFile.parentFile.mkdirs()
            if (!outputFile.exists()) outputFile.createNewFile()
        }
        comment =
            "this file is automatically created in the build process DO NOT MANUALLY MODIFY THIS FILE"
        val vc = project.extensions.getByType(VersionCatalogsExtension::class.java).named("libs")
        type.writeProperties(this, vc)
        properties.load(FileInputStream(propertiesFile))
    }
}

And here’s how it gets called:

class MyCustomProjectPlugin : Plugin {
companion object {
var versions: Properties = Properties()
var libraries: Properties = Properties()
var plugins: Properties = Properties()
var bundles: Properties = Properties()
}

override fun apply(project: Project) {
    //println("is root " + project.name)
    if (project == project.rootProject) {
        println("MyCustomPlugin applied")
        createPropertiesFile(project, PropertyType.versions, versions)
        createPropertiesFile(project, PropertyType.libraries, libraries)
        createPropertiesFile(project, PropertyType.plugins, plugins)
        createPropertiesFile(project, PropertyType.bundles, bundles)

The disadvantage of this approach is that you don’t get the typesafe behavior of using the libs.versions.toml file directly. I’ve written custom plugins that get version information that is consistent with the version data contained in the libs.versions.toml file. The libs key is used as the key for the corresponding properties file. Although not shown, I also write the properties file to the resources directory, which means my runtime also has access to all the build version information.

I also create a version number for the toml file and store it in the toml file. That version number gets manually changed each time I make a change to the toml file. I actually wish that the Gradle Team would support the libs.versions.toml as a versioned artifact that could be retrieved from a repository. I also wish they’d allow root project reference plugin using version numbers obtained from the libs.version.toml file. I hate hardwiring version data in any build scripts. That contributes to unmanageable build script maintenance. I just don’t see how that would violate Idempotency if all version data is derived from the versions catalog. I certainly understand the argument that version data derived from environment variables potentially breaks Idempotency, but I don’t see that with using versions catalog data.

I’ve got a fairly simple work around.
I have a custom plug that reads the libs.version.toml and converts each entry section into a Properties file that I make globally available.

That’s actually totally unnecessary. There is already a built-in way to access the version catalog type-unsafe. Just get the VersionCatalogsExtension and you can access the catalog by string keys.

The disadvantage of this approach is that you don’t get the typesafe behavior of using the libs.versions.toml file directly.

Same limitation for the built-in way.
In the according GitHub issue I actually show a hacky work-around how you can actually use the type-safe accessors in convention plugins though.

I actually wish that the Gradle Team would support the libs.versions.toml as a versioned artifact that could be retrieved from a repository.

That always was supported. There is even a plugin to create a published catalog that you can then depend upon in other projects.

I also wish they’d allow root project reference plugin using version numbers obtained from the libs.version.toml file.

Not sure what you mean. Sounds like supported functionality, unless I misunderstood.

If I recall correctly, VersionCatalogsExtension assumes you know the catalog key name. For most situations that’s perfectly fine. In my case, I wanted to get the version for all dependencies with the assumption that you don’t know all the dependencies version key names. This allows me to store the version number of all dependencies associated with the project without knowing the version key names. I use it to report the version number of all dependencies for a project in an About dialog. I believe you have to do an iteration technic if you’re trying to achieve that objective.

I thought that the libs.versions.toml file is resolved at the initialization phase, and my assumption is that would put a damper on having a versioned libs.versions.toml file. Sound’s like your saying that’s not the case. Gradle documentation explaining versions catalogs seems to be lacking. I’m not aware of the plugin that can be used to create a publish catalog. I’ll try to look for it! Sound’s like that’s what I’m looking for. Any pointers to the use of this plugin would be greatly appreciated.

On the last point, my root project end’s up having a plugin block looking something like this:

plugins {
id(“com.android.application”) version “7.2.0” apply false
id(“org.jetbrains.kotlin.android”) version “1.5.31” apply false
}

What I like it to look like is:

plugins {
id(“com.android.application”) version libs.versions.android.application apply false
id(“org.jetbrains.kotlin.android”) version libs.versions.jetbrain.kotlin.android apply false
}

If I try the second format I always get some kind of error indicating that “val is not allowed in this context”, and the only solution is for me to use a literal hardwired version number. The gradle documentation seems to indicate that this is done to preserve Idempotency. I do understand what that concept refers to, but I fail to understand why getting version information from the version catalog in this context would break Idempotency. Isn’t that the point of the version catalog? My desired goal is to have no literal version data in any of my build scripts; but in this context, I can’t seem to do that. Is there something I doing wrong or misunderstanding? (Usually that’s the case!)

I do greatly appreciate your feedback. It was very thoughtful!

Yes, and the VersionCatalogsExtension provides you all the aliases, so you can also easily iterate over all entries and also all catalogs without any problem or the need to know all aliases in advance.

Not sure where you mean it is lacking.
It’s all documented on Sharing dependency versions between projects, includeing the plugin to publish and reuse a version catalog: Sharing dependency versions between projects

And why not simply

plugins {
    alias(libs.plugins.android.application) apply false
    alias(libs.plugins.jetbrain.kotlin.android) apply false
}

?

@broot did you found any updates to this ?

What I wrote here is still the current state.