Custom Settings plugin using libs.versions.toml from classpath

Help/Discuss
1

Hi

I am looking develop a custom standalone Settings plugin that builds up a version catalog to be used by a lot other builds and I would like to be able to use libs.versions.toml from the bundled plugin classpath not the downstream project source folders, the format of the “libs.versions.toml” is clear simple and elegant and it will make maintenance by my others easier over time and prefer this over the Groovy DSL version of it.
The solution that I am currently landing on is the following:

import org.gradle.api.Plugin;

import org.gradle.api.initialization.Settings;

import org.jetbrains.annotations.NotNull;

 

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.nio.file.Files;

import java.nio.file.Path;

import java.util.Objects;

 

public class VersionCatalogPlugin implements Plugin<Settings> {

  @Override

  public void apply(@NotNull Settings settings) {

    try {

      Path tomlFile = getFile(settings);

      settings.getDependencyResolutionManagement().versionCatalogs(catalogs -> {

        catalogs.create("libs", catalog -> {

          catalog.from(tomlFile.toFile());

        });

      });

    } catch (Exception e) {

      throw new RuntimeException("Failed to load version catalog", e);

    }

  }

 

  private @NotNull Path getFile(Settings settings) throws IOException {

    Path tomlFile;

    ClassLoader classLoader = getClass().getClassLoader();

    try (InputStream resource = Objects.requireNonNull(classLoader.getResourceAsStream("libs.versions.toml"))) {

      tomlFile = settings.getRootDir().toPath().resolve("build/libs.versions.toml");

      if (Files.exists(tomlFile)) {

        Files.delete(tomlFile);

      }

      resource.transferTo(new FileOutputStream(tomlFile.toFile()));

    }

    return tomlFile;

  }

}

What I have found is that the above code fails because the from method is expecting to read from something that it is a dependency notation is there any why to load the toml file from the plugin classpath directly ?

Thank you, kindly
Luis Oscar

2

Is there a reason you do not simply publish the toml file as it is intended to be and then just use the from in the consumer builds instead of applying the settings plugin?

3

The reason not to publish the toml it is because I am looking to serve about 60 code repositories using this plugin and I need a central place provide recommended library versions some of this libraries are internal other are 3rd party. The toml would have to be copied to 60 code repositories and have to maintain it in many places.

4

I didn’t think you got me right.
You already intend to publish the TOML as part of your settings plugin.
But the idiomatic was is to publish the TOML itself as artifact and then the consumers can use it using from just like the would apply your settings plugin.
No copying around involved.

1 Like
5
6

I have tried to use the recomendation provided in the document in our internal Artifactory by publishing but could not get it to work at retrieved it looks like something odd to with type of matching being apply for dependency coming from the remote repository either not sure exactly what possibility HTTP issues.

7

I’m sorry, but I did not get most of your last post, might be my lack of English.

If you didn’t get it to work, you might have done something wrong.
I’d be happy to help you getting the idiomatic way working instead of trying to help you reinventing the wheel in an awkward and harder to maintain way. :slight_smile:

What exact problem do you have when trying to use the idiomatic approach?

8

Hi

Bellow is was the error I got when trying to use the way recommended in the documentation,
not sure if this is because I am publishing the TOML file in Artifactory in the same bundle as the plugin.

C:\work\enterprise-integration\vhi-ei-gradle-plugin-test>gradlew clean build                                                                                                                                                                 

[Incubating] Problems report is available at: file:///C:/work/enterprise-integration/vhi-ei-gradle-plugin-test/build/reports/problems/problems-report.html                                                                                   

                                                                                                                                                                                                                                              

FAILURE: Build failed with an exception.                                                                                                                                                                                                     

                                                                                                                                                                                                                                              

* What went wrong:                                                                                                                                                                                                                           

Could not resolve all artifacts for configuration 'incomingCatalogForLibs0'.                                                                                                                                                                 

> Could not resolve ie.vhi.integrationhub:ei-gradle-plugin:3.4.5-SNAPSHOT.                                                                                                                                                                   

  Required by:                                                                                                                                                                                                                               

      unspecified:unspecified:unspecified                                                                                                                                                                                                    

   > No matching variant of ie.vhi.integrationhub:ei-gradle-plugin:3.4.5-SNAPSHOT:20250425.131252-12 was found. The consumer was configured to find attribute 'org.gradle.category' with value 'platform', attribute 'org.gradle.usage' with v

alue 'version-catalog' but:                                                                                                                                                                                                                  

       - Variant 'apiElements':                                                                                                                                                                                                              

           - Incompatible because this component declares attribute 'org.gradle.category' with value 'library', attribute 'org.gradle.usage' with value 'java-api' and the consumer needed attribute 'org.gradle.category' with value 'platfor

m', attribute 'org.gradle.usage' with value 'version-catalog'                                                                                                                                                                                

       - Variant 'runtimeElements':                                                                                                                                                                                                          

           - Incompatible because this component declares attribute 'org.gradle.category' with value 'library', attribute 'org.gradle.usage' with value 'java-runtime' and the consumer needed attribute 'org.gradle.category' with value 'pla

tform', attribute 'org.gradle.usage' with value 'version-catalog'
9

At those coordinates the TOML is not properly published.
I’m also not sure whether you can publish then at the same coordinates or maybe need to use different coordinates for plugin and TOML.

10

Ok, I gave it a try.
You cannot publish two components at the same time from one project.
But you can adjust one component to add the other for the publishing.
So for example in Kotlin DSL this works to publish a Java library and a TOML at the same time to the same coordinates:

publishing {
    publications {
        val libraryAndVersionCatalog by creating(MavenPublication::class) {
            val java by components.getting(AdhocComponentWithVariants::class) {
                addVariantsFromConfiguration(configurations.versionCatalogElements.get()) {}
            }
            from(java)
        }
    }
}
11

Hi

I am currently using the Gradle DSL I have tried to the bellow but it did not worked.

pluginMaven(MavenPublication) {
  addVariantsFromConfiguration(configurations.versionCatalogElements.get())
}

Addttionally I was able to get it to work in the custom plugin using the following:

import org.gradle.api.Plugin;
import org.gradle.api.file.FileCollection;
import org.gradle.api.initialization.Settings;
import org.jetbrains.annotations.NotNull;

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Objects;

public class VersionCatalogSettingsPlugin implements Plugin<Settings> {
  @Override
  public void apply(@NotNull Settings settings) {
    try {
      extractLibraryCatalogFile(settings);
      settings.getDependencyResolutionManagement().versionCatalogs(catalogs -> {
        FileCollection fileCollection = settings.getLayout().getSettingsDirectory().files("build/libs.versions.toml");
        catalogs.create("libs", catalog -> catalog.from(fileCollection));
      });
    } catch (Exception e) {
      throw new RuntimeException("Failed to load version catalog", e);
    }
  }

  private void extractLibraryCatalogFile(Settings settings) throws IOException {
    ClassLoader classLoader = getClass().getClassLoader();
    try (InputStream resource = Objects.requireNonNull(classLoader.getResourceAsStream("libs.versions.toml"))) {
      Path tomlFile = settings.getRootDir().toPath().resolve("build/libs.versions.toml");
      if (Files.exists(tomlFile)) {
        Files.delete(tomlFile);
      }
      resource.transferTo(new FileOutputStream(tomlFile.toFile()));
    }
  }
}
12

I am currently using the Gradle DSL

You probably mean Groovy DSL.
The Kotlin DSL is also a Gradle DSL.

I strongly recommend switching to Kotlin DSL.
By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and amazingly better IDE support if you use a good IDE like IntelliJ IDEA or Android Studio.

I have tried to the bellow but it did not worked.

Yes, of course not, Kotlin DSL would have helped. :smiley:
In my snippet I configure the java component.
You try to configure the pluginMaven publication, and try to call a non-existing method.
Should be something like

components.java {
   addVariantsFromConfiguration(configurations.versionCatalogElements) {}
}
1 Like
13

Hi @Vampire

Thank you for all your help.

The last suggestion worked to fix the publishing side, but I have found issues further on the consumption side, since it was now having issues pick up the existing plugin it looks like for this to work I would need to create a self contained project just to publish the TOML file to Artifactory and that looks to be a little bit of an overkill for me at the moment.

I have found that the other solution using the custom settings plugin that expands the TOML locally in the build directory of the downstream project more simple and convenient the only reservation I have at the moment it is using the BuildLayout API marked as incubating in the Settings interface.

Using the custom setting plugin I can make the TOML injection in downstream optionally configurable allow the plugin consumers to disable it in case they need to define their own version of the TOML file, plus the usage is simpler since I don’t have to configure repositories more than once in the settings.gradle file since I have found that to use the loading of the TOML file from the remote repository I also need to add the remote repository in the dependencyResolutionManagement section and this is uggly and inconvinient since I alredy have the same repositories defined in the top of pluginManagement section of the same file.

14

but I have found issues further on the consumption side, since it was now having issues pick up the existing plugin

Again, what issues.
From the top of my head I don’t see any issue that should arise in consuming either the plugin or the toml.

allow the plugin consumers to disable it in case they need to define their own version of the TOML file

With the proper publishing it is just as optional, because it is only used if downstream uses it

plus the usage is simpler since I don’t have to configure repositories more than once in the settings.gradle file since I have found that to use the loading of the TOML file from the remote repository I also need to add the remote repository in the dependencyResolutionManagement section and this is uggly and inconvinient since I alredy have the same repositories defined in the top of pluginManagement section of the same file.

Don’t you anyway need that repository to resolve your libraries?
Or is that plugin the only thing you consume from Artifactory.
If you also consume dependencies from there, you should already have the repository in dependencyResolutionManagement, so this should not make any difference.

Also, if anyway still the settings plugin is applied, you can also add the dependencyResolutionManagement repositories from within the settings plugin.

And finally, you could also do something like

pluginManagement {
    [repositories, dependencyResolutionManagement.repositories].each {
       it.tap {
          maven {
             name = 'EMPIC Nexus'
             url = 'https://nexus.empic.de/repository/maven'
          }
       }
    }
}

But hey, use whatever works for you, you have to live with the consequences in the end. :slight_smile:

15

Regarding the original question, we also had the problem accessing the toml from inside the jar. We solved it in a rather crude way in our settings-plugin:

override fun apply(settings: Settings) {
        var javaClazz = this.javaClass
        settings.dependencyResolutionManagement.versionCatalogs {
            create("libs") {
                val tmpPath = File.createTempFile("temp", "toml").toPath()
                val tomlStream = javaClazz.getResourceAsStream("libs.versions.toml") ?: throw NullPointerException("libs.versions.toml nicht im Classpath gefunden")
                tomlStream.use {
                    Files.copy(
                        tomlStream, tmpPath, StandardCopyOption.REPLACE_EXISTING
                    )
                }
                from(objectFactory.fileCollection().from(tmpPath))
                tmpPath.toFile().deleteOnExit()
            }
        }
    }

Although this works right now for us, I’ll definitely look at Bjorns suggestions, too.

1 Like
16

Thank you for sharing yes I had considered this option also, but I end up creating a separate project to deploy the TOML into Nexus as I belive that this will leverage the gradle caching and snapshoot versioning of the catalog TOML file. It does sound a bit overkill to create a project just to deploy a TOML to Nexus

17

It’s not really overkill. :smiley:
But should als not really be necessary if you have the plugin too.
Publishing as additional variant as I showed should work just fine and have the same caching and so on.
Separate project makes sense for example if you want to version and release it separately.