Dynamically changing module groupid during dependency resolution

Help/Discuss
1

Hi,

I’m using Gradle in a slightly unusual way. I’m downloading external build artifacts (zips) using dependency management. The Ivy repository defines the transitive dependencies of them. The declaration of these dependencies come from an non Gradle source (it’s an xml file). That’s parsed and defined as Gradle dependencies at build time. There are 2 groupIds that these dependencies can have but in the xml definition there’s nothing to say which one it comes from.

So I might have a dependency on

main:core-module:5.1
main:other-module:5.1
extension:extension-module:1.3

but in the xml definition it’s just listed as

core-module
other-module
extension-module

There are other settings that specify that the main group should use version 5.1 and the extension group should be 1.3

Currently when I’m adding the dependencies to Gradle after parsing the xml I check a config file that says whether a module should be found in the extension group (there are no name conflicts as far as I’m aware) and if it is then add the dependency extension:<module-name>:<extension-version> otherwise add main:<module-name>:<extension-version>
This configuration file is now getting pretty unwieldy and is really just replicating information that could be found from the repository.

What I’ve been trying to understand is if I can customise DependencyGraphBuilder and specifically (I think) a custom DependencyToComponentIdResolver on the ComponentResolversChain. My thinking being that it would first attempt to look up on the default main group but if that fails due to a 404 it would then try looking up with my custom resolver which would use an alternative groupId.

Or is there some other mechanism for dynamically changing groupId but on if the declared one cannot be found. I don’t believe that dependency substitutions will work for me because they are fixed. You need to know ahead of time what the substitutions are and I don’t until trying to resolve them.

I’d appreciate any pointers or just letting me know that I’m completely barking up the wrong tree.

Many thanks,
Andy

2

Maybe you could just define your repository in a clever way instead.
Something like

ivy("https://example.com/ivy/") {
    patternLayout {
        artifact("main/[module]/5.1/[artifact]-5.1.[ext]")
        artifact("extension/[module]/1.3/[artifact]-1.3.[ext]")
        ivy("main/[module]/5.1/ivy.xml")
        ivy("extension/[module]/1.3/ivy.xml")
    }
}

Then Gradle should automatically try both and you can just define the dependencies as

implementation(":core-module")
implementation(":other-module")
implementation(":extension-module")

If a name clash would happen, afair the first pattern you list will win, so main would win.

3

Thanks, this was a helpful pointer. It doesn’t quite work yet but it gives me another way to attempt to try to resolve.

It’s correctly checking in multiple locations on the repository until it gets a success response from the Ivy metadata url for the matching group. However I then see:

> Could not resolve :core-module.
            > inconsistent module metadata found. Descriptor: main:core-module:5.1 Errors: bad group: expected='' found='main'
              bad version: expected='' found='5.1'

Now actually this might not be a problem because the repository doesn’t really need to know dependencies to specific groups and versions, it just needs to know the module and then the build can specify the version to be used. I’d have to republish everything but it might work out better.

I’ve also found that resolutionStrategy does get called again(?) after it fails to find the dependencies the first time so I do something like

configurations.all {
	resolutionStrategy.eachDependency { DependencyResolveDetails details ->
		if (details.requested.name =~ /known-pattern-for-extension/) {
			details.useTarget("extension:${details.requested.name}:1.3")
		}
	}
}

then it works without any republishing (and my repository group filters can still work), but that still depends on me knowing the names of all of the ‘extension’ modules. The DependencyResolveDetails doesn’t seem to include any property for “has been resolved successfully” for me to check instead.

4

Did I understand correctly, that the Ivy files do not contain dependency information, or you don’t need it? Then you can also use

metadataSources {
    artifact()
}

and remove the ivy.xml patterns.
Then it will only search for the pure jar and should not get the inconsistency error afair.

5

They do but I was assuming that if I don’t specify the versions then it would be resolved by Gradle in the same way.
e.g. my Ivy metadata would look like

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0">
  <info organisation="" module="core-module" revision="" status="release"/>
  <configurations/>
  <publications>
    <artifact name="core-module" type="zip" ext="zip"/>
  </publications>
  <dependencies>
    <dependency org="" name="other-module" rev=""/>
  </dependencies>
</ivy-module>

I haven’t tried yet but I can foresee issues with the publishing of the artifacts this way. All of those attributes that I’m blanking are required according to the Ivy docs.

Also, I now realise that my comment about resolutionStrategy.eachDependencies being called during resolution is only for the transitive dependencies being pulled in after the declared ones. It’s not that it calls it again if the initial dependency resolution fails. My build was working because I was explicitly telling it to change the group for specific modules before resolution! :man_facepalming:

I’ll continue to look at the multiple repository definitions. Thanks again for the input.

1 Like