Proper way to resolve transitive modules with and without Scala variants

Help/Discuss
1

Say we have a large dependency graph in which Scala dependencies foo:bar:5.0 and foo:bar_2.10:6.0 are pulled in transitively. These two modules are the same for all intents and purposes, but can’t natively be resolved by Gradle. As far as I can tell, there’s no Gradle API to directly flatten these two prior to dependency resolution.

Our current approach is to add a dummy Action<DependencyResolveDetails that parses all transitive dependencies, and stores them in memory for other dependency resolution actions to use. We then can force all instances foo:bar:5.0 into foo:bar_2.10:6.0, picking the one with the highest version.

There are two points we would like help with:

  1. Is there an easier/cleaner way to go through all of our transitive dependencies prior to dependency resolution? We would like to avoid resolving dependencies multiple times.
  2. Is our rule of flattening varianted dependencies into the highest version in-line with Gradle’s expected module resolution?
2

If I understand correctly, you would like Gradle to enter conflict resolution for foo:bar:5.0 and foo:bar_2.10:6.0.

And for the moment, it does not since they have different module name.

On its way to 5.0 a feature was added to Gradle that could help in this case: capabilities

It enables dependencies, with potential different group:name to declare they provide a common feature and thus cause dependency resolution to conflict resolve them because capabilities have a version just like dependencies.

And by default Gradle resolution will pick the higher version during conflict resolution.

So in this case, the consumer would enhance the dependency metadata to make that conflict resolution happen:

dependencies {
    components {
        all { details ->
            // Apply the below to all dependencies so filter the capability addition
            if (details.id.group == 'foo' && (details.id.name == 'bar' || details.id.name.startsWith('bar_')) {
                allVariants {
                    withCapabilities {
                        addCapability('org', 'bar', details.id.version)
                    }
                }
            }
        }
    }
}

Unfortunately, there is not yet documentation describing this feature.

So please let us know if that helps resolving your problem.

Note that we are also considering improving the Gradle scala plugin to support this convention of having the supported Scala version encoded in the artifact name.

3

I think you can use a module replacement rule

Eg

dependencies {
    modules {
        module("foo:bar") {
            replacedBy("foo:bar_2", "bar was renamed to bar_2")
        }
    }
}
4

Thanks for getting back to me, @Lance @ljacomet

In our case, we don’t necessarily know the group or module name prior to resolution, and require a more flexible replacement strategy that can deduce replaceable modules during resolution time. It sounds like this may not be supported yet, so I’d like to throw my hat in for integrating this into a future version of the Scala plugin :wink:. Using the Play framework in a large dependency graph has made dynamic Scala resolution a priority for us, and our ad-hoc solutions are… less-than-ideal.

5

Perhaps you could use a component selection rule similar to here

6

Hmmm… seems similar to what we’re already doing. I’ll play around with it, thanks!