Best strategy to reuse dependency declarations, including exclusions?

Help/Discuss
1

The current situation:

  • We have a large collection of module-name-version-exclusions data which is currently codified in a single location - a Libraries class in our build plugin.
  • We have a plugin which uses resolutionStrategy.force to force all the versions we use.
  • We’re on Gradle 6.8 but just about to update to 7.x (hopefully within a week).

The issue right now:

  • I’d like to replace the force usages with something newer to rid ourselves of the deprecation warnings. Though this apparently wasn’t removed in Gradle 7, I’d still like to kill it. (But because it’s still in Gradle 7, we now have some more time to think about the best solution.)

Non-solutions identified so far:

  • Using ComponentMetadataRule to remove the excluded libraries doesn’t work for us. It does produce the right dependency graph - but a BOM we create as part of the build is then missing the <exclude> entries.
  • Using dependencySubstitution to override the dependency versions being looked up sort of works, but results in a very hard to maintain build script. We end up with around 100 substitutions in total, if I consider multiple version substitutions for the same artifact to be a single substitution (because I wrote a utility method to do that). I also wouldn’t be surprised if the generated BOM were wrong as well, but I didn’t make it that far into testing it.
  • Using a TOML version catalog lets us codify all the versions in a cleaner way, but results in duplication - because exclusions cannot be codified in the same way, we end up with all the dependencies declared in the TOML file and then a second copy declared in Libraries which has all the exclusion rules. It’s also kind of a sad solution because using a version catalog usually gives us a nice typesafe API, but then we ignore that whole API and make people go through our own. The TOML catalog plus a new feature where we can also declare exclusions in there would be pretty close to perfect for us.

Current ideas on the table:

  1. Can we declare all versions and exclusions in a platform project, and then import that project?
    Initial impressions say that it may not be possible. If this were possible, it would give us a central place to put everything while also being a very conventional way to do things in Gradle.

  2. Can we declare all versions and exclusions in a Maven BOM, and then import that BOM? If I do that, will the exclusions in the BOM be obeyed? And can I pull from a Maven BOM in a local file, rather than pulling one as an artifact? If this were possible, we’d at least keep our centralised definition, and we’d be using standard ways to do it, with the only downside being having to deal with the XML format and having a little piece of Maven permanently embedded in our Gradle project.

  3. Do we just keep our centralised class but change it to declare all versions as strictly()? We keep our very shady Gradle build, but with the minimum changes from what we currently have, and because we’re still using our Libraries class, we get to keep using our typesafe accessors. Potential traps:

    • because it’s per-library, someone might add a new library to the list and forget to add the !! on it.
    • there might be entries in our list which are only used transitively - in which case making the version strict only works if we actually add an explicit dependency - we’re planning to use dependencySubstitution as a fallback for any of these which appear on the way.

I’m looking for tips on direction is most likely to work because it’s a decently long time to make the change each time just to find out whether it will work or not. At the moment, my impression is that option 3 might be the next one we try, because it’s the minimum code change. But we have some time to plan things right now.

Or perhaps there is an option we haven’t yet thought of?

How do other people with large (>100 dependencies) projects deal with dependency definition?

I know many times people have said that the reason why the exclusion is declared at the time you add the dependency is because different projects might want to exclude different things. That sounds plausible, but we have not yet encountered a case for it. However, we do have dependencies where we have to exclude 10+ of their transitive dependencies, and don’t want to repeat that whenever it occurs.

2

Can you do something like

configurations {
  compileClasspath.resolutionStrategy {
    eachDependency { DependencyResolveDetails details ->
      // use this loop to gather the various versions for the group/artifacts you are interested in
    }
    eachDependency { DependencyResolveDetails details ->
      // use the data gathered in the first loop to pick the best version (call details.useTarget) 
    }
  }
}
3

That will not first process all dependencies in the first block, then all in the second block, but more for each dependency first do the first block, then the second block, doesn’t it?