Stop resolution on first flatDir match


(Geoff Morrison) #1

Our build includes many artifacts in flatDir repositories, and a few artifacts in Maven repositories. Our configuration lists all flatDir repos before all Maven repos. This works – but it’s oh so slow. From debug logs, I see that ExternalResourceResolver finds the artifact but no metadata, so it generates metadata from the artifact name. Because the module has been located, the search should end there, but RepositoryChainComponentMetaDataResolver explicitly keeps searching if the module descriptor it locates was generated. This means it falls through to the Maven repository, which in our case is somewhat slow to return a negative response. Multiplied by a few hundred dependencies, this can make dependency resolution take up to 30 minutes, all for the benefit of a small number of Maven-hosted dependencies.

Subsequent builds are pretty quick because of the missing artifact cache, but we do a lot of building on clean VMs.

Obviously there are a lot of things we could do better – move jars from flatDir to Maven so metadata can be cached, make our Maven repo not so slow – but these all take time. Is there something we can do in the short term to improve this situation?

Other details that might be useful: we’re running Gradle 2.13. We don’t depend on any dynamic versions or changing modules.