Dependency substitution configuration problem

I have a subproject in a multi-project build which produces a jar of resources. Due to issues with the test runner that depends on these resources, I need to extract the jar into the test classpath (so they are on disk instead of accessed through zipfilesystem in java).

The resource jar is also published, so that the plugin I’m building can be used externally. I then use dependency substitution in my local build to point at the resource project directly. Here is an example of how I setup a copy task, which my test and integTest tasks then depend on. I want to extract that out to an isolated configuration helper, so that I can reuse this in more places.

I thought this was working correctly, but then figured out that the existing and isolated code only work because the :core project (which the plugin also depends on through dependency substitution) depends on the resources jar as well (but directly on the project, with a separate named configuration).

Removing that indirect dependency, I then get this failure from dependency substitution:

A problem occurred evaluating project ':plugins'.
> Failed to apply plugin [id 'elasticsearch.esplugin']
   > Could not resolve all dependencies for configuration ':plugins:analysis-icu:restSpec'.
      > Module version org.elasticsearch:analysis-icu:3.0.0-SNAPSHOT, configuration 'restSpec' declares a dependency on configuration 'default' which is not declared in the module descriptor for org.elasticsearch:rest-api-spec:3.0.0-SNAPSHOT

I know dependency substitution currently cannot specify a configuration to depend on. Can anyone explain exactly what is going on? Why did it work when the :core project depended on the resources jar through a separate configuration? Why is it complaining now when the default? FYI, the resources jar is a one line gradle file. It simply applies the java plugin, because it just needs to create a jar file with the given resources.

I still haven’t discovered the root cause or a workaround here, but I did notice that if I change the name of the configuration that directly depends on the :rest-api-spec project in :core, there is still no resolve problem. So it seems to be that any configuration in the dependent project (not even one that is pulled in by the default, or named similarly) needs to have the direct project dependency…

Bump? Anyone? This is a huge pain as it means an external project depending on this must also setup a direct dependency on this special project to pull it in for tests…

I’m trying to reproduce that problem with your build, but I’m not seeing the same failure when I’ve tried taking out the dependency for rest-api-spec in core or renaming the configuration in core (I assumed restSpec2 -> restSpec).

From the error, it seems like the dependency had not been substituted, but that’s a guess.

Is there an earlier commit I could look at?

The dependency on rest spec needs to be there, but the substitution is when the problem occurs. Try this changing the following in core/build.gradle:

-  restSpec2 project("${projectsPrefix}:rest-api-spec") // for rest tests
+  restSpec2 "org.elasticsearch:rest-api-spec:${version}" //project("${projectsPrefix}:rest-api-spec") // for rest tests

Substitution specified in the root build.gradle should replace that with the local project. But running any task now fails with the previously mentioned error.

Aha, thanks!

Looking at the full stacktrace (-s), I see something suspicious:

at build_e3im14386dsfq74qj3hbyp3gc.run(elasticsearch/core/build.gradle:116)

You’re triggering dependency resolution during configuration, I’ll have to look a little harder to tell if this is something that could be warned/nagged about or prevented. I think there’s some sort of loop that’s preventing the substitution.

Your copyRestSpec Copy task is causing it because calling buildDependencies triggers a resolve (to find if other dependencies have been substituted) and asPath triggers a resolve (to resolve the configuration into a set of files).

Configuration is a Buildable, which means Gradle can infer dependencies, so you can just use dependsOn: configurations.restSpec2 instead of using buildDependencies.

Extracting the zip is a little trickier because we need to avoid iterating over the collection during configuration, so you can use a Closure to defer it:

from { /* stuff */ }

I would probably use zipTree(configurations.restSpec2.singleFile) instead of asPath, since you only expect a single file or configurations.restSpec2.collect { zipTree(it) } if you ever thought you’d extract multiple zips.

@sterling Thank you!! Dependinging on the configuration itself instead of buildDependencies, and wrapping the from in a closure indeed fixes the problem. Having a warning about this with dependency substitution would be great! I’ve hit this a couple times now in different contexts, so it seems easy to do, while not intentional.