Correct way to resolve a configuration from another project in 5.x?

Help/Discuss
1

I’ve got a gradle build with two projects. :foo is an interactive application. :ship has some custom tasks related to deployment. One task in particular from :ship started to generate a warning in Gradle 5.1:

(file is in subproject :ship)

class FooTask extends DefaultTask {
	FooTask() {
		dependsOn(':foo:classes')
	}

	@TaskAction
	void doAction() {
		def cp = project.project(':foo').sourceSets.main.runtimeClasspath
		project.javaexec({
			classpath = cp
			main = 'fooMain'
		})
	}
}

I am getting this warning:

The configuration :foo:runtimeClasspath was resolved without accessing the project in a safe manner.  This may happen when
 a configuration is resolved from a thread not managed by Gradle or from a different project.  See https://docs.gradle.org/5.1/userguide/tro
ubleshooting_dependency_resolution.html#configuration_resolution_constraints for more details. This behaviour has been deprecated and is sch
eduled to be removed in Gradle 6.0.
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveToStateOrLater(DefaultConfiguration.java:534)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$2200(DefaultConfiguration.java:135)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getSelectedArtifacts(DefaultConfiguration.java:1158)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:1147)
        at org.gradle.api.internal.file.AbstractFileCollection.iterator(AbstractFileCollection.java:72)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.iterator(DefaultConfiguration.java:462)
        at com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:384)
        at com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.addAll(ImmutableCollection.java:476)
        at com.google.common.collect.ImmutableSet$Builder.addAll(ImmutableSet.java:518)
        at org.gradle.api.internal.file.CompositeFileCollection.getFiles(CompositeFileCollection.java:80)
        at org.gradle.api.internal.file.CompositeFileCollection.iterator(CompositeFileCollection.java:67)
        at org.gradle.util.CollectionUtils.join(CollectionUtils.java:561)
        at org.gradle.process.internal.JavaExecHandleBuilder.getAllJvmArgs(JavaExecHandleBuilder.java:59)
        at org.gradle.process.internal.JavaExecHandleBuilder.getAllArguments(JavaExecHandleBuilder.java:225)
        at org.gradle.process.internal.AbstractExecHandleBuilder.build(AbstractExecHandleBuilder.java:135)
        at org.gradle.process.internal.JavaExecHandleBuilder.build(JavaExecHandleBuilder.java:242)
        at org.gradle.process.internal.DefaultJavaExecAction.execute(DefaultJavaExecAction.java:34)
        at org.gradle.api.internal.file.DefaultFileOperations.javaexec(DefaultFileOperations.java:227)
        at org.gradle.api.internal.project.DefaultProject.javaexec(DefaultProject.java:1103)
        at org.gradle.api.internal.project.DefaultProject.javaexec(DefaultProject.java:1098)
        at org.gradle.api.Project$javaexec$2.call(Unknown Source)
        ...

The error message helpfully references the docs for Constraints on configuration resolution which say:

Configurations need to be resolved safely when crossing project boundaries … Gradle can usually manage this safe access, but the configuration needs to be accessed in a way that enables Gradle to do so. There are a number of ways a configuration might be resolved unsafely … e.g. A task from one project directly resolves a configuration in another project.

The trouble is, the docs don’t say what makes a resolution safe. They list a couple unsafe examples (one of which I am clearly violating), but the docs:

  • don’t recommend a solution
  • don’t describe what proper operation is, so that I can find a solution myself

My first instinct is that there is now no way for one project to depend on the resolved configuration of another project. If so, fair enough, but let me describe my usecase for why I hope there is a way to resolve a peer project’s dependencies.

  • subproject :foo is an interactive application
  • subproject :ship has tasks that headlessly automates :foo for certain maintenance tasks, along with other generic maintenance tasks

Of course, some of the tasks in :ship could move to :foo if they have to. But the tasks are actually about completing tasks relating to shipping, and they’re coupled with other shipping tasks that don’t have anything to do with :foo. Likewise, :foo is an otherwise totally normal java project unrelated to shipping, and it would be a little messy if some random shipping logic has to get stuffed into its project.

Perhaps there can be a way to explicitly opt-in and say in my settings.gradle that :ship depends on :foo, so it’s okay if :ship uses configurations from :foo? I’m shooting in the dark here, because the docs don’t explicitly say what is allowed, just a few examples of what isn’t…

3

It might help to declare the FileCollection as a task input. Eg.

class FooTask extends DefaultTask {
    @InputFiles
    FileCollection classpath

    @TaskAction
    void doAction() {
        def cp = classpath 
        project.javaexec {
            classpath = cp
            main = 'fooMain'
        }
    }

build.gradle

task foo(type: FooTask) {
    dependsOn ':foo:classes'
    classpath = project(':foo').sourceSets.main.runtimeClasspath
}
4

Thanks for tips, but didn’t work so far.

class FooTask extends DefaultTask {
	@InputFiles
	FileCollection cp

	FooTask() {
		dependsOn(':foo:classes')
		cp = project.project(':foo').sourceSets.main.runtimeClasspath
	}

Also, keeping the constructor empty and doing it like this also did not work.

task foo(type: FooTask) {
    dependsOn ':foo:classes'
    classpath = project(':foo').sourceSets.main.runtimeClasspath
}
5

How’s a both this

foo/build.gradle

configurations {
   runtimeClasspath {
      builtBy 'classes' 
   } 
} 
dependencies {
   runtimeClasspath sourceSets.main.runtimeClasspath 
}

other/build.gradle

configurations {
   fooRuntimeClasspath
} 
dependencies {
   fooRuntimeClasspath project(path: ':foo', configuration: 'runtimeClasspath') 
} 
task foo(type: FooTask) {
   classpath = configurations.fooRuntimeClasspath
}
6

I think as a simplification to this (assuming foo is a regular Java project):

In foo/build.gradle:
Nothing extra/special.

In ship/build.gradle:

configurations {
   fooRuntimeClasspath
} 
dependencies {
   fooRuntimeClasspath project(':foo') 
} 
task foo(type: FooTask) {
   classpath = configurations.fooRuntimeClasspath
}
7

Is there any reason that you can’t treat foo like a regular project dependency (sort of like Lance said below with my simplification)? If you depend on a Java project, you’ll get the runtime classpath.

We can provide better guidance here for sure, but it’s difficult to prescribe a fits all solution because it depends on what you’re doing.

In general,

  • Don’t reach across into other projects to grab project specific information (e.g., configurations, tasks, etc). This can cause unexpected failures that may be intermittent (deadlocks, concurrency issues).
  • Where you need to share artifacts across project boundaries, use dependency management. Gradle is then responsible for making sure project state is locked/managed.

I’ll see what we can add for 5.2.

8

Thanks, this last suggestion worked. Here is the pre 5.0 code:

class FooTask extends DefaultTask {
	FooTask() {
		dependsOn(':foo:classes')
	}

	@TaskAction
	void doAction() {
		def cp = project.project(':foo').sourceSets.main.runtimeClasspath
		project.javaexec({
			classpath = cp
...

And post 5.0:

configurations {
	fooTaskClasspath
}
dependencies {
	fooTaskClasspath project(':foo')
}
class FooTask extends DefaultTask {
	FooTask() {
		// this explicit dependency is required (edit at 11:41am PST 1/10/2019)
		dependsOn(':foo:jar')
	}

	@TaskAction
	void doAction() {
		project.javaexec({
			classpath = project.configurations.fooTaskClasspath

The pre-5.0 method is more composable - all the logic can be encapsulated within one task class. The post-5.0 method requires project-global configurations that cannot be encapsulated within the task.

I understand that making configuration faster means making it lazier which means limiting the scope of possible interactions. But composing subprojects on top of other subprojects is an essential mechanism. It seems that in 5.x, the only legal way for subprojects to compose is via project dependencies, which is a very narrow pipe.

I also think there’s something wrong that all available advice still has the word “generally”. There ought to be some way to say “this is the restriction”, not just “generally, this is the restriction”. E.g.

A subproject may not refer to another subproject in any way except:

  • by declaring a project dependency, e.g. dependencies { compile project(':other') }
  • by declaring a configuration dependency in settings.gradle, e.g.
    include 'a'
    include 'b', dependsOn 'a'
    • warning: configuration dependencies will disable all lazy evaluation within dependee projects if any configuration within a depender project needs to evaluated. Prefer project dependencies wherever possible.

There’s no “generally”, there’s a hard and fast rule which allows rich composability between projects while still restricting interproject communication enough to allow aggressive optimizations.

9

If you declared configurations.fooTaskClasspath as a task input you shouldn’t need dependsOn(':foo:classes')

10

Thanks, good to know.