Dependency on subproject not resolved

Help/Discuss
1

I have a multimodule build in which one module creates an application via the “application” plugin and another module executes the created application. However, executing the second module does not trigger creation of the application zip and results in the following error:

Cannot expand ZIP ‘C:\myproject\application\build\distributions\application-1.0.0-SNAPSHOT.zip’ as it does not exist.

I have the following configuration in the second module:

configurations {
   theApplication
}

dependencies {
	theApplication project(path: ':application', configuration: 'archives')
}
	
task extractApplication(type: Copy) {
  from { 
  	configurations.theApplication.findAll { it.name.endsWith '.zip' }.collect { zipTree(it) }
  }
  into "$buildDir/tmp"
}

Why running the “extractApplication” task does not trigger creation of the application archives and how to fix it?

Edit: I managed to work around the problem by adding the dependency to the “extractApplication” task:

dependsOn(“:application:assembleDist”)

Is it the correct solution or there is a better one?

2

That is not the correct solution. The correct solution is to instrument the :application project like this:

configurations {
  theApplication
}
artifacts {
  theApplication assembleDist
}

Then in your second module you change the code to this:

configurations {
   theApplication
}

dependencies {
	theApplication project(path: ':application', configuration: 'theApplication')
}
	
task extractApplication(type: Copy) {
  dependsOn configurations.theApplication

  from {
    zipTree(configurations.theApplication.singleFile)
  }
  into "$buildDir/tmp"
}
1 Like
3

Thanks for the reply.

Can you elaborate why declaring dependency on configuration is better than on a task?

4

Because it uses the Gradle preferred mechanism to link outputs of one task to inputs of another. Gradle relies on the ability to correctly define the input/output of any one task and uses this information to generate task dependency graphs. Directly using the dependsOn mechanism with a task accomplished the same thing but it causes your project to be intimately coupled with the structure of the sibling project. Using the configuration names as handles is better because you no longer assume any special knowledge of how the artifacts are constructed, you just need to know there is an output artifact declared for the configuration named in the dependency. In essence the project, like the task is able to declare its outputs so sibling projects can make references.

This also works to make the correct POM files with usage of dependencies, the way you were doing it the dependencies of the project you are calling into are not visible to the consumer.

1 Like
5

Hmm, as of 11/12/2024 on gradle 8.5 and 8.11, this solution doesn’t work for me. Here’s an example project that has the same issue.

settings.gradle

rootProject.name = 'test-sibling-deps'
include 'projectA'
include 'projectB'

projectA/build.gradle

plugins {
    id 'java'
}

configurations {
    staticResources
}

dependencies {
    staticResources project(path: ":projectB", configuration: "staticResources")
}

tasks.register("copyStaticResources", Copy) {
    dependsOn configurations.staticResources
    var archive = configurations.staticResources.find {
        println it
        it.name.startsWith("projectB")
    }
    if (archive != null) {
        from zipTree(archive).matching {
            include 'staticB.txt'
        }.files
        into layout.buildDirectory.dir("resources/main")
    }
}

tasks.processResources.dependsOn tasks.copyStaticResources

projectB/build.gradle

plugins {
    id 'java'
}

configurations {
    staticResources
}

artifacts {
    staticResources jar
}

There are two more empty files that exist: projectA/src/main/resources/staticA.txt and projectB/src/main/resources/staticB.txt.

When trying to run gradlew build from the root project directory, I get the following:

A problem occurred evaluating project ':projectA'.
> Could not create task ':projectA:copyStaticResources'.
   > Cannot expand ZIP 'C:\...\test-sibling-deps\projectB\build\libs\projectB.jar' as it does not exist.

Shouldn’t the dependsOn configurations.staticResources indicate to gradle that projectB needs to be built before its static resources can be resolved?

6

Yes, but you call zipTree at configuration time when the task was not able to run yet.
Maybe you need to use not a Copy task, but a copy { ... } closure within doLast { ... } action so that you only do it at execution time.

Btw. you should definitely not copy into layout.buildDirectory.dir("resources/main"). That is the output directory of another task. Having overlapping outputs between task is an extremely bad idea and causes all sorts of problems, from silently wrong builds, to flaky builds, to always failing builds, depending on the exact details, and only occasionally if you are lucky a half-way correct result, and even then maybe poisened build cache entries and so on.

Instead define a separate dedicated output directory for the task that is only owned by that task (don’t forget to declare it as output directory if you moved the logic to a copy { ... } - or after having a dedicated output directory better a sync { ... } - closure. Then define the task as source directory for the main source set like sourceSets { main { resources { srcDir(copyStaticResources) } } } and remove the manual faulty task dependency. Then also all tasks needing resources have the necessary task dependency that they are missing right now and also get the resources accordingly wihtout polluting the output directory of another task.

1 Like
7

Hiya Bjorn - I’ve tried the following based off your suggestions, I don’t think I’ve got it quite right as this is still failing:

plugins {
    id 'java'
}

configurations {
    staticResources
}

dependencies {
    staticResources project(path: ":projectB", configuration: "staticResources")
}

tasks.register("copyStaticResources")

tasks.copyStaticResources.doLast {
    copy {
        var archive = configurations.staticResources.find {
            it.name.startsWith("projectB")
        }
        if (archive != null) {
            from zipTree(archive).matching {
                include 'staticB.txt'
            }.files
            into layout.buildDirectory.dir("copied")
        }
    }
}

sourceSets {
    main {
        resources {
            srcDir(copyStaticResources)
        }
    }
}

UPDATE, this seems to work, though I don’t think its what you had in mind as I no longer am defining the sourceSets srcDir:

plugins {
    id 'java'
}

configurations {
    staticResources
}

dependencies {
    staticResources project(path: ":projectB", configuration: "staticResources")
}


tasks.processResources.dependsOn configurations.staticResources
tasks.processResources.doLast {
    copy {
        var archive = configurations.staticResources.find {
            it.name.startsWith("projectB")
        }
        if (archive != null) {
            from zipTree(archive).matching {
                include 'staticB.txt'
            }.files
            into layout.buildDirectory.dir("copied")
        }
    }
}

UPDATE 2 maybe this next iteration better matches what Bjorn suggested

plugins {
    id 'java'
}

configurations {
    staticResources
}

dependencies {
    staticResources project(path: ":projectB", configuration: "staticResources")
}

tasks.register('copyStaticResources') {
    dependsOn configurations.staticResources
    outputs.dir(layout.buildDirectory.dir("copied"))
    doLast {
        copy {
            var archive = configurations.staticResources.find {
                it.name.startsWith("projectB")
            }
            if (archive != null) {
                from zipTree(archive).matching {
                    include 'staticB.txt'
                }.files
                into layout.buildDirectory.dir("copied")
            }
        }
    }
}

sourceSets {
    main {
        resources {
            srcDir(copyStaticResources)
        }
    }
}

tasks.processResources.dependsOn tasks.copyStaticResources
8

First things first, I’m not called Bjorn. I it is Björn or Bjoern. :wink:

tasks.register(“copyStaticResources”)
tasks.copyStaticResources.doLast {

This is a bad idea generally.
By using tasks.register you leverage task-configuration avoidance which is good.
But right in the next line you break task-configuration avoidance causing the task to always be realized.
Instead do def copyStaticResources = tasks.register("copyStaticResources") { ... } so you only configure the task if necessary and right away have a reference to the task provider that you then can use as source directory.

tasks.copyStaticResources.doLast {
copy {

You only do the zipTree and so on at execution time now, but you missed the part about properly declaring inputs and outputs of the task I mentioned.
You should always declare the inputs and outputs of a task properly, otherwise you can neither wire task outputs and inputs together, do not have the necessary task dependencies implicitly, and also the task can never be up-to-date.
In the task configuration but outside the doLast { ... } use inputs.... and outputs.... to declare the inputs and outputs of the task.

Once you fixed that, it should start working as expected I think.

UPDATE, this seems to work, though I don’t think its what you had in mind as I no longer am defining the sourceSets srcDir:

Yeah, no, that’s not what I meant, see above.
Additionally here now you break task-configuration avoidance of the processResources task, and you will miss your resource at execution time as it no longer lands in the final artifact.
If you add a doLast action to the processResources task, it would be ok to write to the output directory of the processResources task, as the action is now part of that task.
But the first variant, fixed of the missing parts is imho preferable in your situation.

UPDATE 2 maybe this next iteration better matches what Bjorn suggested

Almost. I might forgot to mention it explicitly, but any explicit dependsOn where on the left-hand side is not a lifecycle task is a code smell and a sign that you somewhere do something not properly. Instead - as said above - use inputs.... like inputs.files(configurations.staticResources), then the task can for example also properly be up-to-date. The need for explicit dependsOn with non-lifecycle tasks most often means that you did not properly wire task outputs and task inputs together.

So the last version should almost be fine, except for the wrong dependsOn that should be an inputs... and the superfluous dependsOn in the last line.

1 Like
9

As per Björn’s suggestions, here’s what works and (hopefully) follows gradle best practices.

plugins {
    id 'java'
}

configurations {
    staticResources
}

dependencies {
    staticResources project(path: ":projectB", configuration: "staticResources")
}

def copyStaticResources = tasks.register('copyStaticResources') {
    inputs.files(configurations.staticResources)
    outputs.dir(layout.buildDirectory.dir("copied"))
    doLast {
        copy {
            var archive = configurations.staticResources.find {
                it.name.startsWith("projectB")
            }
            if (archive != null) {
                from zipTree(archive).matching {
                    include 'staticB.txt'
                }.files
                into layout.buildDirectory.dir("copied")
            }
        }
    }
}

sourceSets {
    main {
        resources {
            srcDir(copyStaticResources)
        }
    }
}

Thank you so much for your help @Vampire!

10

hopefully follows gradle best practices

Actually, almost. :smiley:
How your staticResources configuration is created is legacy and discouraged actually. :smiley:
A configuration should have either the role of resolving dependencies, or providing things for consumers, or being designated to declaring dependencies.
The syntax you used creates a configuration that can do all three for backwards-compatibility reasons.
There are incubating convenience helpers that let you create properly roled configurations (configurations.dependencyScope, configurations.consumable, configurations.resolvable) or you can set the canBe... properites accordingly.
So to use such properly roled configurations you would create two, one that is neither consumable nor resolvable to declare the dependency, and one that is only resolvable that extends the first one and is used to resolve the dependency. But you could for now also keep it like that if you prefer, that is just a semantic thing, not a functional difference.

And one personal recommendation, you should consider switching to Kotlin DSL. By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and amazingly better IDE support if you use a good IDE like IntellliJ IDEA or Android Studio. :slight_smile: