Composite build doesn't find tasks with Gradle library directory structure ("lib")

Help/Discuss
1

I have a composite build with project ‘A’ including other builds (includeBuild) ‘B’, ‘C’, … I use it to “forward” tasks in project’s A build.gradle, e.g.,

dependencyUpdates.dependsOn gradle.includedBuilds*.task(':dependencyUpdates')

This works fine, but if I include a Gradle Java library setup (gradle init for library) “Library”, it contains an extra “lib” folder where build.gradle is located. Now Gradle starts to complain with

Task with path ‘:dependencyUpdates’ not found in project ‘:Library’

I could include the project “Library” using the subfolder “Library/lib”, but then I get an error with Buildship:

Multiple projects in the build are located in the same directory:

Is there a way to ‘includeBuild’ a library layout correctly ? Thanks!

2

dependencyUpdates is a task of the gradle-versions-plugin.
You do not apply it to the project you created, but try to depend on it from A.
Apply the plugin in the root project there too, then it should probably work.

3

Hallo Björn, thank you for your response!

Either I misunderstand what you are trying to tell me, but I think the question is different?

Gradle versions higher than 6.6.1 have a lib folder when creating a library setup. So the way things worked with composite builds do not work any longer when using includeBuild with a library build that was created with Gradle > 6.6.1. The reason is that build.gradle is now in the “lib” sub-directory which is no longer found by project “A” in my example.

  • If I move build.gradle out of “lib” into the parent directory, things start to work. But it is not how Gradle intends to setup a library project.
  • If I directly includeBuild “Library/lib” things start to work.

Both workarounds don’t seem right to me. Especially because Eclipse Buildship start giving me an error if I it encounters includeBuild "Library/lib"

In particular:

org.eclipse.buildship.core.internal.UnsupportedConfigurationException: Multiple projects in the build are located in the same directory: /Users/bro/Documents/repositories/calimero-usb/lib
	at org.eclipse.buildship.core.internal.workspace.ValidateProjectLocationOperation.run(ValidateProjectLocationOperation.java:52)
	at org.eclipse.buildship.core.internal.DefaultGradleBuild$SynchronizeOperation.runInToolingApi(DefaultGradleBuild.java:230)
	at org.eclipse.buildship.core.internal.operation.DefaultToolingApiOperationManager$WorkspaceRunnableAdapter.run(DefaultToolingApiOperationManager.java:58)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2382)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2407)
	at org.eclipse.buildship.core.internal.operation.DefaultToolingApiOperationManager.run(DefaultToolingApiOperationManager.java:39)
	at org.eclipse.buildship.core.internal.DefaultGradleBuild$SynchronizeOperation.run(DefaultGradleBuild.java:195)
	at org.eclipse.buildship.core.internal.DefaultGradleBuild.synchronize(DefaultGradleBuild.java:103)
	at org.eclipse.buildship.core.internal.workspace.SynchronizationJob.runInToolingApi(SynchronizationJob.java:64)
	at org.eclipse.buildship.core.internal.workspace.SynchronizationJob.runInToolingApi(SynchronizationJob.java:30)
	at org.eclipse.buildship.core.internal.operation.ToolingApiJob$1.runInToolingApi(ToolingApiJob.java:54)
	at org.eclipse.buildship.core.internal.operation.DefaultToolingApiOperationManager$WorkspaceRunnableAdapter.run(DefaultToolingApiOperationManager.java:58)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2382)
	at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2407)
	at org.eclipse.buildship.core.internal.operation.DefaultToolingApiOperationManager.run(DefaultToolingApiOperationManager.java:39)
	at org.eclipse.buildship.core.internal.operation.ToolingApiJob.run(ToolingApiJob.java:65)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
4

Gradle versions higher than 6.6.1 have a lib folder when creating a library setup.

Well, that’s just a default that is generated.
For the case you need to add other modules in the future to not need to then maybe move the code from the root project to a subproject.
I personally never follow that layout, but have everything in the root project if I only have that one project and move things around if need comes up.

So the way things worked with composite builds do not work any longer when using includeBuild with a library build that was created with Gradle > 6.6.1

That project layout does not change too much in the handling of included builds, but if you want to depend on a task called dependencyUpdates in the root project of all included builds then just make sure your included builds have the layout your including build expects, or change your logic in the including build if you prefer to follow the default layout of the init task.

  • If I move build.gradle out of “lib” into the parent directory, things start to work. But it is not how Gradle intends to setup a library project.

As I said, that is just some default layout, that makes it easier / cleaner / whatever to add additional subprojects later if needed. If you don’t want this layout for your projects, just use your own layout like moving everything to the root project, that’s perfectly fine. Gradle has no strong opinion on that. It is just a very questionable convention with which I personally for example do not agree.

  • If I directly includeBuild “Library/lib” things start to work.

That sounds very strange, unless you also have a “settings” script in Library/lib which you should not have if lib is already part of the build define in the setting script in Library. One project should never ever ever ever ever ever ever be included in more than one build, but only ever composite build used to share them.

How buildship behaves I cannot say much to, as I would never for the life of me ever work with Eclipse more then 10 seconds to test something for a colleague in the past and bless god, noone here uses Eclipse anymore. :slight_smile:
But it sounds like you indeed have lib included with include in multiple builds that you then combine using includeBuild like having include("lib") in calimero-usb/settings.gradle(.kts) and additionally a calimero-usb/lib/settings.gradle(.kts) which already is the thing you should never ever … do, and then have in one composite build includeBuild("calimero-usb") and includeBuild("calimero-usb/lib") which makes it even worse.

What you actually asked about was, that in your including build you try to depend on a task called dependencyUpdates in the root project. I have no idea where this task is coming from, but typically it is coming from Ben Manes’ gradle-versions-plugin. If you apply that plugin (or any other plugin createing such a task) to the lib project, the task is only available in the lib project, not in the root project. If you instead apply the plugin in the root project, or create task with that name in the root project that depends on the task in the lib project, or move the lib project contents to the root project, those are all legitimate ways to make the root project have a task with that name and thus your logic in the including build working with your included build.

5

That makes sense!

I experimented a little bit, and adding a “dummy” build.gradle in the root project (the parent where “lib” is), actually makes Gradle happy, and Eclipse too :slight_smile: . (I am in no way intricate with Gradle, but it looks to me that not having an auto-generated build.gradle in the root project of a Gradle-generated multi-project layout is wrong.) Anyway, in the future I will just remove that “lib” convention, it seems a little roundabout to me.

The thing about Eclipse is, that my projects are usually libraries, so users also use Eclipse; IntelliJ works fine wrt the problem statement.

That sounds very strange, unless you also have a “settings” script in Library/lib

No I don’t.

Thanks for your help, much appreciated!

6

Can you knit an MCVE that demonstrates this problem?
You might have hit a bug in Gradle that should be fixed.
Build scripts are optional for a project.
A project not having a build script should be perfectly fine if you don’t need to do any configuration.
It is still a valid Gradle project if it is declared by the settings script.
So auto-generating an empty build script by init is not the right thing to do.
Instead it should just work properly. :slight_smile:

7

@Vampire I created a setup which shows this, but when I try to attach it, I get “Sorry, new users can not upload attachments.”

Anyway, the steps are like this:

  • We want to create a composite build, where the “conductor” project directs composites “app” and “library” which are in sibling directories.
  1. create a folder “conductor”: use gradle init for basic (groovy or kotlin doesn’t matter) **
  2. create a sibling folder “app”: use gradle init for java application.
  3. in the conductor build.gradle add
plugins {
	id 'java-library'
}
clean.dependsOn gradle.includedBuilds*.task(':clean')
assemble.dependsOn gradle.includedBuilds*.task(':assemble')

and add to settings.gradle

includeBuild '../app'

So, now we have a composite build where conductor conducts “app”

  1. in “conductor”, doing gradlew clean assemble should just work for java stuff

  2. now go to another sibling directory “library”

  3. again use gradle init but for java => library.

  4. In conductor settings.gradle add another composite

includeBuild '../library'
  1. in “conductor”, doing gradlew clean assemble will fail:
* What went wrong:
Could not determine the dependencies of task ':clean'.
> Task with path ':clean' not found in project ':library'.

I do understand that the Gradle “library” directory does not need a build.gradle in the root project by itself, but it is already set up as multi-project build (it has include(‘lib’) in its settings file), so IMO the multi-project setup should work when called out of the box from a top-level perspective?

** I accidentally ran this command in the wrong folder, used Ctrl^C and ran into this issue. Merging the fix would be nice, because if you don’t closely watch the folder you will wonder where the additional Gradle files on hard disk come from afterwards. Also, after deleting the whole folder and redoing the steps, Gradle just recreated the folders without asking?!

8

Can you maybe share it on GitHub?
Or you can send it via https://swisstransfer.ch/ or similar and I’ll attach it for you.

9

https://www.swisstransfer.com/d/25e5c359-1e1f-42fd-8e7d-8e097678a0e9

10

gradle-lib-composite.zip (5.8 MB)

11

Task with path ‘:clean’ not found in project ‘:library’.

Of course this fails, that is expected in this setup.

In app you do include('app') which includes an empty subproject, as you do not have anything in app/app and also do not configure it through some other means (which you shouldn’t anyway), you could just leave out the include('app') and have the same result as you do everything in the root project.

In library you have the root project, which effectively is empty like app/app and the subproject lib which you include using include('lib') and where you do your logic.

In your conductor you include the builds in app and library which includes app with its root project : (in the context of the composite build :app) and its empty subproject :app (in the context of the composite build :app:app) and library with its empty root project : (in the context of the composite build :library) and its subproject :lib (in the context of the composite build :library:lib).

Then in conductor you depend on the tasks clean and assemble in the root projects of all included builds. This works for app as there you have the logic in the root project and the subproject is empty, but not for library as there the root project does not have theses tasks but only the subproject, hence it complains that you want to depend on :library:clean and :library:assemble which do not exist.

If you just add an empty build.gradle to library/ this will also not change anything, as the tasks would still not be there. With the single line plugins { id 'base' } within library/build.gradle it will build successfully as the base plugin adds the clean and assemble tasks.

But even then, it would not do what you expect, because you just depend on clean and assemble from the root project. So unless you make the root project tasks depend on the according subproject tasks, the clean and assemble tasks of the :library:lib project would not be executed at all.

12

In app you do include('app') which includes an empty subproject

This was not intended, but was a mistake already known by this issue. I never manually edited the directory or any of its files. I am sorry it’s included in the zip.

and the subproject lib which you include using include('lib') and where you do your logic

I didn’t include anything. Gradle is doing all of this! That’s my pet peeve here. I did accept your answer before, because I do concur the “lib” subdirectory is maybe not the solution I want. And in the state it is presented, it doesn’t work with composite builds (without adding superfluous files).

So auto-generating an empty build script by init is not the right thing to do.
Instead it should just work properly. :slight_smile:

That’s what he said :stuck_out_tongue:

Out of scope (no need to answer):
So, having tried and knowing that plugins { id 'base' } is not really ideal, and currently I repeat all the plugins I need in the root directory in a plugins { ... } block (in build.gradle in “library”), is there any read-up or how-to which makes this less repetitive?

13

I didn’t include anything.

Didn’t mean to blame you for anything, just explaining the situation.

Gradle is doing all of this!

Not “all”. The “conductor” was done by you and is the “culprit”, as it depends on a root project task of a build that does not exist. :slight_smile:

And in the state it is presented, it doesn’t work with composite builds (without adding superfluous files).

It works perfectly fine with composite builds.
The problem are not composite builds.
If you for example depend on the lib in there by coordinates, it works exactly like designed.
The only problem is, that it does not match your expectations in “conductor”, that all your included builds have the tasks clean and assemble in their root project.

That’s what he said :stuck_out_tongue:

Yes, and it does just work properly as designed.
Again, it just does not meet your expectations in “conductor”.

Again, I don’t want to blame anyone.
I just try to explain you why it works (or not works) like it is and where the problem is in your MCVE that makes it not work, and that the “work-around” you did, does not do what you expect it to do.

is there any read-up or how-to which makes this less repetitive?

Not sure what you mean.
As explained above, duplicating the plugins you apply in :libraries:lib within :libraries will make the tasks you expect from conductor to be available, but it will not do what you expect, as you only call the tasks from the root project, unless you then wire down from the root project to the subproject too.

To be less repetetive, just don’t use a subproject, but directly the root project as already concluded above?

Generally spoken, so avoid repetition in build logic, you should use convention plugins, for example implemented as precompiled script plugins, that you then apply where you want their effect to be present and that then applies the conventions you want to have.