Gradle task execution runs a subjproject task on its own, rather than in the context of the whole multiproject

I have a Gradle multiproject, flat structure, something like this:

  • master
  • sub1
  • sub2

From the Gradle Tasks view I try to launch a task for sub1 (like: sub1:jar, for instance), but the build fails because it seems like it’s trying to execute sub1:jar on its own, without processing the whole multiproject. Indeed, the master project contains a gradle.properties file, some configuration code in allprojects block and so on. However, these don’t get processed and the task execution fails (for instance because it can’t find a property defined in that master gradle.properties file).
At first I thought this were a regression of the latest Eclipse/Buildship version, but I tested this accross multiple versions of Eclipse/Buildship and with many of our multiprojects and I can state for sure that this problem only occurs on a single Gradle multiproject among all our many multiprojects we have (all using the same structure).
What can be causing this? I can’t really understand what is different here…

As a workaround I found out that changing the launch configuration for that task in order to set the working dir as ${workspace_loc:/master} (instead of the automatically generated ${workspace_loc:/sub1}) and the task to sub1:jar (instead of the generated jar) fixes the problem. But this is annoying because every time I use the Gradle Tasks view to start a task, a new launch configuration is created using the default settings and execution fails.

This is driving me crazy.
Thanks in advance for any help.

Are those projects where it works Gradle versions <7 and the build where it is failing on >=7?

From the upgrade notes:

Removal of searching for settings files in ‘master’ directories

Gradle no longer supports discovering the settings file in a directory named master in a sibling directory. If your build still uses this deprecated feature, consider refactoring the build to have the root directory match the physical root of the project hierarchy. You can find more information about how to structure a Gradle build or a composition of builds in the user manual. Alternatively, you can still run tasks in builds like this by invoking the build from the master directory only using a fully qualified path to the task.

Besides that using allprojects { ... }, subprojects { ... } and so on are anyway legacy and bad practice. You should consider switching to convention plugins instead.

Hi Björn,
thanks for pointing this out.
However no: I have working builds on 7.4 and 7.4.2, while the non-working build is on 7.6.1. Also, I tried to downgrade the Gradle wrapper on the non-working multiproject to version 7.4.2, but this still fails. There must be something different, I can’t understand what it is.

Also, let me say that the referenced sentence in the upgrade notes is quite obscure. I mean, in 7.1 (not in 7.0!) Gradle tried to deprecate the flat multiproject layout (which seems like not being even mentioned any more in the manual), however on popular demand the deprecation was removed from Gradle 7.4.
The master project is the natural location for the settings file in such a multiproject layout, because master is indeed the root project. So, I personally find that notice in the Gradle 6->7 upgrade notes confusing and contradictory… And, in particular, all was working fine on our side, even with Gradle 7, unless I hit this particular multiproject that is driving me crazy now. And we do have settings.gradle and gradle.properties in master project.

@donat do you have any suggestion on what I could check to understand what is going on here?

Thanks a lot in advance.

You should really not ping random people just because you think they might be able to help in an open community if that person is not already involved in the discussion or you really need something personal of that person, that is pretty rude. Almost as rude as DMing someone with questions that could be posted the same in a public place dedicated for it. :wink:

Also, let me say that the referenced sentence in the upgrade notes is quite obscure. I mean, in 7.1 (not in 7.0!) Gradle tried to deprecate the flat multiproject layout (which seems like not being even mentioned any more in the manual), however on popular demand the deprecation was removed from Gradle 7.4.

https://docs.gradle.org/current/userguide/upgrading_version_7.html#non_hierarchical_project_structures

If you are using the one-page version of the docs, you always have to remember that no all pages that should be included in the one-page version are included, some like that one are missing, so if missing something, try to find it with the site search instead of Ctrl+F.

And, in particular, all was working fine on our side, even with Gradle 7, unless I hit this particular multiproject that is driving me crazy now. And we do have settings.gradle and gradle.properties in master project.

Actually, I’d assume you might just not notice the others are broken as you maybe do not do the bad cross-project configuration. But hard to guess without investigating your actual projects.

I did a quick test with Gradle 8.1.1:

$ master/gradlew -p master projects

> Task :projects

------------------------------------------------------------
Root project 'showcase'
------------------------------------------------------------

Root project 'showcase'
\--- Project ':foo'

To see a list of the tasks of a project, run gradlew <project-path>:tasks
For example, try running gradlew :foo:tasks

BUILD SUCCESSFUL in 22s
1 actionable task: 1 executed
$ master/gradlew -p foo projects

> Task :projects

------------------------------------------------------------
Root project 'foo'
------------------------------------------------------------

Root project 'foo'
No sub-projects

To see a list of the tasks of a project, run gradlew <project-path>:tasks
For example, try running gradlew :tasks

BUILD SUCCESSFUL in 9s
1 actionable task: 1 executed

So try to run the projects task in one of the builds you deem working and then let’s see what it says. :slight_smile:

I’m not “pinging random people”, I’m asking Donat which is the main lead of Buildship, since this problem happens in Buildship only and I discussed with him many other times for other problems and also helped to solve them. I’m not forcing him to reply if he doesn’t want to and I definitely don’t think I’ve been rude. If so, I’d appreciate Donat to say that.

I used the link you provided me to get to the documentation, which points to the one page version of the manual, and then tried Ctrl+F. I didn’t know it was not including the whole manual. I’m used to other projects documentations, like the Spring ones, where the single page manual is just… the whole manual being put in a single page :wink:
Still, even if I use the site search for “includeFlat” I don’t find any mention to the flat multiproject structure any more apart from the “deprecation” reference you’re pointing to and links to the DSL reference. So it still seems like any reference to the flat multiproject layout has gone from the user manual.

The “bad cross-project configuration” is just this:

allprojects {
	repositories {
		mavenLocal()
		maven {
			url myMavenRepoUrl
			allowInsecureProtocol true
		}

		flatDir name: 'flatDirLib', dirs: "../sub1/lib"
	}
	// define the group id for all projects
	group = 'my.group.id'
}

And this is in the non-working multiproject just like it is in the working ones. Plus gradle.properties and settings.gradle, once again in both the working multiprojects and in the non-working one.
Having to write a plugin for the above sounds overkill to me, honestly.

This is what it says on a working multiproject:

D:\co\DCS-FATTURAPA-trunk>master\gradlew -p master projects

> Task :projects

------------------------------------------------------------
Root project 'DCS-FATTURAPA'
------------------------------------------------------------

Root project 'DCS-FATTURAPA'
+--- Project ':fatturapa-api'
+--- Project ':fatturapa-jpa'
+--- Project ':fatturapa-rest'
+--- Project ':fatturapa-sdi'
+--- Project ':fatturapa-service'
+--- Project ':fatturapa-storage'
+--- Project ':fatturapa-utility-invio'
+--- Project ':fatturapa-utility-ricezione'
+--- Project ':fatturapa-web-common'
+--- Project ':fatturapa-web-invio'
\--- Project ':fatturapa-web-ricezione'

To see a list of the tasks of a project, run gradlew <project-path>:tasks
For example, try running gradlew :fatturapa-api:tasks

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed

D:\co\DCS-FATTURAPA-trunk>master\gradlew -p fatturapa-web-invio projects

FAILURE: Build failed with an exception.

* Where:
Build file 'D:\co\DCS-FATTURAPA-trunk\fatturapa-web-invio\build.gradle' line: 25

* What went wrong:
A problem occurred evaluating root project 'fatturapa-web-invio'.
> Project with path ':fatturapa-api' could not be found in root project 'fatturapa-web-invio'.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 3s

And this is what Buildship does when I double click on, say, fatturapa-web-invio:war task from the Gradle Tasks view:

As you can see, sibling projects are configured, their required tasks are run and the build goes all green.

Now, let’s do the same of the non-working multiproject:

mauro@hppb ~/svn/DCS-CONSERVAZIONE-trunk $ master/gradlew -p master projects

> Task :projects

------------------------------------------------------------
Root project 'DCS-CONSERVAZIONE'
------------------------------------------------------------

Root project 'DCS-CONSERVAZIONE'
+--- Project ':conservazione-annoto'
+--- Project ':conservazione-annoto-sql'
+--- Project ':conservazione-client'
+--- Project ':conservazione-commons'
+--- Project ':conservazione-daemons'
\--- Project ':conservazione-server' - Conservazione Server

To see a list of the tasks of a project, run gradlew <project-path>:tasks
For example, try running gradlew :conservazione-annoto:tasks

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed

mauro@hppb ~/svn/DCS-CONSERVAZIONE-trunk $ master/gradlew -p conservazione-daemons projects

FAILURE: Build failed with an exception.

* Where:
Build file '/home/mauro/svn/DCS-CONSERVAZIONE-trunk/conservazione-daemons/build.gradle' line: 25

* What went wrong:
A problem occurred evaluating root project 'conservazione-daemons'.
> Could not get unknown property 'springVersion' for object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s

Seems like quite similar to me. Please note this is failing not because it doesn’t find a dependency project (because this particular subproject has no dependency over sibling projects) but because it doesn’t find springVersion property which is defined in master/gradle.properties.

Then, if I run, say, conservazione-daemons:war task from the Gradle Tasks view:

immagine

As you can see, it doesn’t configure the other subprojects. What goes wrong is printed in the Console view:

Once again the problem on the springVersion property that is not resolved.

As I said from the beginning: it seems like Buildship is running the task with “-p subproject” on this multiproject, while it is doing it with “-p master” in the working ones (which is the expected behaviour).

The file .settings/org.eclipse.buildship.core.prefs is similar as well in both situations. In particular, it contains this:

connection.project.dir=../master

in both the subproject of the working multiproject where I run the task successfully and in the one of the non-working multiproject where task execution fails. So it seems like settings are in place to tell Buildship that a root project exists and is in master. Indeed, dependency resolution and project setup in the workspace is fine, the only problem I have is with task execution from the Gradle Tasks view.

I also tried to remove all the projects from the workspace of the non-working multiproject and re-import them from scratch using the “Import Existing Gradle Multiproject” wizard, hoping this could fix any initial setup issue, but it didn’t.

I used the link you provided me to get to the documentation, which points to the one page version of the manual, and then tried Ctrl+F. I didn’t know it was not including the whole manual. I’m used to other projects documentations, like the Spring ones, where the single page manual is just… the whole manual being put in a single page :wink:

Yes, I fully agree that this is a doc bug: Improvements for the single-page documentation. · Issue #16760 · gradle/gradle · GitHub :slight_smile:

Still, even if I use the site search for “includeFlat” I don’t find any mention to the flat multiproject structure any more apart from the “deprecation” reference you’re pointing to and links to the DSL reference. So it still seems like any reference to the flat multiproject layout has gone from the user

Yeah, I guess while supporting the existing users of it, they still want to not advertise it. Or it is a bug that they removed the deprecation but not reintroduced the according documentation in https://github.com/gradle/gradle/pull/19135.

The “bad cross-project configuration” is just this:

The “problem” is, that all these APIs like subprojects { ... }, allprojects { ... }, and so on immediately introduce project coupling. Besides generally making builds harder to understand and harder to maintain (not sooo much in that concrete case, but generally), the project coupling works against more sophisticated features like upcoming project isolation or configure-on-demand. Thus the idiomatic way is to use convention plugins that mitigate all that.

Having to write a plugin for the above sounds overkill to me, honestly.

Writing a plugin actually is pretty trivial, especially if you use precompiled script plugins.
They look almost like normal build scripts and allow you to quickly and easily write convention plugins without much ceremony.

Actually, in the snippet you showed I would actually not use a convention plugin either, but I would put the group = my.group.id to the gradle.properties of the root project where it then is in effect for all projects that do not overwrite it and move the repository delcarations to dependencyResolutionManagement { repositories { ... } } in the settings script, where you can define repositories for all projecst in the build and also can set a property that prevents projects to add additional repositories individually to make sure you have all repositories declared centrally. :slight_smile:

As I said from the beginning: it seems like Buildship is running the task with “-p subproject” on this multiproject, while it is doing it with “-p master” in the working ones (which is the expected behaviour).

Ah, sorry, I somehow missed this fact or did not interpret what you wrote in that way. I thought it works to do master\gradlew -p fatturapa-web-invio war in the working project but not in the non-working one. So I guess this also does not work from the commandline in the “working” project.

Then I’m out, as I don’t use Eclipse.
Sorry again, that I misinterpreted your problem.

I agree with you in general, but I still think that there are cases in which the newly recommended way of doing certain things in Gradle is indeed uselessly complicating an otherwise simple configuration by virtue of an “ideological” point of view.

In this particular case, a multiproject, after all, is a multiproject. It’s not just a set of completely independent projects, but it’s a set of strictly related projects. I agree that decoupling is a good thing and that configuration not put into each individual build.gradle file could be missed if an observer goes there and tries to understand what is going on. However, some simple configuration like the above makes a lot of sense to be put in a “central” section of a multiproject. This said:

Thanks for pointing this out. I will investigate on this which sounds quite a reasonable (and as simple) alternative way to do that.

If anyone else can help with the Eclipse/Buildship problem, he/she is welcome :slight_smile:
Otherwise I may try to open an issue on GitHub, but I would like to understand if there’s something obvious I’m missing before doing that.

1 Like