Decoupled projects vs. cross-project configuration


(Björn Kautler) #1

The userguide says that the preferred method of doing common build configuration in a multi-project build is by doing cross-project configuration by using allprojects{} and subprojects{}. But later in the userguide, it is written that this will couple the projects and thus make usage of new features like parallel project execution and others non-usable.

So I have two questions regarding this: 1. Is there an alternate recommended way of applying common build logic and why isn’t it instead recommended in the userguide? 2. Is there some functionality to verify whether there is a coupling of projects present, not only by means of allprojects{} and subprojects{} but also by other means and if not, could this maybe get implemented?


(Luke Daley) #2

It really depends on what you want to use. Parallel build imposes some constraints, while configure-on-demand imposes more.

For parallel build, you run into problems if you try and change the state of other project during execution time. Doing so during configuration time is ok. This means that it’s ok to use allprojects {} , subprojects {}, project() {} etc as long as it’s during configuration time.

For configure-on-demand, it gets more complicated because not al build configuration is executed. In that case, the only configuration injection that is possible is from the very root project down (because the root project is always evaluated). If this is too coarse, the only other approach at this time is to use a script plugin that contains commonality that you explicitly apply from each project that needs it. You can centralise the config, but end up duplicating a lot of apply() statements.

So the short answer is: There’s not silver bullet alternative at this time.

Some general rules:

  1. Try to limit configuration injection that is not top down 2. Avoid changing the configuration of other projects at execution time 3. Avoid accessing the configuration of other projects (other than top down configuration injection)

For #3, the only cross project communication protocol that configure-on-demand mode supports right now is the use of project dependencies.

Hope that helps.


(Luke Daley) #3

That’s a bug. Raised as GRADLE-2777.


(Björn Kautler) #4

I guess this reply was meant for http://forums.gradle.org/gradle/topics/is_it_intended_that_in_compare_gradle_builds_the_absence_of_a_jar_in_both_builds_is_considered_non_equal ?


(Luke Daley) #5

Yes, it was. Thanks.


(Björn Kautler) #6

Thanks, but then I still have one question and two requests:

Question:

You say that in the root project you can use allprojects{} and subprojects{} without a problem as it is always evaluated. If I have a project structure like:

a -> aa -> -> aaa -> -> aab -> ab -> ac

Can it be that with configure-on-demand only a and aaa is configured, or is aa also always configured if aaa is configured. Becuase in the latter case you could also use subprojects{} and allprojects{} in the aa project without a problem and this would also adhere to your general rule 3 of only doing top down configuration injection.

Requests:

  1. The documentation should be improved, as http://www.gradle.org/docs/current/userguide/userguide_single.html#sec:decoupled_projects states that allprojects and subprojects already cause a coupling and you cannot use the features that need decoupled projects. A more detailed definition like you gave here (thanks for that) in the userguide would be nice. Either in the section I mentioned, and / or in the chapters about the specific features that need a certain level of decoupledness.

  2. My original point 2: “Is there some functionality to verify whether there is a coupling of projects present, not only by means of allprojects{} and subprojects{} but also by other means and if not, could this maybe get implemented?”, just extended by checking for certain levels of decoupledness, maybe having as result “feature a is possible, feature b not because projects a and b are coupled in file x line y”. I see that this check cannot be done when the feature that needs it is used, as with configure-on-demand e. g. you cannot know whether a coupling is present in the parts that are not configured yet.


(Szczepan Faber) #7

Hey,

If you run “gradle a:aa:aaa” then only “a” and “aaa” are configured. Other words, we only handle the root project ‘specially’. This will change in future when we add better ways for common configuration. Then we may not configure the root project by default.

I agree with both of your requests. Feel free to submit a pull request with improved documentation :)\

Detecting when projects are incorrectly coupled is already on our roadmap. It is required for unincubating both: parallel and COD.

Hope that helps!


(Björn Kautler) #8

Ah, thanks, that is interesting to know. :slight_smile:

Regarding the pull request, I already did for two other issues at https://github.com/gradle/gradle/pull/153, but it doesn’t get pulled, at least not until now. Besides that, maybe someone who has more knowledge about the topic should improve the documentation, I could only re-write what Luke said here.


(Luke Daley) #9

Sorry, that should have been pulled earlier. I just did it. Many thanks!

Regarding the docs for this CoD, I always find it valuable to have docs from the perspective of someone who has more distance on it than the developers. Even if you were to just submit your interpretation of my comments it would be helpful.


(Björn Kautler) #10

Ok, I’ll have a look whether I have time to write something up, thanks for pulling. :slight_smile:


(Michael Brand) #11

I’ve taken a different approach to “configure on demand”. If I have configuration that is common configuration across projects I pull the configuration out into a separate file and included it in each “demanding” project by using “apply from”.

My thinking was to treat my gradle scripts like an OO hierarchy where the parent contains only functionality that is common to all children. “Apply from” files allow me to reuse functionality in only those children that require it. This effectively makes common functionality a plug-in that can be used in much the same way. The solution is simple and elegant.


(Thibault Kruse) #12

I am also a bit disappointed about the documentation here. I know you’ll accept pull requests for improvements, but I cannot write stuff that I cannnot understand from the existing docmentation. Using gradle 1.11 in the following.

What i miss first is a definition of a subproject. So far I gather that anything defined in the settings.gradle counts as a subproject, even if no such folder exists (which I consider a bug, there should be a failure in case of a typo).

It seems that I can organise my subprojects using non-project folders, i.e.

include ‘collection/subproject1’ adds a subproject named ‘collection/subproject1’ with correct subfolder resolution but no subproject ‘collection’, whereas

include ‘collection:subproject1’ implicitly adds two subprojects. I did not see this documented.

The settings command includeFlat is not discussed at all in the multi-module chapter.

It seems that subprojects do not get added recursively, i.e. root - subproject – settings.gradle [include ‘subsubproject’] - settings.gradle [include ‘subproject’]

does not work as might be expected. root in this case does not know about subsubproject.

The command ‘gradle -q projects’ could well be shown and used in the chapter on subprojects at least once.

Regarding coupling, it is still not clear to me whether the real-life example in “56.6.3. Real life examples” has coupled or non-coupled subprojects. I believe the usage of allprojects {} and subprojects {} is only at configuration time (but I a m not even sure about that), and I don’t know whether the usage of “subprojects.each {project…” causes coupling.

Next what I miss is organization of version numbers of dependencies, such as:

depencencies {

compile “org.springframework:spring-core:$SPRING_VERSION”

testCompile “org.springframework:spring-test:$SPRING_VERSION”

compile “org.springframework:spring-context:$SPRING_VERSION”

compile “org.springframework:spring-context-support:$SPRING_VERSION”

compile “org.springframework:spring-beans:$SPRING_VERSION”

compile “org.springframework:spring-aop:$SPRING_VERSION”

compile “org.springframework:spring-aspects:$SPRING_VERSION”

compile “org.springframework:spring-tx:$SPRING_VERSION” } Obviously I want all subprojects using spring to use the same version, and I also want this to work with ‘gradle dependencies’. I also think things look ‘cleaner’ if every subproject has it’s own build.gradle, rather than configuring subprojects in the root projects.

With subprojects, ‘gradle dependencies’ gives me output like: compile - Compile classpath for source set ‘main’. — org.apache.commons:commons-lang3:3.3.0 FAILED

Next, on my trials, I notice that if the root project defines dependencies like: project(‘collection:subproject1’) {

apply plugin: ‘java’

dependencies {

compile(“org.hibernate:hibernate-validator:4.3.1.Final”)

compile(“org.json:json:20140107”)

} }

but the given subproject has an own settings.gradle, the dependencies are not listed when running ‘gradle dependencies’. I cannot decide whether this is a feature or a bug. The role of a settings.gradle in a subproject is mysterious.

Another problem I had was to define a common release number for all decoupled subprojects. I guess that can be done using allprojects {} in the configuration context without coupling the subprojects, But I did not see that in the user guide yet, so +1 for 'Vampire’s request do add this to the docs.


(Thibault Kruse) #13

BTW else I also used the approach suggested mzbrand, I have separate gradle files that I use in the subprojects with ‘apply from …’. That seems more explicit to me and thus to be more understandable by gradle novices than the implicit magic of gradle inheritance. So I have one version.gradle file defining the group and version for all subprojects and the common versions of dependencies (like SPRING_VERSION), and another called java-builds.gradle defining plugins, tasks, task dependencies and common util java dependencies (like slf4J) for java subprojects.


(Thibault Kruse) #14

Actually to even the meager output of gradle dependencies, I needed to add this to my root project:

dependencies {
    runtime(project(':subproject1'))
    testRuntime project(':subproject1').sourceSets.test.output
    runtime(project(':subproject2'))
    testRuntime project(':subproject2').sourceSets.test.output
}

still not even sure whether I can write that in a generic way without causing coupling


(Thibault Kruse) #15

As far as I can see, the settings.gradle is central to the relationship between root project and subproject. Essentially, no subproject should be allowed to have it’s own settings.gradle, so I consider it a bug that gradle does not raise an error when that is the case. And the documentation should define that there is no recursion in the relationship, that the root settings.gradle defines all subproject, and that each subproject recognizes it’s root project by walking up the file tree to the first folder having a settings. gradle, but only if that one mentions this subproject as a subproject.


(Thibault Kruse) #16

Seems the “FAILED” output I got for ‘gradle dependencies’ was due to a flaw in my repository declaration location, so forget about that.