Multi-project builds with empty levels

I have a hierarchy like so:

trunk/
–build.gradle
–settings.gradle
–group1/
----projectA/
----projectB/
----projectC/
–group2/
----projectD/
----projectE/

This is how it appears in source control/on the filesystem. My settings.gradle only includes leaf nodes, as you can imagine. For the most typical tasks, like clean/test/build/javadoc/etc., everything works the way I expect: if I issue the instruction at the top level, the intermediate levels are mostly ignored, and the “real” projects are built with the appropriate command.

However, there are a few funny things which happen. First, the build task actually creates a jar for group1 and group2, even though there isn’t anything in it (besides a trivial manifest.mf). Second, the maven-publish task tries to upload this bogus project to the repo. Third, I created some custom helper tasks to show the various reports which are generated (JUnit, Javadoc, PMD, etc.). If I issue these commands at the top level, they fail when they hit the group level, because no documents exist there. I had to add some code to ignore this level.

I assume that my workaround for my custom tasks is the proper one, and that, in fact, all of the tasks are really evaluating projects at all levels, but trying to do something sensible at each one, even if there is no real work to do. But I would like to have a principled way of saying: “The group level only exists for organizational purposes. Skip these projects during recursive multi-builds.” Ideally, this mechanism could be implemented across all tasks. Unfortunately, my Groovy-fu and overall Gradle understanding is far too limited to come up with this mechanism myself. I have seen examples where it is possible to hook into all tasks, but I would also like a solution that not only works, but is The Right Way™. Thanks for any tips.

Dave

First, the build task actually creates a jar for group1 and group2, even though there isn’t anything in it (besides a trivial manifest.mf). Second, the maven-publish task tries to upload this bogus project to the repo. Third, I created some custom helper tasks to show the various reports which are generated (JUnit, Javadoc, PMD, etc.).

You are probably applying the relevant plugins e.g. the Java plugin to all subprojects in the top-level root project. Try to be more selective by filtering the set of projects you are using for configuration injection.

The group level only exists for organizational purposes. Skip these projects during recursive multi-builds.

You might want to think about breaking out the projects by groups into separate VCS repositories. Alternatively, just run the build from the directory-level of the group or directory reference the group project when executing the Gradle command e.g. gradle :group1:build. Then you don’t execute all tasks for all subprojects.

You are probably applying the relevant plugins e.g. the Java plugin to
all subprojects in the top-level root project. Try to be more selective
by filtering the set of projects you are using for configuration
injection.

Ok. I’m not really sure how to do that. I put my generic build file off to the side, like so:

gradle/
–build.gradle { subprojects { apply plugin: ‘java’ … all the main build logic … } }
trunk/
–build.gradle { apply from: “…/gradle/build.gradle” }

It is true that the root project does everything in subprojects { … }, which includes the group layer, unfortunately, But I’m trying to avoid putting boilerplate at the group level, and I also want to be able to do a global build of everything from the top level.

You might want to think about breaking out the projects by groups into separate VCS repositories. Alternatively, just run the build from the directory-level of the group or directory reference the group project when executing the Gradle command e.g. gradle :group1:build. Then you don’t execute all tasks for all subprojects.

I’d rather not use separate repos because that is really inconvenient in my environment. What I was hoping is that I could run builds at the individual project level, the group level, or the root level, and the appropriate scope of projects would get rebuilt. I should also mention that the projects all inter-depend across groups.

If you could give me a hint on how to do the filtering, I think that is what I prefer. Thanks.

1 Like

It is true that the root project does everything in subprojects { … }, which includes the group layer, unfortunately, But I’m trying to avoid putting boilerplate at the group level, and I also want to be able to do a global build of everything from the top level.

It would look similar to this depending on your needs:

def javaProjects() {
    subprojects.findAll { subproject -> 
       !subproject.name.startsWith('group')
    }
}

configure(javaProjects()) {
    apply plugin: 'java'
}

Let me see if I understand what is going on in your example. Right now, I have:

subprojects {
// do everything that I want to be inherited by all projects
}

This mostly works because all the leaf nodes end up with this logic, but my intermediate layer does as well. I assume that subprojects is a method which takes a closure, and that closure is what is implementing all of my build logic.

Your javaProjects() method returns a list of only the leaf node projects, because subprojects includes everything below the current project, all the way down.

Then, I presume that my code is basically equivalent to:

configure(subprojects) {
// do everything that I want to be inherited by all projects
}

and that your code just limits subprojects to the ones I’m actually interested in configuring. Is that correct? Thanks for the help!

Also, is it more efficient to define a function than a closure? I get the impression that closures are more preferred in gradle, but probably just because I haven’t identified a pattern for what should be a function vs. a closure.