Composite Build with multi-projects

Help/Discuss
1

Hello everyone,

We have two multi-projects that we want to build with composite-build. I uploaded an example code to GitHub (https://github.com/pwoelfle/gradle-composite-build-multiprojects).

There we have an application multi-project that has two sub-projects and another library multi-project that also has two sub-projects. The sub-projects of the application use the library sub-projects as binary dependency. The root projects of both multi-projects have not applied the Java plugin.

Using a separate composite-build project that simply includes both multi-projects as include-build, we want to build both at the same time with the benefits of dependency substitution. This case is like in the documentation (https://docs.gradle.org/current/userguide/composite_builds.html#separate_composite) only with multi-projects.

When we execute the clean or build task that is forwarded to the included-builds on the composite-build project, an exception occurs that the task is not found on the included-build project.

If we add the Java plugin to the root projects of the multi-projects, no exception occurs, but only the clean or build task on the root project is executed.

Is this operation not supported yet or did we missed something?

Thanks for any advices :slight_smile:

2

Task execution in included builds is not yet supported.

3

This might have changed since then. Right now (4.7) it is possible to reference tasks of subprojects from an included build:

task clean {
    dependsOn gradle.includedBuild('project-lib').task(':lib-one:clean')
    dependsOn gradle.includedBuild('project-lib').task(':lib-two:clean')
    dependsOn gradle.includedBuild('project-application').task(':subproject-one:clean')
    dependsOn gradle.includedBuild('project-application').task(':subproject-two:clean')
}

.../composite-build$ gradlew clean --console=plain
> Task :project-lib:lib-two:clean UP-TO-DATE
> Task :project-lib:lib-one:clean UP-TO-DATE
> Task :project-application:subproject-two:clean UP-TO-DATE
> Task :project-application:subproject-one:clean UP-TO-DATE
> Task :clean

However this is pretty tedious to maintain since you have to know the internal structure of the included projects. Rather than breaking encapsulation it’s possible to define a clear interface, let’s say each included project must have a cleanFull task (separate from rootProject’s clean, because it leaves it possible to call :clean without executing all subproject’s ...:clean).

// project-lib/build.gradle
task cleanFull { dependsOn subprojects*.task(':clean') }
// project-application/build.gradle
task cleanFull { dependsOn subprojects*.task(':clean') }
// composite-build/build.gradle
task clean { dependsOn gradle.includedBuilds*.task(':cleanFull') }

and then the same command executes the same tasks (with a bit of extra delegation)

.../composite-build$ gradlew clean --console=plain
> Task :project-lib:lib-one::clean UP-TO-DATE
> Task :project-lib:lib-two::clean UP-TO-DATE
> Task :project-lib:cleanFull UP-TO-DATE
> Task :project-application:subproject-one::clean UP-TO-DATE
> Task :project-application:subproject-two::clean UP-TO-DATE
> Task :project-application:cleanFull UP-TO-DATE
> Task :clean

Note that usually we would probably want the included projects’ rootProject:clean to also execute, in that case simply replace subprojects with allprojects and then we get:

.../composite-build$ gradlew clean --console=plain
> Task :project-lib::clean UP-TO-DATE
> Task :project-lib:lib-one::clean UP-TO-DATE
> Task :project-lib:lib-two::clean UP-TO-DATE
> Task :project-lib:cleanFull UP-TO-DATE
> Task :project-application::clean UP-TO-DATE
> Task :project-application:subproject-one::clean UP-TO-DATE
> Task :project-application:subproject-two::clean UP-TO-DATE
> Task :project-application:cleanFull UP-TO-DATE
> Task :clean

This approach also allows for each included project to decide what cleanFull means separately.

Big thank you to @pwoelfle for the example project! Credit is all yours, you were pretty close; love this use of *. syntax.

4

Something is wrong with the above syntax, in my own project I get this warning:

The task name ':clean' contains at least one of the following characters: [ , /, \, :, <, >, ", ?, *, |]. This has been deprecated and is scheduled to be removed in Gradle 5.0.
        at build_3exurscyyqkncbej3cesp295r$_run_closure3.doCall(included\build.gradle:40)
        (Run with --stacktrace to get the full stack trace of this deprecation warning.)

Using dependsOn subprojects*.tasks*.findByPath('clean') solved this problem.

5

@TWiStErRob That’s when you explicitly set it up, but @pwoelfle was asking about build automatically calling the build task in all included builds. This is not supported as of today.

6

Oh, I see, so sub-project-like behavior with included builds.

I hope my examples are still useful for someone coming to this thread.
This was the only example I found for multi-project multi-build setup.

7

Here is another option you could use. You can grab all of the task names that were specified on the command line:

def startParameterTasks = project.gradle.startParameter.taskNames.collect()

Once you have that list, you can programmatically add them to all of your included builds:

(I’m using this in a plugin)

startParameterTasks.each {taskName ->
    project.getTasksByName(taskName, false).each { task ->
        task.dependsOn project.gradle.includedBuilds*.task(":$taskName")
    }
}

You can do other interesting things with this logic too. Here is what I’m currently doing… I created some logic based on one of three behaviors specified from the command line:

./gradlew <tasks> -P task.behavior=default|all|none

I then added logic based on those three values (as an enum; no value is default):

  • none: don’t pass any tasks specified on the command line to included builds
  • all: pass all tasks specified on the command line to included builds
  • default: pass only default tasks that are specified in a list and that were specified on the command line

Additionally, you can put all of this into your own plugin and make it standard across all of your projects. This bit of code, in my opinion, is a good workaround for one of the current composite build limitations.