Multi-module (multi-project) builds are straight forward, since they are described by a single settings.gradle file, but attempting to aggregate them into larger comprehensive builds is not well supported at this time.
The new parallel build capability does a good job of describing decoupled items, so let’s assume that all projects/modules are decoupled between independent multi-module builds, such that logically there is a DAG also at the multi-module to other multi-module(s) level. This guarantees that combining the underlying graphs would still result in a DAG. Therefore “a” could be worked on completely independently, and then “b” just requires “a” to also be part of the larger combined checkout of work items.
There currently is no such thing as a “combined.gradle”, and it’s actually very difficult to attempt to blend settings information programmatically. It could automatically remap the actual directory paths for the logically named modules – if there are no naming collisions.
You can make this work by hand-crafting the settings.gradle from “b” and replicating the information in “a” (plus manually re-mapping the directory paths), but that requires all commands be run relative to the “b” tree only and clearly code duplication is rarely a good pattern.
Are combined multi-module builds still being targeted for a future release? This also goes hand-in-hand with the potential ability to specify a dependency as a direct module/project output or an externally resolved dependency if the referenced project not directly part of the multi-module build tree.
I thought work was probably already being done in this area, but haven’t seen any active chatter on it either here or the mailing lists for quite some time. I suspect I’m going to have to create some custom form of composite multi-module build support soon, and would very much like to steer the concepts and direction to align with the longer term official solution.
Should I post ideas and hurdles here in the forums, on the dev mailing list, or as part of a gist somewhere? As noted, this isn’t a trivial concept, and has many possible implementations that fundamentally change the lowest-level core, so hashing out the details is highly desired.
but haven’t seen any active chatter on it either here or the mailing lists for quite some time.
We still have much foundational work to do, so it’s not something we are actively discussing. We are currently working on publication though and it does cut into this area. As the project decoupling and publication work continue in parallel this will eventually come to the fore. The mailing list is where the discussions will take place.
Should I post ideas and hurdles here in the forums, on the dev mailing list, or as part of a gist somewhere?
If you hit hurdles, this is the place to try and resolve them. If you come up with novel techniques that you can use to implement this in your builds, it’s best to share them here so others can benefit. If you want to participate in the design of this feature in Gradle itself then the dev list is the place.
has many possible implementations that fundamentally change the lowest-level core, so hashing out the details is highly desired
Unfortunately, it’s not realistic to try and sync up with what the eventual solution will be at the fine level. The fundamentals will be the same though:
Don’t use project names/paths as identifiers in your build, use some strategy to dereference them to their real path in the “current” build (i.e. the project path may change depending on how dynamic your composite project is) 2. Don’t reach across project boundaries except for parents pushing configuration down to children (based on the assumption that a child always has the same parent and can’t exist without it) in any other way than project dependencies. 3. Make as few assumptions about what is outside the immediate project and direct parental chain as possible. 4. Use a physical layout that can be “parsed” dynamically in the settngs.gradle
At a high level, that’s all the advice I can offer at this time.
It will be very helpful to incorporate your experiences/learnings into the design of this capability so I’m looking forward to seeing how it goes for you.
A year has passed since the last word on this topic. We’re very interested in how this has progressed since then, having a project with 500+modules and wanting to find clean ways of improving performance, perhaps by finding independent DAGs as suggested.