How to prevent included builds from rebuilding?

Help/Discuss
1

Hi Gradle community,

I’d like to share a solution I found for preventing included builds from rebuilding every time, as it was challenging to discover, and I believe others may benefit from it.

My Use Case: Building gtdsync with Dependencies on Freeplane

I’m developing an add-on called GTD Sync for Freeplane, a mind-mapping application that has a complex plugin-based structure. Due to Freeplane’s architecture (an OSGi framework for plugins), it’s not suitable to package it as a (local) Maven repository dependency. Although Freeplane provides binary JARs for use through the Freeplane Gradle plugin, this doesn’t include the source code.

Since I need source-level access to Freeplane’s classes for code completion, source lookup, and documentation in the IDE, using Freeplane’s source directly as an included build has been the most effective approach. This allows me to access all of Freeplane’s dependencies directly within my IDE without needing to manually add them.

Project Structure

Here’s an overview of the directory structure:

git/
├── gtdsync/                       # Root project directory for the GTD Sync add-on
│   ├── settings.gradle            # Includes freeplane_root as an included build
│   ├── build.gradle               # Adds compileOnly dependencies to Freeplane
│   └── (other GTD Sync files)
└── freeplane_root/                # Freeplane source code as an included build
    ├── settings.gradle
    ├── build.gradle
    ├── freeplane/                 # Main Freeplane project
    ├── freeplane_api/             # Freeplane API
    ├── freeplane_framework/       # Freeplane framework subproject
    └── (several subprojects for Freeplane plugins)

In gtdsync/settings.gradle, I include freeplane_root as an included build:

rootProject.name = 'gtdsync'
includeBuild('../freeplane_root')

And in gtdsync/build.gradle, I add dependencies to the Freeplane source:

dependencies {
    compileOnly 'freeplane_root:freeplane'
    compileOnly 'freeplane_root:freeplane_api'
    compileOnly 'freeplane_root:freeplane_framework'
    // Other Freeplane plugins...
}

Problem: Slow Builds Due to Rebuilding the Included Build

With this setup, packageAddon (part of the Freeplane Gradle plugin) and test tasks can take a considerable amount of time. Every time I run these tasks, Gradle seems to trigger a rebuild of freeplane_root, even though its source hasn’t changed. (I always work with the unchanged source code of a stable release and will rebuild manually when I have the need to switch to another stable release.) This creates a large task graph that slows down the build.

Solutions Tried

Initially, I tried various approaches to control or disable freeplane_root task execution from gtdsync/build.gradle:

  1. Setting onlyIf conditions for tasks in freeplane_root to prevent them from running.
  2. Conditional dependency configurations to isolate Freeplane dependencies.
  3. Custom configurations to separate freeplane_root dependencies from the main and test classpaths.
  4. Gradle properties to conditionally disable tasks in freeplane_root based on a flag.

All of these attempts were based on the assumption that freeplane_root tasks were part of gtdsync’s task graph, but this wasn’t the case. I realized that the rebuild of freeplane_root is actually triggered by defining any dependency on freeplane_root in gtdsync/build.gradle. This dependency forces Gradle to check if freeplane_root is up-to-date and, if necessary, to rebuild it.

Solution: Use --no-rebuild (-a)

The effective solution is to add the -a or --no-rebuild option on the command line:

./gradlew test -a
./gradlew packageAddon -a

According to the Gradle documentation, this flag stops buildSrc from rebuilding, but in practice, it also stops included builds from rebuilding. This flag has been working well for me, as it keeps freeplane_root from rebuilding, making test and packageAddon tasks much faster.

Feature Requests

To improve the experience with included builds, I have two requests:

  1. Documentation Update: It would be very helpful to update the Gradle documentation to explicitly mention that --no-rebuild (-a) also applies to included builds, not just buildSrc.

  2. Feature Request for build.gradle Control: A built-in method to prevent specific included builds from rebuilding would be incredibly useful. This could look something like:

    gradle.includedBuild('freeplane_root').noRebuild()

    for a specific included build, or

    gradle.includedBuilds.noRebuild()

    to apply to all included builds.

These features would allow developers to better manage large projects with included builds, improving performance without needing command-line flags each time.

Thank you for considering these suggestions, and I hope this post helps others facing similar challenges!

2

–no-rebuild stops tasks from rebuilding. It doesnt matter if theses tasks are defined in buildSrc, an included build, adhoc, part of a plugin from the pluginrepository or else… If it would be documented anywhere that this flag only prevents tasks defined in buildSrc from rebuilding that would be a fat bug, but i dont believe thats the case unless you can provide a link.

3

This is the link.

I am sorry that I clearly did not quote/read it very well. This is the exact text:

-a, --no-rebuild
Do not rebuild project dependencies. Useful for debugging and fine-tuning buildSrc, but can lead to wrong results. Use with caution!

It speaks in general of not rebuilding project dependencies and gives a buildSrc use case as an example. This is in agreement with what I observe. In case of:

./gradlew test -a

I see the following task graph:

> Task :compileJava NO-SOURCE
> Task :compileGroovy UP-TO-DATE
> Task :processResources NO-SOURCE
> Task :classes UP-TO-DATE
> Task :compileTestJava NO-SOURCE
> Task :compileTestGroovy UP-TO-DATE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test

So the task rebuilds the project itself, in this case everything is up to date, but not it’s dependencies, in this case an included build. You seem to say that it stops any rebuilding, but I might be misinterpreting your words.

Now I have re-read and know this, it is clear that this is the solution to my problem. I still am of the opinion that this solution was hard to find, but at the moment I do not have a suggestion to improve it.

4

Ok that is indeed one of the lines in the documentation.

But i have to admit I am fairly confused about your situation in general… if “freeplane” was written somewhat reasonable its task should simply print UP-TO-DATE and should not execute if not necessary. But i am not familiar with their build process…

This dependency forces Gradle to check if freeplane_root is up-to-date and, if necessary, to rebuild it.

Like yes thats generally whats supposed to happen. Except it should just determine that its up to date and not rebuild it. Is there a good reason your freeplane_root goes out of date the entire time?

Since you have already included it in your build-> you can also edit freeplane tasks yourself and edit their inputs so their tasks are up to date. Or hack around by disabling their tasks: But you probably need to that in your copy of the freeplane_root.

If I guess correctly your situation is just a bit odd because freeplane doesnt seem to have any maven publication which you could normally use to avoid builds…

But generally yes --no-rebuild might be the right thing for you

6

Solution: Use --no-rebuild (-a)

This flag can sometimes be a solution, but only if you know exactly what you are doing.
This is a very dangerous flag.
It does not only affect included builds, it also affects projects within your main build.
So if your main build has projects A, B, and C, C depends on B which depends on A.
Now you change code in all three, A, B, and C, but only build C and using that flag.
Then the stale results of A and B are used and those are also not rebuilt.
So be really careful when using that flag.

If an included build gets rebuilt on every execution, you should instead find out why the hell this is happening, for example using -i which should tell for each task why it is rerun. If nothing changed, nothing should be rebuilt except you or that including build did something majorly wrong and broke the up-to-date checks somehow.

  1. Documentation Update: It would be very helpful to update the Gradle documentation to explicitly mention that --no-rebuild (-a) also applies to included builds, not just buildSrc.

If you think so, post a feature request.
But I disagree actually.
The docs clearly state:

Do not rebuild project dependencies. Useful for debugging and fine-tuning buildSrc, but can lead to wrong results. Use with caution!

So it is imho clearly documented what it does (which is not what you stated) and just lists buildSrc development as one of the use-cases where this flag is useful, as this (or also build logic in an included build) is the major use-case for it and it should practically never be used in “normal” Gradle usage. That your included build is most probably just a side-effect of project dependencies not being built.

Feature Request for build.gradle Control : A built-in method to prevent specific included builds from rebuilding would be incredibly useful

Again, if you want to open a feature request, do so, this is a community forum.
But again, Gradle is extremely good in avoiding unncessary work. If it does something, then in most cases only because it is really necessary. If an included build is rebuilt every time, then something is f***ed up and that should be fixed, not some spurious setting added that does not really make sense but only results in incorrect and flaky builds.

Now I have re-read and know this, it is clear that this is the solution to my problem.

… no, definitely not. :wink: