Why Is Multi-Module Gradle So Hard? (Version Catalogs + BuildSrc Frustrations)

Help/Discuss
1

Background

My project originally used a single-module Gradle build. Everything lived in the root build.gradle.kts, with dependencies and plugins managed through a libs.versions.toml version catalog. This setup was simple and worked fine.

As the codebase grew, I wanted to:

  • Split the project into multiple modules.
  • Centralize quality tools (Spotless, SpotBugs, Error Prone + NullAway, JSpecify, JUnit).
  • Keep the build DRY and maintainable.

What I expected to be a straightforward migration turned into a much more painful process than anticipated.

First Attempt: subprojects {}

My first approach, following typical examples (including ChatGPT suggestions), was to use subprojects {} in the root build file:

import net.ltgt.gradle.errorprone.errorprone

repositories {
    mavenCentral()
}

plugins {
    java
    alias(libs.plugins.io.freefair.lombok)
    // https://github.com/diffplug/spotless/tree/main?tab=readme-ov-file
    alias(libs.plugins.com.diffplug.spotless)
    // https://spotbugs.readthedocs.io/en/latest/gradle.html#configure-gradle-plugin
    alias(libs.plugins.com.github.spotbugs)
    // https://github.com/tbroyer/gradle-errorprone-plugin
    alias(libs.plugins.net.ltgt.errorprone)
}

subprojects {

    // Core Java + quality plugins for ALL modules
    apply(plugin = libs.plugins.com.diffplug.spotless.get().pluginId)
    apply(plugin = libs.plugins.com.github.spotbugs.get().pluginId)
    apply(plugin = libs.plugins.net.ltgt.errorprone.get().pluginId)
    apply(plugin = libs.plugins.io.freefair.lombok.get().pluginId)


    repositories {
        mavenCentral()
    }

    dependencies {

        // JSpecify: annotations only; don't ship them at runtime
        compileOnly(libs.org.jspecify.jspecify)
        testCompileOnly(libs.org.jspecify.jspecify)

        // Error Prone + NullAway for ALL modules
        errorprone(libs.com.google.errorprone.error.prone.core)
        errorprone(libs.com.uber.nullaway.nullaway)

        // JUnit (shared default)
        testImplementation(platform(libs.org.junit.junit.bom))
        testImplementation(libs.org.junit.jupiter.junit.jupiter)
        testRuntimeOnly(libs.org.junit.platform.junit.platform.launcher)
        testImplementation(libs.org.assertj.assertj.core)
    }

    java {
        sourceCompatibility = JavaVersion.VERSION_24
        targetCompatibility = JavaVersion.VERSION_24
        modularity.inferModulePath = true
    }

    // https://github.com/diffplug/spotless/tree/main/plugin-gradle#palantir-java-format
    spotless {
        java {
            importOrder()
            removeUnusedImports()
            trimTrailingWhitespace()
            endWithNewline()
            formatAnnotations()
            palantirJavaFormat().formatJavadoc(true)
        }
    }

    // https://spotbugs-gradle-plugin.netlify.app/com/github/spotbugs/snom/spotbugsextension
    spotbugs {
        toolVersion = "4.9.4"
        effort = com.github.spotbugs.snom.Effort.MAX
        reportLevel = com.github.spotbugs.snom.Confidence.LOW
        showProgress = false // TODO: enable before commiting
        ignoreFailures = true // TODO: disable before commiting
        reportsDir = file("${layout.buildDirectory}/reports/spotbugs")
        projectName = name
        release = version.toString()
        maxHeapSize = "512m"
        jvmArgs = listOf("-Duser.language=en")
    }

    // Error Prone + NullAway for all Java compilations
    tasks.withType<JavaCompile>().configureEach {
        options.apply {
            encoding = "UTF-8"

            errorprone {
                // Do NOT disable all checks — keep the defaults and enable NullAway
                check("NullAway", net.ltgt.gradle.errorprone.CheckSeverity.ERROR)
                option("NullAway:JSpecifyMode", "true")
                option("NullAway:OnlyNullMarked", "true")
            }

            // Example: make tests less strict (disable NullAway for test sources only)
            if (name.contains("test", ignoreCase = true)) {
                errorprone.disable("NullAway")
            }

        }

    }

    tasks.withType<Javadoc> {
        options.encoding = "UTF-8"
    }

    tasks.test {
        useJUnitPlatform()
    }

    // Keep code formatted before build
    tasks.build {
        dependsOn(tasks.spotlessApply)
    }
}

But running this immediately failed with:

Build file 'C:\..\build.gradle.kts' line: 25

Extension with name 'libs' does not exist. Currently registered extension names: [ext]

Even though libs.versions.toml was still there. After going back and forth with ChatGPT and trying fixes like adding:

pluginManagement {
    repositories {
        gradlePluginPortal()
        mavenCentral()
    }
}

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories { mavenCentral() }
    versionCatalogs {
        create("libs") {
            from(files("gradle/libs.versions.toml")) // points to your TOML
        }
    }
}

…to settings.gradle.kts, nothing worked. At that point I gave up on ChatGPT, turned to Google, and eventually found Sharing Build Logic in BuildSrc. That seemed to be a better fit, so I restructured the project as described in the docs.

Second Attempt: buildSrc

I set up buildSrc and followed the official instructions, assuming things would just work. My example-common.gradle.kts looked like this:

group = "com.example"
version = "1.3.0-SNAPSHOT"
description = "realExample"

repositories {
    mavenCentral()
}

plugins {
    java
    alias(libs.plugins.io.freefair.lombok)
    // https://github.com/diffplug/spotless/tree/main?tab=readme-ov-file
    alias(libs.plugins.com.diffplug.spotless)
    // https://spotbugs.readthedocs.io/en/latest/gradle.html#configure-gradle-plugin
    alias(libs.plugins.com.github.spotbugs)
    // https://github.com/tbroyer/gradle-errorprone-plugin
    alias(libs.plugins.net.ltgt.errorprone)
}

dependencies {

    // JSpecify: annotations only; don't ship them at runtime
    compileOnly(libs.org.jspecify.jspecify)
    testCompileOnly(libs.org.jspecify.jspecify)

    // Error Prone + NullAway for ALL modules
    errorprone(libs.com.google.errorprone.error.prone.core)
    errorprone(libs.com.uber.nullaway.nullaway)

    // JUnit (shared default)
    testImplementation(platform(libs.org.junit.junit.bom))
    testImplementation(libs.org.junit.jupiter.junit.jupiter)
    testRuntimeOnly(libs.org.junit.platform.junit.platform.launcher)
    testImplementation(libs.org.assertj.assertj.core)
}
java {
    sourceCompatibility = JavaVersion.VERSION_24
    targetCompatibility = JavaVersion.VERSION_24
    modularity.inferModulePath = true
}

// https://github.com/diffplug/spotless/tree/main/plugin-gradle#palantir-java-format
spotless {
    java {
        importOrder()
        removeUnusedImports()
        trimTrailingWhitespace()
        endWithNewline()
        formatAnnotations()
        palantirJavaFormat().formatJavadoc(true)
    }
}

// https://spotbugs-gradle-plugin.netlify.app/com/github/spotbugs/snom/spotbugsextension
spotbugs {
    toolVersion = "4.9.4"
    effort = com.github.spotbugs.snom.Effort.MAX
    reportLevel = com.github.spotbugs.snom.Confidence.LOW
    showProgress = false // TODO: enable before commiting
    ignoreFailures = true // TODO: disable before commiting
    reportsDir = file("${layout.buildDirectory}/reports/spotbugs")
    projectName = name
    release = version.toString()
    maxHeapSize = "512m"
    jvmArgs = listOf("-Duser.language=en")
}

// Error Prone + NullAway for all Java compilations
tasks.withType<JavaCompile>().configureEach {
    options.apply {
        encoding = "UTF-8"

        errorprone {
            // Do NOT disable all checks — keep the defaults and enable NullAway
            check("NullAway", net.ltgt.gradle.errorprone.CheckSeverity.ERROR)
            option("NullAway:JSpecifyMode", "true")
            option("NullAway:OnlyNullMarked", "true")
        }

        // Example: make tests less strict (disable NullAway for test sources only)
        if (name.contains("test", ignoreCase = true)) {
            errorprone.disable("NullAway")
        }

    }

}

tasks.withType<Javadoc> {
    options.encoding = "UTF-8"
}

tasks.test {
    useJUnitPlatform()
}

// Keep code formatted before build
tasks.build {
    dependsOn(tasks.spotlessApply)
}

But this led to new errors:

e: file:///C:/../buildSrc/build/kotlin-dsl/plugins-blocks/extracted/example-common.gradle.kts:11:11 Unresolved reference 'libs'.
e: file:///C:/../buildSrc/build/kotlin-dsl/plugins-blocks/extracted/example-common.gradle.kts:11:16 'fun Project.plugins(block: PluginDependenciesSpec.() -> Unit): Nothing' is deprecated. The plugins {} block must not be used here. If you need to apply a plugin imperatively, please use apply<PluginType>() or apply(plugin = "id") instead.

So back to Google again—this time about version catalogs. I came across Version Catalogs and tried adjusting settings.gradle.kts to include:

dependencyResolutionManagement {
    versionCatalogs {
        create("libs") {
            from(files("../gradle/libs.versions.toml"))
        }
    }
}

Even after all this effort, I’m still running into errors.

It’s 4 a.m. now, and I’m completely exhausted. This process should be so much easier. All I want is to keep using the plugins from the version catalog in my convention plugin—without having to manually copy everything over or rely on the strange methods in the Using a catalog in buildSrc, which I don’t fully understand.

2

Why Is Multi-Module Gradle So Hard?

Probably because you started trusting trash-in-trash-out tool too much and got frustrated by it not working, because actually it is not really that hard. :smiley:

My first approach, following typical examples (including ChatGPT suggestions), was to use subprojects {} in the root build file:

“typical examples” of highly discouraged bad practices.
Times where subprojects { ... } was recommended are gone for years.
Any form of cross-project configuration (subprojects { ... }, allprojects { ... }, project(...) { ... ], or any other form) immediately introduces project coupling which works against more sophisticated Gradle features and optimizations, and is highly discouraged.

In your case, at the time the subprojects { ... } code is evaluated, the libs extension is not yet present. One could surely see that as bug and you should maybe report it if there is no report about it yet.
It could well be that this was not found yet, as it is very bad practice anyway.

You could work-around this problem by saving the libs from the outer project to a variable that you use then like

val libsToo = libs
subprojects {
    println(libsToo.plugins.foo.get().pluginId)
}

But as I said, the whole subprojects { ,,, } is a bad idea anyway.

Instead to centralize build logic you should use convention plugins, implemented in buildSrc or - what I prefer - an included build, for example implemented as precompiled Kotlin DSL script plugin, as you already discovered.

After going back and forth with ChatGPT and trying fixes like adding:

That is just reconfiguring what is the conventional default anyway, so it effectively is a no-op.

But this led to new errors:

In a Kotlin DSL precompiled script plugin you cannot use the libs accessor due to conceptional issues.
Imagine buildSrc as a standalone build that builds a plugin, because that indeed is the reality.
Imagine (now for real) that this standalone build is built separately and release separately.
Now imagine this separately built plugin is applied to your build.
Essentially, this is what happens, just that Gradle automatically builds the plugin for you and uses it without the need for publishing.

The version catalog is define by the build that applies the plugin - your main build.
It defines what entries are there.
So at the time the plugin is compiled it cannot know which entries are in the version catalog, so it cannot generate the accessors that you try to use.

In a composite build or buildSrc situation like you have, you could use my hack-around from Make generated type-safe version catalogs accessors accessible from precompiled script plugins · Issue #15383 · gradle/gradle · GitHub to make the accessors work in Kotlin DSL precompiled script plugins at least for dependencies. For plugins they are not too useful anyway there.

Alternatively, I believe there is also a plugin mentioned in that issue that someone developed that does the generation another time for the plugin project instead of reusing the Gradle-generated ones with my hack-around.

Alternatively, you can use the string-y access to the version catalog which is the “official” way, by getting the VersionCatalogsExtension and from that get the version catalog and from that the dependencies.

I came across Version Catalogs and tried adjusting settings.gradle.kts to include:

This makes the version catalog of the main build available to the buildSrc build, so you can use the libs accessors in the build scripts of the buildSrc build. But that does not change anything for the code you build inside that project unless you use my hack-around.

Regarding the plugins in the precompiled script plugin, you anyway need to declare them as dependencies of the built that builds your plugin. So the last snippet you mentioned you still need if you want to use the version catalog for that. And so you the “strange methods” whyever you call them strange is exactly what you need. If your plugin wants to apply other 3rd party plugins, you need to declare them as dependencies of your plugin project and then in the convention plugin, you can just apply them by simple ID without version, as the version is coming from the dependency.

3

Thanks for the detailed reply, I really appreciate it.

After thinking about it for a while, I realized you’re right. my frustration was more with ChatGPT not being able to solve the issue directly. That said, a lot of the problems came from Gradle itself, which added to my confusion. For example:

–

When a programmer turns to ChatGPT, it’s usually because they don’t know the topic well and need a quick solution. It might not always be the best approach, but it’s often the most accessible one. even if the generated advice sometimes leads to practices that aren’t ideal, like the example above.

–

This is something I honestly expected Gradle to handle automatically. For example, a command to set up a buildSrc folder and make it accessible by default (especially for convention plugins) would feel natural, since that’s how it works in a standard Gradle build. Without this, it’s not obvious that the behavior would differ, and I didn’t suspect it was an issue.

–

I called them “strange” because from my perspective, I suddenly had a buildSrc project inside my build with its own separate world view. containing a plugin that looks like a build file, which itself also has a build.gradle.kts. That was confusing at first.

For example, I didn’t know that I should declare plugins I normally use in the root plugins block as dependencies inside buildSrc/build.gradle.kts:

repositories {
    gradlePluginPortal()
    mavenCentral()
}

plugins {
    `kotlin-dsl`
}

dependencies {

    implementation(plugin(libs.plugins.com.diffplug.spotless))
    implementation(plugin(libs.plugins.com.github.spotbugs))
    implementation(plugin(libs.plugins.net.ltgt.errorprone))
    implementation(plugin(libs.plugins.io.freefair.lombok))

}

// Helper function that transforms a Gradle Plugin alias from a
// Version Catalog into a valid dependency notation for buildSrc
fun plugin(plugin: Provider<PluginDependency>) =
    plugin.map { "${it.pluginId}:${it.pluginId}.gradle.plugin:${it.version}" }

And then move my previous build logic into the convention plugin rather than keeping it in the build.gradle.kts. For example:

import net.ltgt.gradle.errorprone.errorprone

repositories {
    mavenCentral()
}

plugins {
    id("io.freefair.lombok")
    id("com.diffplug.spotless")
    id("com.github.spotbugs")
    id("net.ltgt.errorprone")
    id("java")
}

val libs = extensions.getByType(VersionCatalogsExtension::class.java).named("libs")

dependencies {

    // JSpecify: annotations only; don't ship them at runtime
    compileOnly(libs.findLibrary("org-jspecify-jspecify").get())
    testCompileOnly(libs.findLibrary("org-jspecify-jspecify").get())

    // Error Prone + NullAway for ALL modules
    errorprone(libs.findLibrary("com-google-errorprone-error-prone-core").get())
    errorprone(libs.findLibrary("com-uber-nullaway-nullaway").get())

    // JUnit (shared default)
    testImplementation(platform(libs.findLibrary("org-junit-junit-bom").get()))
    testImplementation(libs.findLibrary("org-junit-jupiter-junit-jupiter").get())
    testRuntimeOnly(libs.findLibrary("org-junit-platform-junit-platform-launcher").get())
    testImplementation(libs.findLibrary("org-assertj-assertj-core").get())
}

java {
    sourceCompatibility = JavaVersion.VERSION_24
    targetCompatibility = JavaVersion.VERSION_24
    modularity.inferModulePath = true
}

// https://github.com/diffplug/spotless/tree/main/plugin-gradle#palantir-java-format
spotless {
    java {
        importOrder()
        removeUnusedImports()
        trimTrailingWhitespace()
        endWithNewline()
        formatAnnotations()
        palantirJavaFormat().formatJavadoc(true)
    }
}

// https://spotbugs-gradle-plugin.netlify.app/com/github/spotbugs/snom/spotbugsextension
spotbugs {
    toolVersion = "4.9.4"
    effort = com.github.spotbugs.snom.Effort.MAX
    reportLevel = com.github.spotbugs.snom.Confidence.LOW
    showProgress = false // TODO: enable before commiting
    ignoreFailures = true // TODO: disable before commiting
    reportsDir = file("${layout.buildDirectory}/reports/spotbugs")
    projectName = name
    release = version.toString()
    maxHeapSize = "512m"
    jvmArgs = listOf("-Duser.language=en")
}

// Error Prone + NullAway for all Java compilations
tasks.withType<JavaCompile>().configureEach {
    options.apply {
        encoding = "UTF-8"

        errorprone {
            // Do NOT disable all checks — keep the defaults and enable NullAway
            check("NullAway", net.ltgt.gradle.errorprone.CheckSeverity.ERROR)
            option("NullAway:JSpecifyMode", "true")
            option("NullAway:OnlyNullMarked", "true")
        }

        // make tests less strict (disable NullAway for test sources only)
        if (name.contains("test", ignoreCase = true)) {
            errorprone.disable("NullAway")
        }

    }

}

tasks.withType<Javadoc> {
    options.encoding = "UTF-8"
}

tasks.test {
    useJUnitPlatform()
}

// Keep code formatted before build
tasks.build {
    dependsOn(tasks.spotlessApply)
}

If you look at it this way, how would I reasonably know that the convention plugin must both declare these other plugins as dependencies and then reference them in its plugins block before I can use them inside the plugin itself? That connection wasn’t intuitive at all.

4

That said, a lot of the problems came from Gradle itself, which added to my confusion.

Yeah, sure, I didn’t want to imply Gradle does not have problems. :smiley:
Of course it has bugs and it also still has a high traction, and thus things that can be good practice today can be bad practice tomorrow, that’s the price of evolution. :slight_smile:

When a programmer turns to ChatGPT, it’s usually because they don’t know the topic well and need a quick solution.

Well, the problem is, that this is exactly a case where you should never ask an “AI” tool. :smiley:
AI tools of today are very good in giving answers that look correct, but utterly bad in giving answers that are correct.
If you don’t know something you should never ask an AI tool, at least not without thorough research afterwards.
They are oversimplified just next-word-guessers, and for example code they produce also often does not compile because they use methods that never existed and never will exist in the used API and so on.
They are perfectly fine if you are too lazy (the good kind) to do something yourself even though you could have done it yourself, as you always need to understand what you are given and need to be able to evaluate whether it is good and makes sense or not.
Trusting the answer of an AI bot is like googling and just accepting the first result, taking it for granted.
:man_shrugging: :slight_smile:

This is something I honestly expected Gradle to handle automatically.

Again, it is impossible to handle in a generic way.
The contents of the version catalog are controlled by the build applying the plugins.
The build that builds the plugins simply cannot know what the contents will be, unless you tell it to because you know better than Gradle that you are in a special situation where some invariants are granted.

For example, a command to set up a buildSrc folder and make it accessible by default (especially for convention plugins) would feel natural, since that’s how it works in a standard Gradle build. Without this, it’s not obvious that the behavior would differ, and I didn’t suspect it was an issue.

So you think it would make sense that the Gradle folks provide a way to generate a project that violates the design decisions they made when designing the feature? Does not really sound senseful to me. :smiley:

containing a plugin that looks like a build file,

Noone forces you to use precompiled script plugins, you can also use Java, or Groovy, or standard Kotlin, or Scala, or any other JVM language to implement your plugin. The precompiled script plugins are just syntactic sugar so that it looks very similar to a build script and thus lowers the hurdles to properly centralize build logic instead of doing cross-project configuration. :slight_smile:

For example, I didn’t know that I should declare plugins I normally use in the root plugins block as dependencies inside buildSrc/build.gradle.kts :-1:

Well, yeah, it is something new to you to build a plugin and you probably didn’t read the relevant part of the documentation that explains it, of course this can be confusing then. :smiley:

But what you do is, you develop a plugin in quasi separate build, that plugin has other plugins as dependencies so you declare them as dependencies in the build building those plugins. It is just like declaring a dependency on a Java library before using that library in your Java code. :slight_smile:

1 Like