Help with collapsing sourceSet-s in a composite build

Help/Discuss
1

Hi,

I’m building a multi-repo multi-project Java app and I ran into an issue with properly collapsing source sets. I know one possible answer is- “don’t collapse, keep them as separate subprojects”. I’m trying to understand if there is a solution that involves collapsing.

In repo A, subproject A1, I have source sets: base, generator, generated, and main, with the following relevant bits:

dependencies {
   generatorImplementation sourceSets.base.output
   generatedImplementation sourceSets.base.output
   api sourceSets.base.output
   api sourceSets.generated.output
}
jar {
   from sourceSets.base.output
   from sourceSets.generated.output
}

What I’m trying to achieve is to compile the 3 source sets (base, generated, main) independently, run the generator in the 4th (generator) source set, and package everything up in a single jar.

Above, I use api instead of implementation because the latter generates IDE (not Gradle!) errors in a composite build. Concretely, I have a second repo B, with its own subproject B2, which has A:A1 as implementation dependency. I usually build A & B as a composite build, so that during the compilation of B, A’s products are replaced by Gradle instead of looking into a repo. If I replace api by implementation above, the IDE starts showing errors in B2 files because (to the best of my understanding…) while replacing artifacts, it ends up using A1’s main source set build products (classes) only instead of A1’s jar, and the IDE thinks the A1 generated source set build products are runtimeOnly dependencies. What I find strange is that only the IDE (IntelliJ) is confused about this, Gradle compiles B2 just fine, even with implementation instead of api.

However, I have another problem, which is the topic of this post: I have another subproject B:Bdist which is a distribution, declaring B:B2 as implementation dependency. When I assemble B:Bdist from a composite build, Gradle ends up packaging A1 generated source set class files inside the distribution zip, instead of packaging the A1 jar, I assume because of how the artifact replacing works. If I publish A:A1 to Maven local and build B separately, the dist zip looks fine, containing only the A1 jar.

TLDR Question- Is there a way to tell Gradle in a composite build to build and use the A1 jar from build/libs instead of sifting through build/classes/java/main while ignoring (at least when it is wrapped by the IDE…) build/classes/java/generated?

Non-TLDR Question- Can you think of a way to fix this, while keeping the source set collapsing and the composite build?

Thanks!

2

I tried to skin the cat another way. Instead of adding the base and generated source set outputs to the Jar directly, I added them as additional main source set outputs, and declared them also as compileOnly dependencies:

sourceSets {
    base {
    }
    generator {
    }
    generated {
        java {
            srcDirs += [new File(generatorOutputDir, "java")]
        }
        resources {
            srcDirs += [new File(generatorOutputDir, "resources")]
        }
    }
    main {
        output.dir(sourceSets.base.output)
        output.dir(sourceSets.generated.output)
    }
}
dependencies {
    generatorImplementation sourceSets.base.output
    generatedImplementation sourceSets.base.output
    compileOnly sourceSets.base.output
    compileOnly sourceSets.generated.output
}

This works ok at first, meaning that the A.A1 jar correctly contains all of base, generated, and main classes. However, I’m getting another error from A.A2 which has A.A1 as both annotationProcessor and implementation dependency:

// A.A2
dependencies {
    annotationProcessor project (":A:A1")
    implementation project(":A:A1")
}

The (wrong) compilation command line I see looks like:

... -processorpath ...;A1.jar;... -classpath ...;$buildDir\classes\java\main;....

Why are the entries different?? I want the A1.jar dependency not the path to the main source set output dir only.

3

I’ve isolated the problem- A:A1 is a java-library which, as per https://docs.gradle.org/current/userguide/java_library_plugin.html#sec:java_library_classes_usage, causes consumers such as A:A2 to use its classes instead of its jar. I have another question but I will use a separate thread, as this one contains too much irrelevant info.