Right way to generate sources in Gradle 7

I have found only quite old posts on the topic of generated sources. What is the right way to do it in Gradle 7? How do I tell it that some task generates the source code?

Usually I simply added dependencies for compileJava (and probably sourcesJar) tasks on the code-generating task. Is this the idiomatic way or there is something better?

Example:

plugins {
    id("java")
    id("maven-publish")
}

repositories {
    mavenCentral()
}

java {
    withSourcesJar()
}

tasks.register("copyTemplates", Copy) {
    from("templates")
    into("src/main/java")
}

tasks.named("compileJava").configure {
    dependsOn("copyTemplates")
}
tasks.named("sourcesJar").configure {
    dependsOn("copyTemplates")
}
2 Likes

Don’t mix sources under version control (src/main/java) and generated sources in one folder. Even worse, do NOT put generated sources under version control - this is an anti-pattern and it makes everything just more complicated and fragile.

We place generated sources in a folder below the target build folder; e.g., build/generated/main/java. You will easily get used to that if you start accepting that they are not really sources: they are products (or artefacts) that have been generated dynamically during a build from real sources and are therefore not sources in the first place. Put in other words, they can be produced any time from the sources that are the basis for their generation (of course, provided you have a working build).

Considering generated Java “sources”, the folder in which you store them can then be easily added as another source folder:

sourceSets {
   main {
      java {
         srcDir "${buildDir}/generated/main/java"
      }
   }
}

As the build folder is usually ignored by version control, placing them below this folder also ensures that they are not inadvertently put under version control.

1 Like

Basically what @twwwt said, but more.

You should not only not mix generated and non-generated code.
You should also not share output folders among multiple tasks or up-to-date checks and caching might be disturbed.
So latest when you have mutliple code generation tasks, give each task a separate output folder, for example layout.buildDirectory.dir("generated/sources/$name/main/java").

Additionally it is preferrable to use implicit task dependencies over explicit task dependencies.
That way every task that needs sources automatically depends on the needed tasks and you don’t need manual task dependencies.
You can achieve that by using the task or task provider as srcDir directly.
It is almost always a sign of antipattern if you configure paths manually except for inputs / outputs of tasks.

So the full example would be:

plugins {
    id("java")
    id("maven-publish")
}

repositories {
    mavenCentral()
}

java {
    withSourcesJar()
}

def copyTemplates = tasks.register("copyTemplates", Copy) {
    from("templates")
    into(layout.buildDirectory.dir("generated/sources/$name/main/java"))
}

sourceSets {
    main {
        java {
            srcDir(copyTemplates)
        }
    }
}
2 Likes

We treat javac warnings as errors, and the generated code contains such warnings, so we cannot add the sources to the main source set and have them compiled that way.

I found How to add generated sources to compile classpath without adding them to the main sourceSet? - #3 by Benjamin_Manes that tries to solve that problem, but in our context we also use dependency locking, and solving it the way that post suggests it highlighted a problem: the dependencies from the code generation configuration leaks out to the main classpath (without solving the problem, the classes are still not present in the war file).

In essence, I think we would want a separate source set for the generated code, build it in that source set, and then include the .class files in the resulting war file as classes. Or alternatively build a JAR out of the generated code, and include it that way.

@slovdahl The last time I did this, I separated it out so as to not invalidate the build cache. Then in the actual jar it pulls in all of the classes so that it’s not a dependency. Hope that helps.

1 Like

Thanks for this! I think I managed to solve it now. Not exactly in the same way your suggestion, but I got inspired by it, and the documentation on Sharing outputs between projects.

I solved it by producing a JAR too, something like this:

configurations {
    codeGen
    codeGenJars {
        canBeConsumed = true
        canBeResolved = false
    }
}

sourceSets {
    codeGen {
        java {
            srcDirs = ["${buildDir}/generated-sources/codeGen"]
        }
    }
}

task generateCode {
    // ..
}

compileCodeGenJava {
    dependsOn generateCode
    classpath = configurations.codeGen
}

task codeGenJar(type: Jar) {
    archiveBaseName = 'codegen-jar'

    from sourceSets.codeGen.output
}

artifacts {
    codeGenJars(codeGenJar)
}

dependencies {
    implementation project(path: project.path, configuration: 'codeGenJars')
}

Some remarks:

  • Your configuration codeGen is considered legacy and deprecated as it is consumable and resolvable. You should always set only one to true or both to false (the latter if you follow the example from built-in plugins that you have a separate configuration for declaring dependencies and then extended from that for resolving). That by default both are true is just due to backwards compatibility reasons.
  • Still prefer implicit task dependencies over explicit ones. Do not manually have dependsOn and a path configured manually, but use srcDir(generateCode) and the outputs of the task will be considered source folders and the task dependencies added automatically where necessary.
1 Like

Thanks for the input, very much appreciated!

So for the first point, this would be better, right? At least if I want to keep the number of configurations down.

configurations {
    codeGen {
        canBeConsumed = false
        canBeResolved = true
    }

    codeGenJars {
        canBeConsumed = true
        canBeResolved = false
    }
}

Regarding your second point, is this what you mean?

sourceSets {
    codeGen {
        java {
            srcDir(generateCode)
        }
    }
}

task generateCode {
    // ..
}

compileCodeGenJava {
    classpath = configurations.codeGen
}

I had some issues getting it to work until I realized this obviously does not work any more (because the newly introduced circular dependency):

task generateCode {
    doLast {
        File sourceDir = sourceSets.codeGen.java.srcDirs.iterator().next()
        // ...
    }
}

With this it works:

task generateCode {
    outputs.dir(layout.buildDirectory.dir('generated-sources'))

    doLast {
        File sourceDir = file(layout.buildDirectory.dir('generated-sources'))
        // ...
    }
}

Is it somehow possible to de-duplicate the generated-sources definition? :thinking:

1 Like

So for the first point, this would be better, right? At least if I want to keep the number of configurations down.

Exaclty

Regarding your second point, is this what you mean?

Exactly

I had some issues getting it to work until I realized this obviously does not work any more (because the newly introduced circular dependency):

Well, you omitted that part in your original example code.
But actually, I had it kind of covered with “the outputs of the task will be considered source folders” which implies, that the task has to properly define its outputs to work properly. Actually you should also define your inputs properly, then your task can be up-to-date, or even cached if the generation is costly and you mark it as cacheable.

Is it somehow possible to de-duplicate the generated-sources definition? :thinking:

task generateCode {
    def output = layout.buildDirectory.dir('generated-sources')
    outputs.dir(output)

    doLast {
        File sourceDir = output.get().asFile
        // ...
    }
}

Btw. if you have multiple tasks that generate code, use separate directories, task outputs should not be overlapping.
And consider using task configuration avoidance to save configuration time. :wink: (tasks.register('generateCode') { ... })

1 Like