Resuming Gradle incremental build in another workspace/other machine

Help/Discuss
1

Continuing the discussion from Is it possible to resume Gradle incremental build on other machine?:

It is more than continuing the build on another machine, it could also be continuing the build pipeline in another workspace.

I find it useful have a fork my build pipeline in Jenkins, and run some stages in parallel. I have stages that take hours, and even days to complete. When I fork the pipeline, instead of redoing the initial compile in each fork, which is already several hours, I would like to copy the workspace, and continue the pipeline in another Jenkins job.

I could run tasks in parallel, but the Jenkins job would not return until all the stages are built. We need to make decisions about the pipeline status more incrementally.

2

I want to provide a way to reproduce the problem of incremental builds in build pipelines, and I think a bug should be opened for this use case if it does not exist already. This use case mimics what people tend to do in Jenkins, for example when they archive workspaces with the intention of continuing the build in another stage of the build pipeline.

First the build.gradle file:

class Generate extends DefaultTask {
    @Input 
    int fileCount = 10
    @OutputDirectory
    File generatedFileDir = project.file("${project.buildDir}/generated")
    @TaskAction
    public void perform() {
        for (int i=0; i<fileCount; i++) {
            new File(generatedFileDir, "${i}.txt").text = i
        }
    }
}

task generator(type: Generate) {
    fileCount = 5
}
 
task zip(type: Zip) {
    from generator
}

We build the generator output: gradle generator

:generator
BUILD SUCCESSFUL
Total time: 1.875 secs

We move (or copy) the workspace:

cd ..
mv gradle-resume-build-1 gradle-resume-build-2

And we continue the build: gradle zip

:generator
:zip
BUILD SUCCESSFUL
Total time: 1.907 secs

The generator task was re-executed. The incremental aspect has been lost with the parent directory rename.

Is there bug on the bug tracker for this issue? If not , can one be opened please?

3

I believe the reason for this behavior is that file hashes are stored with absolute paths in .gradle/fileSnapshots.bin. If you archive your project workspace and extract it in a different Jenkins job then paths might not line up anymore and therefore the task actions have to be executed again.

As a workaround I’d exclude the tasks you don’t want to execute in in subsequent jobs with -x <taskName>.

4

I am having the same issue as well with Gradle + Jenkins. I’ve temporarily solved this problem by having multiple Jenkins projects share the same workspace. This works for now but is not a great long-term solution.

Is there a way to tell Gradle to use relative paths when hashing? Or any other workarounds besides -x <taskName> ?