Zip task with lazy `from` property

Help/Discuss
1

Hi there,

I need to zip some files but getting the list of them needs to call an expensive operation. I tried to wrap the Zip’s from property in a closure but I noticed Gradle is evaluating it even though the task was not called.

task zipExpensiveListOfFiles(type: Zip) {
    from {
      println 'Getting the list of files to be zipped is very expensive...'
      'foo'
    }
  into 'bar'
}

Then I tried to set the from in a doFirst block:

task zipExpensiveListOfFiles(type: Zip) {
  doFirst {
    from {
      println 'Getting the list of files to be zipped is very expensive...'
      'foo'
    }
  }
  into 'bar'
}

This way Gradle didn’t evaluate the from's closure but when I called the task via command line it didn’t execute it because the task has no source files:

:zipExpensiveListOfFiles
Skipping task ':zipExpensiveListOfFiles' as it has no source files.
:zipExpensiveListOfFiles UP-TO-DATE
:zipExpensiveListOfFiles (Thread[main,5,main]) completed. Took 0.029 secs.

Is there a way to accomplish what I need?

Thanks

2

I think you have to (1)create an intermediate copy task before zip task, and (2)use Project#copy method, and (3)let Zip task depends on intermediate copy task.

// (1)intermediate task to collect files
task myCopy {
    description = 'Collects files, which requires expensive calculation.'
    outputs.file file("${buildDir}/myCopy")
    doLast {
        def list = someExpensiveListingFiles()
        // (2)use Project#copy method
        copy {
            from files(list)
            into file("${buildDir}/myCopy")
        }
    }
}
// (3)zip task depends on intermediate copy task
task myZip(type: Zip, dependsOn: ['myCopy']) {
    group = 'My Archiving'
    description = 'Archives files whose collection requires an expensive calculation.'
    from tasks['myCopy']
    into file("${buildDir}/bar")
}

With this script, run task myZip. Then you can obtain zipped files.

1 Like
3

Example is available via gist.

In this sample script, archive objects is unknown while evaluating build script.

4

Hi @tanob,

This is one of those special situations where you have to conditionally configure something based on what we need to eventually execute. The way to do that is to look at the TaskExecutionGraph and configure the Zip task when we know the task will be executed.

Here’s a slimmed down example:

apply plugin: 'base'

task myZip(type: Zip) {
    from("file2")
}

task A(dependsOn: myZip)
task B

def expensiveMethod() {
    println "Calling expensive method"
    return [ "file1" ]
}

gradle.taskGraph.whenReady { graph ->
    if (graph.hasTask(tasks.myZip)) {
        tasks.myZip {
            from(expensiveMethod())
        }
    }
}

You’ll notice that if you run gradle B, the expensive method is not called. If you run gradle A or gradle myZip, it is.

4 Likes