Best way to handle up-to-date mechanism unpacking zip file with read-only files

Old Forum
1

I’m running into two problems with a simple task to unzip a dependency:

  1. The up-to-date mechanism is not working out of the box.

The first time run, a temporary directory is created where the dependency is unzipped (through the filetree() mechanism. Then the files from that temp directory are coped to my target directory. However, the second time this is run, the filetree mechanism attempts to expand the zip into the expandedArchives directory again.

  1. Further, when this unzip happens a second time, it fails because the zip file has readOnly files and you can’t unzip over files that are marked as readOnly.

Here is an example of my code:

apply plugin: 'base'
  configurations {
  config
}
  dependencies {
  config "my.org:config:0.9-SNAPSHOT@zip"
}
  task unpackConfig(type: Copy, dependsOn: configurations.config) {
  from zipTree(configurations.config.singleFile)
  into "${buildDir}/config-src"
}

Here’s the first run where everything works fine:

gradle clean unpackConfig
:clean
:unpackConfig

Here’s an example where running a second time -

gradle unpackConfig
:unpackConfig
  FAILURE: Build failed with an exception.
  * What went wrong:
Could not expand ZIP '/Users/dlethin/.gradle/caches/modules-2/files-2.1/my.org/config/0.9-SNAPSHOT/56c11d4393acf57b9c61530022399a9967c73b86/config-0.9-SNAPSHOT.zip'.
> Could not copy zip entry /Users/dlethin/.gradle/caches/modules-2/files-2.1/my.org.config/config/0.9-SNAPSHOT/56c11d4393acf57b9c61530022399a9967c73b86/config-0.9-SNAPSHOT.zip!apps/app1/latest/app.properties to '/Users/dlethin/myproject/build/tmp/expandedArchives/config-0.9-SNAPSHOT.zip_17tsbb99s44a56674imboe47l1/apps/app1/latest/app.properties'.
  Caused by: java.io.FileNotFoundException: /Users/dlethin/myproject/build/tmp/expandedArchives/config-0.9-SNAPSHOT.zip_17tsbb99s44a56674imboe47l1/apps/app1/latest/app.properties (Permission denied)

So what I’m looking for:

  1. don’t unzip if we’ve already unzipped the latest version of the dependency – ie, get up-to-date incremental build to work. 2. if the source zip file timestamp is later then the unzipped directory, first delete the unzipped directory to get around the issue that there are read-only files.

Is there an easy way to get this work for or do I have to roll my own task rather than using copy task with ziptree().

Any suggestions?

Thanks.

Doug

2

I found this forum entry which appears to get me part way -

http://forums.gradle.org/gradle/topics/ziptree_unable_to_overwrite_temporary_read_only_files

It seem if I use the suggested doLast closure to my copy task where the unzip happens to add write capability, then that seems to get me past the error:

task unpackConfig(type: Copy, dependsOn: configurations.config) {
  from zipTree(configurations.config.singleFile)
  into "${buildDir}/config-src"
    doLast {
    exec {
      commandLine = "chmod -R u+w $buildDir/tmp/expandedArchives".tokenize()
    }
  }
}

Here’s the run –

gradle unpackConfig
:unpackConfig UP-TO-DATE
  BUILD SUCCESSFUL
  Total time: 1.005 secs

Is there any way to conditionally first delete the archive directory if it turns out the unpacked zip file is NOT up-to-date? I want to make sure that if there were files that might have been deleted from the zip, they don’t show up in the expanded directory.

Thanks.

Doug

3

Either delete in a ‘doFirst’, or better use a ‘Sync’ task.

4

Thanks, I will give those suggestions a try.

Regarding the workaround I discovered and posted above to add a doLast to mark files as writable, to get around the FileNotFoundException I’m getting in my build, I was too quick to claim the workaround works.

It’s true that the second time i run the task, it correctly marks the task as UP-TO-DATE.

However, the third time I run it, it fails again with the original error I was trying to work around.

I’m at a loss how to get up-to-date functionality working unpacking zip files.

It seems that that open issue GRADLE-2959 is the key to getting this to work?

Thanks.

Doug

5

Doesn’t using a ‘Sync’ task solve your problem? If so, it’s a much cleaner solution. Also, ‘zipTree(…)’ should be wrapped with curly braces so that the configuration only gets resolved at execution time. If you want the unzipping to happen less often, perhaps make it a separate task.

6

Using Sync in theory will solve my problem of not having to inject a doFirst closure to delete the target directory if the unzip task is considered out-of-date and will run.

The problem I’m still running into appears to be bug GRADLE-2959.

I’m going to experiment with perhaps rolling my own unzip task which wraps around ant.unzip. Then I don’t have to use Project.zipTree(). We’ll see if that helps.

Thanks.

7

It should be possible to overcome GRADLE-2959 with something like ‘doFirst { delete “$buildDir/tmp” }’. I don’t think this will result in the archive getting extracted more often than before.