Unzipping dependencies, but only when they change


(Scott Palmer) #1

Is there a built-in task for efficiently unzipping dependencies, along the lines of the Maven dependency plugin, like a C SDK with headers and libs? Maven will write out a tag file to use as a timestamp to compare the asset in the M2 cache with so it only unzips the asset again if the asset was updated. I have used code like this, but I would have to roll the timestamp checking myself… seems like this would be useful in the core Gradle libs:

task explodeDeps << {
        tree = zipTree(configurations.compile.fileCollection{dep -> dep.name == 'myartifactname'}.singleFile)
        copy {
            into "."
            from tree
        }
    }

(Peter Niederwieser) #2

Gradle’s generic up-to-date mechanism should already have you covered, and in a more reliable way than Maven (using content hashing for both the zip and the unzipped content). The simplest way to tap into this is to use the Copy task instead of the copy method.


(Scott Palmer) #3

Thanks for your quick reply.

I’ve read the “Working With Files” section of the tutorial, but it doesn’t mention anything related to conditionally extracting a zip with the Copy task, based on hashes, dates or any other criteria.

Can you point me to a sample of how this is done?

I had to dig deep with google just to get the method I posted above, as it was non-obvious how to get access to the cached zip file in the first place.


(Peter Niederwieser) #4

It is a generic mechanism for all tasks that declare their inputs and outputs. Like many others, the Copy task already does this for you. The Gradle user guide has more on this.

task explodeDeps(type: Copy) {
  from zipTree(...)
  into projectDir // don't use "."; might be better to unzip into a "fresh" dir rather than the project dir

If you run this task twice in a row, it should be shown as up-to-date on the second run. It would be interesting to know if the up-to-date check is fast enough in your case.


(Scott Palmer) #5

Well I just tried it and the explodeDeps task runs every time. I never see the UP-TO-DATE message.

task explodeDeps(type: Copy) {
        from(zipTree(configurations.nativesdk.fileCollection{dep -> dep.name == 'drcisolib'}.singleFile));
        into projectDir;
    }

------------------------------------------------------------ Gradle 1.0-milestone-7 ------------------------------------------------------------

Gradle build time: Thursday, January 5, 2012 10:24:43 AM UTC Groovy: 1.8.4 Ant: Apache Ant™ version 1.8.2 compiled on December 20 2010 Ivy: 2.2.0 JVM: 1.6.0_29 (Apple Inc. 20.4-b02-402) OS: Mac OS X 10.7.2 x86_64


(Peter Niederwieser) #6

Try a target directory other than projectDir (say “$projectDir/exploded”).


(Scott Palmer) #7

That did the trick, thanks…

Now, will it work if I extract multiple zips to the same folder, where all of the zips will contain a single root folder so nothing will conflict or overlap. E.g. I want to extract to $projectDir/SDKs several zips such that the result would look like:

  ./SDKs/

sdk_zip1/

sdk_zip2/

Can I make that from() section automatically include all of the zips in a specific configuration? I’m thinking of putting all of the C/C++ SDK dependencies in a “navitesdks” configuration so hopefully I can do something simplified like:

task explodeDeps(type: Copy) {
        from(zipTree(configurations.nativesdk.fileCollection));
        into "$projectDir/SDKs";
}

That’s why I was originally using the “copy { }” in a generic task, because I assumed I would have a loop of some sort within the task to extract multiple archives.


(Luke Daley) #8

In putting together the following code, I discovered some bugs in the latest Gradle (1.0-milestone-7) that prevent it from working.

task explodeDeps(type: Copy, dependsOn: configurations.nativesdk) { task ->
  into "sdks"
  configurations.nativesdk.incoming.afterResolve { ResolvableDependencies incoming ->
    incoming.files.each { File file ->
      if (file.name.endsWith(".zip")) {
        from (zipTree(file)) {
          into file.name - ".zip"
        }
      }
    }
  }
}

I can’t find a good way to do this in milestone-7 unfortunately. One of the other guys might though.

The above code will work with the next Gradle release.


(Scott Palmer) #9

Luke, you are the man!

Thanks a million. Not being a gradle/groovy expert, I’m sure it would have taken me quite some time to come up with something like that. I probably never would actually, given that it doesn’t work in milestone-7. I would have assumed it was my fault if I even managed to get something that was supposed to work :slight_smile:

I will simply have to be happy that in asking for help, I managed to get some bugs fixed :slight_smile:


(Scott Palmer) #10

I just tried the code above and it claims that the explodeDeps task is up-to-date even when the folder (sdks/whatever) has been deleted.


(Scott Palmer) #11

Actually it seems to be worse than that. The code just isn’t running… I added prints and they don’t happen… until I add back my original from() part… then suddenly the rest of the code starts to run.


(nm2501) #12

I would like to use this approach, but it’s not quite working for me. I have the zip being unzipped to [root]/sdks, but I’d like it to be unzipped to [root]/[name of zip file without extension]. The code I’m using is:

project(':framework') {
        task explodeDeps(type: Copy, dependsOn: configurations.compile) { task ->
        from zipTree(configurations.compile.singleFile)
        into "../sdks"
        configurations.compile.incoming.afterResolve { ResolvableDependencies incoming ->
            incoming.files.each { File file ->
                if (file.name.endsWith(".zip")) {
                    from (zipTree(file)) {
                       into file.name - ".zip"
                    }
                }
            }
        }
    }
}

Any help would be much appreciated.


(Daryl Teo) #13

I also am looking at this… is there no way to specify that the contents of the zip should be the dependency, and not the zip itself?