Solution for copying a subdirectory wihtin a Zip into another Zip?


(Mike Meessen) #1

Hi there,

I’m having trouble with this for quite some time now. I finally came to a working solution, but it’s really sluggish, so I’m wondering if there’s a way to accomplish what I want more elegantly.

Here’s our use case: We have a multi-project build that builds a hand full of command-line java applications that are all part of one big system. We’re using the gradle application plugin on each of the application-building subprojects. Now, what we need is one big Zip that contains all of the applications together with some additional stuff. So that “master” Zip should contain one “applications” directory which contains the “bin”, “lib” and “etc” directories directly. These directories should then contain the union of those of the individual applications. So, for example, “bin” should contain all of our applications’ start scripts, “lib” the union of libraries of all applications and so on. What bugs me and prevents me from fulfilling it in one simple Zip task, is the intermediate directory with the application name + version that’s put in each “distZip” by the application plugin. I can’t seem to find a way to get rid of that directory elegantly while putting everything in the master zip.

Here’s my working solution, which I don’t want to keep since it’s so ugly. The key line is “tmpDestinationDir.eachDir { from(it) }” which descends one level to omit the “[applicationName]-[version]” directory:

root project’s build.gradle:

def distProjects = [':subproject1', ':subproject2', ':subproject3']
distProjects.each {
 evaluationDependsOn(it)
}
  task assembleApplications() {
 def distTasks = distProjects.collect { projName ->
   project(projName).tasks.getByName('distZip')
 }
 dependsOn distTasks
   destinationDir = new File( project.buildDir, 'tmp/applications' )
   doFirst {
  tmpDestinationDir = new File( project.buildDir, 'tmp/applications-projects' )
  copy {
   into tmpDestinationDir
   distTasks.each { distTask ->
    distTask.outputs.files.each { from zipTree(it) }
   }
  }
    // Create destinationDir var in closer scope so the copy spec can access it -.-
  def destinationDir = destinationDir
  copy {
   into destinationDir
   tmpDestinationDir.eachDir { from(it) }
  }
    }
}
  task masterZip(type: Zip, dependsOn: assembleApplications) {
       //bunch of other from's...
 from ( assembleApplications.destinationDir ) {
  into 'applications'
 }
}

Any idea how I can accomplish what I need more in a simpler, more elegant and gradle-like way?

Best regards, Mike


(Luke Daley) #2

It should be as simple as this:

def distProjects = [':subproject1', ':subproject2', ':subproject3']
distProjects.each { evaluationDependsOn(it) }
  task masterZip(type: Zip) {
    distProjects.each { with it.applicationDistribution }
}

Give that a go and let me know if it works. If it does, I’ll elaborate on why :slight_smile:


(Mike Meessen) #3

Wow, that was fast… again :slight_smile:

with one slight modification:

def distProjectNames = [':subproject1', ':subproject2', ':subproject3']
distProjectNames.each { evaluationDependsOn(it) }
def distProjects = distProjectNames.collect { project(it) }
task masterZip(type: Zip) {
    distProjects.each { with it.applicationDistribution }
}

that almost works - ok, I would’ve NEVER found that, can’t wait for your explanation!

The last issue I see, is that the resulting Zip file is a bit odd: It contains the same file multiple times in the same folder if it existed in the originating locations. I think this is allowed by the Zip spec, but I think gradle should make sure the zip contains the same file only once as this is totally unusual and unpacking might cause a lot of trouble… should we consider that a gradle bug?

Best regards, Mike


(Luke Daley) #4

With the duplicates, it’s a bit unclear what the default behaviour should be. I guess at least we should provide a way to easily de-dup. As a workaround, you could try assembling the content with a copy task first, and then just zipping that…

def distProjectNames = [':subproject1', ':subproject2', ':subproject3']
distProjectNames.each { evaluationDependsOn(it) }
def distProjects = distProjectNames.collect { project(it) }
task master(type: Copy {
   distProjects.each { with it.applicationDistribution }
}
task masterZip(type: Zip) {
    from master
}

The application plugin adds an ‘applicationDistribution’ property to the ‘Project’.

This is a ‘CopySpec’ which is used as the source of the application zip.

So, you can just use this as a copy spec in any other copy operation by using the dsl:org.gradle.api.tasks.Copy:with(CopySpec) method.

Gradle can infer the task dependencies based on the ‘applicationDistribution’ copy spec.


(Mike Meessen) #5

Ok, so here’s my working final result:

def distProjectNames = [':subproject1', ':subproject2', ':subproject3']
distProjectNames.each { evaluationDependsOn(it) }
def distProjects = distProjectNames.collect { project(it) }
task master(type: Copy) {
   distProjects.each { with it.applicationDistribution }
   into new File( project.buildDir, 'tmp/applications' )
}
task masterZip(type: Zip) {
    from master
}

About the multiple zip entries: imho, having the zip task ensure uniqueness would be a reasonable default as that’s what 99,9% of users / use cases would expect / need.

Anyway, having the option to ensure uniqueness in the first place would be the most important. Should I create a ticket for that? or do you want to do it?

Thanks again for your support, always highly appreciated!

Best regards, Mike


(Luke Daley) #6

Glad it’s all working for you.

If you could raise a new problem entry in this forum about the zip duplicates issue that would be appreciated. Just needs a few words and then a link to this entry.

From there I’ll export it to JIRA and link it back.


(Mike Meessen) #7

Done: http://forums.gradle.org/gradle/topics/add_an_option_to_avoid_duplicate_entries_when_creating_a_zip_file

Thanks again for your support.

Best regards, Mike